2024-07-27T02:53:17.685 INFO:root:teuthology version: 0.0.1.dev379+g53ce146 2024-07-27T02:53:17.710 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-07-27T02:53:17.801 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-07-26_23:45:44-fs-wip-pdonnell-testing-20240726.202642-debug-distro-default-smithi/7820977 branch: wip-pdonnell-testing-20240726.202642-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/fsync-tester}} email: pdonnell@redhat.com first_in_suite: false job_id: '7820977' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-07-26_23:45:44-fs-wip-pdonnell-testing-20240726.202642-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240726.202642-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_bal_replicate_threshold: 1 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: af8fdbf079811fe158ce62927bd145c35b74d4b8 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: af8fdbf079811fe158ce62927bd145c35b74d4b8 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240726.202642-debug sha1: af8fdbf079811fe158ce62927bd145c35b74d4b8 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 6029 sha1: af8fdbf079811fe158ce62927bd145c35b74d4b8 sleep_before_teardown: 0 subset: 246/512 suite: fs suite_branch: wip-pdonnell-testing-20240726.202642-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_af8fdbf079811fe158ce62927bd145c35b74d4b8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: af8fdbf079811fe158ce62927bd145c35b74d4b8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi067.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHptxK0B83hcHJLP2SKzB7VBLYWjEHCHzc6GIS3LgninERtt92ucW8bKlqiISfa1CbysOEPXoiaFrpVv0iK1YwA= smithi149.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKIfqcRHlA5gvPAVf43Kp29LjGY/IlwJHJral594hWNrBz0xHQ3VeSya6gDc7BjYzLEVI25wY5oOKA7RpKhAYUE= smithi175.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGJmhXHGIn5h8wsUIS1qktwFIBQrWTQaFmaI80gl8/98GUjr5VyqXl/7/tYJJwfb5ibnSTsOZb5Eu8rspbvJR4E= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - suites/fsync-tester.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 53ce1462e129f6eb4071986336534c740fdebd31 timestamp: 2024-07-26_23:45:44 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.4074525 2024-07-27T02:53:17.802 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_af8fdbf079811fe158ce62927bd145c35b74d4b8/qa; will attempt to use it 2024-07-27T02:53:17.803 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_af8fdbf079811fe158ce62927bd145c35b74d4b8/qa/tasks 2024-07-27T02:53:17.803 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-07-27T02:53:17.805 INFO:teuthology.task.internal:Checking packages... 2024-07-27T02:53:17.826 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'af8fdbf079811fe158ce62927bd145c35b74d4b8' 2024-07-27T02:53:17.826 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-07-27T02:53:17.826 INFO:teuthology.packaging:ref: None 2024-07-27T02:53:17.826 INFO:teuthology.packaging:tag: None 2024-07-27T02:53:17.826 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240726.202642-debug 2024-07-27T02:53:17.826 INFO:teuthology.packaging:sha1: af8fdbf079811fe158ce62927bd145c35b74d4b8 2024-07-27T02:53:17.826 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240726.202642-debug 2024-07-27T02:53:17.965 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-5358.gaf8fdbf0 2024-07-27T02:53:17.967 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-07-27T02:53:17.972 INFO:teuthology.task.internal:no buildpackages task found 2024-07-27T02:53:17.972 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-07-27T02:53:17.977 INFO:teuthology.task.internal:Saving configuration 2024-07-27T02:53:17.997 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-07-27T02:53:18.004 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-07-27T02:53:18.029 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-07-26_23:45:44-fs-wip-pdonnell-testing-20240726.202642-debug-distro-default-smithi/7820977', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-27 02:48:13.320152', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHptxK0B83hcHJLP2SKzB7VBLYWjEHCHzc6GIS3LgninERtt92ucW8bKlqiISfa1CbysOEPXoiaFrpVv0iK1YwA='} 2024-07-27T02:53:18.049 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-07-26_23:45:44-fs-wip-pdonnell-testing-20240726.202642-debug-distro-default-smithi/7820977', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-27 02:48:13.321650', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKIfqcRHlA5gvPAVf43Kp29LjGY/IlwJHJral594hWNrBz0xHQ3VeSya6gDc7BjYzLEVI25wY5oOKA7RpKhAYUE='} 2024-07-27T02:53:18.074 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-07-26_23:45:44-fs-wip-pdonnell-testing-20240726.202642-debug-distro-default-smithi/7820977', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-07-27 02:48:13.322942', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGJmhXHGIn5h8wsUIS1qktwFIBQrWTQaFmaI80gl8/98GUjr5VyqXl/7/tYJJwfb5ibnSTsOZb5Eu8rspbvJR4E='} 2024-07-27T02:53:18.074 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-07-27T02:53:18.080 INFO:teuthology.task.internal:roles: ubuntu@smithi067.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-07-27T02:53:18.080 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-07-27T02:53:18.080 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-07-27T02:53:18.080 INFO:teuthology.run_tasks:Running task console_log... 2024-07-27T02:53:18.173 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fca955efac0>, signals=[15]) 2024-07-27T02:53:18.174 INFO:teuthology.run_tasks:Running task internal.connect... 2024-07-27T02:53:18.179 INFO:teuthology.task.internal:Opening connections... 2024-07-27T02:53:18.179 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi067.front.sepia.ceph.com 2024-07-27T02:53:18.180 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-27T02:53:18.261 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi149.front.sepia.ceph.com 2024-07-27T02:53:18.262 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-27T02:53:18.348 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2024-07-27T02:53:18.349 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-27T02:53:18.434 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-07-27T02:53:18.440 DEBUG:teuthology.orchestra.run.smithi067:> uname -m 2024-07-27T02:53:18.459 INFO:teuthology.orchestra.run.smithi067.stdout:x86_64 2024-07-27T02:53:18.459 DEBUG:teuthology.orchestra.run.smithi067:> cat /etc/os-release 2024-07-27T02:53:18.516 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="CentOS Stream" 2024-07-27T02:53:18.517 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="9" 2024-07-27T02:53:18.517 INFO:teuthology.orchestra.run.smithi067.stdout:ID="centos" 2024-07-27T02:53:18.517 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE="rhel fedora" 2024-07-27T02:53:18.517 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="9" 2024-07-27T02:53:18.517 INFO:teuthology.orchestra.run.smithi067.stdout:PLATFORM_ID="platform:el9" 2024-07-27T02:53:18.517 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-27T02:53:18.517 INFO:teuthology.orchestra.run.smithi067.stdout:ANSI_COLOR="0;31" 2024-07-27T02:53:18.517 INFO:teuthology.orchestra.run.smithi067.stdout:LOGO="fedora-logo-icon" 2024-07-27T02:53:18.517 INFO:teuthology.orchestra.run.smithi067.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-27T02:53:18.517 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://centos.org/" 2024-07-27T02:53:18.517 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-27T02:53:18.518 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-27T02:53:18.518 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-27T02:53:18.519 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server 2024-07-27T02:53:18.536 DEBUG:teuthology.orchestra.run.smithi149:> uname -m 2024-07-27T02:53:18.554 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2024-07-27T02:53:18.554 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release 2024-07-27T02:53:18.611 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream" 2024-07-27T02:53:18.611 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="9" 2024-07-27T02:53:18.611 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos" 2024-07-27T02:53:18.612 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora" 2024-07-27T02:53:18.612 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="9" 2024-07-27T02:53:18.612 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el9" 2024-07-27T02:53:18.612 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-27T02:53:18.612 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2024-07-27T02:53:18.612 INFO:teuthology.orchestra.run.smithi149.stdout:LOGO="fedora-logo-icon" 2024-07-27T02:53:18.612 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-27T02:53:18.612 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/" 2024-07-27T02:53:18.612 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-27T02:53:18.612 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-27T02:53:18.612 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-27T02:53:18.613 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2024-07-27T02:53:18.636 DEBUG:teuthology.orchestra.run.smithi175:> uname -m 2024-07-27T02:53:18.653 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2024-07-27T02:53:18.654 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2024-07-27T02:53:18.710 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="CentOS Stream" 2024-07-27T02:53:18.710 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="9" 2024-07-27T02:53:18.710 INFO:teuthology.orchestra.run.smithi175.stdout:ID="centos" 2024-07-27T02:53:18.710 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="rhel fedora" 2024-07-27T02:53:18.711 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="9" 2024-07-27T02:53:18.711 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el9" 2024-07-27T02:53:18.711 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="CentOS Stream 9" 2024-07-27T02:53:18.711 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2024-07-27T02:53:18.711 INFO:teuthology.orchestra.run.smithi175.stdout:LOGO="fedora-logo-icon" 2024-07-27T02:53:18.711 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-07-27T02:53:18.711 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://centos.org/" 2024-07-27T02:53:18.711 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-07-27T02:53:18.711 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-07-27T02:53:18.711 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-07-27T02:53:18.711 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2024-07-27T02:53:18.736 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-07-27T02:53:18.746 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-07-27T02:53:18.754 INFO:teuthology.task.internal:Checking for old test directory... 2024-07-27T02:53:18.754 DEBUG:teuthology.orchestra.run.smithi067:> test '!' -e /home/ubuntu/cephtest 2024-07-27T02:53:18.757 DEBUG:teuthology.orchestra.run.smithi149:> test '!' -e /home/ubuntu/cephtest 2024-07-27T02:53:18.759 DEBUG:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2024-07-27T02:53:18.775 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-07-27T02:53:18.780 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-07-27T02:53:18.780 DEBUG:teuthology.orchestra.run.smithi067:> test -z $(ls -A /var/lib/ceph) 2024-07-27T02:53:18.815 DEBUG:teuthology.orchestra.run.smithi149:> test -z $(ls -A /var/lib/ceph) 2024-07-27T02:53:18.818 DEBUG:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2024-07-27T02:53:19.025 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-07-27T02:53:19.108 INFO:teuthology.run_tasks:Running task kernel... 2024-07-27T02:53:19.125 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-07-27T02:53:19.126 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-07-27T02:53:19.126 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-07-27T02:53:19.126 DEBUG:teuthology.orchestra.run.smithi067:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-27T02:53:19.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-27T02:53:19.144 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2024-07-27T02:53:19.203 INFO:teuthology.orchestra.run.smithi067.stdout:5.14.0-437.el9.x86_64 2024-07-27T02:53:19.204 INFO:teuthology.task.kernel:Running kernel on smithi067: 5.14.0-437.el9.x86_64 2024-07-27T02:53:19.204 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum install -y kernel 2024-07-27T02:53:21.453 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-07-27T02:53:26.070 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - AppStream 5.8 MB/s | 20 MB 00:03 2024-07-27T02:53:33.705 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - CRB 1.8 MB/s | 6.5 MB 00:03 2024-07-27T02:53:35.480 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 18 kB 00:00 2024-07-27T02:53:36.650 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 21 MB/s | 22 MB 00:01 2024-07-27T02:53:42.257 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-07-27T02:53:44.337 INFO:teuthology.orchestra.run.smithi067.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-07-27T02:53:44.338 INFO:teuthology.orchestra.run.smithi067.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-07-27T02:53:44.422 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-07-27T02:53:44.432 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-07-27T02:53:44.432 INFO:teuthology.orchestra.run.smithi067.stdout: Package Architecture Version Repository Size 2024-07-27T02:53:44.432 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-07-27T02:53:44.433 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-07-27T02:53:44.433 INFO:teuthology.orchestra.run.smithi067.stdout: kernel x86_64 5.14.0-480.el9 baseos 1.5 M 2024-07-27T02:53:44.433 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-07-27T02:53:44.433 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-core x86_64 5.14.0-480.el9 baseos 17 M 2024-07-27T02:53:44.433 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules x86_64 5.14.0-480.el9 baseos 36 M 2024-07-27T02:53:44.433 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules-core x86_64 5.14.0-480.el9 baseos 30 M 2024-07-27T02:53:44.433 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T02:53:44.433 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-07-27T02:53:44.433 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-07-27T02:53:44.433 INFO:teuthology.orchestra.run.smithi067.stdout:Install 4 Packages 2024-07-27T02:53:44.433 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T02:53:44.434 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 84 M 2024-07-27T02:53:44.434 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 126 M 2024-07-27T02:53:44.434 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-07-27T02:53:44.833 INFO:teuthology.orchestra.run.smithi067.stdout:(1/4): kernel-5.14.0-480.el9.x86_64.rpm 4.5 MB/s | 1.5 MB 00:00 2024-07-27T02:53:46.500 INFO:teuthology.orchestra.run.smithi067.stdout:(2/4): kernel-core-5.14.0-480.el9.x86_64.rpm 8.5 MB/s | 17 MB 00:02 2024-07-27T02:53:51.941 INFO:teuthology.orchestra.run.smithi067.stdout:(3/4): kernel-modules-5.14.0-480.el9.x86_64.rpm 4.8 MB/s | 36 MB 00:07 2024-07-27T02:53:52.808 INFO:teuthology.orchestra.run.smithi067.stdout:(4/4): kernel-modules-core-5.14.0-480.el9.x86_6 3.8 MB/s | 30 MB 00:07 2024-07-27T02:53:52.808 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-07-27T02:53:52.808 INFO:teuthology.orchestra.run.smithi067.stdout:Total 10 MB/s | 84 MB 00:08 2024-07-27T02:53:53.577 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-07-27T02:53:53.998 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-07-27T02:53:53.998 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-07-27T02:53:55.035 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-07-27T02:53:55.035 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-07-27T02:53:57.650 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-07-27T02:53:58.333 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-modules-core-5.14.0-480.el9.x86_64 1/4 2024-07-27T02:53:58.425 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-core-5.14.0-480.el9.x86_64 2/4 2024-07-27T02:54:01.196 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-core-5.14.0-480.el9.x86_64 2/4 2024-07-27T02:54:01.351 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-modules-5.14.0-480.el9.x86_64 3/4 2024-07-27T02:54:06.556 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-modules-5.14.0-480.el9.x86_64 3/4 2024-07-27T02:54:06.619 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-5.14.0-480.el9.x86_64 4/4 2024-07-27T02:54:11.840 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-modules-core-5.14.0-480.el9.x86_64 4/4 2024-07-27T02:54:43.307 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-core-5.14.0-480.el9.x86_64 4/4 2024-07-27T02:54:43.349 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-modules-5.14.0-480.el9.x86_64 4/4 2024-07-27T02:54:43.713 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-5.14.0-480.el9.x86_64 4/4 2024-07-27T02:54:43.714 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-5.14.0-480.el9.x86_64 1/4 2024-07-27T02:54:43.714 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-core-5.14.0-480.el9.x86_64 2/4 2024-07-27T02:54:43.714 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-modules-5.14.0-480.el9.x86_64 3/4 2024-07-27T02:54:44.517 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-modules-core-5.14.0-480.el9.x86_64 4/4 2024-07-27T02:54:44.518 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T02:54:44.518 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-07-27T02:54:44.518 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-5.14.0-480.el9.x86_64 2024-07-27T02:54:44.518 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-core-5.14.0-480.el9.x86_64 2024-07-27T02:54:44.518 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules-5.14.0-480.el9.x86_64 2024-07-27T02:54:44.518 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules-core-5.14.0-480.el9.x86_64 2024-07-27T02:54:44.518 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T02:54:44.518 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-07-27T02:54:44.900 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q kernel | sort -rV | head -n 1 2024-07-27T02:54:44.989 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-5.14.0-480.el9.x86_64 2024-07-27T02:54:44.989 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-480.el9.x86_64 2024-07-27T02:54:44.989 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-480.el9.x86_64 2024-07-27T02:54:44.989 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-07-27T02:54:44.990 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-07-27T02:54:44.990 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-07-27T02:54:44.990 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi067.front.sepia.ceph.com, path=None, version=distro) 2024-07-27T02:54:44.990 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q kernel | sort -rV | head -n 1 2024-07-27T02:54:45.048 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-5.14.0-480.el9.x86_64 2024-07-27T02:54:45.048 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-480.el9.x86_64 2024-07-27T02:54:45.049 DEBUG:teuthology.orchestra.run.smithi067:> sudo rpm -qi grub2-tools 2024-07-27T02:54:45.088 INFO:teuthology.orchestra.run.smithi067.stdout:Name : grub2-tools 2024-07-27T02:54:45.088 INFO:teuthology.orchestra.run.smithi067.stdout:Epoch : 1 2024-07-27T02:54:45.088 INFO:teuthology.orchestra.run.smithi067.stdout:Version : 2.06 2024-07-27T02:54:45.088 INFO:teuthology.orchestra.run.smithi067.stdout:Release : 68.el9 2024-07-27T02:54:45.088 INFO:teuthology.orchestra.run.smithi067.stdout:Architecture: x86_64 2024-07-27T02:54:45.088 INFO:teuthology.orchestra.run.smithi067.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-07-27T02:54:45.088 INFO:teuthology.orchestra.run.smithi067.stdout:Group : Unspecified 2024-07-27T02:54:45.088 INFO:teuthology.orchestra.run.smithi067.stdout:Size : 8264254 2024-07-27T02:54:45.088 INFO:teuthology.orchestra.run.smithi067.stdout:License : GPLv3+ 2024-07-27T02:54:45.089 INFO:teuthology.orchestra.run.smithi067.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-07-27T02:54:45.089 INFO:teuthology.orchestra.run.smithi067.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-07-27T02:54:45.089 INFO:teuthology.orchestra.run.smithi067.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-07-27T02:54:45.089 INFO:teuthology.orchestra.run.smithi067.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-07-27T02:54:45.089 INFO:teuthology.orchestra.run.smithi067.stdout:Packager : builder@centos.org 2024-07-27T02:54:45.089 INFO:teuthology.orchestra.run.smithi067.stdout:Vendor : CentOS 2024-07-27T02:54:45.089 INFO:teuthology.orchestra.run.smithi067.stdout:URL : http://www.gnu.org/software/grub/ 2024-07-27T02:54:45.089 INFO:teuthology.orchestra.run.smithi067.stdout:Summary : Support tools for GRUB. 2024-07-27T02:54:45.089 INFO:teuthology.orchestra.run.smithi067.stdout:Description : 2024-07-27T02:54:45.089 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T02:54:45.089 INFO:teuthology.orchestra.run.smithi067.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-07-27T02:54:45.089 INFO:teuthology.orchestra.run.smithi067.stdout:customizable bootloader with modular architecture. It supports a rich 2024-07-27T02:54:45.089 INFO:teuthology.orchestra.run.smithi067.stdout:variety of kernel formats, file systems, computer architectures and 2024-07-27T02:54:45.089 INFO:teuthology.orchestra.run.smithi067.stdout:hardware devices. 2024-07-27T02:54:45.089 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T02:54:45.090 INFO:teuthology.orchestra.run.smithi067.stdout:This subpackage provides tools for support of all platforms. 2024-07-27T02:54:45.090 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-07-27T02:54:45.090 INFO:teuthology.task.kernel:Updating grub on smithi067 to boot 5.14.0-480.el9.x86_64 2024-07-27T02:54:45.091 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-07-27T02:54:45.741 INFO:teuthology.orchestra.run.smithi067.stderr:Generating grub configuration file ... 2024-07-27T02:54:47.072 INFO:teuthology.orchestra.run.smithi067.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-07-27T02:54:47.104 INFO:teuthology.orchestra.run.smithi067.stderr:done 2024-07-27T02:54:47.107 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2024-07-27T02:54:47.122 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.piZWsr4492 2024-07-27T02:54:47.122 DEBUG:teuthology.orchestra.run.smithi067:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.piZWsr4492 2024-07-27T02:54:47.188 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.piZWsr4492 2024-07-27T02:54:47.326 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.piZWsr4492 is 6KB 2024-07-27T02:54:47.375 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.piZWsr4492 2024-07-27T02:54:47.389 DEBUG:teuthology.orchestra.run.smithi067:> sudo /bin/ls /boot/loader/entries || true 2024-07-27T02:54:47.452 INFO:teuthology.orchestra.run.smithi067.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-07-27T02:54:47.452 INFO:teuthology.orchestra.run.smithi067.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-07-27T02:54:47.452 INFO:teuthology.orchestra.run.smithi067.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-07-27T02:54:47.452 INFO:teuthology.orchestra.run.smithi067.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-480.el9.x86_64.conf 2024-07-27T02:54:47.453 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-480.el9.x86_64 2024-07-27T02:54:47.584 DEBUG:teuthology.orchestra.run.smithi067:> sudo shutdown -r now 2024-07-27T02:55:17.586 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-07-27T02:55:17.587 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-07-27T02:55:17.588 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-27T02:55:52.035 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2024-07-27T02:56:01.036 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-07-27T02:56:01.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-27T02:56:32.847 DEBUG:teuthology.orchestra.run.smithi067:> true 2024-07-27T02:56:33.283 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-07-27T02:56:33.283 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-480.el9.x86_64"... 2024-07-27T02:56:33.283 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2024-07-27T02:56:33.298 INFO:teuthology.orchestra.run.smithi067.stdout:5.14.0-480.el9.x86_64 2024-07-27T02:56:33.298 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-480.el9.x86_64 vs 5.14.0-480.el9.x86_64 2024-07-27T02:56:33.298 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-07-27T02:56:33.299 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-07-27T02:56:34.299 DEBUG:teuthology.parallel:result is None 2024-07-27T02:56:34.300 INFO:teuthology.run_tasks:Running task internal.base... 2024-07-27T02:56:34.308 INFO:teuthology.task.internal:Creating test directory... 2024-07-27T02:56:34.308 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-27T02:56:34.311 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-27T02:56:34.313 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-07-27T02:56:34.331 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-07-27T02:56:34.371 INFO:teuthology.run_tasks:Running task internal.archive... 2024-07-27T02:56:34.382 INFO:teuthology.task.internal:Creating archive directory... 2024-07-27T02:56:34.382 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-27T02:56:34.385 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-27T02:56:34.387 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-07-27T02:56:34.474 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-07-27T02:56:34.481 INFO:teuthology.task.internal:Enabling coredump saving... 2024-07-27T02:56:34.481 DEBUG:teuthology.orchestra.run.smithi149:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-27T02:56:34.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-27T02:56:34.497 DEBUG:teuthology.orchestra.run.smithi175:> test -f /run/.containerenv -o -f /.dockerenv 2024-07-27T02:56:34.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-27T02:56:34.513 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-27T02:56:34.515 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-27T02:56:34.540 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-07-27T02:56:34.580 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-27T02:56:34.591 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-27T02:56:34.597 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-27T02:56:34.609 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-27T02:56:34.610 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-27T02:56:34.632 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-07-27T02:56:34.633 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-07-27T02:56:34.641 INFO:teuthology.task.internal:Configuring sudo... 2024-07-27T02:56:34.641 DEBUG:teuthology.orchestra.run.smithi067:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-27T02:56:34.643 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-27T02:56:34.654 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-07-27T02:56:34.699 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-07-27T02:56:34.709 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-07-27T02:56:34.709 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-27T02:56:34.711 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-27T02:56:34.721 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-07-27T02:56:34.755 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-27T02:56:34.811 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-27T02:56:34.851 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-27T02:56:34.930 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-27T02:56:34.968 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-27T02:56:34.968 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-27T02:56:35.031 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-27T02:56:35.077 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-27T02:56:35.124 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-27T02:56:35.208 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-27T02:56:35.249 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-07-27T02:56:35.249 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-27T02:56:35.315 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-27T02:56:35.360 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-07-27T02:56:35.416 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-27T02:56:35.500 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-07-27T02:56:35.543 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-27T02:56:35.544 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-07-27T02:56:35.611 DEBUG:teuthology.orchestra.run.smithi067:> sudo service rsyslog restart 2024-07-27T02:56:35.613 DEBUG:teuthology.orchestra.run.smithi149:> sudo service rsyslog restart 2024-07-27T02:56:35.615 DEBUG:teuthology.orchestra.run.smithi175:> sudo service rsyslog restart 2024-07-27T02:56:35.654 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-27T02:56:35.687 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-27T02:56:35.688 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-27T02:56:36.159 INFO:teuthology.run_tasks:Running task internal.timer... 2024-07-27T02:56:36.167 INFO:teuthology.task.internal:Starting timer... 2024-07-27T02:56:36.167 INFO:teuthology.run_tasks:Running task pcp... 2024-07-27T02:56:36.193 INFO:teuthology.run_tasks:Running task selinux... 2024-07-27T02:56:36.201 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-07-27T02:56:36.202 DEBUG:teuthology.orchestra.run.smithi067:> sudo service auditd rotate 2024-07-27T02:56:36.286 INFO:teuthology.orchestra.run.smithi067.stdout:Rotating logs: 2024-07-27T02:56:36.288 DEBUG:teuthology.orchestra.run.smithi149:> sudo service auditd rotate 2024-07-27T02:56:36.345 INFO:teuthology.orchestra.run.smithi149.stdout:Rotating logs: 2024-07-27T02:56:36.347 DEBUG:teuthology.orchestra.run.smithi175:> sudo service auditd rotate 2024-07-27T02:56:36.406 INFO:teuthology.orchestra.run.smithi175.stdout:Rotating logs: 2024-07-27T02:56:36.407 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-07-27T02:56:36.408 DEBUG:teuthology.orchestra.run.smithi067:> /usr/sbin/getenforce 2024-07-27T02:56:36.430 INFO:teuthology.orchestra.run.smithi067.stdout:Permissive 2024-07-27T02:56:36.431 DEBUG:teuthology.orchestra.run.smithi149:> /usr/sbin/getenforce 2024-07-27T02:56:36.457 INFO:teuthology.orchestra.run.smithi149.stdout:Permissive 2024-07-27T02:56:36.457 DEBUG:teuthology.orchestra.run.smithi175:> /usr/sbin/getenforce 2024-07-27T02:56:36.479 INFO:teuthology.orchestra.run.smithi175.stdout:Permissive 2024-07-27T02:56:36.479 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi067.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi149.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi175.front.sepia.ceph.com': 'permissive'} 2024-07-27T02:56:36.479 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-27T02:56:36.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-27T02:56:36.507 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-27T02:56:36.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-27T02:56:36.534 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-27T02:56:36.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-27T02:56:36.561 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-07-27T02:56:36.561 DEBUG:teuthology.orchestra.run.smithi067:> sudo /usr/sbin/setenforce permissive 2024-07-27T02:56:36.597 DEBUG:teuthology.orchestra.run.smithi149:> sudo /usr/sbin/setenforce permissive 2024-07-27T02:56:36.625 DEBUG:teuthology.orchestra.run.smithi175:> sudo /usr/sbin/setenforce permissive 2024-07-27T02:56:36.664 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-07-27T02:56:36.674 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-07-27T02:56:36.685 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-07-27T02:56:36.686 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi067.front.sepia.ceph.com,smithi149.front.sepia.ceph.com,smithi175.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-07-27T03:01:21.159 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi067.front.sepia.ceph.com'), Remote(name='ubuntu@smithi149.front.sepia.ceph.com'), Remote(name='ubuntu@smithi175.front.sepia.ceph.com')] 2024-07-27T03:01:21.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-07-27T03:01:21.161 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-27T03:01:21.242 DEBUG:teuthology.orchestra.run.smithi067:> true 2024-07-27T03:01:21.328 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-07-27T03:01:21.328 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-07-27T03:01:21.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-27T03:01:21.403 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-07-27T03:01:21.486 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-07-27T03:01:21.486 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-07-27T03:01:21.486 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-07-27T03:01:21.561 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-07-27T03:01:21.645 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-07-27T03:01:21.646 INFO:teuthology.run_tasks:Running task clock... 2024-07-27T03:01:21.657 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-07-27T03:01:21.658 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-27T03:01:21.658 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-27T03:01:21.661 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-27T03:01:21.661 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-27T03:01:21.663 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-07-27T03:01:21.663 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-27T03:01:21.693 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-27T03:01:21.693 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-27T03:01:21.708 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-27T03:01:21.710 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-27T03:01:21.714 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-07-27T03:01:21.727 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-07-27T03:01:21.732 INFO:teuthology.orchestra.run.smithi149.stderr:sudo: ntpd: command not found 2024-07-27T03:01:21.737 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: ntpd: command not found 2024-07-27T03:01:21.743 INFO:teuthology.orchestra.run.smithi149.stdout:506 Cannot talk to daemon 2024-07-27T03:01:21.749 INFO:teuthology.orchestra.run.smithi067.stdout:506 Cannot talk to daemon 2024-07-27T03:01:21.749 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: ntpd: command not found 2024-07-27T03:01:21.757 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-27T03:01:21.760 INFO:teuthology.orchestra.run.smithi175.stdout:506 Cannot talk to daemon 2024-07-27T03:01:21.764 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-27T03:01:21.770 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-27T03:01:21.773 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-07-27T03:01:21.779 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-27T03:01:21.785 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-07-27T03:01:21.833 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-07-27T03:01:21.835 INFO:teuthology.orchestra.run.smithi067.stderr:bash: line 1: ntpq: command not found 2024-07-27T03:01:21.836 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-27T03:01:21.836 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-07-27T03:01:21.836 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-27T03:01:21.837 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-27T03:01:21.837 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-27T03:01:21.837 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-27T03:01:21.839 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-27T03:01:21.839 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2024-07-27T03:01:21.839 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-27T03:01:21.839 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-27T03:01:21.839 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-27T03:01:21.839 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-27T03:01:21.846 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-07-27T03:01:21.850 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-27T03:01:21.850 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-07-27T03:01:21.850 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-27T03:01:21.850 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-27T03:01:21.850 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-27T03:01:21.850 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-07-27T03:01:21.851 INFO:teuthology.run_tasks:Running task install... 2024-07-27T03:01:21.859 DEBUG:teuthology.task.install:project ceph 2024-07-27T03:01:21.859 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'af8fdbf079811fe158ce62927bd145c35b74d4b8'}} 2024-07-27T03:01:21.859 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'af8fdbf079811fe158ce62927bd145c35b74d4b8'} 2024-07-27T03:01:21.859 INFO:teuthology.task.install:Using flavor: default 2024-07-27T03:01:21.864 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-07-27T03:01:21.864 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-07-27T03:01:21.865 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'af8fdbf079811fe158ce62927bd145c35b74d4b8', 'tag': None, 'wait_for_package': False} 2024-07-27T03:01:21.865 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=af8fdbf079811fe158ce62927bd145c35b74d4b8 2024-07-27T03:01:21.867 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'af8fdbf079811fe158ce62927bd145c35b74d4b8', 'tag': None, 'wait_for_package': False} 2024-07-27T03:01:21.868 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=af8fdbf079811fe158ce62927bd145c35b74d4b8 2024-07-27T03:01:21.868 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'af8fdbf079811fe158ce62927bd145c35b74d4b8', 'tag': None, 'wait_for_package': False} 2024-07-27T03:01:21.869 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=af8fdbf079811fe158ce62927bd145c35b74d4b8 2024-07-27T03:01:22.019 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240726.202642-debug/af8fdbf079811fe158ce62927bd145c35b74d4b8/centos/9/flavors/default/ 2024-07-27T03:01:22.019 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5358.gaf8fdbf0 2024-07-27T03:01:22.022 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240726.202642-debug/af8fdbf079811fe158ce62927bd145c35b74d4b8/centos/9/flavors/default/ 2024-07-27T03:01:22.022 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5358.gaf8fdbf0 2024-07-27T03:01:22.026 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240726.202642-debug/af8fdbf079811fe158ce62927bd145c35b74d4b8/centos/9/flavors/default/ 2024-07-27T03:01:22.026 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5358.gaf8fdbf0 2024-07-27T03:01:22.135 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240726.202642-debug/af8fdbf079811fe158ce62927bd145c35b74d4b8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240726.202642-debug/af8fdbf079811fe158ce62927bd145c35b74d4b8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240726.202642-debug/af8fdbf079811fe158ce62927bd145c35b74d4b8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-27T03:01:22.136 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-07-27T03:01:22.136 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-27T03:01:22.144 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240726.202642-debug/af8fdbf079811fe158ce62927bd145c35b74d4b8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240726.202642-debug/af8fdbf079811fe158ce62927bd145c35b74d4b8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240726.202642-debug/af8fdbf079811fe158ce62927bd145c35b74d4b8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-27T03:01:22.145 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-27T03:01:22.145 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-27T03:01:22.161 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-07-27T03:01:22.161 DEBUG:teuthology.orchestra.run.smithi149:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/af8fdbf079811fe158ce62927bd145c35b74d4b8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-27T03:01:22.171 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-07-27T03:01:22.171 DEBUG:teuthology.orchestra.run.smithi067:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/af8fdbf079811fe158ce62927bd145c35b74d4b8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-27T03:01:22.206 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240726.202642-debug/af8fdbf079811fe158ce62927bd145c35b74d4b8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240726.202642-debug/af8fdbf079811fe158ce62927bd145c35b74d4b8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240726.202642-debug/af8fdbf079811fe158ce62927bd145c35b74d4b8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-07-27T03:01:22.207 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-27T03:01:22.207 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-07-27T03:01:22.225 DEBUG:teuthology.orchestra.run.smithi149:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-27T03:01:22.231 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-07-27T03:01:22.231 DEBUG:teuthology.orchestra.run.smithi175:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/af8fdbf079811fe158ce62927bd145c35b74d4b8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-07-27T03:01:22.239 DEBUG:teuthology.orchestra.run.smithi067:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-27T03:01:22.295 DEBUG:teuthology.orchestra.run.smithi175:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-07-27T03:01:22.298 DEBUG:teuthology.orchestra.run.smithi149:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-27T03:01:22.315 DEBUG:teuthology.orchestra.run.smithi067:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-27T03:01:22.361 INFO:teuthology.orchestra.run.smithi149.stdout:check_obsoletes = 1 2024-07-27T03:01:22.363 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum clean all 2024-07-27T03:01:22.367 DEBUG:teuthology.orchestra.run.smithi175:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-07-27T03:01:22.380 INFO:teuthology.orchestra.run.smithi067.stdout:check_obsoletes = 1 2024-07-27T03:01:22.381 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum clean all 2024-07-27T03:01:22.429 INFO:teuthology.orchestra.run.smithi175.stdout:check_obsoletes = 1 2024-07-27T03:01:22.431 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum clean all 2024-07-27T03:01:22.587 INFO:teuthology.orchestra.run.smithi149.stdout:45 files removed 2024-07-27T03:01:22.615 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-07-27T03:01:22.633 INFO:teuthology.orchestra.run.smithi067.stdout:45 files removed 2024-07-27T03:01:22.653 INFO:teuthology.orchestra.run.smithi175.stdout:45 files removed 2024-07-27T03:01:22.660 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-07-27T03:01:22.681 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-07-27T03:01:23.205 INFO:teuthology.orchestra.run.smithi149.stdout:ceph packages for x86_64 256 kB/s | 83 kB 00:00 2024-07-27T03:01:23.259 INFO:teuthology.orchestra.run.smithi175.stdout:ceph packages for x86_64 261 kB/s | 83 kB 00:00 2024-07-27T03:01:23.276 INFO:teuthology.orchestra.run.smithi067.stdout:ceph packages for x86_64 258 kB/s | 83 kB 00:00 2024-07-27T03:01:23.547 INFO:teuthology.orchestra.run.smithi149.stdout:ceph noarch packages 80 kB/s | 24 kB 00:00 2024-07-27T03:01:23.593 INFO:teuthology.orchestra.run.smithi175.stdout:ceph noarch packages 82 kB/s | 24 kB 00:00 2024-07-27T03:01:23.635 INFO:teuthology.orchestra.run.smithi067.stdout:ceph noarch packages 79 kB/s | 24 kB 00:00 2024-07-27T03:01:23.866 INFO:teuthology.orchestra.run.smithi149.stdout:ceph source packages 6.8 kB/s | 1.9 kB 00:00 2024-07-27T03:01:23.926 INFO:teuthology.orchestra.run.smithi175.stdout:ceph source packages 6.4 kB/s | 1.9 kB 00:00 2024-07-27T03:01:23.994 INFO:teuthology.orchestra.run.smithi067.stdout:ceph source packages 6.1 kB/s | 1.9 kB 00:00 2024-07-27T03:01:24.553 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-07-27T03:01:25.276 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 6.2 MB/s | 8.2 MB 00:01 2024-07-27T03:01:26.764 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.2 MB 00:02 2024-07-27T03:01:27.611 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 16 MB/s | 20 MB 00:01 2024-07-27T03:01:29.255 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - AppStream 5.6 MB/s | 20 MB 00:03 2024-07-27T03:01:29.524 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 12 MB/s | 20 MB 00:01 2024-07-27T03:01:34.063 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - CRB 7.2 MB/s | 6.5 MB 00:00 2024-07-27T03:01:34.830 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 5.9 MB/s | 6.5 MB 00:01 2024-07-27T03:01:34.830 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 2.3 MB/s | 6.5 MB 00:02 2024-07-27T03:01:35.531 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - Extras packages 97 kB/s | 18 kB 00:00 2024-07-27T03:01:35.661 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 18 kB 00:00 2024-07-27T03:01:35.833 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 76 kB/s | 18 kB 00:00 2024-07-27T03:01:36.985 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 17 MB/s | 22 MB 00:01 2024-07-27T03:01:37.066 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 17 MB/s | 22 MB 00:01 2024-07-27T03:01:37.120 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 19 MB/s | 22 MB 00:01 2024-07-27T03:01:42.422 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-07-27T03:01:42.493 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-07-27T03:01:42.633 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-07-27T03:01:44.672 INFO:teuthology.orchestra.run.smithi175.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-27T03:01:44.673 INFO:teuthology.orchestra.run.smithi175.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-27T03:01:44.684 INFO:teuthology.orchestra.run.smithi175.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-07-27T03:01:44.685 INFO:teuthology.orchestra.run.smithi175.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-07-27T03:01:44.686 INFO:teuthology.orchestra.run.smithi175.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-07-27T03:01:44.687 INFO:teuthology.orchestra.run.smithi175.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-07-27T03:01:44.688 INFO:teuthology.orchestra.run.smithi175.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-07-27T03:01:44.691 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-07-27T03:01:44.691 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-07-27T03:01:44.755 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-07-27T03:01:44.770 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-07-27T03:01:44.770 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-07-27T03:01:44.770 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-07-27T03:01:44.770 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-07-27T03:01:44.770 INFO:teuthology.orchestra.run.smithi175.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-07-27T03:01:44.770 INFO:teuthology.orchestra.run.smithi175.stdout: ceph x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 6.6 k 2024-07-27T03:01:44.770 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 5.5 M 2024-07-27T03:01:44.770 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 872 k 2024-07-27T03:01:44.770 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 148 k 2024-07-27T03:01:44.771 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 1.5 M 2024-07-27T03:01:44.771 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 149 k 2024-07-27T03:01:44.771 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 14 M 2024-07-27T03:01:44.771 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 7.4 M 2024-07-27T03:01:44.771 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 49 k 2024-07-27T03:01:44.771 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 10 M 2024-07-27T03:01:44.771 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 44 M 2024-07-27T03:01:44.771 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 274 k 2024-07-27T03:01:44.771 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 770 k 2024-07-27T03:01:44.771 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 225 k 2024-07-27T03:01:44.771 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-shell x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 27 k 2024-07-27T03:01:44.772 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 18 k 2024-07-27T03:01:44.772 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-07-27T03:01:44.772 INFO:teuthology.orchestra.run.smithi175.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-07-27T03:01:44.772 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-07-27T03:01:44.772 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 32 k 2024-07-27T03:01:44.772 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2 x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 766 k 2024-07-27T03:01:44.772 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 127 k 2024-07-27T03:01:44.772 INFO:teuthology.orchestra.run.smithi175.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-07-27T03:01:44.772 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-07-27T03:01:44.772 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-07-27T03:01:44.772 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-07-27T03:01:44.772 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-07-27T03:01:44.773 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-07-27T03:01:44.773 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 188 k 2024-07-27T03:01:44.773 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 371 k 2024-07-27T03:01:44.773 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 355 k 2024-07-27T03:01:44.773 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 116 k 2024-07-27T03:01:44.773 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 86 k 2024-07-27T03:01:44.773 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 3.1 M 2024-07-27T03:01:44.773 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 170 k 2024-07-27T03:01:44.773 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-07-27T03:01:44.773 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrading: 2024-07-27T03:01:44.773 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-07-27T03:01:44.773 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-07-27T03:01:44.773 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-07-27T03:01:44.774 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-07-27T03:01:44.774 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-07-27T03:01:44.774 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-07-27T03:01:44.774 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-07-27T03:01:44.774 INFO:teuthology.orchestra.run.smithi175.stdout: librados2 x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 3.4 M 2024-07-27T03:01:44.774 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1 x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 3.1 M 2024-07-27T03:01:44.774 INFO:teuthology.orchestra.run.smithi175.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-07-27T03:01:44.774 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-07-27T03:01:44.774 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-07-27T03:01:44.774 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-07-27T03:01:44.774 INFO:teuthology.orchestra.run.smithi175.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-07-27T03:01:44.774 INFO:teuthology.orchestra.run.smithi175.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-07-27T03:01:44.775 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-07-27T03:01:44.775 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 20 M 2024-07-27T03:01:44.775 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 34 k 2024-07-27T03:01:44.775 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 2.3 M 2024-07-27T03:01:44.775 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 284 k 2024-07-27T03:01:44.775 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 4.9 M 2024-07-27T03:01:44.775 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 18 M 2024-07-27T03:01:44.775 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 16 k 2024-07-27T03:01:44.775 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 25 k 2024-07-27T03:01:44.775 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-27T03:01:44.775 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-27T03:01:44.775 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-27T03:01:44.775 INFO:teuthology.orchestra.run.smithi175.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-07-27T03:01:44.776 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-27T03:01:44.776 INFO:teuthology.orchestra.run.smithi175.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-07-27T03:01:44.776 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-07-27T03:01:44.776 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-07-27T03:01:44.776 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 158 k 2024-07-27T03:01:44.776 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-27T03:01:44.776 INFO:teuthology.orchestra.run.smithi175.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-27T03:01:44.776 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-27T03:01:44.776 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-27T03:01:44.776 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-27T03:01:44.776 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1 x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 504 k 2024-07-27T03:01:44.776 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-27T03:01:44.776 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2 x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 5.3 M 2024-07-27T03:01:44.777 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-27T03:01:44.777 INFO:teuthology.orchestra.run.smithi175.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-07-27T03:01:44.777 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-27T03:01:44.777 INFO:teuthology.orchestra.run.smithi175.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-07-27T03:01:44.777 INFO:teuthology.orchestra.run.smithi175.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-07-27T03:01:44.777 INFO:teuthology.orchestra.run.smithi175.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-07-27T03:01:44.777 INFO:teuthology.orchestra.run.smithi175.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-27T03:01:44.777 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-27T03:01:44.777 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-07-27T03:01:44.777 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-07-27T03:01:44.777 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-07-27T03:01:44.777 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-07-27T03:01:44.778 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-27T03:01:44.778 INFO:teuthology.orchestra.run.smithi175.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-07-27T03:01:44.778 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-27T03:01:44.778 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-27T03:01:44.778 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-27T03:01:44.778 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 45 k 2024-07-27T03:01:44.778 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 138 k 2024-07-27T03:01:44.778 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-07-27T03:01:44.778 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-27T03:01:44.778 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-27T03:01:44.779 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-27T03:01:44.779 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-07-27T03:01:44.779 INFO:teuthology.orchestra.run.smithi175.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-07-27T03:01:44.779 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-27T03:01:44.779 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-27T03:01:44.779 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth noarch 1:2.32.0-1.el9 epel 216 k 2024-07-27T03:01:44.779 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-07-27T03:01:44.779 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-07-27T03:01:44.779 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-27T03:01:44.779 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-27T03:01:44.779 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-07-27T03:01:44.779 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-27T03:01:44.779 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-27T03:01:44.780 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-27T03:01:44.780 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-27T03:01:44.780 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-27T03:01:44.780 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-27T03:01:44.780 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-27T03:01:44.780 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-27T03:01:44.780 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-27T03:01:44.780 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-27T03:01:44.780 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-27T03:01:44.780 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-27T03:01:44.780 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-27T03:01:44.780 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-27T03:01:44.780 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-07-27T03:01:44.781 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-27T03:01:44.781 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-27T03:01:44.781 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-27T03:01:44.781 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-27T03:01:44.781 INFO:teuthology.orchestra.run.smithi175.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-07-27T03:01:44.781 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-27T03:01:44.781 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-27T03:01:44.781 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-27T03:01:44.781 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-27T03:01:44.781 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-07-27T03:01:44.781 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-27T03:01:44.781 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-27T03:01:44.781 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-27T03:01:44.782 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-27T03:01:44.782 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-27T03:01:44.782 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-27T03:01:44.782 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-27T03:01:44.782 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-27T03:01:44.782 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-27T03:01:44.782 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-07-27T03:01:44.782 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-27T03:01:44.782 INFO:teuthology.orchestra.run.smithi175.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-07-27T03:01:44.782 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-27T03:01:44.782 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-27T03:01:44.782 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-27T03:01:44.782 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-27T03:01:44.783 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-07-27T03:01:44.783 INFO:teuthology.orchestra.run.smithi175.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-07-27T03:01:44.783 INFO:teuthology.orchestra.run.smithi175.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-07-27T03:01:44.783 INFO:teuthology.orchestra.run.smithi175.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-27T03:01:44.783 INFO:teuthology.orchestra.run.smithi175.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-27T03:01:44.783 INFO:teuthology.orchestra.run.smithi175.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-07-27T03:01:44.783 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-27T03:01:44.783 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-07-27T03:01:44.783 INFO:teuthology.orchestra.run.smithi175.stdout:Installing weak dependencies: 2024-07-27T03:01:44.783 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-07-27T03:01:44.783 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-27T03:01:44.783 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-07-27T03:01:44.783 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-07-27T03:01:44.784 INFO:teuthology.orchestra.run.smithi175.stdout:Install 148 Packages 2024-07-27T03:01:44.784 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrade 12 Packages 2024-07-27T03:01:44.784 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-27T03:01:44.785 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 232 M 2024-07-27T03:01:44.785 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-07-27T03:01:44.785 INFO:teuthology.orchestra.run.smithi149.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-27T03:01:44.786 INFO:teuthology.orchestra.run.smithi149.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-27T03:01:44.786 INFO:teuthology.orchestra.run.smithi149.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-07-27T03:01:44.786 INFO:teuthology.orchestra.run.smithi149.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-07-27T03:01:44.786 INFO:teuthology.orchestra.run.smithi149.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-07-27T03:01:44.786 INFO:teuthology.orchestra.run.smithi149.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-07-27T03:01:44.787 INFO:teuthology.orchestra.run.smithi149.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-07-27T03:01:44.789 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-07-27T03:01:44.790 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-07-27T03:01:44.855 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-07-27T03:01:44.872 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-07-27T03:01:44.873 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-07-27T03:01:44.873 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-07-27T03:01:44.873 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-07-27T03:01:44.873 INFO:teuthology.orchestra.run.smithi149.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-07-27T03:01:44.873 INFO:teuthology.orchestra.run.smithi149.stdout: ceph x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 6.6 k 2024-07-27T03:01:44.873 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 5.5 M 2024-07-27T03:01:44.873 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 872 k 2024-07-27T03:01:44.873 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 148 k 2024-07-27T03:01:44.874 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 1.5 M 2024-07-27T03:01:44.874 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 149 k 2024-07-27T03:01:44.874 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 14 M 2024-07-27T03:01:44.874 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 7.4 M 2024-07-27T03:01:44.874 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 49 k 2024-07-27T03:01:44.874 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 10 M 2024-07-27T03:01:44.874 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 44 M 2024-07-27T03:01:44.874 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 274 k 2024-07-27T03:01:44.874 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 770 k 2024-07-27T03:01:44.874 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 225 k 2024-07-27T03:01:44.874 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-shell x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 27 k 2024-07-27T03:01:44.874 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 18 k 2024-07-27T03:01:44.874 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-07-27T03:01:44.875 INFO:teuthology.orchestra.run.smithi149.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-07-27T03:01:44.875 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-07-27T03:01:44.875 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 32 k 2024-07-27T03:01:44.875 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2 x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 766 k 2024-07-27T03:01:44.875 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 127 k 2024-07-27T03:01:44.875 INFO:teuthology.orchestra.run.smithi149.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-07-27T03:01:44.875 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-07-27T03:01:44.875 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-07-27T03:01:44.875 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-07-27T03:01:44.875 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-07-27T03:01:44.875 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-07-27T03:01:44.875 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 188 k 2024-07-27T03:01:44.876 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 371 k 2024-07-27T03:01:44.876 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 355 k 2024-07-27T03:01:44.876 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 116 k 2024-07-27T03:01:44.876 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 86 k 2024-07-27T03:01:44.876 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 3.1 M 2024-07-27T03:01:44.876 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 170 k 2024-07-27T03:01:44.876 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-07-27T03:01:44.876 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrading: 2024-07-27T03:01:44.876 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-07-27T03:01:44.876 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-07-27T03:01:44.876 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-07-27T03:01:44.876 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-07-27T03:01:44.876 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-07-27T03:01:44.877 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-07-27T03:01:44.877 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-07-27T03:01:44.877 INFO:teuthology.orchestra.run.smithi149.stdout: librados2 x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 3.4 M 2024-07-27T03:01:44.877 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1 x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 3.1 M 2024-07-27T03:01:44.877 INFO:teuthology.orchestra.run.smithi149.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-07-27T03:01:44.877 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-07-27T03:01:44.877 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-07-27T03:01:44.877 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-07-27T03:01:44.877 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-07-27T03:01:44.877 INFO:teuthology.orchestra.run.smithi149.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-07-27T03:01:44.877 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-07-27T03:01:44.877 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 20 M 2024-07-27T03:01:44.877 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 34 k 2024-07-27T03:01:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 2.3 M 2024-07-27T03:01:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 284 k 2024-07-27T03:01:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 4.9 M 2024-07-27T03:01:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 18 M 2024-07-27T03:01:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 16 k 2024-07-27T03:01:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 25 k 2024-07-27T03:01:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-27T03:01:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-27T03:01:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-27T03:01:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-07-27T03:01:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-27T03:01:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-07-27T03:01:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-07-27T03:01:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-07-27T03:01:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 158 k 2024-07-27T03:01:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-27T03:01:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-27T03:01:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-27T03:01:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-27T03:01:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-27T03:01:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1 x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 504 k 2024-07-27T03:01:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-27T03:01:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2 x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 5.3 M 2024-07-27T03:01:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-27T03:01:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-07-27T03:01:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-27T03:01:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-07-27T03:01:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-07-27T03:01:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-07-27T03:01:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-27T03:01:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-27T03:01:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-07-27T03:01:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-07-27T03:01:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-07-27T03:01:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-07-27T03:01:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-27T03:01:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-07-27T03:01:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-27T03:01:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-27T03:01:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-27T03:01:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 45 k 2024-07-27T03:01:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 138 k 2024-07-27T03:01:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-07-27T03:01:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-27T03:01:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-27T03:01:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-27T03:01:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-07-27T03:01:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-07-27T03:01:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-27T03:01:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-27T03:01:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth noarch 1:2.32.0-1.el9 epel 216 k 2024-07-27T03:01:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-07-27T03:01:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-07-27T03:01:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-27T03:01:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-27T03:01:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-07-27T03:01:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-27T03:01:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-27T03:01:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-27T03:01:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-27T03:01:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-27T03:01:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-27T03:01:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-27T03:01:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-27T03:01:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-27T03:01:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-27T03:01:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-27T03:01:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-27T03:01:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-27T03:01:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-27T03:01:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-07-27T03:01:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-27T03:01:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-27T03:01:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-27T03:01:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-27T03:01:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-07-27T03:01:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-27T03:01:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-27T03:01:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-27T03:01:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-27T03:01:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-07-27T03:01:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-27T03:01:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-27T03:01:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-27T03:01:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-27T03:01:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-27T03:01:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-27T03:01:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-27T03:01:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-27T03:01:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-27T03:01:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-07-27T03:01:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-27T03:01:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-07-27T03:01:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-27T03:01:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-27T03:01:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-27T03:01:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-27T03:01:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-07-27T03:01:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-07-27T03:01:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-07-27T03:01:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-27T03:01:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-27T03:01:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-07-27T03:01:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-27T03:01:44.886 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-07-27T03:01:44.886 INFO:teuthology.orchestra.run.smithi149.stdout:Installing weak dependencies: 2024-07-27T03:01:44.886 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-07-27T03:01:44.886 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-07-27T03:01:44.886 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-07-27T03:01:44.886 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-07-27T03:01:44.886 INFO:teuthology.orchestra.run.smithi149.stdout:Install 148 Packages 2024-07-27T03:01:44.886 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrade 12 Packages 2024-07-27T03:01:44.886 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-07-27T03:01:44.887 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 232 M 2024-07-27T03:01:44.887 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-07-27T03:01:45.047 INFO:teuthology.orchestra.run.smithi067.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-27T03:01:45.048 INFO:teuthology.orchestra.run.smithi067.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-07-27T03:01:45.060 INFO:teuthology.orchestra.run.smithi067.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-07-27T03:01:45.061 INFO:teuthology.orchestra.run.smithi067.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-07-27T03:01:45.061 INFO:teuthology.orchestra.run.smithi067.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-07-27T03:01:45.062 INFO:teuthology.orchestra.run.smithi067.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-07-27T03:01:45.064 INFO:teuthology.orchestra.run.smithi067.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-07-27T03:01:45.067 INFO:teuthology.orchestra.run.smithi067.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-07-27T03:01:45.068 INFO:teuthology.orchestra.run.smithi067.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-07-27T03:01:45.137 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-07-27T03:01:45.162 INFO:teuthology.orchestra.run.smithi067.stdout:======================================================================================= 2024-07-27T03:01:45.162 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2024-07-27T03:01:45.162 INFO:teuthology.orchestra.run.smithi067.stdout:======================================================================================= 2024-07-27T03:01:45.162 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-07-27T03:01:45.162 INFO:teuthology.orchestra.run.smithi067.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-07-27T03:01:45.162 INFO:teuthology.orchestra.run.smithi067.stdout: ceph x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 6.6 k 2024-07-27T03:01:45.162 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 5.5 M 2024-07-27T03:01:45.163 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 872 k 2024-07-27T03:01:45.163 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 148 k 2024-07-27T03:01:45.163 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 1.5 M 2024-07-27T03:01:45.163 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 149 k 2024-07-27T03:01:45.163 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 14 M 2024-07-27T03:01:45.163 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 7.4 M 2024-07-27T03:01:45.163 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 49 k 2024-07-27T03:01:45.163 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 10 M 2024-07-27T03:01:45.163 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 44 M 2024-07-27T03:01:45.163 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 274 k 2024-07-27T03:01:45.163 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 770 k 2024-07-27T03:01:45.163 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-mirror x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 225 k 2024-07-27T03:01:45.163 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-shell x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 27 k 2024-07-27T03:01:45.163 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-top noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 18 k 2024-07-27T03:01:45.164 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-07-27T03:01:45.164 INFO:teuthology.orchestra.run.smithi067.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-07-27T03:01:45.164 INFO:teuthology.orchestra.run.smithi067.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-07-27T03:01:45.164 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 32 k 2024-07-27T03:01:45.164 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2 x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 766 k 2024-07-27T03:01:45.164 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 127 k 2024-07-27T03:01:45.164 INFO:teuthology.orchestra.run.smithi067.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-07-27T03:01:45.164 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-07-27T03:01:45.164 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-07-27T03:01:45.164 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-07-27T03:01:45.165 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-07-27T03:01:45.165 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-07-27T03:01:45.165 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 188 k 2024-07-27T03:01:45.165 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 371 k 2024-07-27T03:01:45.165 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 355 k 2024-07-27T03:01:45.166 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 116 k 2024-07-27T03:01:45.166 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 86 k 2024-07-27T03:01:45.166 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 3.1 M 2024-07-27T03:01:45.166 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 170 k 2024-07-27T03:01:45.166 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-07-27T03:01:45.166 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrading: 2024-07-27T03:01:45.166 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-07-27T03:01:45.166 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-07-27T03:01:45.166 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-07-27T03:01:45.166 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-07-27T03:01:45.166 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-07-27T03:01:45.166 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-07-27T03:01:45.166 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-07-27T03:01:45.167 INFO:teuthology.orchestra.run.smithi067.stdout: librados2 x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 3.4 M 2024-07-27T03:01:45.167 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1 x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 3.1 M 2024-07-27T03:01:45.167 INFO:teuthology.orchestra.run.smithi067.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-07-27T03:01:45.167 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-07-27T03:01:45.167 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-07-27T03:01:45.167 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-07-27T03:01:45.167 INFO:teuthology.orchestra.run.smithi067.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-07-27T03:01:45.167 INFO:teuthology.orchestra.run.smithi067.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-07-27T03:01:45.167 INFO:teuthology.orchestra.run.smithi067.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-07-27T03:01:45.167 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 20 M 2024-07-27T03:01:45.167 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 34 k 2024-07-27T03:01:45.167 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 2.3 M 2024-07-27T03:01:45.167 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 284 k 2024-07-27T03:01:45.167 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 4.9 M 2024-07-27T03:01:45.168 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 18 M 2024-07-27T03:01:45.168 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5358.gaf8fdbf0.el9 ceph-noarch 16 k 2024-07-27T03:01:45.168 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 25 k 2024-07-27T03:01:45.168 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-07-27T03:01:45.168 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-07-27T03:01:45.168 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-07-27T03:01:45.168 INFO:teuthology.orchestra.run.smithi067.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-07-27T03:01:45.168 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-07-27T03:01:45.168 INFO:teuthology.orchestra.run.smithi067.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-07-27T03:01:45.168 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-07-27T03:01:45.168 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-07-27T03:01:45.168 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 158 k 2024-07-27T03:01:45.168 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-07-27T03:01:45.168 INFO:teuthology.orchestra.run.smithi067.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-07-27T03:01:45.169 INFO:teuthology.orchestra.run.smithi067.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-07-27T03:01:45.169 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-07-27T03:01:45.169 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-07-27T03:01:45.169 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1 x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 504 k 2024-07-27T03:01:45.169 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-07-27T03:01:45.169 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2 x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 5.3 M 2024-07-27T03:01:45.169 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-07-27T03:01:45.169 INFO:teuthology.orchestra.run.smithi067.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-07-27T03:01:45.169 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-07-27T03:01:45.169 INFO:teuthology.orchestra.run.smithi067.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-07-27T03:01:45.169 INFO:teuthology.orchestra.run.smithi067.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-07-27T03:01:45.169 INFO:teuthology.orchestra.run.smithi067.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-07-27T03:01:45.169 INFO:teuthology.orchestra.run.smithi067.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-07-27T03:01:45.169 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-07-27T03:01:45.169 INFO:teuthology.orchestra.run.smithi067.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-07-27T03:01:45.170 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-07-27T03:01:45.170 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-07-27T03:01:45.170 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-07-27T03:01:45.170 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-07-27T03:01:45.170 INFO:teuthology.orchestra.run.smithi067.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-07-27T03:01:45.170 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-07-27T03:01:45.170 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-07-27T03:01:45.170 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-07-27T03:01:45.170 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 45 k 2024-07-27T03:01:45.170 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common x86_64 2:19.0.0-5358.gaf8fdbf0.el9 ceph 138 k 2024-07-27T03:01:45.170 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-07-27T03:01:45.170 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-07-27T03:01:45.170 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-07-27T03:01:45.171 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-07-27T03:01:45.171 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-07-27T03:01:45.171 INFO:teuthology.orchestra.run.smithi067.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-07-27T03:01:45.171 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-07-27T03:01:45.171 INFO:teuthology.orchestra.run.smithi067.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-07-27T03:01:45.171 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth noarch 1:2.32.0-1.el9 epel 216 k 2024-07-27T03:01:45.171 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-07-27T03:01:45.171 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-07-27T03:01:45.171 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-07-27T03:01:45.171 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-07-27T03:01:45.171 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-07-27T03:01:45.171 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-07-27T03:01:45.171 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-07-27T03:01:45.171 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-07-27T03:01:45.172 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-07-27T03:01:45.172 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-07-27T03:01:45.172 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-07-27T03:01:45.172 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-07-27T03:01:45.172 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-07-27T03:01:45.172 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-07-27T03:01:45.172 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-07-27T03:01:45.172 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-07-27T03:01:45.172 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-07-27T03:01:45.172 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-07-27T03:01:45.172 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-07-27T03:01:45.172 INFO:teuthology.orchestra.run.smithi067.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-07-27T03:01:45.172 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-07-27T03:01:45.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-07-27T03:01:45.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-07-27T03:01:45.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-07-27T03:01:45.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-07-27T03:01:45.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-07-27T03:01:45.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-07-27T03:01:45.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-07-27T03:01:45.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-07-27T03:01:45.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-07-27T03:01:45.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-07-27T03:01:45.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-07-27T03:01:45.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-07-27T03:01:45.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-07-27T03:01:45.173 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-07-27T03:01:45.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-07-27T03:01:45.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-07-27T03:01:45.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-07-27T03:01:45.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-07-27T03:01:45.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-07-27T03:01:45.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-07-27T03:01:45.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-07-27T03:01:45.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-07-27T03:01:45.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-07-27T03:01:45.174 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-07-27T03:01:45.175 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-07-27T03:01:45.175 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-07-27T03:01:45.175 INFO:teuthology.orchestra.run.smithi067.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-07-27T03:01:45.175 INFO:teuthology.orchestra.run.smithi067.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-07-27T03:01:45.175 INFO:teuthology.orchestra.run.smithi067.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-07-27T03:01:45.175 INFO:teuthology.orchestra.run.smithi067.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-07-27T03:01:45.175 INFO:teuthology.orchestra.run.smithi067.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-07-27T03:01:45.175 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-07-27T03:01:45.175 INFO:teuthology.orchestra.run.smithi067.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-07-27T03:01:45.175 INFO:teuthology.orchestra.run.smithi067.stdout:Installing weak dependencies: 2024-07-27T03:01:45.175 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-07-27T03:01:45.175 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:01:45.175 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-07-27T03:01:45.175 INFO:teuthology.orchestra.run.smithi067.stdout:======================================================================================= 2024-07-27T03:01:45.176 INFO:teuthology.orchestra.run.smithi067.stdout:Install 148 Packages 2024-07-27T03:01:45.176 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrade 12 Packages 2024-07-27T03:01:45.176 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:01:45.176 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 232 M 2024-07-27T03:01:45.176 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-07-27T03:01:45.661 INFO:teuthology.orchestra.run.smithi149.stdout:(1/160): ceph-19.0.0-5358.gaf8fdbf0.el9.x86_64. 33 kB/s | 6.6 kB 00:00 2024-07-27T03:01:45.667 INFO:teuthology.orchestra.run.smithi067.stdout:(1/160): ceph-19.0.0-5358.gaf8fdbf0.el9.x86_64. 33 kB/s | 6.6 kB 00:00 2024-07-27T03:01:45.897 INFO:teuthology.orchestra.run.smithi175.stdout:(1/160): ceph-19.0.0-5358.gaf8fdbf0.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-07-27T03:01:46.027 INFO:teuthology.orchestra.run.smithi149.stdout:(2/160): ceph-base-19.0.0-5358.gaf8fdbf0.el9.x8 9.7 MB/s | 5.5 MB 00:00 2024-07-27T03:01:46.092 INFO:teuthology.orchestra.run.smithi067.stdout:(2/160): ceph-base-19.0.0-5358.gaf8fdbf0.el9.x8 8.8 MB/s | 5.5 MB 00:00 2024-07-27T03:01:46.094 INFO:teuthology.orchestra.run.smithi149.stdout:(3/160): ceph-immutable-object-cache-19.0.0-535 2.2 MB/s | 148 kB 00:00 2024-07-27T03:01:46.126 INFO:teuthology.orchestra.run.smithi067.stdout:(3/160): ceph-fuse-19.0.0-5358.gaf8fdbf0.el9.x8 1.9 MB/s | 872 kB 00:00 2024-07-27T03:01:46.151 INFO:teuthology.orchestra.run.smithi067.stdout:(4/160): ceph-immutable-object-cache-19.0.0-535 2.5 MB/s | 148 kB 00:00 2024-07-27T03:01:46.161 INFO:teuthology.orchestra.run.smithi149.stdout:(4/160): ceph-fuse-19.0.0-5358.gaf8fdbf0.el9.x8 1.7 MB/s | 872 kB 00:00 2024-07-27T03:01:46.376 INFO:teuthology.orchestra.run.smithi067.stdout:(5/160): ceph-mgr-19.0.0-5358.gaf8fdbf0.el9.x86 6.7 MB/s | 1.5 MB 00:00 2024-07-27T03:01:46.687 INFO:teuthology.orchestra.run.smithi149.stdout:(5/160): ceph-common-19.0.0-5358.gaf8fdbf0.el9. 16 MB/s | 20 MB 00:01 2024-07-27T03:01:46.760 INFO:teuthology.orchestra.run.smithi067.stdout:(6/160): ceph-mon-19.0.0-5358.gaf8fdbf0.el9.x86 13 MB/s | 4.9 MB 00:00 2024-07-27T03:01:46.805 INFO:teuthology.orchestra.run.smithi175.stdout:(2/160): ceph-fuse-19.0.0-5358.gaf8fdbf0.el9.x8 961 kB/s | 872 kB 00:00 2024-07-27T03:01:46.895 INFO:teuthology.orchestra.run.smithi149.stdout:(6/160): ceph-mds-19.0.0-5358.gaf8fdbf0.el9.x86 2.8 MB/s | 2.3 MB 00:00 2024-07-27T03:01:46.947 INFO:teuthology.orchestra.run.smithi175.stdout:(3/160): ceph-immutable-object-cache-19.0.0-535 1.0 MB/s | 148 kB 00:00 2024-07-27T03:01:46.954 INFO:teuthology.orchestra.run.smithi149.stdout:(7/160): ceph-mon-19.0.0-5358.gaf8fdbf0.el9.x86 18 MB/s | 4.9 MB 00:00 2024-07-27T03:01:46.960 INFO:teuthology.orchestra.run.smithi067.stdout:(7/160): ceph-mds-19.0.0-5358.gaf8fdbf0.el9.x86 2.7 MB/s | 2.3 MB 00:00 2024-07-27T03:01:47.504 INFO:teuthology.orchestra.run.smithi149.stdout:(8/160): ceph-radosgw-19.0.0-5358.gaf8fdbf0.el9 19 MB/s | 10 MB 00:00 2024-07-27T03:01:47.546 INFO:teuthology.orchestra.run.smithi149.stdout:(9/160): ceph-mgr-19.0.0-5358.gaf8fdbf0.el9.x86 1.1 MB/s | 1.5 MB 00:01 2024-07-27T03:01:47.571 INFO:teuthology.orchestra.run.smithi149.stdout:(10/160): ceph-selinux-19.0.0-5358.gaf8fdbf0.el 378 kB/s | 25 kB 00:00 2024-07-27T03:01:47.638 INFO:teuthology.orchestra.run.smithi149.stdout:(11/160): cephfs-mirror-19.0.0-5358.gaf8fdbf0.e 3.3 MB/s | 225 kB 00:00 2024-07-27T03:01:47.705 INFO:teuthology.orchestra.run.smithi149.stdout:(12/160): cephfs-shell-19.0.0-5358.gaf8fdbf0.el 405 kB/s | 27 kB 00:00 2024-07-27T03:01:47.772 INFO:teuthology.orchestra.run.smithi149.stdout:(13/160): libcephfs-devel-19.0.0-5358.gaf8fdbf0 477 kB/s | 32 kB 00:00 2024-07-27T03:01:47.856 INFO:teuthology.orchestra.run.smithi149.stdout:(14/160): libcephfs2-19.0.0-5358.gaf8fdbf0.el9. 9.0 MB/s | 766 kB 00:00 2024-07-27T03:01:47.906 INFO:teuthology.orchestra.run.smithi149.stdout:(15/160): libcephsqlite-19.0.0-5358.gaf8fdbf0.e 3.1 MB/s | 158 kB 00:00 2024-07-27T03:01:47.956 INFO:teuthology.orchestra.run.smithi149.stdout:(16/160): librados-devel-19.0.0-5358.gaf8fdbf0. 2.5 MB/s | 127 kB 00:00 2024-07-27T03:01:48.007 INFO:teuthology.orchestra.run.smithi149.stdout:(17/160): libradosstriper1-19.0.0-5358.gaf8fdbf 9.9 MB/s | 504 kB 00:00 2024-07-27T03:01:48.064 INFO:teuthology.orchestra.run.smithi175.stdout:(4/160): ceph-common-19.0.0-5358.gaf8fdbf0.el9. 8.2 MB/s | 20 MB 00:02 2024-07-27T03:01:48.069 INFO:teuthology.orchestra.run.smithi067.stdout:(8/160): ceph-common-19.0.0-5358.gaf8fdbf0.el9. 7.6 MB/s | 20 MB 00:02 2024-07-27T03:01:48.215 INFO:teuthology.orchestra.run.smithi149.stdout:(18/160): librgw2-19.0.0-5358.gaf8fdbf0.el9.x86 25 MB/s | 5.3 MB 00:00 2024-07-27T03:01:48.222 INFO:teuthology.orchestra.run.smithi175.stdout:(5/160): ceph-mgr-19.0.0-5358.gaf8fdbf0.el9.x86 9.6 MB/s | 1.5 MB 00:00 2024-07-27T03:01:48.253 INFO:teuthology.orchestra.run.smithi067.stdout:(9/160): ceph-osd-19.0.0-5358.gaf8fdbf0.el9.x86 12 MB/s | 18 MB 00:01 2024-07-27T03:01:48.257 INFO:teuthology.orchestra.run.smithi149.stdout:(19/160): python3-ceph-argparse-19.0.0-5358.gaf 1.1 MB/s | 45 kB 00:00 2024-07-27T03:01:48.278 INFO:teuthology.orchestra.run.smithi067.stdout:(10/160): ceph-selinux-19.0.0-5358.gaf8fdbf0.el 121 kB/s | 25 kB 00:00 2024-07-27T03:01:48.306 INFO:teuthology.orchestra.run.smithi175.stdout:(6/160): ceph-mds-19.0.0-5358.gaf8fdbf0.el9.x86 1.7 MB/s | 2.3 MB 00:01 2024-07-27T03:01:48.307 INFO:teuthology.orchestra.run.smithi149.stdout:(20/160): python3-ceph-common-19.0.0-5358.gaf8f 2.7 MB/s | 138 kB 00:00 2024-07-27T03:01:48.337 INFO:teuthology.orchestra.run.smithi067.stdout:(11/160): cephfs-mirror-19.0.0-5358.gaf8fdbf0.e 3.8 MB/s | 225 kB 00:00 2024-07-27T03:01:48.366 INFO:teuthology.orchestra.run.smithi149.stdout:(21/160): python3-cephfs-19.0.0-5358.gaf8fdbf0. 3.1 MB/s | 188 kB 00:00 2024-07-27T03:01:48.395 INFO:teuthology.orchestra.run.smithi067.stdout:(12/160): cephfs-shell-19.0.0-5358.gaf8fdbf0.el 463 kB/s | 27 kB 00:00 2024-07-27T03:01:48.433 INFO:teuthology.orchestra.run.smithi149.stdout:(22/160): python3-rados-19.0.0-5358.gaf8fdbf0.e 5.5 MB/s | 371 kB 00:00 2024-07-27T03:01:48.453 INFO:teuthology.orchestra.run.smithi067.stdout:(13/160): libcephfs-devel-19.0.0-5358.gaf8fdbf0 545 kB/s | 32 kB 00:00 2024-07-27T03:01:48.500 INFO:teuthology.orchestra.run.smithi149.stdout:(23/160): python3-rbd-19.0.0-5358.gaf8fdbf0.el9 5.2 MB/s | 355 kB 00:00 2024-07-27T03:01:48.562 INFO:teuthology.orchestra.run.smithi067.stdout:(14/160): libcephfs2-19.0.0-5358.gaf8fdbf0.el9. 6.9 MB/s | 766 kB 00:00 2024-07-27T03:01:48.566 INFO:teuthology.orchestra.run.smithi149.stdout:(24/160): python3-rgw-19.0.0-5358.gaf8fdbf0.el9 1.7 MB/s | 116 kB 00:00 2024-07-27T03:01:48.612 INFO:teuthology.orchestra.run.smithi067.stdout:(15/160): libcephsqlite-19.0.0-5358.gaf8fdbf0.e 3.1 MB/s | 158 kB 00:00 2024-07-27T03:01:48.617 INFO:teuthology.orchestra.run.smithi149.stdout:(25/160): rbd-fuse-19.0.0-5358.gaf8fdbf0.el9.x8 1.7 MB/s | 86 kB 00:00 2024-07-27T03:01:48.623 INFO:teuthology.orchestra.run.smithi175.stdout:(7/160): ceph-mon-19.0.0-5358.gaf8fdbf0.el9.x86 12 MB/s | 4.9 MB 00:00 2024-07-27T03:01:48.671 INFO:teuthology.orchestra.run.smithi067.stdout:(16/160): librados-devel-19.0.0-5358.gaf8fdbf0. 2.1 MB/s | 127 kB 00:00 2024-07-27T03:01:48.763 INFO:teuthology.orchestra.run.smithi067.stdout:(17/160): libradosstriper1-19.0.0-5358.gaf8fdbf 5.4 MB/s | 504 kB 00:00 2024-07-27T03:01:48.775 INFO:teuthology.orchestra.run.smithi149.stdout:(26/160): rbd-mirror-19.0.0-5358.gaf8fdbf0.el9. 20 MB/s | 3.1 MB 00:00 2024-07-27T03:01:48.842 INFO:teuthology.orchestra.run.smithi149.stdout:(27/160): rbd-nbd-19.0.0-5358.gaf8fdbf0.el9.x86 2.5 MB/s | 170 kB 00:00 2024-07-27T03:01:48.909 INFO:teuthology.orchestra.run.smithi149.stdout:(28/160): ceph-grafana-dashboards-19.0.0-5358.g 514 kB/s | 34 kB 00:00 2024-07-27T03:01:48.923 INFO:teuthology.orchestra.run.smithi175.stdout:(8/160): ceph-base-19.0.0-5358.gaf8fdbf0.el9.x8 1.7 MB/s | 5.5 MB 00:03 2024-07-27T03:01:48.985 INFO:teuthology.orchestra.run.smithi149.stdout:(29/160): ceph-mgr-cephadm-19.0.0-5358.gaf8fdbf 1.9 MB/s | 149 kB 00:00 2024-07-27T03:01:48.990 INFO:teuthology.orchestra.run.smithi175.stdout:(9/160): ceph-selinux-19.0.0-5358.gaf8fdbf0.el9 378 kB/s | 25 kB 00:00 2024-07-27T03:01:49.205 INFO:teuthology.orchestra.run.smithi067.stdout:(18/160): ceph-radosgw-19.0.0-5358.gaf8fdbf0.el 4.6 MB/s | 10 MB 00:02 2024-07-27T03:01:49.305 INFO:teuthology.orchestra.run.smithi067.stdout:(19/160): librgw2-19.0.0-5358.gaf8fdbf0.el9.x86 9.7 MB/s | 5.3 MB 00:00 2024-07-27T03:01:49.330 INFO:teuthology.orchestra.run.smithi067.stdout:(20/160): python3-ceph-argparse-19.0.0-5358.gaf 361 kB/s | 45 kB 00:00 2024-07-27T03:01:49.397 INFO:teuthology.orchestra.run.smithi067.stdout:(21/160): python3-ceph-common-19.0.0-5358.gaf8f 1.5 MB/s | 138 kB 00:00 2024-07-27T03:01:49.423 INFO:teuthology.orchestra.run.smithi067.stdout:(22/160): python3-cephfs-19.0.0-5358.gaf8fdbf0. 2.0 MB/s | 188 kB 00:00 2024-07-27T03:01:49.498 INFO:teuthology.orchestra.run.smithi067.stdout:(23/160): python3-rados-19.0.0-5358.gaf8fdbf0.e 3.6 MB/s | 371 kB 00:00 2024-07-27T03:01:49.557 INFO:teuthology.orchestra.run.smithi067.stdout:(24/160): python3-rbd-19.0.0-5358.gaf8fdbf0.el9 2.6 MB/s | 355 kB 00:00 2024-07-27T03:01:49.582 INFO:teuthology.orchestra.run.smithi067.stdout:(25/160): python3-rgw-19.0.0-5358.gaf8fdbf0.el9 1.4 MB/s | 116 kB 00:00 2024-07-27T03:01:49.616 INFO:teuthology.orchestra.run.smithi067.stdout:(26/160): rbd-fuse-19.0.0-5358.gaf8fdbf0.el9.x8 1.4 MB/s | 86 kB 00:00 2024-07-27T03:01:49.644 INFO:teuthology.orchestra.run.smithi149.stdout:(30/160): ceph-mgr-dashboard-19.0.0-5358.gaf8fd 21 MB/s | 14 MB 00:00 2024-07-27T03:01:49.682 INFO:teuthology.orchestra.run.smithi067.stdout:(27/160): rbd-nbd-19.0.0-5358.gaf8fdbf0.el9.x86 2.5 MB/s | 170 kB 00:00 2024-07-27T03:01:49.766 INFO:teuthology.orchestra.run.smithi067.stdout:(28/160): ceph-grafana-dashboards-19.0.0-5358.g 412 kB/s | 34 kB 00:00 2024-07-27T03:01:49.799 INFO:teuthology.orchestra.run.smithi175.stdout:(10/160): ceph-radosgw-19.0.0-5358.gaf8fdbf0.el 8.8 MB/s | 10 MB 00:01 2024-07-27T03:01:49.858 INFO:teuthology.orchestra.run.smithi067.stdout:(29/160): ceph-mgr-cephadm-19.0.0-5358.gaf8fdbf 1.6 MB/s | 149 kB 00:00 2024-07-27T03:01:49.899 INFO:teuthology.orchestra.run.smithi175.stdout:(11/160): cephfs-mirror-19.0.0-5358.gaf8fdbf0.e 2.2 MB/s | 225 kB 00:00 2024-07-27T03:01:49.917 INFO:teuthology.orchestra.run.smithi067.stdout:(30/160): rbd-mirror-19.0.0-5358.gaf8fdbf0.el9. 9.2 MB/s | 3.1 MB 00:00 2024-07-27T03:01:49.958 INFO:teuthology.orchestra.run.smithi175.stdout:(12/160): cephfs-shell-19.0.0-5358.gaf8fdbf0.el 462 kB/s | 27 kB 00:00 2024-07-27T03:01:50.016 INFO:teuthology.orchestra.run.smithi175.stdout:(13/160): libcephfs-devel-19.0.0-5358.gaf8fdbf0 546 kB/s | 32 kB 00:00 2024-07-27T03:01:50.018 INFO:teuthology.orchestra.run.smithi149.stdout:(31/160): ceph-mgr-diskprediction-local-19.0.0- 20 MB/s | 7.4 MB 00:00 2024-07-27T03:01:50.094 INFO:teuthology.orchestra.run.smithi149.stdout:(32/160): ceph-mgr-modules-core-19.0.0-5358.gaf 3.7 MB/s | 284 kB 00:00 2024-07-27T03:01:50.150 INFO:teuthology.orchestra.run.smithi175.stdout:(14/160): libcephfs2-19.0.0-5358.gaf8fdbf0.el9. 5.6 MB/s | 766 kB 00:00 2024-07-27T03:01:50.153 INFO:teuthology.orchestra.run.smithi149.stdout:(33/160): ceph-mgr-rook-19.0.0-5358.gaf8fdbf0.e 846 kB/s | 49 kB 00:00 2024-07-27T03:01:50.217 INFO:teuthology.orchestra.run.smithi175.stdout:(15/160): libcephsqlite-19.0.0-5358.gaf8fdbf0.e 2.3 MB/s | 158 kB 00:00 2024-07-27T03:01:50.219 INFO:teuthology.orchestra.run.smithi149.stdout:(34/160): ceph-prometheus-alerts-19.0.0-5358.ga 247 kB/s | 16 kB 00:00 2024-07-27T03:01:50.275 INFO:teuthology.orchestra.run.smithi175.stdout:(16/160): librados-devel-19.0.0-5358.gaf8fdbf0. 2.1 MB/s | 127 kB 00:00 2024-07-27T03:01:50.286 INFO:teuthology.orchestra.run.smithi149.stdout:(35/160): ceph-volume-19.0.0-5358.gaf8fdbf0.el9 4.0 MB/s | 274 kB 00:00 2024-07-27T03:01:50.351 INFO:teuthology.orchestra.run.smithi175.stdout:(17/160): libradosstriper1-19.0.0-5358.gaf8fdbf 6.6 MB/s | 504 kB 00:00 2024-07-27T03:01:50.379 INFO:teuthology.orchestra.run.smithi149.stdout:(36/160): cephadm-19.0.0-5358.gaf8fdbf0.el9.noa 8.2 MB/s | 770 kB 00:00 2024-07-27T03:01:50.429 INFO:teuthology.orchestra.run.smithi149.stdout:(37/160): cephfs-top-19.0.0-5358.gaf8fdbf0.el9. 366 kB/s | 18 kB 00:00 2024-07-27T03:01:50.467 INFO:teuthology.orchestra.run.smithi067.stdout:(31/160): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-07-27T03:01:50.526 INFO:teuthology.orchestra.run.smithi067.stdout:(32/160): ceph-mgr-modules-core-19.0.0-5358.gaf 4.8 MB/s | 284 kB 00:00 2024-07-27T03:01:50.529 INFO:teuthology.orchestra.run.smithi149.stdout:(38/160): fuse-2.9.9-16.el9.x86_64.rpm 799 kB/s | 80 kB 00:00 2024-07-27T03:01:50.576 INFO:teuthology.orchestra.run.smithi067.stdout:(33/160): ceph-mgr-rook-19.0.0-5358.gaf8fdbf0.e 987 kB/s | 49 kB 00:00 2024-07-27T03:01:50.621 INFO:teuthology.orchestra.run.smithi149.stdout:(39/160): libgfortran-11.4.1-3.el9.x86_64.rpm 8.6 MB/s | 803 kB 00:00 2024-07-27T03:01:50.635 INFO:teuthology.orchestra.run.smithi067.stdout:(34/160): ceph-prometheus-alerts-19.0.0-5358.ga 282 kB/s | 16 kB 00:00 2024-07-27T03:01:50.693 INFO:teuthology.orchestra.run.smithi067.stdout:(35/160): ceph-volume-19.0.0-5358.gaf8fdbf0.el9 4.6 MB/s | 274 kB 00:00 2024-07-27T03:01:50.705 INFO:teuthology.orchestra.run.smithi149.stdout:(40/160): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-07-27T03:01:50.805 INFO:teuthology.orchestra.run.smithi149.stdout:(41/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-07-27T03:01:50.809 INFO:teuthology.orchestra.run.smithi175.stdout:(18/160): librgw2-19.0.0-5358.gaf8fdbf0.el9.x86 11 MB/s | 5.3 MB 00:00 2024-07-27T03:01:50.851 INFO:teuthology.orchestra.run.smithi175.stdout:(19/160): python3-ceph-argparse-19.0.0-5358.gaf 1.1 MB/s | 45 kB 00:00 2024-07-27T03:01:50.901 INFO:teuthology.orchestra.run.smithi175.stdout:(20/160): python3-ceph-common-19.0.0-5358.gaf8f 2.7 MB/s | 138 kB 00:00 2024-07-27T03:01:50.952 INFO:teuthology.orchestra.run.smithi175.stdout:(21/160): python3-cephfs-19.0.0-5358.gaf8fdbf0. 3.7 MB/s | 188 kB 00:00 2024-07-27T03:01:51.002 INFO:teuthology.orchestra.run.smithi175.stdout:(22/160): python3-rados-19.0.0-5358.gaf8fdbf0.e 7.3 MB/s | 371 kB 00:00 2024-07-27T03:01:51.052 INFO:teuthology.orchestra.run.smithi175.stdout:(23/160): python3-rbd-19.0.0-5358.gaf8fdbf0.el9 7.0 MB/s | 355 kB 00:00 2024-07-27T03:01:51.094 INFO:teuthology.orchestra.run.smithi067.stdout:(36/160): ceph-test-19.0.0-5358.gaf8fdbf0.el9.x 15 MB/s | 44 MB 00:02 2024-07-27T03:01:51.102 INFO:teuthology.orchestra.run.smithi175.stdout:(24/160): python3-rgw-19.0.0-5358.gaf8fdbf0.el9 2.3 MB/s | 116 kB 00:00 2024-07-27T03:01:51.136 INFO:teuthology.orchestra.run.smithi067.stdout:(37/160): cephadm-19.0.0-5358.gaf8fdbf0.el9.noa 1.7 MB/s | 770 kB 00:00 2024-07-27T03:01:51.153 INFO:teuthology.orchestra.run.smithi175.stdout:(25/160): rbd-fuse-19.0.0-5358.gaf8fdbf0.el9.x8 1.7 MB/s | 86 kB 00:00 2024-07-27T03:01:51.155 INFO:teuthology.orchestra.run.smithi149.stdout:(42/160): python3-cryptography-36.0.1-4.el9.x86 3.5 MB/s | 1.2 MB 00:00 2024-07-27T03:01:51.161 INFO:teuthology.orchestra.run.smithi067.stdout:(38/160): cephfs-top-19.0.0-5358.gaf8fdbf0.el9. 274 kB/s | 18 kB 00:00 2024-07-27T03:01:51.214 INFO:teuthology.orchestra.run.smithi149.stdout:(43/160): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-07-27T03:01:51.297 INFO:teuthology.orchestra.run.smithi149.stdout:(44/160): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-07-27T03:01:51.303 INFO:teuthology.orchestra.run.smithi067.stdout:(39/160): fuse-2.9.9-16.el9.x86_64.rpm 478 kB/s | 80 kB 00:00 2024-07-27T03:01:51.356 INFO:teuthology.orchestra.run.smithi149.stdout:(45/160): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-07-27T03:01:51.378 INFO:teuthology.orchestra.run.smithi175.stdout:(26/160): rbd-mirror-19.0.0-5358.gaf8fdbf0.el9. 14 MB/s | 3.1 MB 00:00 2024-07-27T03:01:51.387 INFO:teuthology.orchestra.run.smithi067.stdout:(40/160): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-07-27T03:01:51.420 INFO:teuthology.orchestra.run.smithi067.stdout:(41/160): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 803 kB 00:00 2024-07-27T03:01:51.428 INFO:teuthology.orchestra.run.smithi175.stdout:(27/160): rbd-nbd-19.0.0-5358.gaf8fdbf0.el9.x86 3.3 MB/s | 170 kB 00:00 2024-07-27T03:01:51.445 INFO:teuthology.orchestra.run.smithi067.stdout:(42/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-07-27T03:01:51.448 INFO:teuthology.orchestra.run.smithi149.stdout:(46/160): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-07-27T03:01:51.470 INFO:teuthology.orchestra.run.smithi175.stdout:(28/160): ceph-grafana-dashboards-19.0.0-5358.g 827 kB/s | 34 kB 00:00 2024-07-27T03:01:51.506 INFO:teuthology.orchestra.run.smithi067.stdout:(43/160): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-07-27T03:01:51.520 INFO:teuthology.orchestra.run.smithi175.stdout:(29/160): ceph-mgr-cephadm-19.0.0-5358.gaf8fdbf 2.9 MB/s | 149 kB 00:00 2024-07-27T03:01:51.554 INFO:teuthology.orchestra.run.smithi067.stdout:(44/160): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-07-27T03:01:51.579 INFO:teuthology.orchestra.run.smithi067.stdout:(45/160): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-07-27T03:01:51.613 INFO:teuthology.orchestra.run.smithi067.stdout:(46/160): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-07-27T03:01:51.639 INFO:teuthology.orchestra.run.smithi067.stdout:(47/160): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-07-27T03:01:51.922 INFO:teuthology.orchestra.run.smithi067.stdout:(48/160): ceph-mgr-dashboard-19.0.0-5358.gaf8fd 6.8 MB/s | 14 MB 00:02 2024-07-27T03:01:51.947 INFO:teuthology.orchestra.run.smithi067.stdout:(49/160): automake-1.16.2-8.el9.noarch.rpm 2.0 MB/s | 689 kB 00:00 2024-07-27T03:01:51.981 INFO:teuthology.orchestra.run.smithi067.stdout:(50/160): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 943 kB 00:00 2024-07-27T03:01:51.998 INFO:teuthology.orchestra.run.smithi067.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-07-27T03:01:52.040 INFO:teuthology.orchestra.run.smithi067.stdout:(52/160): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-07-27T03:01:52.057 INFO:teuthology.orchestra.run.smithi067.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 560 kB/s | 33 kB 00:00 2024-07-27T03:01:52.082 INFO:teuthology.orchestra.run.smithi067.stdout:(54/160): boost-program-options-1.75.0-8.el9.x8 671 kB/s | 107 kB 00:00 2024-07-27T03:01:52.099 INFO:teuthology.orchestra.run.smithi067.stdout:(55/160): flexiblas-openblas-openmp-3.0.4-8.el9 433 kB/s | 18 kB 00:00 2024-07-27T03:01:52.118 INFO:teuthology.orchestra.run.smithi067.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 332 kB/s | 11 kB 00:00 2024-07-27T03:01:52.149 INFO:teuthology.orchestra.run.smithi067.stdout:(57/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-07-27T03:01:52.167 INFO:teuthology.orchestra.run.smithi067.stdout:(58/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-07-27T03:01:52.217 INFO:teuthology.orchestra.run.smithi067.stdout:(59/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-07-27T03:01:52.223 INFO:teuthology.orchestra.run.smithi149.stdout:(47/160): automake-1.16.2-8.el9.noarch.rpm 889 kB/s | 689 kB 00:00 2024-07-27T03:01:52.251 INFO:teuthology.orchestra.run.smithi067.stdout:(60/160): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 662 kB 00:00 2024-07-27T03:01:52.276 INFO:teuthology.orchestra.run.smithi067.stdout:(61/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 434 kB/s | 25 kB 00:00 2024-07-27T03:01:52.301 INFO:teuthology.orchestra.run.smithi067.stdout:(62/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-07-27T03:01:52.326 INFO:teuthology.orchestra.run.smithi067.stdout:(63/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-07-27T03:01:52.360 INFO:teuthology.orchestra.run.smithi067.stdout:(64/160): libtool-2.4.6-46.el9.x86_64.rpm 2.9 MB/s | 581 kB 00:00 2024-07-27T03:01:52.385 INFO:teuthology.orchestra.run.smithi067.stdout:(65/160): openblas-0.3.21-2.el9.x86_64.rpm 602 kB/s | 35 kB 00:00 2024-07-27T03:01:52.411 INFO:teuthology.orchestra.run.smithi067.stdout:(66/160): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-07-27T03:01:52.498 INFO:teuthology.orchestra.run.smithi149.stdout:(48/160): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 943 kB 00:00 2024-07-27T03:01:52.511 INFO:teuthology.orchestra.run.smithi067.stdout:(67/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 35 MB/s | 4.4 MB 00:00 2024-07-27T03:01:52.553 INFO:teuthology.orchestra.run.smithi067.stdout:(68/160): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-07-27T03:01:52.607 INFO:teuthology.orchestra.run.smithi149.stdout:(49/160): boost-program-options-1.75.0-8.el9.x8 988 kB/s | 107 kB 00:00 2024-07-27T03:01:52.636 INFO:teuthology.orchestra.run.smithi067.stdout:(69/160): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2024-07-27T03:01:52.637 INFO:teuthology.orchestra.run.smithi175.stdout:(30/160): ceph-mgr-dashboard-19.0.0-5358.gaf8fd 13 MB/s | 14 MB 00:01 2024-07-27T03:01:52.670 INFO:teuthology.orchestra.run.smithi067.stdout:(70/160): postgresql-contrib-13.14-1.el9.x86_64 5.6 MB/s | 911 kB 00:00 2024-07-27T03:01:52.695 INFO:teuthology.orchestra.run.smithi067.stdout:(71/160): postgresql-private-libs-13.14-1.el9.x 938 kB/s | 133 kB 00:00 2024-07-27T03:01:52.715 INFO:teuthology.orchestra.run.smithi149.stdout:(50/160): elfutils-libelf-devel-0.191-4.el9.x86 201 kB/s | 22 kB 00:00 2024-07-27T03:01:52.745 INFO:teuthology.orchestra.run.smithi067.stdout:(72/160): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-07-27T03:01:52.787 INFO:teuthology.orchestra.run.smithi067.stdout:(73/160): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-07-27T03:01:52.824 INFO:teuthology.orchestra.run.smithi149.stdout:(51/160): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-07-27T03:01:52.863 INFO:teuthology.orchestra.run.smithi067.stdout:(74/160): postgresql-server-13.14-1.el9.x86_64. 26 MB/s | 5.8 MB 00:00 2024-07-27T03:01:52.888 INFO:teuthology.orchestra.run.smithi067.stdout:(75/160): python3-colorama-0.4.6-3.el9.noarch.r 612 kB/s | 61 kB 00:00 2024-07-27T03:01:52.913 INFO:teuthology.orchestra.run.smithi067.stdout:(76/160): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-07-27T03:01:52.924 INFO:teuthology.orchestra.run.smithi149.stdout:(52/160): flexiblas-3.0.4-8.el9.x86_64.rpm 328 kB/s | 33 kB 00:00 2024-07-27T03:01:52.938 INFO:teuthology.orchestra.run.smithi067.stdout:(77/160): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-07-27T03:01:52.964 INFO:teuthology.orchestra.run.smithi067.stdout:(78/160): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-07-27T03:01:52.989 INFO:teuthology.orchestra.run.smithi067.stdout:(79/160): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-07-27T03:01:53.014 INFO:teuthology.orchestra.run.smithi067.stdout:(80/160): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-07-27T03:01:53.039 INFO:teuthology.orchestra.run.smithi067.stdout:(81/160): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-07-27T03:01:53.140 INFO:teuthology.orchestra.run.smithi067.stdout:(82/160): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-07-27T03:01:53.157 INFO:teuthology.orchestra.run.smithi067.stdout:(83/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-07-27T03:01:53.224 INFO:teuthology.orchestra.run.smithi067.stdout:(84/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 5.1 MB 00:00 2024-07-27T03:01:53.229 INFO:teuthology.orchestra.run.smithi175.stdout:(31/160): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-07-27T03:01:53.249 INFO:teuthology.orchestra.run.smithi067.stdout:(85/160): python3-packaging-20.9-5.el9.noarch.r 840 kB/s | 77 kB 00:00 2024-07-27T03:01:53.266 INFO:teuthology.orchestra.run.smithi067.stdout:(86/160): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-07-27T03:01:53.266 INFO:teuthology.orchestra.run.smithi149.stdout:(53/160): ceph-osd-19.0.0-5358.gaf8fdbf0.el9.x8 2.8 MB/s | 18 MB 00:06 2024-07-27T03:01:53.279 INFO:teuthology.orchestra.run.smithi175.stdout:(32/160): ceph-mgr-modules-core-19.0.0-5358.gaf 5.6 MB/s | 284 kB 00:00 2024-07-27T03:01:53.291 INFO:teuthology.orchestra.run.smithi067.stdout:(87/160): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-07-27T03:01:53.317 INFO:teuthology.orchestra.run.smithi067.stdout:(88/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-07-27T03:01:53.338 INFO:teuthology.orchestra.run.smithi175.stdout:(33/160): ceph-mgr-rook-19.0.0-5358.gaf8fdbf0.e 847 kB/s | 49 kB 00:00 2024-07-27T03:01:53.342 INFO:teuthology.orchestra.run.smithi067.stdout:(89/160): python3-protobuf-3.14.0-13.el9.noarch 2.8 MB/s | 269 kB 00:00 2024-07-27T03:01:53.359 INFO:teuthology.orchestra.run.smithi067.stdout:(90/160): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-07-27T03:01:53.384 INFO:teuthology.orchestra.run.smithi067.stdout:(91/160): python3-pyasn1-modules-0.4.8-6.el9.no 2.9 MB/s | 279 kB 00:00 2024-07-27T03:01:53.388 INFO:teuthology.orchestra.run.smithi175.stdout:(34/160): ceph-prometheus-alerts-19.0.0-5358.ga 330 kB/s | 16 kB 00:00 2024-07-27T03:01:53.401 INFO:teuthology.orchestra.run.smithi067.stdout:(92/160): python3-requests-oauthlib-1.3.0-12.el 911 kB/s | 54 kB 00:00 2024-07-27T03:01:53.418 INFO:teuthology.orchestra.run.smithi067.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-07-27T03:01:53.438 INFO:teuthology.orchestra.run.smithi175.stdout:(35/160): ceph-volume-19.0.0-5358.gaf8fdbf0.el9 5.4 MB/s | 274 kB 00:00 2024-07-27T03:01:53.531 INFO:teuthology.orchestra.run.smithi175.stdout:(36/160): cephadm-19.0.0-5358.gaf8fdbf0.el9.noa 8.2 MB/s | 770 kB 00:00 2024-07-27T03:01:53.542 INFO:teuthology.orchestra.run.smithi149.stdout:(54/160): flexiblas-openblas-openmp-3.0.4-8.el9 66 kB/s | 18 kB 00:00 2024-07-27T03:01:53.560 INFO:teuthology.orchestra.run.smithi067.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 279 kB/s | 44 kB 00:00 2024-07-27T03:01:53.581 INFO:teuthology.orchestra.run.smithi175.stdout:(37/160): cephfs-top-19.0.0-5358.gaf8fdbf0.el9. 366 kB/s | 18 kB 00:00 2024-07-27T03:01:53.585 INFO:teuthology.orchestra.run.smithi067.stdout:(95/160): qatlib-24.02.0-1.el9.x86_64.rpm 1.3 MB/s | 221 kB 00:00 2024-07-27T03:01:53.603 INFO:teuthology.orchestra.run.smithi067.stdout:(96/160): qatlib-service-24.02.0-1.el9.x86_64.r 858 kB/s | 36 kB 00:00 2024-07-27T03:01:53.619 INFO:teuthology.orchestra.run.smithi067.stdout:(97/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-07-27T03:01:53.642 INFO:teuthology.orchestra.run.smithi149.stdout:(55/160): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-07-27T03:01:53.653 INFO:teuthology.orchestra.run.smithi067.stdout:(98/160): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-07-27T03:01:53.670 INFO:teuthology.orchestra.run.smithi067.stdout:(99/160): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-07-27T03:01:53.687 INFO:teuthology.orchestra.run.smithi067.stdout:(100/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 2.8 MB/s | 96 kB 00:00 2024-07-27T03:01:53.712 INFO:teuthology.orchestra.run.smithi067.stdout:(101/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-07-27T03:01:53.738 INFO:teuthology.orchestra.run.smithi067.stdout:(102/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 890 kB/s | 45 kB 00:00 2024-07-27T03:01:53.796 INFO:teuthology.orchestra.run.smithi067.stdout:(103/160): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-07-27T03:01:53.798 INFO:teuthology.orchestra.run.smithi175.stdout:(38/160): ceph-osd-19.0.0-5358.gaf8fdbf0.el9.x8 3.2 MB/s | 18 MB 00:05 2024-07-27T03:01:53.823 INFO:teuthology.orchestra.run.smithi175.stdout:(39/160): fuse-2.9.9-16.el9.x86_64.rpm 330 kB/s | 80 kB 00:00 2024-07-27T03:01:53.907 INFO:teuthology.orchestra.run.smithi175.stdout:(40/160): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-07-27T03:01:53.957 INFO:teuthology.orchestra.run.smithi175.stdout:(41/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-07-27T03:01:53.967 INFO:teuthology.orchestra.run.smithi149.stdout:(56/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.9 MB/s | 3.0 MB 00:01 2024-07-27T03:01:54.005 INFO:teuthology.orchestra.run.smithi067.stdout:(104/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-07-27T03:01:54.007 INFO:teuthology.orchestra.run.smithi175.stdout:(42/160): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-07-27T03:01:54.038 INFO:teuthology.orchestra.run.smithi067.stdout:(105/160): abseil-cpp-20211102.0-3.el9.x86_64.r 2.2 MB/s | 548 kB 00:00 2024-07-27T03:01:54.049 INFO:teuthology.orchestra.run.smithi175.stdout:(43/160): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.2 MB 00:00 2024-07-27T03:01:54.059 INFO:teuthology.orchestra.run.smithi149.stdout:(57/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 384 kB/s | 160 kB 00:00 2024-07-27T03:01:54.075 INFO:teuthology.orchestra.run.smithi175.stdout:(44/160): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-07-27T03:01:54.084 INFO:teuthology.orchestra.run.smithi149.stdout:(58/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-07-27T03:01:54.097 INFO:teuthology.orchestra.run.smithi067.stdout:(106/160): gperftools-libs-2.9.1-2.el9.x86_64.r 3.3 MB/s | 309 kB 00:00 2024-07-27T03:01:54.100 INFO:teuthology.orchestra.run.smithi175.stdout:(45/160): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-07-27T03:01:54.125 INFO:teuthology.orchestra.run.smithi175.stdout:(46/160): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-07-27T03:01:54.139 INFO:teuthology.orchestra.run.smithi067.stdout:(107/160): protobuf-compiler-3.14.0-13.el9.x86_ 2.1 MB/s | 863 kB 00:00 2024-07-27T03:01:54.150 INFO:teuthology.orchestra.run.smithi175.stdout:(47/160): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-07-27T03:01:54.164 INFO:teuthology.orchestra.run.smithi067.stdout:(108/160): grpc-data-1.46.7-10.el9.noarch.rpm 155 kB/s | 19 kB 00:00 2024-07-27T03:01:54.214 INFO:teuthology.orchestra.run.smithi067.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-07-27T03:01:54.240 INFO:teuthology.orchestra.run.smithi067.stdout:(110/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-07-27T03:01:54.265 INFO:teuthology.orchestra.run.smithi067.stdout:(111/160): libunwind-1.6.2-1.el9.x86_64.rpm 1.4 MB/s | 67 kB 00:00 2024-07-27T03:01:54.290 INFO:teuthology.orchestra.run.smithi067.stdout:(112/160): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-07-27T03:01:54.341 INFO:teuthology.orchestra.run.smithi067.stdout:(113/160): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-07-27T03:01:54.366 INFO:teuthology.orchestra.run.smithi067.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-07-27T03:01:54.400 INFO:teuthology.orchestra.run.smithi067.stdout:(115/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-07-27T03:01:54.409 INFO:teuthology.orchestra.run.smithi149.stdout:(59/160): librdkafka-1.6.1-102.el9.x86_64.rpm 1.8 MB/s | 662 kB 00:00 2024-07-27T03:01:54.425 INFO:teuthology.orchestra.run.smithi067.stdout:(116/160): python3-cachetools-4.2.4-1.el9.noarc 548 kB/s | 32 kB 00:00 2024-07-27T03:01:54.450 INFO:teuthology.orchestra.run.smithi067.stdout:(117/160): python3-certifi-2023.05.07-4.el9.noa 279 kB/s | 14 kB 00:00 2024-07-27T03:01:54.476 INFO:teuthology.orchestra.run.smithi067.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-07-27T03:01:54.509 INFO:teuthology.orchestra.run.smithi067.stdout:(119/160): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-07-27T03:01:54.518 INFO:teuthology.orchestra.run.smithi149.stdout:(60/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 235 kB/s | 25 kB 00:00 2024-07-27T03:01:54.534 INFO:teuthology.orchestra.run.smithi067.stdout:(120/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-07-27T03:01:54.560 INFO:teuthology.orchestra.run.smithi067.stdout:(121/160): python3-google-auth-2.32.0-1.el9.noa 4.2 MB/s | 216 kB 00:00 2024-07-27T03:01:54.585 INFO:teuthology.orchestra.run.smithi067.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-07-27T03:01:54.611 INFO:teuthology.orchestra.run.smithi067.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 425 kB/s | 11 kB 00:00 2024-07-27T03:01:54.627 INFO:teuthology.orchestra.run.smithi149.stdout:(61/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-07-27T03:01:54.636 INFO:teuthology.orchestra.run.smithi067.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 710 kB/s | 18 kB 00:00 2024-07-27T03:01:54.661 INFO:teuthology.orchestra.run.smithi067.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 923 kB/s | 23 kB 00:00 2024-07-27T03:01:54.686 INFO:teuthology.orchestra.run.smithi067.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 776 kB/s | 19 kB 00:00 2024-07-27T03:01:54.711 INFO:teuthology.orchestra.run.smithi067.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 789 kB/s | 20 kB 00:00 2024-07-27T03:01:54.735 INFO:teuthology.orchestra.run.smithi149.stdout:(62/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-07-27T03:01:54.779 INFO:teuthology.orchestra.run.smithi067.stdout:(128/160): libarrow-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 4.4 MB 00:00 2024-07-27T03:01:54.804 INFO:teuthology.orchestra.run.smithi067.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-07-27T03:01:54.829 INFO:teuthology.orchestra.run.smithi067.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-07-27T03:01:54.844 INFO:teuthology.orchestra.run.smithi149.stdout:(63/160): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-07-27T03:01:54.850 INFO:teuthology.orchestra.run.smithi175.stdout:(48/160): automake-1.16.2-8.el9.noarch.rpm 950 kB/s | 689 kB 00:00 2024-07-27T03:01:54.854 INFO:teuthology.orchestra.run.smithi067.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-07-27T03:01:54.896 INFO:teuthology.orchestra.run.smithi067.stdout:(132/160): python3-grpcio-1.46.7-10.el9.x86_64. 5.6 MB/s | 2.0 MB 00:00 2024-07-27T03:01:54.911 INFO:teuthology.orchestra.run.smithi149.stdout:(64/160): libtool-2.4.6-46.el9.x86_64.rpm 703 kB/s | 581 kB 00:00 2024-07-27T03:01:54.917 INFO:teuthology.orchestra.run.smithi175.stdout:(49/160): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-07-27T03:01:54.930 INFO:teuthology.orchestra.run.smithi067.stdout:(133/160): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-07-27T03:01:54.945 INFO:teuthology.orchestra.run.smithi149.stdout:(65/160): openblas-0.3.21-2.el9.x86_64.rpm 351 kB/s | 35 kB 00:00 2024-07-27T03:01:54.955 INFO:teuthology.orchestra.run.smithi067.stdout:(134/160): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-07-27T03:01:54.959 INFO:teuthology.orchestra.run.smithi175.stdout:(50/160): boost-program-options-1.75.0-8.el9.x8 990 kB/s | 107 kB 00:00 2024-07-27T03:01:54.981 INFO:teuthology.orchestra.run.smithi067.stdout:(135/160): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-07-27T03:01:55.006 INFO:teuthology.orchestra.run.smithi067.stdout:(136/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-07-27T03:01:55.018 INFO:teuthology.orchestra.run.smithi175.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 216 kB/s | 22 kB 00:00 2024-07-27T03:01:55.031 INFO:teuthology.orchestra.run.smithi067.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-07-27T03:01:55.065 INFO:teuthology.orchestra.run.smithi067.stdout:(138/160): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-07-27T03:01:55.085 INFO:teuthology.orchestra.run.smithi175.stdout:(52/160): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-07-27T03:01:55.090 INFO:teuthology.orchestra.run.smithi067.stdout:(139/160): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-07-27T03:01:55.115 INFO:teuthology.orchestra.run.smithi067.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-07-27T03:01:55.118 INFO:teuthology.orchestra.run.smithi175.stdout:(53/160): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-07-27T03:01:55.141 INFO:teuthology.orchestra.run.smithi067.stdout:(141/160): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-07-27T03:01:55.166 INFO:teuthology.orchestra.run.smithi067.stdout:(142/160): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-07-27T03:01:55.183 INFO:teuthology.orchestra.run.smithi067.stdout:(143/160): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-07-27T03:01:55.208 INFO:teuthology.orchestra.run.smithi067.stdout:(144/160): python3-zc-lockfile-2.0-10.el9.noarc 799 kB/s | 20 kB 00:00 2024-07-27T03:01:55.233 INFO:teuthology.orchestra.run.smithi067.stdout:(145/160): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-07-27T03:01:55.259 INFO:teuthology.orchestra.run.smithi067.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-07-27T03:01:55.284 INFO:teuthology.orchestra.run.smithi067.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-07-27T03:01:55.335 INFO:teuthology.orchestra.run.smithi067.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-07-27T03:01:55.385 INFO:teuthology.orchestra.run.smithi175.stdout:(54/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 10 MB/s | 3.0 MB 00:00 2024-07-27T03:01:55.411 INFO:teuthology.orchestra.run.smithi175.stdout:(55/160): flexiblas-openblas-openmp-3.0.4-8.el9 62 kB/s | 18 kB 00:00 2024-07-27T03:01:55.495 INFO:teuthology.orchestra.run.smithi175.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-07-27T03:01:55.520 INFO:teuthology.orchestra.run.smithi175.stdout:(57/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-07-27T03:01:55.543 INFO:teuthology.orchestra.run.smithi067.stdout:(149/160): librbd1-19.0.0-5358.gaf8fdbf0.el9.x8 12 MB/s | 3.1 MB 00:00 2024-07-27T03:01:55.595 INFO:teuthology.orchestra.run.smithi175.stdout:(58/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 454 kB/s | 45 kB 00:00 2024-07-27T03:01:55.637 INFO:teuthology.orchestra.run.smithi175.stdout:(59/160): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-07-27T03:01:55.677 INFO:teuthology.orchestra.run.smithi067.stdout:(150/160): NetworkManager-1.48.2-2.el9.x86_64.r 6.8 MB/s | 2.3 MB 00:00 2024-07-27T03:01:55.704 INFO:teuthology.orchestra.run.smithi175.stdout:(60/160): libtool-2.4.6-46.el9.x86_64.rpm 5.2 MB/s | 581 kB 00:00 2024-07-27T03:01:55.729 INFO:teuthology.orchestra.run.smithi175.stdout:(61/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 277 kB/s | 25 kB 00:00 2024-07-27T03:01:55.735 INFO:teuthology.orchestra.run.smithi067.stdout:(151/160): librados2-19.0.0-5358.gaf8fdbf0.el9. 7.1 MB/s | 3.4 MB 00:00 2024-07-27T03:01:55.737 INFO:teuthology.orchestra.run.smithi149.stdout:(66/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2024-07-27T03:01:55.761 INFO:teuthology.orchestra.run.smithi067.stdout:(152/160): NetworkManager-team-1.48.2-2.el9.x86 488 kB/s | 41 kB 00:00 2024-07-27T03:01:55.796 INFO:teuthology.orchestra.run.smithi175.stdout:(62/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 516 kB/s | 47 kB 00:00 2024-07-27T03:01:55.838 INFO:teuthology.orchestra.run.smithi175.stdout:(63/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-07-27T03:01:55.897 INFO:teuthology.orchestra.run.smithi175.stdout:(64/160): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-07-27T03:01:55.902 INFO:teuthology.orchestra.run.smithi067.stdout:(153/160): elfutils-debuginfod-client-0.191-4.e 259 kB/s | 37 kB 00:00 2024-07-27T03:01:55.938 INFO:teuthology.orchestra.run.smithi175.stdout:(65/160): openblas-0.3.21-2.el9.x86_64.rpm 351 kB/s | 35 kB 00:00 2024-07-27T03:01:55.944 INFO:teuthology.orchestra.run.smithi067.stdout:(154/160): NetworkManager-libnm-1.48.2-2.el9.x8 4.5 MB/s | 1.8 MB 00:00 2024-07-27T03:01:55.970 INFO:teuthology.orchestra.run.smithi067.stdout:(155/160): elfutils-libelf-0.191-4.el9.x86_64.r 3.0 MB/s | 207 kB 00:00 2024-07-27T03:01:56.003 INFO:teuthology.orchestra.run.smithi067.stdout:(156/160): elfutils-libs-0.191-4.el9.x86_64.rpm 4.3 MB/s | 260 kB 00:00 2024-07-27T03:01:56.020 INFO:teuthology.orchestra.run.smithi149.stdout:(67/160): postgresql-13.14-1.el9.x86_64.rpm 5.6 MB/s | 1.6 MB 00:00 2024-07-27T03:01:56.079 INFO:teuthology.orchestra.run.smithi067.stdout:(157/160): openssl-3.2.2-2.el9.x86_64.rpm 12 MB/s | 1.4 MB 00:00 2024-07-27T03:01:56.104 INFO:teuthology.orchestra.run.smithi067.stdout:(158/160): NetworkManager-tui-1.48.2-2.el9.x86_ 671 kB/s | 247 kB 00:00 2024-07-27T03:01:56.139 INFO:teuthology.orchestra.run.smithi175.stdout:(66/160): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-07-27T03:01:56.145 INFO:teuthology.orchestra.run.smithi149.stdout:(68/160): postgresql-contrib-13.14-1.el9.x86_64 7.1 MB/s | 911 kB 00:00 2024-07-27T03:01:56.179 INFO:teuthology.orchestra.run.smithi067.stdout:(159/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-07-27T03:01:56.213 INFO:teuthology.orchestra.run.smithi067.stdout:(160/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 8.1 MB/s | 1.1 MB 00:00 2024-07-27T03:01:56.218 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-07-27T03:01:56.218 INFO:teuthology.orchestra.run.smithi067.stdout:Total 21 MB/s | 232 MB 00:11 2024-07-27T03:01:56.254 INFO:teuthology.orchestra.run.smithi149.stdout:(69/160): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-07-27T03:01:56.289 INFO:teuthology.orchestra.run.smithi175.stdout:(67/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-07-27T03:01:56.331 INFO:teuthology.orchestra.run.smithi175.stdout:(68/160): postgresql-13.14-1.el9.x86_64.rpm 8.4 MB/s | 1.6 MB 00:00 2024-07-27T03:01:56.406 INFO:teuthology.orchestra.run.smithi175.stdout:(69/160): postgresql-contrib-13.14-1.el9.x86_64 7.7 MB/s | 911 kB 00:00 2024-07-27T03:01:56.423 INFO:teuthology.orchestra.run.smithi175.stdout:(70/160): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-07-27T03:01:56.665 INFO:teuthology.orchestra.run.smithi175.stdout:(71/160): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-07-27T03:01:56.699 INFO:teuthology.orchestra.run.smithi175.stdout:(72/160): protobuf-3.14.0-13.el9.x86_64.rpm 3.7 MB/s | 1.0 MB 00:00 2024-07-27T03:01:56.766 INFO:teuthology.orchestra.run.smithi175.stdout:(73/160): python3-attrs-20.3.0-7.el9.noarch.rpm 886 kB/s | 87 kB 00:00 2024-07-27T03:01:56.891 INFO:teuthology.orchestra.run.smithi175.stdout:(74/160): python3-colorama-0.4.6-3.el9.noarch.r 490 kB/s | 61 kB 00:00 2024-07-27T03:01:56.963 INFO:teuthology.orchestra.run.smithi149.stdout:(70/160): ceph-test-19.0.0-5358.gaf8fdbf0.el9.x 4.6 MB/s | 44 MB 00:09 2024-07-27T03:01:56.966 INFO:teuthology.orchestra.run.smithi175.stdout:(75/160): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-07-27T03:01:56.992 INFO:teuthology.orchestra.run.smithi175.stdout:(76/160): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-07-27T03:01:57.067 INFO:teuthology.orchestra.run.smithi175.stdout:(77/160): python3-jinja2-2.11.3-6.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-07-27T03:01:57.092 INFO:teuthology.orchestra.run.smithi175.stdout:(78/160): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-07-27T03:01:57.167 INFO:teuthology.orchestra.run.smithi175.stdout:(79/160): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-07-27T03:01:57.193 INFO:teuthology.orchestra.run.smithi175.stdout:(80/160): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-07-27T03:01:57.268 INFO:teuthology.orchestra.run.smithi175.stdout:(81/160): python3-markupsafe-1.1.1-12.el9.x86_6 346 kB/s | 35 kB 00:00 2024-07-27T03:01:57.321 INFO:teuthology.orchestra.run.smithi149.stdout:(71/160): postgresql-server-13.14-1.el9.x86_64. 5.4 MB/s | 5.8 MB 00:01 2024-07-27T03:01:57.422 INFO:teuthology.orchestra.run.smithi149.stdout:(72/160): python3-attrs-20.3.0-7.el9.noarch.rpm 876 kB/s | 87 kB 00:00 2024-07-27T03:01:57.435 INFO:teuthology.orchestra.run.smithi175.stdout:(82/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 5.1 MB 00:00 2024-07-27T03:01:57.460 INFO:teuthology.orchestra.run.smithi175.stdout:(83/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 250 kB 00:00 2024-07-27T03:01:57.527 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-07-27T03:01:57.756 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-07-27T03:01:57.756 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-07-27T03:01:57.830 INFO:teuthology.orchestra.run.smithi149.stdout:(73/160): protobuf-3.14.0-13.el9.x86_64.rpm 1.2 MB/s | 1.0 MB 00:00 2024-07-27T03:01:57.835 INFO:teuthology.orchestra.run.smithi175.stdout:(84/160): ceph-test-19.0.0-5358.gaf8fdbf0.el9.x 4.9 MB/s | 44 MB 00:08 2024-07-27T03:01:57.860 INFO:teuthology.orchestra.run.smithi175.stdout:(85/160): python3-oauthlib-3.1.1-5.el9.noarch.r 523 kB/s | 222 kB 00:00 2024-07-27T03:01:57.885 INFO:teuthology.orchestra.run.smithi175.stdout:(86/160): python3-packaging-20.9-5.el9.noarch.r 181 kB/s | 77 kB 00:00 2024-07-27T03:01:57.930 INFO:teuthology.orchestra.run.smithi149.stdout:(74/160): python3-colorama-0.4.6-3.el9.noarch.r 614 kB/s | 61 kB 00:00 2024-07-27T03:01:57.936 INFO:teuthology.orchestra.run.smithi175.stdout:(87/160): python3-prettytable-0.7.2-27.el9.noar 425 kB/s | 42 kB 00:00 2024-07-27T03:01:57.986 INFO:teuthology.orchestra.run.smithi175.stdout:(88/160): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-07-27T03:01:58.012 INFO:teuthology.orchestra.run.smithi175.stdout:(89/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-07-27T03:01:58.037 INFO:teuthology.orchestra.run.smithi175.stdout:(90/160): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-07-27T03:01:58.039 INFO:teuthology.orchestra.run.smithi149.stdout:(75/160): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-07-27T03:01:58.087 INFO:teuthology.orchestra.run.smithi175.stdout:(91/160): python3-pytz-2021.1-5.el9.noarch.rpm 510 kB/s | 51 kB 00:00 2024-07-27T03:01:58.112 INFO:teuthology.orchestra.run.smithi175.stdout:(92/160): python3-requests-oauthlib-1.3.0-12.el 534 kB/s | 54 kB 00:00 2024-07-27T03:01:58.148 INFO:teuthology.orchestra.run.smithi149.stdout:(76/160): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-07-27T03:01:58.229 INFO:teuthology.orchestra.run.smithi175.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 293 kB/s | 42 kB 00:00 2024-07-27T03:01:58.248 INFO:teuthology.orchestra.run.smithi149.stdout:(77/160): python3-jsonpatch-1.21-16.el9.noarch. 257 kB/s | 26 kB 00:00 2024-07-27T03:01:58.254 INFO:teuthology.orchestra.run.smithi175.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 311 kB/s | 44 kB 00:00 2024-07-27T03:01:58.398 INFO:teuthology.orchestra.run.smithi149.stdout:(78/160): python3-babel-2.9.1-2.el9.noarch.rpm 6.1 MB/s | 6.0 MB 00:00 2024-07-27T03:01:58.424 INFO:teuthology.orchestra.run.smithi149.stdout:(79/160): python3-jsonpointer-2.0-4.el9.noarch. 109 kB/s | 19 kB 00:00 2024-07-27T03:01:58.485 INFO:teuthology.orchestra.run.smithi175.stdout:(95/160): qatlib-service-24.02.0-1.el9.x86_64.r 158 kB/s | 36 kB 00:00 2024-07-27T03:01:58.507 INFO:teuthology.orchestra.run.smithi149.stdout:(80/160): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-07-27T03:01:58.532 INFO:teuthology.orchestra.run.smithi149.stdout:(81/160): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-07-27T03:01:58.641 INFO:teuthology.orchestra.run.smithi149.stdout:(82/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-07-27T03:01:58.739 INFO:teuthology.orchestra.run.smithi175.stdout:(96/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-07-27T03:01:58.741 INFO:teuthology.orchestra.run.smithi149.stdout:(83/160): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-07-27T03:01:58.764 INFO:teuthology.orchestra.run.smithi175.stdout:(97/160): qatlib-24.02.0-1.el9.x86_64.rpm 413 kB/s | 221 kB 00:00 2024-07-27T03:01:58.781 INFO:teuthology.orchestra.run.smithi175.stdout:(98/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 162 kB/s | 47 kB 00:00 2024-07-27T03:01:58.848 INFO:teuthology.orchestra.run.smithi175.stdout:(99/160): socat-1.7.4.1-6.el9.x86_64.rpm 2.7 MB/s | 304 kB 00:00 2024-07-27T03:01:58.850 INFO:teuthology.orchestra.run.smithi149.stdout:(84/160): python3-packaging-20.9-5.el9.noarch.r 714 kB/s | 77 kB 00:00 2024-07-27T03:01:58.873 INFO:teuthology.orchestra.run.smithi175.stdout:(100/160): uuid-1.6.2-55.el9.x86_64.rpm 629 kB/s | 58 kB 00:00 2024-07-27T03:01:58.898 INFO:teuthology.orchestra.run.smithi175.stdout:(101/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 817 kB/s | 96 kB 00:00 2024-07-27T03:01:58.958 INFO:teuthology.orchestra.run.smithi149.stdout:(85/160): python3-prettytable-0.7.2-27.el9.noar 390 kB/s | 42 kB 00:00 2024-07-27T03:01:58.967 INFO:teuthology.orchestra.run.smithi175.stdout:(102/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 536 kB/s | 64 kB 00:00 2024-07-27T03:01:59.067 INFO:teuthology.orchestra.run.smithi149.stdout:(86/160): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-07-27T03:01:59.101 INFO:teuthology.orchestra.run.smithi175.stdout:(103/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 197 kB/s | 45 kB 00:00 2024-07-27T03:01:59.141 INFO:teuthology.orchestra.run.smithi175.stdout:(104/160): lua-devel-5.4.4-4.el9.x86_64.rpm 92 kB/s | 22 kB 00:00 2024-07-27T03:01:59.176 INFO:teuthology.orchestra.run.smithi149.stdout:(87/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-07-27T03:01:59.232 INFO:teuthology.orchestra.run.smithi175.stdout:(105/160): abseil-cpp-20211102.0-3.el9.x86_64.r 4.1 MB/s | 548 kB 00:00 2024-07-27T03:01:59.249 INFO:teuthology.orchestra.run.smithi175.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-07-27T03:01:59.274 INFO:teuthology.orchestra.run.smithi175.stdout:(107/160): gperftools-libs-2.9.1-2.el9.x86_64.r 2.3 MB/s | 309 kB 00:00 2024-07-27T03:01:59.302 INFO:teuthology.orchestra.run.smithi175.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-07-27T03:01:59.325 INFO:teuthology.orchestra.run.smithi175.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-07-27T03:01:59.326 INFO:teuthology.orchestra.run.smithi149.stdout:(88/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.2 MB/s | 5.1 MB 00:00 2024-07-27T03:01:59.351 INFO:teuthology.orchestra.run.smithi175.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-07-27T03:01:59.360 INFO:teuthology.orchestra.run.smithi149.stdout:(89/160): python3-pyasn1-modules-0.4.8-6.el9.no 1.5 MB/s | 279 kB 00:00 2024-07-27T03:01:59.384 INFO:teuthology.orchestra.run.smithi175.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-07-27T03:01:59.426 INFO:teuthology.orchestra.run.smithi175.stdout:(112/160): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-07-27T03:01:59.427 INFO:teuthology.orchestra.run.smithi149.stdout:(90/160): python3-pytz-2021.1-5.el9.noarch.rpm 508 kB/s | 51 kB 00:00 2024-07-27T03:01:59.461 INFO:teuthology.orchestra.run.smithi149.stdout:(91/160): python3-requests-oauthlib-1.3.0-12.el 535 kB/s | 54 kB 00:00 2024-07-27T03:01:59.493 INFO:teuthology.orchestra.run.smithi175.stdout:(113/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.6 MB/s | 839 kB 00:00 2024-07-27T03:01:59.527 INFO:teuthology.orchestra.run.smithi175.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.4 MB/s | 43 kB 00:00 2024-07-27T03:01:59.552 INFO:teuthology.orchestra.run.smithi175.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-07-27T03:01:59.561 INFO:teuthology.orchestra.run.smithi149.stdout:(92/160): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-07-27T03:01:59.577 INFO:teuthology.orchestra.run.smithi175.stdout:(116/160): python3-certifi-2023.05.07-4.el9.noa 566 kB/s | 14 kB 00:00 2024-07-27T03:01:59.602 INFO:teuthology.orchestra.run.smithi175.stdout:(117/160): python3-cheroot-8.6.0-4.el9.noarch.r 6.8 MB/s | 172 kB 00:00 2024-07-27T03:01:59.627 INFO:teuthology.orchestra.run.smithi175.stdout:(118/160): python3-asyncssh-2.13.2-5.el9.noarch 2.7 MB/s | 548 kB 00:00 2024-07-27T03:01:59.661 INFO:teuthology.orchestra.run.smithi149.stdout:(93/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-07-27T03:01:59.661 INFO:teuthology.orchestra.run.smithi175.stdout:(119/160): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-07-27T03:01:59.728 INFO:teuthology.orchestra.run.smithi175.stdout:(120/160): libarrow-9.0.0-11.el9.x86_64.rpm 9.2 MB/s | 4.4 MB 00:00 2024-07-27T03:01:59.753 INFO:teuthology.orchestra.run.smithi175.stdout:(121/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-07-27T03:01:59.770 INFO:teuthology.orchestra.run.smithi149.stdout:(94/160): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-07-27T03:01:59.779 INFO:teuthology.orchestra.run.smithi175.stdout:(122/160): python3-google-auth-2.32.0-1.el9.noa 1.8 MB/s | 216 kB 00:00 2024-07-27T03:01:59.804 INFO:teuthology.orchestra.run.smithi175.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-07-27T03:01:59.830 INFO:teuthology.orchestra.run.smithi175.stdout:(124/160): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-07-27T03:01:59.855 INFO:teuthology.orchestra.run.smithi175.stdout:(125/160): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-07-27T03:01:59.878 INFO:teuthology.orchestra.run.smithi149.stdout:(95/160): qatlib-service-24.02.0-1.el9.x86_64.r 332 kB/s | 36 kB 00:00 2024-07-27T03:01:59.880 INFO:teuthology.orchestra.run.smithi175.stdout:(126/160): python3-jaraco-collections-3.0.0-8.e 459 kB/s | 23 kB 00:00 2024-07-27T03:01:59.906 INFO:teuthology.orchestra.run.smithi175.stdout:(127/160): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-07-27T03:01:59.931 INFO:teuthology.orchestra.run.smithi175.stdout:(128/160): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-07-27T03:01:59.956 INFO:teuthology.orchestra.run.smithi175.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-07-27T03:01:59.973 INFO:teuthology.orchestra.run.smithi175.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 4.7 MB/s | 79 kB 00:00 2024-07-27T03:01:59.978 INFO:teuthology.orchestra.run.smithi149.stdout:(96/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 474 kB/s | 47 kB 00:00 2024-07-27T03:01:59.998 INFO:teuthology.orchestra.run.smithi175.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-07-27T03:02:00.032 INFO:teuthology.orchestra.run.smithi175.stdout:(132/160): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-07-27T03:02:00.038 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-07-27T03:02:00.039 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-07-27T03:02:00.057 INFO:teuthology.orchestra.run.smithi175.stdout:(133/160): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-07-27T03:02:00.082 INFO:teuthology.orchestra.run.smithi175.stdout:(134/160): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-07-27T03:02:00.087 INFO:teuthology.orchestra.run.smithi149.stdout:(97/160): socat-1.7.4.1-6.el9.x86_64.rpm 2.7 MB/s | 304 kB 00:00 2024-07-27T03:02:00.108 INFO:teuthology.orchestra.run.smithi175.stdout:(135/160): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-07-27T03:02:00.133 INFO:teuthology.orchestra.run.smithi175.stdout:(136/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-07-27T03:02:00.158 INFO:teuthology.orchestra.run.smithi175.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-07-27T03:02:00.196 INFO:teuthology.orchestra.run.smithi149.stdout:(98/160): uuid-1.6.2-55.el9.x86_64.rpm 534 kB/s | 58 kB 00:00 2024-07-27T03:02:00.209 INFO:teuthology.orchestra.run.smithi175.stdout:(138/160): python3-grpcio-1.46.7-10.el9.x86_64. 4.2 MB/s | 2.0 MB 00:00 2024-07-27T03:02:00.250 INFO:teuthology.orchestra.run.smithi175.stdout:(139/160): python3-rsa-4.9-2.el9.noarch.rpm 644 kB/s | 59 kB 00:00 2024-07-27T03:02:00.276 INFO:teuthology.orchestra.run.smithi175.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 535 kB/s | 36 kB 00:00 2024-07-27T03:02:00.296 INFO:teuthology.orchestra.run.smithi149.stdout:(99/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rpm 957 kB/s | 96 kB 00:00 2024-07-27T03:02:00.301 INFO:teuthology.orchestra.run.smithi175.stdout:(141/160): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-07-27T03:02:00.326 INFO:teuthology.orchestra.run.smithi175.stdout:(142/160): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-07-27T03:02:00.352 INFO:teuthology.orchestra.run.smithi175.stdout:(143/160): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-07-27T03:02:00.377 INFO:teuthology.orchestra.run.smithi175.stdout:(144/160): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-07-27T03:02:00.394 INFO:teuthology.orchestra.run.smithi175.stdout:(145/160): python3-zc-lockfile-2.0-10.el9.noarc 478 kB/s | 20 kB 00:00 2024-07-27T03:02:00.396 INFO:teuthology.orchestra.run.smithi149.stdout:(100/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 636 kB/s | 64 kB 00:00 2024-07-27T03:02:00.419 INFO:teuthology.orchestra.run.smithi175.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-07-27T03:02:00.444 INFO:teuthology.orchestra.run.smithi175.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-07-27T03:02:00.495 INFO:teuthology.orchestra.run.smithi175.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-07-27T03:02:00.496 INFO:teuthology.orchestra.run.smithi149.stdout:(101/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 449 kB/s | 45 kB 00:00 2024-07-27T03:02:00.589 INFO:teuthology.orchestra.run.smithi149.stdout:(102/160): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-07-27T03:02:00.687 INFO:teuthology.orchestra.run.smithi175.stdout:(149/160): NetworkManager-1.48.2-2.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-07-27T03:02:00.745 INFO:teuthology.orchestra.run.smithi175.stdout:(150/160): NetworkManager-libnm-1.48.2-2.el9.x8 31 MB/s | 1.8 MB 00:00 2024-07-27T03:02:00.764 INFO:teuthology.orchestra.run.smithi149.stdout:(103/160): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-07-27T03:02:00.787 INFO:teuthology.orchestra.run.smithi175.stdout:(151/160): NetworkManager-team-1.48.2-2.el9.x86 987 kB/s | 41 kB 00:00 2024-07-27T03:02:00.829 INFO:teuthology.orchestra.run.smithi175.stdout:(152/160): NetworkManager-tui-1.48.2-2.el9.x86_ 5.8 MB/s | 247 kB 00:00 2024-07-27T03:02:00.839 INFO:teuthology.orchestra.run.smithi149.stdout:(104/160): abseil-cpp-20211102.0-3.el9.x86_64.r 7.2 MB/s | 548 kB 00:00 2024-07-27T03:02:00.871 INFO:teuthology.orchestra.run.smithi175.stdout:(153/160): elfutils-debuginfod-client-0.191-4.e 888 kB/s | 37 kB 00:00 2024-07-27T03:02:00.873 INFO:teuthology.orchestra.run.smithi149.stdout:(105/160): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-07-27T03:02:00.905 INFO:teuthology.orchestra.run.smithi175.stdout:(154/160): elfutils-libelf-0.191-4.el9.x86_64.r 6.1 MB/s | 207 kB 00:00 2024-07-27T03:02:00.906 INFO:teuthology.orchestra.run.smithi149.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 587 kB/s | 19 kB 00:00 2024-07-27T03:02:00.946 INFO:teuthology.orchestra.run.smithi175.stdout:(155/160): elfutils-libs-0.191-4.el9.x86_64.rpm 6.1 MB/s | 260 kB 00:00 2024-07-27T03:02:01.005 INFO:teuthology.orchestra.run.smithi175.stdout:(156/160): librbd1-19.0.0-5358.gaf8fdbf0.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-07-27T03:02:01.032 INFO:teuthology.orchestra.run.smithi149.stdout:(107/160): libarrow-9.0.0-11.el9.x86_64.rpm 35 MB/s | 4.4 MB 00:00 2024-07-27T03:02:01.047 INFO:teuthology.orchestra.run.smithi175.stdout:(157/160): openssl-3.2.2-2.el9.x86_64.rpm 13 MB/s | 1.4 MB 00:00 2024-07-27T03:02:01.057 INFO:teuthology.orchestra.run.smithi149.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-07-27T03:02:01.082 INFO:teuthology.orchestra.run.smithi149.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-07-27T03:02:01.106 INFO:teuthology.orchestra.run.smithi175.stdout:(158/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 19 MB/s | 1.1 MB 00:00 2024-07-27T03:02:01.107 INFO:teuthology.orchestra.run.smithi149.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-07-27T03:02:01.133 INFO:teuthology.orchestra.run.smithi149.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-07-27T03:02:01.175 INFO:teuthology.orchestra.run.smithi149.stdout:(112/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-07-27T03:02:01.198 INFO:teuthology.orchestra.run.smithi175.stdout:(159/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-07-27T03:02:01.208 INFO:teuthology.orchestra.run.smithi149.stdout:(113/160): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-07-27T03:02:01.225 INFO:teuthology.orchestra.run.smithi149.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-07-27T03:02:01.251 INFO:teuthology.orchestra.run.smithi149.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-07-27T03:02:01.273 INFO:teuthology.orchestra.run.smithi175.stdout:(160/160): librados2-19.0.0-5358.gaf8fdbf0.el9. 4.0 MB/s | 3.4 MB 00:00 2024-07-27T03:02:01.276 INFO:teuthology.orchestra.run.smithi149.stdout:(116/160): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-07-27T03:02:01.280 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-07-27T03:02:01.280 INFO:teuthology.orchestra.run.smithi175.stdout:Total 14 MB/s | 232 MB 00:16 2024-07-27T03:02:01.301 INFO:teuthology.orchestra.run.smithi149.stdout:(117/160): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-07-27T03:02:01.335 INFO:teuthology.orchestra.run.smithi149.stdout:(118/160): python3-cherrypy-18.6.1-2.el9.noarch 10 MB/s | 358 kB 00:00 2024-07-27T03:02:01.369 INFO:teuthology.orchestra.run.smithi149.stdout:(119/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-07-27T03:02:01.394 INFO:teuthology.orchestra.run.smithi149.stdout:(120/160): python3-google-auth-2.32.0-1.el9.noa 8.5 MB/s | 216 kB 00:00 2024-07-27T03:02:01.461 INFO:teuthology.orchestra.run.smithi149.stdout:(121/160): python3-grpcio-1.46.7-10.el9.x86_64. 31 MB/s | 2.0 MB 00:00 2024-07-27T03:02:01.486 INFO:teuthology.orchestra.run.smithi149.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-07-27T03:02:01.511 INFO:teuthology.orchestra.run.smithi149.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 425 kB/s | 11 kB 00:00 2024-07-27T03:02:01.537 INFO:teuthology.orchestra.run.smithi149.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 707 kB/s | 18 kB 00:00 2024-07-27T03:02:01.562 INFO:teuthology.orchestra.run.smithi149.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 933 kB/s | 23 kB 00:00 2024-07-27T03:02:01.579 INFO:teuthology.orchestra.run.smithi149.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-07-27T03:02:01.596 INFO:teuthology.orchestra.run.smithi149.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 1.2 MB/s | 20 kB 00:00 2024-07-27T03:02:01.763 INFO:teuthology.orchestra.run.smithi149.stdout:(128/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.0 MB/s | 16 MB 00:02 2024-07-27T03:02:01.830 INFO:teuthology.orchestra.run.smithi149.stdout:(129/160): python3-kubernetes-26.1.0-2.el9.noar 4.4 MB/s | 1.0 MB 00:00 2024-07-27T03:02:01.855 INFO:teuthology.orchestra.run.smithi149.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 3.4 MB/s | 79 kB 00:00 2024-07-27T03:02:01.880 INFO:teuthology.orchestra.run.smithi149.stdout:(131/160): python3-logutils-0.3.5-21.el9.noarch 396 kB/s | 46 kB 00:00 2024-07-27T03:02:01.956 INFO:teuthology.orchestra.run.smithi149.stdout:(132/160): openblas-openmp-0.3.21-2.el9.x86_64. 689 kB/s | 4.7 MB 00:07 2024-07-27T03:02:01.981 INFO:teuthology.orchestra.run.smithi149.stdout:(133/160): python3-natsort-7.1.1-5.el9.noarch.r 459 kB/s | 58 kB 00:00 2024-07-27T03:02:02.031 INFO:teuthology.orchestra.run.smithi149.stdout:(134/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.8 MB/s | 90 kB 00:00 2024-07-27T03:02:02.056 INFO:teuthology.orchestra.run.smithi149.stdout:(135/160): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-07-27T03:02:02.082 INFO:teuthology.orchestra.run.smithi149.stdout:(136/160): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2024-07-27T03:02:02.107 INFO:teuthology.orchestra.run.smithi149.stdout:(137/160): python3-pyperclip-1.8.2-2.el9.noarch 386 kB/s | 28 kB 00:00 2024-07-27T03:02:02.132 INFO:teuthology.orchestra.run.smithi149.stdout:(138/160): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-07-27T03:02:02.157 INFO:teuthology.orchestra.run.smithi149.stdout:(139/160): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-07-27T03:02:02.183 INFO:teuthology.orchestra.run.smithi149.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-07-27T03:02:02.208 INFO:teuthology.orchestra.run.smithi149.stdout:(141/160): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-07-27T03:02:02.233 INFO:teuthology.orchestra.run.smithi149.stdout:(142/160): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-07-27T03:02:02.258 INFO:teuthology.orchestra.run.smithi149.stdout:(143/160): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-07-27T03:02:02.284 INFO:teuthology.orchestra.run.smithi149.stdout:(144/160): python3-zc-lockfile-2.0-10.el9.noarc 799 kB/s | 20 kB 00:00 2024-07-27T03:02:02.309 INFO:teuthology.orchestra.run.smithi149.stdout:(145/160): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-07-27T03:02:02.334 INFO:teuthology.orchestra.run.smithi149.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-07-27T03:02:02.362 INFO:teuthology.orchestra.run.smithi149.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-07-27T03:02:02.412 INFO:teuthology.orchestra.run.smithi149.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-07-27T03:02:02.525 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-07-27T03:02:02.718 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-07-27T03:02:02.718 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-07-27T03:02:02.763 INFO:teuthology.orchestra.run.smithi149.stdout:(149/160): librados2-19.0.0-5358.gaf8fdbf0.el9. 7.9 MB/s | 3.4 MB 00:00 2024-07-27T03:02:02.971 INFO:teuthology.orchestra.run.smithi149.stdout:(150/160): librbd1-19.0.0-5358.gaf8fdbf0.el9.x8 5.2 MB/s | 3.1 MB 00:00 2024-07-27T03:02:03.021 INFO:teuthology.orchestra.run.smithi149.stdout:(151/160): NetworkManager-1.48.2-2.el9.x86_64.r 3.8 MB/s | 2.3 MB 00:00 2024-07-27T03:02:03.080 INFO:teuthology.orchestra.run.smithi149.stdout:(152/160): NetworkManager-team-1.48.2-2.el9.x86 376 kB/s | 41 kB 00:00 2024-07-27T03:02:03.122 INFO:teuthology.orchestra.run.smithi149.stdout:(153/160): NetworkManager-tui-1.48.2-2.el9.x86_ 2.4 MB/s | 247 kB 00:00 2024-07-27T03:02:03.148 INFO:teuthology.orchestra.run.smithi149.stdout:(154/160): elfutils-debuginfod-client-0.191-4.e 549 kB/s | 37 kB 00:00 2024-07-27T03:02:03.189 INFO:teuthology.orchestra.run.smithi149.stdout:(155/160): NetworkManager-libnm-1.48.2-2.el9.x8 4.2 MB/s | 1.8 MB 00:00 2024-07-27T03:02:03.231 INFO:teuthology.orchestra.run.smithi149.stdout:(156/160): elfutils-libelf-0.191-4.el9.x86_64.r 1.9 MB/s | 207 kB 00:00 2024-07-27T03:02:03.257 INFO:teuthology.orchestra.run.smithi149.stdout:(157/160): elfutils-libs-0.191-4.el9.x86_64.rpm 2.3 MB/s | 260 kB 00:00 2024-07-27T03:02:03.551 INFO:teuthology.orchestra.run.smithi149.stdout:(158/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 3.7 MB/s | 1.1 MB 00:00 2024-07-27T03:02:03.715 INFO:teuthology.orchestra.run.smithi149.stdout:(159/160): openssl-3.2.2-2.el9.x86_64.rpm 2.6 MB/s | 1.4 MB 00:00 2024-07-27T03:02:03.907 INFO:teuthology.orchestra.run.smithi149.stdout:(160/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 3.5 MB/s | 2.4 MB 00:00 2024-07-27T03:02:03.914 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-07-27T03:02:03.914 INFO:teuthology.orchestra.run.smithi149.stdout:Total 12 MB/s | 232 MB 00:19 2024-07-27T03:02:04.567 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-07-27T03:02:04.567 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-07-27T03:02:05.169 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-07-27T03:02:05.367 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-07-27T03:02:05.368 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-07-27T03:02:06.730 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-07-27T03:02:06.834 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-07-27T03:02:06.874 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-07-27T03:02:06.928 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-07-27T03:02:06.969 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-07-27T03:02:06.994 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-07-27T03:02:07.057 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-07-27T03:02:07.230 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-07-27T03:02:07.231 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-07-27T03:02:07.426 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-07-27T03:02:07.553 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librados2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 8/172 2024-07-27T03:02:07.674 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 8/172 2024-07-27T03:02:07.704 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 9/172 2024-07-27T03:02:07.784 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephfs2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 9/172 2024-07-27T03:02:07.824 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rados-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 10/172 2024-07-27T03:02:07.869 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-07-27T03:02:08.071 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-07-27T03:02:08.124 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-07-27T03:02:08.151 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephsqlite-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 14/172 2024-07-27T03:02:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 14/172 2024-07-27T03:02:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libradosstriper1-2:19.0.0-5358.gaf8fdbf0.el9.x86 15/172 2024-07-27T03:02:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5358.gaf8fdbf0.el9.x86 15/172 2024-07-27T03:02:08.353 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-27T03:02:08.373 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-27T03:02:08.514 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-27T03:02:08.609 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-27T03:02:08.868 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-07-27T03:02:09.027 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-27T03:02:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-27T03:02:09.187 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-27T03:02:09.264 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-27T03:02:09.475 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-27T03:02:09.590 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-07-27T03:02:09.619 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-07-27T03:02:09.653 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/172 2024-07-27T03:02:09.729 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/172 2024-07-27T03:02:09.781 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/172 2024-07-27T03:02:09.821 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-packaging-20.9-5.el9.noarch 26/172 2024-07-27T03:02:09.864 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-07-27T03:02:09.897 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-07-27T03:02:09.924 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-07-27T03:02:09.996 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-07-27T03:02:10.047 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 31/172 2024-07-27T03:02:10.085 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-07-27T03:02:10.129 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-07-27T03:02:10.208 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-07-27T03:02:10.239 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-common-2:19.0.0-5358.gaf8fdbf0.el9. 35/172 2024-07-27T03:02:10.286 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-argparse-2:19.0.0-5358.gaf8fdbf0.el 36/172 2024-07-27T03:02:10.311 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cephfs-2:19.0.0-5358.gaf8fdbf0.el9.x86_6 37/172 2024-07-27T03:02:10.387 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-07-27T03:02:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-07-27T03:02:10.507 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-07-27T03:02:10.652 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-07-27T03:02:10.708 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-07-27T03:02:10.977 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-07-27T03:02:11.043 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-07-27T03:02:11.081 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-07-27T03:02:11.143 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-07-27T03:02:11.177 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-07-27T03:02:11.202 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-07-27T03:02:11.245 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-07-27T03:02:11.274 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-07-27T03:02:11.362 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-07-27T03:02:11.397 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-07-27T03:02:11.434 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-07-27T03:02:11.475 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-07-27T03:02:11.512 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librados-devel-2:19.0.0-5358.gaf8fdbf0.el9.x86_6 48/172 2024-07-27T03:02:11.555 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 49/172 2024-07-27T03:02:11.585 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-07-27T03:02:11.697 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librados2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 8/172 2024-07-27T03:02:11.729 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-07-27T03:02:11.790 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 51/172 2024-07-27T03:02:11.814 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 8/172 2024-07-27T03:02:11.840 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 9/172 2024-07-27T03:02:11.886 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-07-27T03:02:11.917 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 9/172 2024-07-27T03:02:11.944 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rados-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 10/172 2024-07-27T03:02:11.947 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 53/172 2024-07-27T03:02:11.977 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-07-27T03:02:12.022 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-07-27T03:02:12.134 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-07-27T03:02:12.165 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-07-27T03:02:12.218 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-07-27T03:02:12.246 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephsqlite-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 14/172 2024-07-27T03:02:12.318 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/172 2024-07-27T03:02:12.334 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 14/172 2024-07-27T03:02:12.363 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libradosstriper1-2:19.0.0-5358.gaf8fdbf0.el9.x86 15/172 2024-07-27T03:02:12.369 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-07-27T03:02:12.403 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-07-27T03:02:12.409 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5358.gaf8fdbf0.el9.x86 15/172 2024-07-27T03:02:12.431 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-27T03:02:12.447 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-07-27T03:02:12.454 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-27T03:02:12.489 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-07-27T03:02:12.623 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-27T03:02:12.635 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-07-27T03:02:12.681 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-07-27T03:02:12.707 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-27T03:02:12.723 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-07-27T03:02:12.825 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-07-27T03:02:12.864 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-google-auth-1:2.32.0-1.el9.noarch 65/172 2024-07-27T03:02:12.906 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-07-27T03:02:12.921 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-07-27T03:02:12.938 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-07-27T03:02:12.963 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-07-27T03:02:13.046 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-07-27T03:02:13.048 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-27T03:02:13.092 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-27T03:02:13.181 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-27T03:02:13.186 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-07-27T03:02:13.220 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-07-27T03:02:13.248 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-07-27T03:02:13.268 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-27T03:02:13.283 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-07-27T03:02:13.322 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-07-27T03:02:13.361 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-07-27T03:02:13.391 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-07-27T03:02:13.426 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-07-27T03:02:13.459 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-07-27T03:02:13.479 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-27T03:02:13.501 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-07-27T03:02:13.535 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-07-27T03:02:13.590 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-07-27T03:02:13.618 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-07-27T03:02:13.630 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-07-27T03:02:13.644 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-07-27T03:02:13.665 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/172 2024-07-27T03:02:13.666 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-07-27T03:02:13.697 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-07-27T03:02:13.711 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-07-27T03:02:13.729 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/172 2024-07-27T03:02:13.749 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-07-27T03:02:13.774 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/172 2024-07-27T03:02:13.803 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-07-27T03:02:13.804 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-07-27T03:02:13.807 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-packaging-20.9-5.el9.noarch 26/172 2024-07-27T03:02:13.837 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-07-27T03:02:13.851 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-07-27T03:02:13.858 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-27T03:02:13.862 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-07-27T03:02:13.883 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-07-27T03:02:13.906 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-07-27T03:02:13.911 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-07-27T03:02:13.980 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-07-27T03:02:14.034 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 31/172 2024-07-27T03:02:14.065 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-07-27T03:02:14.108 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-07-27T03:02:14.184 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-07-27T03:02:14.218 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-common-2:19.0.0-5358.gaf8fdbf0.el9. 35/172 2024-07-27T03:02:14.255 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-argparse-2:19.0.0-5358.gaf8fdbf0.el 36/172 2024-07-27T03:02:14.282 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cephfs-2:19.0.0-5358.gaf8fdbf0.el9.x86_6 37/172 2024-07-27T03:02:14.319 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-07-27T03:02:14.352 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-07-27T03:02:14.372 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librados2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 8/172 2024-07-27T03:02:14.397 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-07-27T03:02:14.472 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-07-27T03:02:14.498 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 8/172 2024-07-27T03:02:14.522 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 9/172 2024-07-27T03:02:14.602 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-07-27T03:02:14.618 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephfs2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 9/172 2024-07-27T03:02:14.625 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-07-27T03:02:14.643 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rados-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 10/172 2024-07-27T03:02:14.650 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-07-27T03:02:14.676 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-07-27T03:02:14.678 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-07-27T03:02:14.712 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-07-27T03:02:14.815 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-07-27T03:02:14.868 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-07-27T03:02:14.874 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-07-27T03:02:14.913 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-07-27T03:02:14.925 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-07-27T03:02:14.953 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephsqlite-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 14/172 2024-07-27T03:02:15.004 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-07-27T03:02:15.040 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-07-27T03:02:15.059 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 14/172 2024-07-27T03:02:15.088 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libradosstriper1-2:19.0.0-5358.gaf8fdbf0.el9.x86 15/172 2024-07-27T03:02:15.141 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5358.gaf8fdbf0.el9.x86 15/172 2024-07-27T03:02:15.155 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-27T03:02:15.180 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-07-27T03:02:15.181 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-07-27T03:02:15.214 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-07-27T03:02:15.237 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-07-27T03:02:15.254 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-07-27T03:02:15.283 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-07-27T03:02:15.318 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-07-27T03:02:15.319 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-07-27T03:02:15.321 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-27T03:02:15.376 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-07-27T03:02:15.406 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librados-devel-2:19.0.0-5358.gaf8fdbf0.el9.x86_6 48/172 2024-07-27T03:02:15.414 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-07-27T03:02:15.448 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 49/172 2024-07-27T03:02:15.599 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-07-27T03:02:15.600 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-07-27T03:02:15.668 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 51/172 2024-07-27T03:02:15.759 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-07-27T03:02:15.779 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-27T03:02:15.823 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 53/172 2024-07-27T03:02:15.832 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-07-27T03:02:15.896 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-07-27T03:02:15.927 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-27T03:02:15.936 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-07-27T03:02:15.972 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/172 2024-07-27T03:02:16.016 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-27T03:02:16.019 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-07-27T03:02:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-07-27T03:02:16.055 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-07-27T03:02:16.099 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-07-27T03:02:16.140 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-07-27T03:02:16.140 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-07-27T03:02:16.220 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-07-27T03:02:16.275 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-07-27T03:02:16.317 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-07-27T03:02:16.327 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-07-27T03:02:16.362 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-07-27T03:02:16.370 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-07-27T03:02:16.414 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/172 2024-07-27T03:02:16.457 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-07-27T03:02:16.488 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/172 2024-07-27T03:02:16.492 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-google-auth-1:2.32.0-1.el9.noarch 65/172 2024-07-27T03:02:16.549 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-07-27T03:02:16.549 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/172 2024-07-27T03:02:16.574 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-07-27T03:02:16.598 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-packaging-20.9-5.el9.noarch 26/172 2024-07-27T03:02:16.599 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-07-27T03:02:16.642 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/172 2024-07-27T03:02:16.673 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/172 2024-07-27T03:02:16.674 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-07-27T03:02:16.701 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/172 2024-07-27T03:02:16.772 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/172 2024-07-27T03:02:16.809 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-07-27T03:02:16.825 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 31/172 2024-07-27T03:02:16.848 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-07-27T03:02:16.854 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/172 2024-07-27T03:02:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 33/172 2024-07-27T03:02:16.975 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 34/172 2024-07-27T03:02:17.017 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-common-2:19.0.0-5358.gaf8fdbf0.el9. 35/172 2024-07-27T03:02:17.053 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-argparse-2:19.0.0-5358.gaf8fdbf0.el 36/172 2024-07-27T03:02:17.081 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cephfs-2:19.0.0-5358.gaf8fdbf0.el9.x86_6 37/172 2024-07-27T03:02:17.086 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-07-27T03:02:17.120 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-07-27T03:02:17.153 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 38/172 2024-07-27T03:02:17.160 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-07-27T03:02:17.195 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : luarocks-3.9.2-5.el9.noarch 39/172 2024-07-27T03:02:17.199 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-07-27T03:02:17.229 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-07-27T03:02:17.255 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-07-27T03:02:17.271 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 40/172 2024-07-27T03:02:17.288 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-07-27T03:02:17.331 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-07-27T03:02:17.365 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-07-27T03:02:17.407 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 41/172 2024-07-27T03:02:17.455 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-07-27T03:02:17.465 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 42/172 2024-07-27T03:02:17.470 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-07-27T03:02:17.503 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-07-27T03:02:17.534 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-07-27T03:02:17.550 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-07-27T03:02:17.577 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-07-27T03:02:17.611 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-07-27T03:02:17.638 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-07-27T03:02:17.693 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-27T03:02:17.955 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-07-27T03:02:17.990 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librbd1-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 104/172 2024-07-27T03:02:18.007 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rsa-4.9-2.el9.noarch 43/172 2024-07-27T03:02:18.086 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 44/172 2024-07-27T03:02:18.086 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 104/172 2024-07-27T03:02:18.112 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rbd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 105/172 2024-07-27T03:02:18.114 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 45/172 2024-07-27T03:02:18.148 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-07-27T03:02:18.150 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 46/172 2024-07-27T03:02:18.207 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 47/172 2024-07-27T03:02:18.246 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librados-devel-2:19.0.0-5358.gaf8fdbf0.el9.x86_6 48/172 2024-07-27T03:02:18.288 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 49/172 2024-07-27T03:02:18.405 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-07-27T03:02:18.449 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-07-27T03:02:18.452 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 50/172 2024-07-27T03:02:18.482 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-07-27T03:02:18.526 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 51/172 2024-07-27T03:02:18.581 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-07-27T03:02:18.616 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/172 2024-07-27T03:02:18.637 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-07-27T03:02:18.675 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-07-27T03:02:18.680 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 53/172 2024-07-27T03:02:18.682 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-07-27T03:02:18.701 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librgw2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 108/172 2024-07-27T03:02:18.754 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-07-27T03:02:18.769 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-07-27T03:02:18.774 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librgw2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 108/172 2024-07-27T03:02:18.792 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-07-27T03:02:18.803 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rgw-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 109/172 2024-07-27T03:02:18.810 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-07-27T03:02:18.828 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 56/172 2024-07-27T03:02:18.875 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-07-27T03:02:18.911 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-07-27T03:02:18.955 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-07-27T03:02:18.988 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-07-27T03:02:18.996 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-07-27T03:02:19.016 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-07-27T03:02:19.095 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-07-27T03:02:19.135 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-07-27T03:02:19.173 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-07-27T03:02:19.214 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-07-27T03:02:19.322 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-07-27T03:02:19.506 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-google-auth-1:2.32.0-1.el9.noarch 65/172 2024-07-27T03:02:19.546 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-07-27T03:02:19.580 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-07-27T03:02:19.605 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-07-27T03:02:19.691 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-07-27T03:02:19.749 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-07-27T03:02:19.816 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-07-27T03:02:19.854 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-07-27T03:02:19.862 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-07-27T03:02:19.890 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-07-27T03:02:19.924 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-07-27T03:02:19.964 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-07-27T03:02:19.995 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-07-27T03:02:20.025 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-07-27T03:02:20.059 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-07-27T03:02:20.092 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-07-27T03:02:20.134 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-07-27T03:02:20.160 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-07-27T03:02:20.255 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-07-27T03:02:20.298 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-07-27T03:02:20.338 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-07-27T03:02:20.442 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-07-27T03:02:20.490 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-27T03:02:21.066 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-07-27T03:02:21.129 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 110/172 2024-07-27T03:02:21.154 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-07-27T03:02:21.166 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-common-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 110/172 2024-07-27T03:02:21.182 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-07-27T03:02:21.215 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-07-27T03:02:21.237 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-07-27T03:02:21.287 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-07-27T03:02:21.320 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-07-27T03:02:21.422 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-07-27T03:02:21.476 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-07-27T03:02:21.520 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-07-27T03:02:21.541 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-07-27T03:02:21.578 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librbd1-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 104/172 2024-07-27T03:02:21.610 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-07-27T03:02:21.656 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-07-27T03:02:21.674 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 104/172 2024-07-27T03:02:21.700 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rbd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 105/172 2024-07-27T03:02:21.736 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-07-27T03:02:21.767 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 110/172 2024-07-27T03:02:21.806 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-base-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 111/172 2024-07-27T03:02:21.835 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-07-27T03:02:21.862 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-07-27T03:02:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-base-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 111/172 2024-07-27T03:02:21.887 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-27T03:02:21.887 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-27T03:02:21.887 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:02:21.921 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-selinux-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 112/172 2024-07-27T03:02:21.941 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-07-27T03:02:22.267 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-07-27T03:02:22.298 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librgw2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 108/172 2024-07-27T03:02:22.379 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librgw2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 108/172 2024-07-27T03:02:22.408 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rgw-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 109/172 2024-07-27T03:02:22.601 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-07-27T03:02:22.711 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-07-27T03:02:23.979 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-07-27T03:02:24.074 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-07-27T03:02:24.093 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-07-27T03:02:24.118 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-07-27T03:02:24.448 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-07-27T03:02:24.490 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librbd1-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 104/172 2024-07-27T03:02:24.585 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 104/172 2024-07-27T03:02:24.611 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rbd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 105/172 2024-07-27T03:02:24.647 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-07-27T03:02:24.669 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 110/172 2024-07-27T03:02:24.710 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-common-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 110/172 2024-07-27T03:02:25.150 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-07-27T03:02:25.184 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librgw2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 108/172 2024-07-27T03:02:25.256 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librgw2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 108/172 2024-07-27T03:02:25.278 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 110/172 2024-07-27T03:02:25.285 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rgw-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 109/172 2024-07-27T03:02:25.321 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-base-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 111/172 2024-07-27T03:02:25.408 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-base-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 111/172 2024-07-27T03:02:25.409 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-27T03:02:25.409 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-27T03:02:25.409 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-27T03:02:25.439 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-selinux-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 112/172 2024-07-27T03:02:27.524 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 110/172 2024-07-27T03:02:27.566 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-common-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 110/172 2024-07-27T03:02:28.139 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 110/172 2024-07-27T03:02:28.178 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-base-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 111/172 2024-07-27T03:02:28.274 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-base-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 111/172 2024-07-27T03:02:28.274 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-07-27T03:02:28.274 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-07-27T03:02:28.274 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-07-27T03:02:28.304 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-selinux-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 112/172 2024-07-27T03:02:33.849 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 112/172 2024-07-27T03:02:33.850 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /sys 2024-07-27T03:02:33.850 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /proc 2024-07-27T03:02:33.850 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /mnt 2024-07-27T03:02:33.850 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /var/tmp 2024-07-27T03:02:33.850 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /home 2024-07-27T03:02:33.850 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /root 2024-07-27T03:02:33.850 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /tmp 2024-07-27T03:02:33.850 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:02:34.042 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-volume-2:19.0.0-5358.gaf8fdbf0.el9.noarch 113/172 2024-07-27T03:02:34.060 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-volume-2:19.0.0-5358.gaf8fdbf0.el9.noarch 113/172 2024-07-27T03:02:34.060 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:34.060 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-07-27T03:02:34.061 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:02:35.704 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-osd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 114/172 2024-07-27T03:02:35.732 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-osd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 114/172 2024-07-27T03:02:35.733 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:35.733 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-27T03:02:35.733 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-27T03:02:35.733 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-27T03:02:35.733 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:02:35.971 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mds-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 115/172 2024-07-27T03:02:35.996 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mds-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 115/172 2024-07-27T03:02:35.996 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:35.996 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-27T03:02:35.996 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-27T03:02:35.996 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-27T03:02:35.996 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:02:36.480 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mon-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 116/172 2024-07-27T03:02:36.505 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mon-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 116/172 2024-07-27T03:02:36.505 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:36.506 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-27T03:02:36.506 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-27T03:02:36.506 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-27T03:02:36.506 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:02:36.619 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-07-27T03:02:36.677 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-07-27T03:02:36.743 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-07-27T03:02:36.936 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-07-27T03:02:36.937 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 112/172 2024-07-27T03:02:36.937 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /sys 2024-07-27T03:02:36.937 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /proc 2024-07-27T03:02:36.937 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /mnt 2024-07-27T03:02:36.937 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /var/tmp 2024-07-27T03:02:36.938 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /home 2024-07-27T03:02:36.938 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /root 2024-07-27T03:02:36.938 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /tmp 2024-07-27T03:02:36.938 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-27T03:02:36.989 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-07-27T03:02:37.070 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-07-27T03:02:37.102 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-volume-2:19.0.0-5358.gaf8fdbf0.el9.noarch 113/172 2024-07-27T03:02:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-volume-2:19.0.0-5358.gaf8fdbf0.el9.noarch 113/172 2024-07-27T03:02:37.119 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:37.119 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-07-27T03:02:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-27T03:02:37.176 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-07-27T03:02:37.320 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-07-27T03:02:37.364 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-07-27T03:02:37.396 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-07-27T03:02:37.423 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-07-27T03:02:37.829 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noarch 128/172 2024-07-27T03:02:37.872 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noarch 128/172 2024-07-27T03:02:37.904 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5358.gaf8fdbf0.e 129/172 2024-07-27T03:02:37.971 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5358.gaf8fdbf0. 130/172 2024-07-27T03:02:37.994 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noa 131/172 2024-07-27T03:02:38.680 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-osd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 114/172 2024-07-27T03:02:38.705 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-osd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 114/172 2024-07-27T03:02:38.706 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:38.706 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-27T03:02:38.706 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-27T03:02:38.706 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-27T03:02:38.706 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-27T03:02:38.939 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mds-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 115/172 2024-07-27T03:02:38.962 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mds-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 115/172 2024-07-27T03:02:38.962 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:38.962 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-27T03:02:38.962 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-27T03:02:38.962 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-27T03:02:38.962 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-27T03:02:39.423 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mon-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 116/172 2024-07-27T03:02:39.446 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mon-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 116/172 2024-07-27T03:02:39.446 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:39.446 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-27T03:02:39.446 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-27T03:02:39.446 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-27T03:02:39.446 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-27T03:02:39.553 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-07-27T03:02:39.610 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-07-27T03:02:39.665 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-07-27T03:02:39.849 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-07-27T03:02:39.898 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-07-27T03:02:39.933 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 112/172 2024-07-27T03:02:39.933 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /sys 2024-07-27T03:02:39.933 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /proc 2024-07-27T03:02:39.933 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /mnt 2024-07-27T03:02:39.933 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /var/tmp 2024-07-27T03:02:39.934 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /home 2024-07-27T03:02:39.934 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /root 2024-07-27T03:02:39.934 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /tmp 2024-07-27T03:02:39.934 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-07-27T03:02:39.976 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-07-27T03:02:40.072 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-07-27T03:02:40.112 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-volume-2:19.0.0-5358.gaf8fdbf0.el9.noarch 113/172 2024-07-27T03:02:40.128 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-volume-2:19.0.0-5358.gaf8fdbf0.el9.noarch 113/172 2024-07-27T03:02:40.128 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:40.129 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-07-27T03:02:40.129 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-07-27T03:02:40.211 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-07-27T03:02:40.256 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-07-27T03:02:40.288 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-07-27T03:02:40.316 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-07-27T03:02:40.769 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noarch 128/172 2024-07-27T03:02:40.797 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noarch 128/172 2024-07-27T03:02:40.838 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5358.gaf8fdbf0.e 129/172 2024-07-27T03:02:40.910 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5358.gaf8fdbf0. 130/172 2024-07-27T03:02:40.937 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noa 131/172 2024-07-27T03:02:41.339 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noa 131/172 2024-07-27T03:02:41.409 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5358.gaf8fdbf0.el9.n 132/172 2024-07-27T03:02:41.690 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-osd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 114/172 2024-07-27T03:02:41.716 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-osd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 114/172 2024-07-27T03:02:41.716 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:41.716 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-07-27T03:02:41.716 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-27T03:02:41.716 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-07-27T03:02:41.716 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-07-27T03:02:41.949 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mds-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 115/172 2024-07-27T03:02:41.972 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mds-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 115/172 2024-07-27T03:02:41.972 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:41.972 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-07-27T03:02:41.972 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-27T03:02:41.972 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-07-27T03:02:41.972 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-07-27T03:02:42.432 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5358.gaf8fdbf0.el9.n 132/172 2024-07-27T03:02:42.433 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mon-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 116/172 2024-07-27T03:02:42.456 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mon-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 116/172 2024-07-27T03:02:42.456 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:42.456 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-07-27T03:02:42.456 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-27T03:02:42.456 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-07-27T03:02:42.456 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-07-27T03:02:42.459 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5358.gaf8 133/172 2024-07-27T03:02:42.571 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-07-27T03:02:42.579 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5358.gaf8 133/172 2024-07-27T03:02:42.629 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-07-27T03:02:42.684 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-07-27T03:02:42.751 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5358.gaf8fdbf0.el 134/172 2024-07-27T03:02:42.778 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 135/172 2024-07-27T03:02:42.804 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 135/172 2024-07-27T03:02:42.804 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:42.804 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-27T03:02:42.804 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-27T03:02:42.805 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-27T03:02:42.805 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:02:42.854 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-rook-2:19.0.0-5358.gaf8fdbf0.el9.noarch 136/172 2024-07-27T03:02:42.866 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5358.gaf8fdbf0.el9.noarch 136/172 2024-07-27T03:02:42.872 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-07-27T03:02:42.926 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-07-27T03:02:42.969 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 137/172 2024-07-27T03:02:43.004 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-07-27T03:02:43.032 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-fuse-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 138/172 2024-07-27T03:02:43.101 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-07-27T03:02:43.243 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-07-27T03:02:43.284 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-07-27T03:02:43.316 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-07-27T03:02:43.344 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-07-27T03:02:43.789 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noarch 128/172 2024-07-27T03:02:43.818 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noarch 128/172 2024-07-27T03:02:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5358.gaf8fdbf0.e 129/172 2024-07-27T03:02:43.915 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5358.gaf8fdbf0. 130/172 2024-07-27T03:02:43.941 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noa 131/172 2024-07-27T03:02:44.060 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-07-27T03:02:44.070 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noa 131/172 2024-07-27T03:02:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-radosgw-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 140/172 2024-07-27T03:02:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 140/172 2024-07-27T03:02:44.115 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:44.115 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-27T03:02:44.115 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-27T03:02:44.115 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-27T03:02:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:02:44.135 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5358.gaf8fdbf0.el9.n 132/172 2024-07-27T03:02:44.157 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5358.gaf8fd 141/172 2024-07-27T03:02:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5358.gaf8fd 141/172 2024-07-27T03:02:44.181 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:44.181 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-27T03:02:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:02:44.232 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 142/172 2024-07-27T03:02:44.257 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 142/172 2024-07-27T03:02:44.257 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:44.257 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-07-27T03:02:44.257 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-27T03:02:44.257 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-27T03:02:44.258 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:02:44.591 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 143/172 2024-07-27T03:02:44.616 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 143/172 2024-07-27T03:02:44.617 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:44.617 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-27T03:02:44.617 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-27T03:02:44.617 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-27T03:02:44.617 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:02:45.097 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5358.gaf8fdbf0.el9.n 132/172 2024-07-27T03:02:45.127 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5358.gaf8 133/172 2024-07-27T03:02:45.240 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5358.gaf8 133/172 2024-07-27T03:02:45.411 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5358.gaf8fdbf0.el 134/172 2024-07-27T03:02:45.446 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 135/172 2024-07-27T03:02:45.469 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 135/172 2024-07-27T03:02:45.469 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:45.469 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-27T03:02:45.469 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-27T03:02:45.469 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-27T03:02:45.469 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-27T03:02:45.513 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-rook-2:19.0.0-5358.gaf8fdbf0.el9.noarch 136/172 2024-07-27T03:02:45.525 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5358.gaf8fdbf0.el9.noarch 136/172 2024-07-27T03:02:45.624 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 137/172 2024-07-27T03:02:45.678 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-fuse-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 138/172 2024-07-27T03:02:46.640 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-07-27T03:02:46.674 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-radosgw-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 140/172 2024-07-27T03:02:46.696 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 140/172 2024-07-27T03:02:46.697 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:46.697 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-27T03:02:46.697 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-27T03:02:46.697 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-27T03:02:46.697 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-27T03:02:46.741 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5358.gaf8fd 141/172 2024-07-27T03:02:46.763 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5358.gaf8fd 141/172 2024-07-27T03:02:46.764 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:46.764 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-27T03:02:46.764 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-27T03:02:46.820 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 142/172 2024-07-27T03:02:46.839 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 142/172 2024-07-27T03:02:46.839 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:46.840 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-07-27T03:02:46.840 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-27T03:02:46.840 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-27T03:02:46.840 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-27T03:02:47.143 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noa 131/172 2024-07-27T03:02:47.159 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 143/172 2024-07-27T03:02:47.182 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 143/172 2024-07-27T03:02:47.182 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:47.182 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-27T03:02:47.182 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-27T03:02:47.182 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-27T03:02:47.182 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-27T03:02:47.206 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5358.gaf8fdbf0.el9.n 132/172 2024-07-27T03:02:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5358.gaf8fdbf0.el9.n 132/172 2024-07-27T03:02:48.198 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5358.gaf8 133/172 2024-07-27T03:02:48.311 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5358.gaf8 133/172 2024-07-27T03:02:48.482 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5358.gaf8fdbf0.el 134/172 2024-07-27T03:02:48.508 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 135/172 2024-07-27T03:02:48.531 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 135/172 2024-07-27T03:02:48.532 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:48.532 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-07-27T03:02:48.532 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-27T03:02:48.532 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-07-27T03:02:48.532 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-07-27T03:02:48.575 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-rook-2:19.0.0-5358.gaf8fdbf0.el9.noarch 136/172 2024-07-27T03:02:48.586 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5358.gaf8fdbf0.el9.noarch 136/172 2024-07-27T03:02:48.686 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 137/172 2024-07-27T03:02:48.749 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-fuse-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 138/172 2024-07-27T03:02:49.221 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-test-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 144/172 2024-07-27T03:02:49.262 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-fuse-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 145/172 2024-07-27T03:02:49.311 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-nbd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 146/172 2024-07-27T03:02:49.343 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-07-27T03:02:49.370 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-07-27T03:02:49.568 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-shell-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 149/172 2024-07-27T03:02:49.607 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-07-27T03:02:49.641 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs-devel-2:19.0.0-5358.gaf8fdbf0.el9.x86_ 151/172 2024-07-27T03:02:49.680 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-07-27T03:02:49.710 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-07-27T03:02:49.711 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-07-27T03:02:49.745 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-radosgw-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 140/172 2024-07-27T03:02:49.756 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-07-27T03:02:49.768 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 140/172 2024-07-27T03:02:49.768 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:49.768 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-07-27T03:02:49.768 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-27T03:02:49.768 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-07-27T03:02:49.768 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-07-27T03:02:49.812 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5358.gaf8fd 141/172 2024-07-27T03:02:49.834 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5358.gaf8fd 141/172 2024-07-27T03:02:49.835 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:49.835 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-07-27T03:02:49.835 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-07-27T03:02:49.888 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 142/172 2024-07-27T03:02:49.910 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 142/172 2024-07-27T03:02:49.910 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:49.911 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-07-27T03:02:49.911 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-27T03:02:49.911 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-07-27T03:02:49.911 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-07-27T03:02:50.247 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 143/172 2024-07-27T03:02:50.270 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 143/172 2024-07-27T03:02:50.270 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-07-27T03:02:50.270 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-07-27T03:02:50.270 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-27T03:02:50.270 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-07-27T03:02:50.270 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-07-27T03:02:50.392 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-27T03:02:50.437 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-27T03:02:50.459 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-27T03:02:51.381 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephfs-top-2:19.0.0-5358.gaf8fdbf0.el9.noarch 156/172 2024-07-27T03:02:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-07-27T03:02:51.479 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-test-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 144/172 2024-07-27T03:02:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-07-27T03:02:51.520 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-fuse-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 145/172 2024-07-27T03:02:51.569 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-nbd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 146/172 2024-07-27T03:02:51.609 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-07-27T03:02:51.610 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-07-27T03:02:51.644 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-07-27T03:02:51.644 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-07-27T03:02:51.678 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-07-27T03:02:51.702 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-27T03:02:51.729 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-07-27T03:02:51.729 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-27T03:02:51.822 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-27T03:02:51.824 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-shell-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 149/172 2024-07-27T03:02:51.855 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-07-27T03:02:51.855 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-27T03:02:51.870 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-07-27T03:02:51.910 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs-devel-2:19.0.0-5358.gaf8fdbf0.el9.x86_ 151/172 2024-07-27T03:02:51.940 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-27T03:02:51.959 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-07-27T03:02:51.964 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-07-27T03:02:51.970 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-27T03:02:51.970 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-27T03:02:51.989 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-07-27T03:02:52.035 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-07-27T03:02:52.174 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-27T03:02:52.174 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-27T03:02:52.250 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-27T03:02:52.274 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-27T03:02:52.301 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-07-27T03:02:52.380 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-27T03:02:52.631 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-27T03:02:52.667 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-27T03:02:52.687 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-27T03:02:53.633 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-top-2:19.0.0-5358.gaf8fdbf0.el9.noarch 156/172 2024-07-27T03:02:53.764 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-07-27T03:02:53.805 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-07-27T03:02:53.888 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-07-27T03:02:53.918 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-07-27T03:02:53.943 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-07-27T03:02:53.969 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-27T03:02:54.003 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-07-27T03:02:54.004 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-27T03:02:54.079 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-27T03:02:54.113 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-07-27T03:02:54.113 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-27T03:02:54.197 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-27T03:02:54.222 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-07-27T03:02:54.227 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-27T03:02:54.227 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-27T03:02:54.407 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-27T03:02:54.407 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-27T03:02:54.475 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-27T03:02:54.501 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-27T03:02:54.527 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-07-27T03:02:54.593 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-test-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 144/172 2024-07-27T03:02:54.599 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-27T03:02:54.633 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-fuse-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 145/172 2024-07-27T03:02:54.675 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-nbd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 146/172 2024-07-27T03:02:54.707 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-07-27T03:02:54.759 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-07-27T03:02:54.948 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-shell-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 149/172 2024-07-27T03:02:55.005 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-07-27T03:02:55.037 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs-devel-2:19.0.0-5358.gaf8fdbf0.el9.x86_ 151/172 2024-07-27T03:02:55.082 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-07-27T03:02:55.113 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-07-27T03:02:55.151 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-07-27T03:02:55.745 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-27T03:02:55.792 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-27T03:02:55.812 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-07-27T03:02:56.667 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-top-2:19.0.0-5358.gaf8fdbf0.el9.noarch 156/172 2024-07-27T03:02:56.777 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-07-27T03:02:56.818 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-07-27T03:02:56.910 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-07-27T03:02:56.947 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-07-27T03:02:56.981 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-07-27T03:02:57.006 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-27T03:02:57.034 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-07-27T03:02:57.034 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-27T03:02:57.117 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-07-27T03:02:57.142 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-07-27T03:02:57.142 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-27T03:02:57.226 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-07-27T03:02:57.251 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-07-27T03:02:57.256 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-27T03:02:57.257 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-27T03:02:57.437 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-07-27T03:02:57.437 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-27T03:02:57.521 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-07-27T03:02:57.546 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-27T03:02:57.572 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-07-27T03:02:57.636 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-27T03:02:58.576 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-27T03:02:58.576 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 1/172 2024-07-27T03:02:58.576 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-base-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2/172 2024-07-27T03:02:58.577 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-common-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 3/172 2024-07-27T03:02:58.577 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-fuse-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 4/172 2024-07-27T03:02:58.577 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5358.gaf8fd 5/172 2024-07-27T03:02:58.577 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mds-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 6/172 2024-07-27T03:02:58.577 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 7/172 2024-07-27T03:02:58.578 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mon-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 8/172 2024-07-27T03:02:58.578 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-osd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 9/172 2024-07-27T03:02:58.578 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-radosgw-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 10/172 2024-07-27T03:02:58.578 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-selinux-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 11/172 2024-07-27T03:02:58.578 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-test-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 12/172 2024-07-27T03:02:58.578 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 13/172 2024-07-27T03:02:58.579 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-shell-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 14/172 2024-07-27T03:02:58.579 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs-devel-2:19.0.0-5358.gaf8fdbf0.el9.x86_ 15/172 2024-07-27T03:02:58.579 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 16/172 2024-07-27T03:02:58.579 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephsqlite-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 17/172 2024-07-27T03:02:58.579 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados-devel-2:19.0.0-5358.gaf8fdbf0.el9.x86_6 18/172 2024-07-27T03:02:58.579 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libradosstriper1-2:19.0.0-5358.gaf8fdbf0.el9.x86 19/172 2024-07-27T03:02:58.580 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librgw2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 20/172 2024-07-27T03:02:58.580 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5358.gaf8fdbf0.el 21/172 2024-07-27T03:02:58.580 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-common-2:19.0.0-5358.gaf8fdbf0.el9. 22/172 2024-07-27T03:02:58.580 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cephfs-2:19.0.0-5358.gaf8fdbf0.el9.x86_6 23/172 2024-07-27T03:02:58.580 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rados-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 24/172 2024-07-27T03:02:58.581 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rbd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 25/172 2024-07-27T03:02:58.581 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rgw-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 26/172 2024-07-27T03:02:58.581 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-fuse-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 27/172 2024-07-27T03:02:58.581 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 28/172 2024-07-27T03:02:58.581 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-nbd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 29/172 2024-07-27T03:02:58.581 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5358.gaf8fdbf0. 30/172 2024-07-27T03:02:58.582 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noa 31/172 2024-07-27T03:02:58.582 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5358.gaf8fdbf0.el9.n 32/172 2024-07-27T03:02:58.582 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5358.gaf8 33/172 2024-07-27T03:02:58.582 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5358.gaf8fdbf0.el 34/172 2024-07-27T03:02:58.582 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5358.gaf8fdbf0.el9.noarch 35/172 2024-07-27T03:02:58.583 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5358.gaf8fdbf0.e 36/172 2024-07-27T03:02:58.583 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-volume-2:19.0.0-5358.gaf8fdbf0.el9.noarch 37/172 2024-07-27T03:02:58.583 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noarch 38/172 2024-07-27T03:02:58.583 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephfs-top-2:19.0.0-5358.gaf8fdbf0.el9.noarch 39/172 2024-07-27T03:02:58.583 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-07-27T03:02:58.583 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-07-27T03:02:58.584 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-07-27T03:02:58.584 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-07-27T03:02:58.584 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-07-27T03:02:58.584 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-07-27T03:02:58.584 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-07-27T03:02:58.584 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-07-27T03:02:58.585 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-07-27T03:02:58.585 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-07-27T03:02:58.585 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-07-27T03:02:58.585 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-07-27T03:02:58.585 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-07-27T03:02:58.585 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-07-27T03:02:58.586 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-07-27T03:02:58.586 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-07-27T03:02:58.586 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-07-27T03:02:58.586 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-07-27T03:02:58.586 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-07-27T03:02:58.586 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-07-27T03:02:58.587 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-07-27T03:02:58.587 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-07-27T03:02:58.587 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-07-27T03:02:58.587 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-07-27T03:02:58.587 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-07-27T03:02:58.587 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-07-27T03:02:58.588 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-07-27T03:02:58.588 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-07-27T03:02:58.588 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-07-27T03:02:58.588 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-07-27T03:02:58.588 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-07-27T03:02:58.588 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-07-27T03:02:58.589 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-07-27T03:02:58.589 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-07-27T03:02:58.589 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-07-27T03:02:58.589 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-07-27T03:02:58.589 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-07-27T03:02:58.590 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-07-27T03:02:58.590 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-07-27T03:02:58.590 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-07-27T03:02:58.590 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-07-27T03:02:58.590 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-07-27T03:02:58.590 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-07-27T03:02:58.591 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-07-27T03:02:58.591 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-07-27T03:02:58.591 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-27T03:02:58.591 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-07-27T03:02:58.591 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-07-27T03:02:58.591 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-07-27T03:02:58.592 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-07-27T03:02:58.592 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-07-27T03:02:58.592 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-07-27T03:02:58.592 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-07-27T03:02:58.592 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-07-27T03:02:58.592 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-07-27T03:02:58.592 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-07-27T03:02:58.593 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-07-27T03:02:58.593 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-07-27T03:02:58.593 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-07-27T03:02:58.593 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-07-27T03:02:58.594 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-07-27T03:02:58.594 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-07-27T03:02:58.594 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-07-27T03:02:58.594 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-07-27T03:02:58.594 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-07-27T03:02:58.595 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-07-27T03:02:58.595 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-07-27T03:02:58.595 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-07-27T03:02:58.595 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-07-27T03:02:58.596 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-07-27T03:02:58.596 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-07-27T03:02:58.596 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-07-27T03:02:58.596 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-07-27T03:02:58.596 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-07-27T03:02:58.597 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-07-27T03:02:58.597 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-07-27T03:02:58.597 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-07-27T03:02:58.597 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-07-27T03:02:58.597 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-07-27T03:02:58.598 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-07-27T03:02:58.598 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-07-27T03:02:58.598 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-07-27T03:02:58.598 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-google-auth-1:2.32.0-1.el9.noarch 122/172 2024-07-27T03:02:58.599 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-07-27T03:02:58.599 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-07-27T03:02:58.599 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-07-27T03:02:58.599 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-07-27T03:02:58.599 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-07-27T03:02:58.600 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-07-27T03:02:58.600 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-07-27T03:02:58.600 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-07-27T03:02:58.600 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-07-27T03:02:58.600 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-07-27T03:02:58.601 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-07-27T03:02:58.601 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-07-27T03:02:58.601 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-07-27T03:02:58.601 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-07-27T03:02:58.601 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-07-27T03:02:58.602 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-07-27T03:02:58.602 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-07-27T03:02:58.602 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-07-27T03:02:58.602 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-07-27T03:02:58.602 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-07-27T03:02:58.603 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-07-27T03:02:58.603 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-07-27T03:02:58.603 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-07-27T03:02:58.603 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-07-27T03:02:58.603 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-07-27T03:02:58.604 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-07-27T03:02:58.604 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 149/172 2024-07-27T03:02:58.604 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-07-27T03:02:58.604 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 151/172 2024-07-27T03:02:58.604 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-07-27T03:02:58.605 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-07-27T03:02:58.605 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-07-27T03:02:58.605 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-07-27T03:02:58.605 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-07-27T03:02:58.605 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-07-27T03:02:58.606 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-07-27T03:02:58.606 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-07-27T03:02:58.606 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-07-27T03:02:58.606 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-07-27T03:02:58.607 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-27T03:02:58.607 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-07-27T03:02:58.607 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-07-27T03:02:58.607 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-07-27T03:02:58.607 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-07-27T03:02:58.608 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-07-27T03:02:58.608 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-07-27T03:02:58.608 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-07-27T03:02:58.608 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-27T03:02:58.608 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-07-27T03:03:00.792 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-27T03:03:00.792 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 1/172 2024-07-27T03:03:00.792 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-base-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2/172 2024-07-27T03:03:00.792 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-common-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 3/172 2024-07-27T03:03:00.792 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-fuse-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 4/172 2024-07-27T03:03:00.792 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5358.gaf8fd 5/172 2024-07-27T03:03:00.792 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mds-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 6/172 2024-07-27T03:03:00.793 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 7/172 2024-07-27T03:03:00.793 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mon-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 8/172 2024-07-27T03:03:00.793 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-osd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 9/172 2024-07-27T03:03:00.793 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-radosgw-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 10/172 2024-07-27T03:03:00.793 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-selinux-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 11/172 2024-07-27T03:03:00.793 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-test-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 12/172 2024-07-27T03:03:00.793 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 13/172 2024-07-27T03:03:00.793 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-shell-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 14/172 2024-07-27T03:03:00.793 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs-devel-2:19.0.0-5358.gaf8fdbf0.el9.x86_ 15/172 2024-07-27T03:03:00.793 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 16/172 2024-07-27T03:03:00.793 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephsqlite-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 17/172 2024-07-27T03:03:00.793 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados-devel-2:19.0.0-5358.gaf8fdbf0.el9.x86_6 18/172 2024-07-27T03:03:00.793 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libradosstriper1-2:19.0.0-5358.gaf8fdbf0.el9.x86 19/172 2024-07-27T03:03:00.793 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librgw2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 20/172 2024-07-27T03:03:00.794 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5358.gaf8fdbf0.el 21/172 2024-07-27T03:03:00.794 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-common-2:19.0.0-5358.gaf8fdbf0.el9. 22/172 2024-07-27T03:03:00.794 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cephfs-2:19.0.0-5358.gaf8fdbf0.el9.x86_6 23/172 2024-07-27T03:03:00.794 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rados-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 24/172 2024-07-27T03:03:00.794 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rbd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 25/172 2024-07-27T03:03:00.794 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rgw-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 26/172 2024-07-27T03:03:00.794 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-fuse-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 27/172 2024-07-27T03:03:00.794 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 28/172 2024-07-27T03:03:00.794 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-nbd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 29/172 2024-07-27T03:03:00.809 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5358.gaf8fdbf0. 30/172 2024-07-27T03:03:00.809 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noa 31/172 2024-07-27T03:03:00.810 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5358.gaf8fdbf0.el9.n 32/172 2024-07-27T03:03:00.810 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5358.gaf8 33/172 2024-07-27T03:03:00.810 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5358.gaf8fdbf0.el 34/172 2024-07-27T03:03:00.810 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5358.gaf8fdbf0.el9.noarch 35/172 2024-07-27T03:03:00.810 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5358.gaf8fdbf0.e 36/172 2024-07-27T03:03:00.810 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-volume-2:19.0.0-5358.gaf8fdbf0.el9.noarch 37/172 2024-07-27T03:03:00.810 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noarch 38/172 2024-07-27T03:03:00.810 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-top-2:19.0.0-5358.gaf8fdbf0.el9.noarch 39/172 2024-07-27T03:03:00.810 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-07-27T03:03:00.810 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-07-27T03:03:00.810 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-07-27T03:03:00.811 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-07-27T03:03:00.811 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-07-27T03:03:00.811 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-07-27T03:03:00.811 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-07-27T03:03:00.811 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-07-27T03:03:00.812 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-07-27T03:03:00.812 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-07-27T03:03:00.812 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-07-27T03:03:00.812 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-07-27T03:03:00.812 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-07-27T03:03:00.813 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-07-27T03:03:00.813 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-07-27T03:03:00.813 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-07-27T03:03:00.813 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-07-27T03:03:00.813 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-07-27T03:03:00.813 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-07-27T03:03:00.814 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-07-27T03:03:00.814 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-07-27T03:03:00.814 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-07-27T03:03:00.814 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-07-27T03:03:00.814 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-07-27T03:03:00.815 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-07-27T03:03:00.815 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-07-27T03:03:00.815 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-07-27T03:03:00.815 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-07-27T03:03:00.815 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-07-27T03:03:00.815 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-07-27T03:03:00.816 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-07-27T03:03:00.816 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-07-27T03:03:00.816 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-07-27T03:03:00.816 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-07-27T03:03:00.816 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-07-27T03:03:00.816 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-07-27T03:03:00.816 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-07-27T03:03:00.816 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-07-27T03:03:00.816 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-07-27T03:03:00.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-07-27T03:03:00.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-07-27T03:03:00.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-07-27T03:03:00.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-07-27T03:03:00.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-07-27T03:03:00.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-07-27T03:03:00.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-27T03:03:00.817 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-07-27T03:03:00.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-07-27T03:03:00.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-07-27T03:03:00.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-07-27T03:03:00.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-07-27T03:03:00.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-07-27T03:03:00.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-07-27T03:03:00.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-07-27T03:03:00.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-07-27T03:03:00.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-07-27T03:03:00.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-07-27T03:03:00.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-07-27T03:03:00.818 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-07-27T03:03:00.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-07-27T03:03:00.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-07-27T03:03:00.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-07-27T03:03:00.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-07-27T03:03:00.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-07-27T03:03:00.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-07-27T03:03:00.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-07-27T03:03:00.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-07-27T03:03:00.819 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-07-27T03:03:00.820 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-07-27T03:03:00.820 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-07-27T03:03:00.820 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-07-27T03:03:00.820 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-07-27T03:03:00.820 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-07-27T03:03:00.821 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-07-27T03:03:00.821 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-07-27T03:03:00.821 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-07-27T03:03:00.821 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-07-27T03:03:00.821 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-07-27T03:03:00.821 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-07-27T03:03:00.822 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-07-27T03:03:00.822 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-07-27T03:03:00.822 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-07-27T03:03:00.822 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-google-auth-1:2.32.0-1.el9.noarch 122/172 2024-07-27T03:03:00.822 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-07-27T03:03:00.823 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-07-27T03:03:00.823 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-07-27T03:03:00.823 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-07-27T03:03:00.823 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-07-27T03:03:00.823 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-07-27T03:03:00.823 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-07-27T03:03:00.824 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-07-27T03:03:00.824 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-07-27T03:03:00.824 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-07-27T03:03:00.824 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-07-27T03:03:00.824 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-07-27T03:03:00.825 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-07-27T03:03:00.825 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-07-27T03:03:00.825 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-07-27T03:03:00.825 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-07-27T03:03:00.825 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-07-27T03:03:00.826 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-07-27T03:03:00.826 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-07-27T03:03:00.826 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-07-27T03:03:00.826 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-07-27T03:03:00.826 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-07-27T03:03:00.827 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-07-27T03:03:00.827 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-07-27T03:03:00.827 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-07-27T03:03:00.827 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-07-27T03:03:00.827 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 149/172 2024-07-27T03:03:00.827 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-07-27T03:03:00.828 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 151/172 2024-07-27T03:03:00.828 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-07-27T03:03:00.828 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-07-27T03:03:00.828 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-07-27T03:03:00.828 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-07-27T03:03:00.829 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-07-27T03:03:00.829 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-07-27T03:03:00.829 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-07-27T03:03:00.829 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-07-27T03:03:00.829 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-07-27T03:03:00.829 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-07-27T03:03:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-27T03:03:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-07-27T03:03:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-07-27T03:03:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-07-27T03:03:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-07-27T03:03:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-07-27T03:03:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-07-27T03:03:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-07-27T03:03:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-27T03:03:00.830 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-07-27T03:03:01.279 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-27T03:03:01.280 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:03:01.280 INFO:teuthology.orchestra.run.smithi067.stdout:Upgraded: 2024-07-27T03:03:01.280 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-07-27T03:03:01.280 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-07-27T03:03:01.280 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-07-27T03:03:01.280 INFO:teuthology.orchestra.run.smithi067.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-07-27T03:03:01.280 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-07-27T03:03:01.280 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-07-27T03:03:01.280 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-07-27T03:03:01.280 INFO:teuthology.orchestra.run.smithi067.stdout: librados2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.280 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.280 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-07-27T03:03:01.281 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-07-27T03:03:01.281 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-07-27T03:03:01.281 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-07-27T03:03:01.281 INFO:teuthology.orchestra.run.smithi067.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-07-27T03:03:01.281 INFO:teuthology.orchestra.run.smithi067.stdout: automake-1.16.2-8.el9.noarch 2024-07-27T03:03:01.281 INFO:teuthology.orchestra.run.smithi067.stdout: bison-3.7.4-5.el9.x86_64 2024-07-27T03:03:01.281 INFO:teuthology.orchestra.run.smithi067.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-07-27T03:03:01.281 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.281 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.281 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.281 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.281 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:01.281 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.282 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.282 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.282 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:01.282 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:01.282 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:01.282 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:01.282 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:01.282 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.282 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.282 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:01.283 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.283 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.283 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.283 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:01.283 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:01.283 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.283 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-shell-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.283 INFO:teuthology.orchestra.run.smithi067.stdout: cephfs-top-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:01.283 INFO:teuthology.orchestra.run.smithi067.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-07-27T03:03:01.283 INFO:teuthology.orchestra.run.smithi067.stdout: flex-2.6.4-9.el9.x86_64 2024-07-27T03:03:01.283 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-27T03:03:01.283 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-27T03:03:01.283 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-27T03:03:01.283 INFO:teuthology.orchestra.run.smithi067.stdout: fuse-2.9.9-16.el9.x86_64 2024-07-27T03:03:01.284 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-27T03:03:01.284 INFO:teuthology.orchestra.run.smithi067.stdout: grpc-data-1.46.7-10.el9.noarch 2024-07-27T03:03:01.284 INFO:teuthology.orchestra.run.smithi067.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-07-27T03:03:01.284 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-9.0.0-11.el9.x86_64 2024-07-27T03:03:01.284 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-07-27T03:03:01.284 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.284 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.284 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.284 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-27T03:03:01.284 INFO:teuthology.orchestra.run.smithi067.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-27T03:03:01.284 INFO:teuthology.orchestra.run.smithi067.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-27T03:03:01.284 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-27T03:03:01.284 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-27T03:03:01.284 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.285 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.285 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-27T03:03:01.285 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.285 INFO:teuthology.orchestra.run.smithi067.stdout: libtool-2.4.6-46.el9.x86_64 2024-07-27T03:03:01.285 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-27T03:03:01.285 INFO:teuthology.orchestra.run.smithi067.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-07-27T03:03:01.285 INFO:teuthology.orchestra.run.smithi067.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-07-27T03:03:01.285 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-27T03:03:01.285 INFO:teuthology.orchestra.run.smithi067.stdout: lua-5.4.4-4.el9.x86_64 2024-07-27T03:03:01.285 INFO:teuthology.orchestra.run.smithi067.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-07-27T03:03:01.285 INFO:teuthology.orchestra.run.smithi067.stdout: luarocks-3.9.2-5.el9.noarch 2024-07-27T03:03:01.285 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-27T03:03:01.285 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-27T03:03:01.285 INFO:teuthology.orchestra.run.smithi067.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-07-27T03:03:01.285 INFO:teuthology.orchestra.run.smithi067.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-07-27T03:03:01.286 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-13.14-1.el9.x86_64 2024-07-27T03:03:01.286 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-07-27T03:03:01.286 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-07-27T03:03:01.286 INFO:teuthology.orchestra.run.smithi067.stdout: postgresql-server-13.14-1.el9.x86_64 2024-07-27T03:03:01.286 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-3.14.0-13.el9.x86_64 2024-07-27T03:03:01.286 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-07-27T03:03:01.286 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-27T03:03:01.286 INFO:teuthology.orchestra.run.smithi067.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-07-27T03:03:01.286 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-27T03:03:01.286 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-27T03:03:01.286 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-27T03:03:01.286 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.287 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.287 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.287 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-07-27T03:03:01.287 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-27T03:03:01.287 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-27T03:03:01.287 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-27T03:03:01.287 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-07-27T03:03:01.287 INFO:teuthology.orchestra.run.smithi067.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-07-27T03:03:01.287 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-27T03:03:01.287 INFO:teuthology.orchestra.run.smithi067.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-27T03:03:01.287 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth-1:2.32.0-1.el9.noarch 2024-07-27T03:03:01.287 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-07-27T03:03:01.287 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-07-27T03:03:01.288 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-27T03:03:01.288 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-27T03:03:01.288 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-27T03:03:01.288 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-27T03:03:01.288 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-27T03:03:01.288 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-27T03:03:01.288 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-27T03:03:01.288 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-27T03:03:01.288 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-27T03:03:01.288 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-27T03:03:01.288 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-27T03:03:01.288 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-27T03:03:01.289 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-27T03:03:01.289 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-27T03:03:01.289 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-27T03:03:01.289 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-27T03:03:01.289 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-27T03:03:01.289 INFO:teuthology.orchestra.run.smithi067.stdout: python3-packaging-20.9-5.el9.noarch 2024-07-27T03:03:01.289 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-27T03:03:01.289 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ply-3.11-14.el9.noarch 2024-07-27T03:03:01.289 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-27T03:03:01.289 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-27T03:03:01.289 INFO:teuthology.orchestra.run.smithi067.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-07-27T03:03:01.289 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-27T03:03:01.289 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-27T03:03:01.290 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-27T03:03:01.290 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-27T03:03:01.290 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-07-27T03:03:01.290 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-27T03:03:01.290 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.290 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.290 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-27T03:03:01.290 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-27T03:03:01.290 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-27T03:03:01.290 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.290 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-27T03:03:01.290 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-27T03:03:01.291 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-27T03:03:01.291 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-27T03:03:01.291 INFO:teuthology.orchestra.run.smithi067.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-27T03:03:01.291 INFO:teuthology.orchestra.run.smithi067.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-07-27T03:03:01.291 INFO:teuthology.orchestra.run.smithi067.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-27T03:03:01.291 INFO:teuthology.orchestra.run.smithi067.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-07-27T03:03:01.291 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-27T03:03:01.291 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-27T03:03:01.291 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-27T03:03:01.291 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-27T03:03:01.291 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-24.02.0-1.el9.x86_64 2024-07-27T03:03:01.291 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-07-27T03:03:01.291 INFO:teuthology.orchestra.run.smithi067.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-07-27T03:03:01.292 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.292 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.292 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:01.292 INFO:teuthology.orchestra.run.smithi067.stdout: re2-1:20211101-3.el9.x86_64 2024-07-27T03:03:01.292 INFO:teuthology.orchestra.run.smithi067.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-27T03:03:01.292 INFO:teuthology.orchestra.run.smithi067.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-27T03:03:01.292 INFO:teuthology.orchestra.run.smithi067.stdout: uuid-1.6.2-55.el9.x86_64 2024-07-27T03:03:01.292 INFO:teuthology.orchestra.run.smithi067.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-07-27T03:03:01.292 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-27T03:03:01.292 INFO:teuthology.orchestra.run.smithi067.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-07-27T03:03:01.293 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:03:01.293 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-07-27T03:03:01.711 DEBUG:teuthology.parallel:result is None 2024-07-27T03:03:03.692 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-27T03:03:03.692 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-27T03:03:03.692 INFO:teuthology.orchestra.run.smithi175.stdout:Upgraded: 2024-07-27T03:03:03.692 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-07-27T03:03:03.692 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-07-27T03:03:03.692 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-07-27T03:03:03.693 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-07-27T03:03:03.693 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-07-27T03:03:03.693 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-07-27T03:03:03.693 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-07-27T03:03:03.693 INFO:teuthology.orchestra.run.smithi175.stdout: librados2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.693 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.694 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-07-27T03:03:03.694 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-07-27T03:03:03.694 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-07-27T03:03:03.694 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-07-27T03:03:03.694 INFO:teuthology.orchestra.run.smithi175.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-07-27T03:03:03.695 INFO:teuthology.orchestra.run.smithi175.stdout: automake-1.16.2-8.el9.noarch 2024-07-27T03:03:03.695 INFO:teuthology.orchestra.run.smithi175.stdout: bison-3.7.4-5.el9.x86_64 2024-07-27T03:03:03.695 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-07-27T03:03:03.695 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.695 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.695 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.696 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.696 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:03.696 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.696 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.696 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.696 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:03.696 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:03.696 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:03.696 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:03.697 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:03.697 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.697 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.697 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:03.697 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.697 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.697 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.697 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:03.697 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:03.697 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.697 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-shell-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.697 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:03.698 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-07-27T03:03:03.698 INFO:teuthology.orchestra.run.smithi175.stdout: flex-2.6.4-9.el9.x86_64 2024-07-27T03:03:03.698 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-27T03:03:03.698 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-27T03:03:03.698 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-27T03:03:03.698 INFO:teuthology.orchestra.run.smithi175.stdout: fuse-2.9.9-16.el9.x86_64 2024-07-27T03:03:03.699 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-27T03:03:03.699 INFO:teuthology.orchestra.run.smithi175.stdout: grpc-data-1.46.7-10.el9.noarch 2024-07-27T03:03:03.699 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-07-27T03:03:03.699 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-9.0.0-11.el9.x86_64 2024-07-27T03:03:03.699 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-07-27T03:03:03.699 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.699 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.699 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.699 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-27T03:03:03.700 INFO:teuthology.orchestra.run.smithi175.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-27T03:03:03.700 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-27T03:03:03.700 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-27T03:03:03.700 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-27T03:03:03.700 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.700 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.700 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-27T03:03:03.700 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.700 INFO:teuthology.orchestra.run.smithi175.stdout: libtool-2.4.6-46.el9.x86_64 2024-07-27T03:03:03.701 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-27T03:03:03.701 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-07-27T03:03:03.701 INFO:teuthology.orchestra.run.smithi175.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-07-27T03:03:03.701 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-27T03:03:03.701 INFO:teuthology.orchestra.run.smithi175.stdout: lua-5.4.4-4.el9.x86_64 2024-07-27T03:03:03.701 INFO:teuthology.orchestra.run.smithi175.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-07-27T03:03:03.701 INFO:teuthology.orchestra.run.smithi175.stdout: luarocks-3.9.2-5.el9.noarch 2024-07-27T03:03:03.701 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-27T03:03:03.701 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-27T03:03:03.701 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-07-27T03:03:03.701 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-07-27T03:03:03.701 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-13.14-1.el9.x86_64 2024-07-27T03:03:03.701 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-07-27T03:03:03.701 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-07-27T03:03:03.702 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server-13.14-1.el9.x86_64 2024-07-27T03:03:03.702 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-3.14.0-13.el9.x86_64 2024-07-27T03:03:03.702 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-07-27T03:03:03.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-27T03:03:03.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-07-27T03:03:03.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-27T03:03:03.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-27T03:03:03.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-27T03:03:03.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.702 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-07-27T03:03:03.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-27T03:03:03.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-27T03:03:03.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-27T03:03:03.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-07-27T03:03:03.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-07-27T03:03:03.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-27T03:03:03.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-27T03:03:03.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth-1:2.32.0-1.el9.noarch 2024-07-27T03:03:03.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-07-27T03:03:03.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-07-27T03:03:03.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-27T03:03:03.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-27T03:03:03.703 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-27T03:03:03.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-27T03:03:03.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-27T03:03:03.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-27T03:03:03.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-27T03:03:03.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-27T03:03:03.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-27T03:03:03.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-27T03:03:03.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-27T03:03:03.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-27T03:03:03.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-27T03:03:03.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-27T03:03:03.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-27T03:03:03.704 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-27T03:03:03.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-27T03:03:03.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging-20.9-5.el9.noarch 2024-07-27T03:03:03.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-27T03:03:03.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply-3.11-14.el9.noarch 2024-07-27T03:03:03.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-27T03:03:03.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-27T03:03:03.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-07-27T03:03:03.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-27T03:03:03.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-27T03:03:03.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-27T03:03:03.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-27T03:03:03.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-07-27T03:03:03.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-27T03:03:03.705 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.706 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.706 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-27T03:03:03.706 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-27T03:03:03.706 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-27T03:03:03.706 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.706 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-27T03:03:03.706 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-27T03:03:03.706 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-27T03:03:03.706 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-27T03:03:03.707 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-27T03:03:03.707 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-07-27T03:03:03.707 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-27T03:03:03.707 INFO:teuthology.orchestra.run.smithi175.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-07-27T03:03:03.707 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-27T03:03:03.708 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-27T03:03:03.708 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-27T03:03:03.708 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-27T03:03:03.708 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-24.02.0-1.el9.x86_64 2024-07-27T03:03:03.708 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-07-27T03:03:03.709 INFO:teuthology.orchestra.run.smithi175.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-07-27T03:03:03.709 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.709 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.709 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:03.709 INFO:teuthology.orchestra.run.smithi175.stdout: re2-1:20211101-3.el9.x86_64 2024-07-27T03:03:03.709 INFO:teuthology.orchestra.run.smithi175.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-27T03:03:03.710 INFO:teuthology.orchestra.run.smithi175.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-27T03:03:03.710 INFO:teuthology.orchestra.run.smithi175.stdout: uuid-1.6.2-55.el9.x86_64 2024-07-27T03:03:03.710 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-07-27T03:03:03.710 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-27T03:03:03.711 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-07-27T03:03:03.711 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-27T03:03:03.711 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-07-27T03:03:03.983 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-27T03:03:03.983 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 1/172 2024-07-27T03:03:03.983 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-base-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2/172 2024-07-27T03:03:03.983 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-common-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 3/172 2024-07-27T03:03:03.983 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-fuse-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 4/172 2024-07-27T03:03:03.983 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5358.gaf8fd 5/172 2024-07-27T03:03:03.983 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mds-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 6/172 2024-07-27T03:03:03.984 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 7/172 2024-07-27T03:03:03.984 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mon-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 8/172 2024-07-27T03:03:03.984 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-osd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 9/172 2024-07-27T03:03:03.984 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-radosgw-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 10/172 2024-07-27T03:03:03.984 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-selinux-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 11/172 2024-07-27T03:03:03.984 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-test-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 12/172 2024-07-27T03:03:03.984 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 13/172 2024-07-27T03:03:03.984 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-shell-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 14/172 2024-07-27T03:03:03.984 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs-devel-2:19.0.0-5358.gaf8fdbf0.el9.x86_ 15/172 2024-07-27T03:03:03.984 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 16/172 2024-07-27T03:03:03.984 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephsqlite-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 17/172 2024-07-27T03:03:03.984 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados-devel-2:19.0.0-5358.gaf8fdbf0.el9.x86_6 18/172 2024-07-27T03:03:03.984 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libradosstriper1-2:19.0.0-5358.gaf8fdbf0.el9.x86 19/172 2024-07-27T03:03:03.985 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librgw2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 20/172 2024-07-27T03:03:03.985 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5358.gaf8fdbf0.el 21/172 2024-07-27T03:03:03.985 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-common-2:19.0.0-5358.gaf8fdbf0.el9. 22/172 2024-07-27T03:03:03.985 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cephfs-2:19.0.0-5358.gaf8fdbf0.el9.x86_6 23/172 2024-07-27T03:03:03.985 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rados-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 24/172 2024-07-27T03:03:03.985 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rbd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 25/172 2024-07-27T03:03:03.985 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rgw-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 26/172 2024-07-27T03:03:03.985 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-fuse-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 27/172 2024-07-27T03:03:03.985 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 28/172 2024-07-27T03:03:03.985 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-nbd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 29/172 2024-07-27T03:03:03.985 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5358.gaf8fdbf0. 30/172 2024-07-27T03:03:03.985 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noa 31/172 2024-07-27T03:03:03.985 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5358.gaf8fdbf0.el9.n 32/172 2024-07-27T03:03:03.986 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5358.gaf8 33/172 2024-07-27T03:03:03.986 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5358.gaf8fdbf0.el 34/172 2024-07-27T03:03:03.986 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5358.gaf8fdbf0.el9.noarch 35/172 2024-07-27T03:03:03.986 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5358.gaf8fdbf0.e 36/172 2024-07-27T03:03:03.986 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-volume-2:19.0.0-5358.gaf8fdbf0.el9.noarch 37/172 2024-07-27T03:03:03.986 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noarch 38/172 2024-07-27T03:03:03.986 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-top-2:19.0.0-5358.gaf8fdbf0.el9.noarch 39/172 2024-07-27T03:03:03.986 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-07-27T03:03:03.986 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-07-27T03:03:03.986 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-07-27T03:03:03.987 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-07-27T03:03:03.987 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-07-27T03:03:03.987 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-07-27T03:03:03.987 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-07-27T03:03:03.987 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-07-27T03:03:03.988 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-07-27T03:03:03.988 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-07-27T03:03:03.988 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-07-27T03:03:03.988 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-07-27T03:03:03.988 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-07-27T03:03:03.988 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-07-27T03:03:03.988 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-07-27T03:03:03.988 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-07-27T03:03:03.988 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-07-27T03:03:03.988 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-07-27T03:03:03.988 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-07-27T03:03:03.988 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-07-27T03:03:03.988 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-07-27T03:03:03.989 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-07-27T03:03:03.989 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-07-27T03:03:03.989 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-07-27T03:03:03.989 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-07-27T03:03:03.989 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-07-27T03:03:03.989 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-07-27T03:03:03.989 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-07-27T03:03:03.989 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-07-27T03:03:03.989 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-07-27T03:03:03.989 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-07-27T03:03:03.989 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-07-27T03:03:03.989 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-07-27T03:03:03.989 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-07-27T03:03:03.989 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-07-27T03:03:03.990 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-07-27T03:03:03.990 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-07-27T03:03:03.990 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-07-27T03:03:03.990 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-07-27T03:03:03.990 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-07-27T03:03:03.990 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-07-27T03:03:03.990 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-07-27T03:03:03.990 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-07-27T03:03:03.990 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-07-27T03:03:03.990 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-07-27T03:03:03.990 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-07-27T03:03:03.990 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-07-27T03:03:03.990 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-07-27T03:03:03.990 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-07-27T03:03:03.991 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-07-27T03:03:03.991 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-07-27T03:03:03.991 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-07-27T03:03:03.991 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-07-27T03:03:03.991 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-07-27T03:03:03.991 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-07-27T03:03:03.991 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-07-27T03:03:03.991 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-07-27T03:03:03.991 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-07-27T03:03:03.991 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-07-27T03:03:03.991 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-07-27T03:03:03.991 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-07-27T03:03:03.991 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-07-27T03:03:03.991 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-07-27T03:03:03.992 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-07-27T03:03:03.992 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-07-27T03:03:03.992 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-07-27T03:03:03.992 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-07-27T03:03:03.992 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-07-27T03:03:03.992 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-07-27T03:03:03.992 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-07-27T03:03:03.992 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-07-27T03:03:03.992 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-07-27T03:03:03.992 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-07-27T03:03:03.994 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-07-27T03:03:03.994 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-07-27T03:03:03.994 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-07-27T03:03:03.994 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-07-27T03:03:03.994 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-07-27T03:03:03.994 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-07-27T03:03:03.994 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-07-27T03:03:03.994 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-07-27T03:03:03.994 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-07-27T03:03:03.994 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-google-auth-1:2.32.0-1.el9.noarch 122/172 2024-07-27T03:03:03.994 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-07-27T03:03:03.994 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-07-27T03:03:03.994 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-07-27T03:03:03.995 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-07-27T03:03:03.995 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-07-27T03:03:03.995 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-07-27T03:03:03.995 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-07-27T03:03:03.995 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-07-27T03:03:03.995 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-07-27T03:03:03.995 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-07-27T03:03:03.995 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-07-27T03:03:03.995 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-07-27T03:03:03.995 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-07-27T03:03:03.995 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-07-27T03:03:03.995 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-07-27T03:03:03.995 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-07-27T03:03:03.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-07-27T03:03:03.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-07-27T03:03:03.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-07-27T03:03:03.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/172 2024-07-27T03:03:03.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-07-27T03:03:03.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-07-27T03:03:03.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-07-27T03:03:03.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-07-27T03:03:03.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-07-27T03:03:03.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-07-27T03:03:03.996 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 149/172 2024-07-27T03:03:03.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-07-27T03:03:03.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 151/172 2024-07-27T03:03:03.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-07-27T03:03:03.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-07-27T03:03:03.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-07-27T03:03:03.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-07-27T03:03:03.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-07-27T03:03:03.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-07-27T03:03:03.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-07-27T03:03:03.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-07-27T03:03:03.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-07-27T03:03:03.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-07-27T03:03:03.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-07-27T03:03:03.997 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-07-27T03:03:03.998 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-07-27T03:03:03.998 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-07-27T03:03:03.998 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-07-27T03:03:03.998 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-07-27T03:03:03.998 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-07-27T03:03:03.998 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-07-27T03:03:03.998 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-07-27T03:03:03.998 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-07-27T03:03:04.106 DEBUG:teuthology.parallel:result is None 2024-07-27T03:03:06.652 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-07-27T03:03:06.652 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-07-27T03:03:06.652 INFO:teuthology.orchestra.run.smithi149.stdout:Upgraded: 2024-07-27T03:03:06.652 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-07-27T03:03:06.652 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-07-27T03:03:06.652 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-07-27T03:03:06.652 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-07-27T03:03:06.652 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-07-27T03:03:06.652 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-07-27T03:03:06.652 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-07-27T03:03:06.652 INFO:teuthology.orchestra.run.smithi149.stdout: librados2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.653 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.653 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-07-27T03:03:06.653 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-07-27T03:03:06.653 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-07-27T03:03:06.653 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-07-27T03:03:06.653 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-07-27T03:03:06.653 INFO:teuthology.orchestra.run.smithi149.stdout: automake-1.16.2-8.el9.noarch 2024-07-27T03:03:06.653 INFO:teuthology.orchestra.run.smithi149.stdout: bison-3.7.4-5.el9.x86_64 2024-07-27T03:03:06.653 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-07-27T03:03:06.653 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.653 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.653 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.653 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.653 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:06.654 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.654 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.654 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.654 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:06.654 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:06.654 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:06.654 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:06.654 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:06.654 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.654 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.654 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:06.654 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.655 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.655 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.655 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:06.655 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:06.655 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.655 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-shell-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.655 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top-2:19.0.0-5358.gaf8fdbf0.el9.noarch 2024-07-27T03:03:06.655 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-07-27T03:03:06.655 INFO:teuthology.orchestra.run.smithi149.stdout: flex-2.6.4-9.el9.x86_64 2024-07-27T03:03:06.655 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-07-27T03:03:06.656 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-07-27T03:03:06.656 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-07-27T03:03:06.656 INFO:teuthology.orchestra.run.smithi149.stdout: fuse-2.9.9-16.el9.x86_64 2024-07-27T03:03:06.656 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-07-27T03:03:06.656 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data-1.46.7-10.el9.noarch 2024-07-27T03:03:06.656 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-07-27T03:03:06.656 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-9.0.0-11.el9.x86_64 2024-07-27T03:03:06.656 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-07-27T03:03:06.656 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.656 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.656 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.656 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-07-27T03:03:06.656 INFO:teuthology.orchestra.run.smithi149.stdout: liboath-2.6.7-2.el9.x86_64 2024-07-27T03:03:06.657 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-07-27T03:03:06.657 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-07-27T03:03:06.657 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-07-27T03:03:06.657 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.657 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.657 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-07-27T03:03:06.657 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.657 INFO:teuthology.orchestra.run.smithi149.stdout: libtool-2.4.6-46.el9.x86_64 2024-07-27T03:03:06.657 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind-1.6.2-1.el9.x86_64 2024-07-27T03:03:06.657 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-07-27T03:03:06.657 INFO:teuthology.orchestra.run.smithi149.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-07-27T03:03:06.657 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-07-27T03:03:06.657 INFO:teuthology.orchestra.run.smithi149.stdout: lua-5.4.4-4.el9.x86_64 2024-07-27T03:03:06.658 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-07-27T03:03:06.658 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks-3.9.2-5.el9.noarch 2024-07-27T03:03:06.658 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-0.3.21-2.el9.x86_64 2024-07-27T03:03:06.658 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-07-27T03:03:06.658 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-07-27T03:03:06.658 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-07-27T03:03:06.658 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-13.14-1.el9.x86_64 2024-07-27T03:03:06.658 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-07-27T03:03:06.658 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-07-27T03:03:06.658 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server-13.14-1.el9.x86_64 2024-07-27T03:03:06.659 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-3.14.0-13.el9.x86_64 2024-07-27T03:03:06.659 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-07-27T03:03:06.659 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-07-27T03:03:06.659 INFO:teuthology.orchestra.run.smithi149.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-07-27T03:03:06.659 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel-2.9.1-2.el9.noarch 2024-07-27T03:03:06.659 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-07-27T03:03:06.659 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-07-27T03:03:06.659 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.659 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.659 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.659 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-07-27T03:03:06.659 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-07-27T03:03:06.660 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-07-27T03:03:06.660 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-07-27T03:03:06.660 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-07-27T03:03:06.660 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-07-27T03:03:06.660 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-07-27T03:03:06.660 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-07-27T03:03:06.660 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth-1:2.32.0-1.el9.noarch 2024-07-27T03:03:06.660 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-07-27T03:03:06.660 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-07-27T03:03:06.660 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-07-27T03:03:06.661 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-07-27T03:03:06.661 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-07-27T03:03:06.661 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-07-27T03:03:06.661 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-07-27T03:03:06.661 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-07-27T03:03:06.661 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-07-27T03:03:06.661 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-07-27T03:03:06.661 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-07-27T03:03:06.661 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-07-27T03:03:06.661 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako-1.1.4-6.el9.noarch 2024-07-27T03:03:06.661 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-07-27T03:03:06.661 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-07-27T03:03:06.661 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-07-27T03:03:06.662 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-07-27T03:03:06.662 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-07-27T03:03:06.662 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-07-27T03:03:06.662 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging-20.9-5.el9.noarch 2024-07-27T03:03:06.662 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-07-27T03:03:06.662 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply-3.11-14.el9.noarch 2024-07-27T03:03:06.662 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend-3.1.0-2.el9.noarch 2024-07-27T03:03:06.662 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-07-27T03:03:06.662 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-07-27T03:03:06.662 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-07-27T03:03:06.662 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-07-27T03:03:06.662 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-07-27T03:03:06.662 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser-2.20-6.el9.noarch 2024-07-27T03:03:06.663 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-07-27T03:03:06.663 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz-2021.1-5.el9.noarch 2024-07-27T03:03:06.663 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.663 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.663 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-07-27T03:03:06.663 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-2.25.1-8.el9.noarch 2024-07-27T03:03:06.663 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-07-27T03:03:06.663 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.663 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes-2.5.1-5.el9.noarch 2024-07-27T03:03:06.663 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa-4.9-2.el9.noarch 2024-07-27T03:03:06.663 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-07-27T03:03:06.663 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-07-27T03:03:06.664 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml-0.10.2-6.el9.noarch 2024-07-27T03:03:06.664 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-07-27T03:03:06.664 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-07-27T03:03:06.664 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-07-27T03:03:06.664 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob-1.8.7-6.el9.noarch 2024-07-27T03:03:06.664 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-07-27T03:03:06.664 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-07-27T03:03:06.664 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-07-27T03:03:06.664 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-24.02.0-1.el9.x86_64 2024-07-27T03:03:06.664 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-07-27T03:03:06.664 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-07-27T03:03:06.664 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.664 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.665 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd-2:19.0.0-5358.gaf8fdbf0.el9.x86_64 2024-07-27T03:03:06.665 INFO:teuthology.orchestra.run.smithi149.stdout: re2-1:20211101-3.el9.x86_64 2024-07-27T03:03:06.665 INFO:teuthology.orchestra.run.smithi149.stdout: socat-1.7.4.1-6.el9.x86_64 2024-07-27T03:03:06.665 INFO:teuthology.orchestra.run.smithi149.stdout: thrift-0.15.0-2.el9.x86_64 2024-07-27T03:03:06.665 INFO:teuthology.orchestra.run.smithi149.stdout: uuid-1.6.2-55.el9.x86_64 2024-07-27T03:03:06.665 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-07-27T03:03:06.665 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-07-27T03:03:06.665 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-07-27T03:03:06.665 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-07-27T03:03:06.665 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-07-27T03:03:07.095 DEBUG:teuthology.parallel:result is None 2024-07-27T03:03:07.096 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=af8fdbf079811fe158ce62927bd145c35b74d4b8 2024-07-27T03:03:07.249 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-27T03:03:07.276 INFO:teuthology.orchestra.run.smithi067.stdout:19.0.0-5358.gaf8fdbf0.el9 2024-07-27T03:03:07.277 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5358.gaf8fdbf0.el9 2024-07-27T03:03:07.277 INFO:teuthology.task.install:The correct ceph version 19.0.0-5358.gaf8fdbf0 is installed. 2024-07-27T03:03:07.279 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=af8fdbf079811fe158ce62927bd145c35b74d4b8 2024-07-27T03:03:07.419 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-27T03:03:07.441 INFO:teuthology.orchestra.run.smithi149.stdout:19.0.0-5358.gaf8fdbf0.el9 2024-07-27T03:03:07.441 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5358.gaf8fdbf0.el9 2024-07-27T03:03:07.442 INFO:teuthology.task.install:The correct ceph version 19.0.0-5358.gaf8fdbf0 is installed. 2024-07-27T03:03:07.443 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=af8fdbf079811fe158ce62927bd145c35b74d4b8 2024-07-27T03:03:07.583 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-07-27T03:03:07.609 INFO:teuthology.orchestra.run.smithi175.stdout:19.0.0-5358.gaf8fdbf0.el9 2024-07-27T03:03:07.610 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5358.gaf8fdbf0.el9 2024-07-27T03:03:07.610 INFO:teuthology.task.install:The correct ceph version 19.0.0-5358.gaf8fdbf0 is installed. 2024-07-27T03:03:07.612 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-07-27T03:03:07.612 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-27T03:03:07.612 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-27T03:03:07.646 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-07-27T03:03:07.646 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-27T03:03:07.672 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-27T03:03:07.673 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-07-27T03:03:07.702 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-07-27T03:03:07.703 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-27T03:03:07.703 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/daemon-helper 2024-07-27T03:03:07.732 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-27T03:03:07.801 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-07-27T03:03:07.801 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/daemon-helper 2024-07-27T03:03:07.826 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-27T03:03:07.891 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-27T03:03:07.892 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/daemon-helper 2024-07-27T03:03:07.920 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-07-27T03:03:07.988 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-07-27T03:03:07.988 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-27T03:03:07.988 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-27T03:03:08.018 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-27T03:03:08.088 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-07-27T03:03:08.088 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-27T03:03:08.112 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-27T03:03:08.177 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-27T03:03:08.177 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/adjust-ulimits 2024-07-27T03:03:08.206 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-07-27T03:03:08.273 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-07-27T03:03:08.273 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-27T03:03:08.274 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/stdin-killer 2024-07-27T03:03:08.304 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-27T03:03:08.374 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-07-27T03:03:08.374 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/stdin-killer 2024-07-27T03:03:08.398 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-27T03:03:08.461 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-27T03:03:08.462 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/stdin-killer 2024-07-27T03:03:08.489 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-07-27T03:03:08.557 INFO:teuthology.run_tasks:Running task cephadm... 2024-07-27T03:03:08.654 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_bal_replicate_threshold': 1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'af8fdbf079811fe158ce62927bd145c35b74d4b8', 'subvols': {'create': 2}} 2024-07-27T03:03:08.654 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 2024-07-27T03:03:08.654 INFO:tasks.cephadm:Cluster fsid is c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:03:08.655 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-07-27T03:03:08.655 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.67', 'mon.b': '172.21.15.149', 'mon.c': '172.21.15.175'} 2024-07-27T03:03:08.655 INFO:tasks.cephadm:First mon is mon.a on smithi067 2024-07-27T03:03:08.655 INFO:tasks.cephadm:First mgr is x 2024-07-27T03:03:08.655 INFO:tasks.cephadm:Normalizing hostnames... 2024-07-27T03:03:08.655 DEBUG:teuthology.orchestra.run.smithi067:> sudo hostname $(hostname -s) 2024-07-27T03:03:08.687 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname $(hostname -s) 2024-07-27T03:03:08.712 DEBUG:teuthology.orchestra.run.smithi175:> sudo hostname $(hostname -s) 2024-07-27T03:03:08.739 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-07-27T03:03:08.740 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=af8fdbf079811fe158ce62927bd145c35b74d4b8 2024-07-27T03:03:08.886 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'af8fdbf079811fe158ce62927bd145c35b74d4b8', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/81580/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-5358-gaf8fdbf0', 'node_name': '172.21.2.5+braggi05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-5358.gaf8fdbf0'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240726.202642-debug/af8fdbf079811fe158ce62927bd145c35b74d4b8/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-07-26 22:46:14.290129', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240726.202642-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240726.202642-debug/af8fdbf079811fe158ce62927bd145c35b74d4b8/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'af8fdbf079811fe158ce62927bd145c35b74d4b8', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/81576/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-5358-gaf8fdbf0', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-5358.gaf8fdbf0'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240726.202642-debug/af8fdbf079811fe158ce62927bd145c35b74d4b8/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-07-26 21:45:24.865708', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240726.202642-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240726.202642-debug/af8fdbf079811fe158ce62927bd145c35b74d4b8/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-07-27T03:03:09.028 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20240726.202642-debug, sha1 af8fdbf079811fe158ce62927bd145c35b74d4b8 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=af8fdbf079811fe158ce62927bd145c35b74d4b8 2024-07-27T03:03:09.030 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240726.202642-debug/af8fdbf079811fe158ce62927bd145c35b74d4b8/centos/9/x86_64/flavors/default/cephadm 2024-07-27T03:03:09.030 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240726.202642-debug/af8fdbf079811fe158ce62927bd145c35b74d4b8/centos/9/x86_64/flavors/default/cephadm 2024-07-27T03:03:09.030 DEBUG:teuthology.orchestra.run.smithi067:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240726.202642-debug/af8fdbf079811fe158ce62927bd145c35b74d4b8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-07-27T03:03:09.598 INFO:teuthology.orchestra.run.smithi067.stdout:-rw-r--r--. 1 ubuntu ubuntu 789644 Jul 27 03:03 /home/ubuntu/cephtest/cephadm 2024-07-27T03:03:09.598 DEBUG:teuthology.orchestra.run.smithi149:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240726.202642-debug/af8fdbf079811fe158ce62927bd145c35b74d4b8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-07-27T03:03:10.131 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r--. 1 ubuntu ubuntu 789644 Jul 27 03:03 /home/ubuntu/cephtest/cephadm 2024-07-27T03:03:10.132 DEBUG:teuthology.orchestra.run.smithi175:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240726.202642-debug/af8fdbf079811fe158ce62927bd145c35b74d4b8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-07-27T03:03:10.677 INFO:teuthology.orchestra.run.smithi175.stdout:-rw-r--r--. 1 ubuntu ubuntu 789644 Jul 27 03:03 /home/ubuntu/cephtest/cephadm 2024-07-27T03:03:10.678 DEBUG:teuthology.orchestra.run.smithi067:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-07-27T03:03:10.698 DEBUG:teuthology.orchestra.run.smithi149:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-07-27T03:03:10.717 DEBUG:teuthology.orchestra.run.smithi175:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-07-27T03:03:10.753 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 on all hosts... 2024-07-27T03:03:10.753 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 pull 2024-07-27T03:03:10.756 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 pull 2024-07-27T03:03:10.759 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 pull 2024-07-27T03:03:11.538 INFO:teuthology.orchestra.run.smithi149.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8... 2024-07-27T03:03:11.670 INFO:teuthology.orchestra.run.smithi175.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8... 2024-07-27T03:03:11.790 INFO:teuthology.orchestra.run.smithi067.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8... 2024-07-27T03:03:42.274 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-27T03:03:42.274 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.0.0-5358-gaf8fdbf0 (af8fdbf079811fe158ce62927bd145c35b74d4b8) squid (dev)", 2024-07-27T03:03:42.274 INFO:teuthology.orchestra.run.smithi175.stdout: "image_id": "b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3", 2024-07-27T03:03:42.274 INFO:teuthology.orchestra.run.smithi175.stdout: "repo_digests": [ 2024-07-27T03:03:42.274 INFO:teuthology.orchestra.run.smithi175.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c" 2024-07-27T03:03:42.275 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-07-27T03:03:42.275 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-27T03:03:48.201 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:03:48.201 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 19.0.0-5358-gaf8fdbf0 (af8fdbf079811fe158ce62927bd145c35b74d4b8) squid (dev)", 2024-07-27T03:03:48.201 INFO:teuthology.orchestra.run.smithi067.stdout: "image_id": "b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3", 2024-07-27T03:03:48.201 INFO:teuthology.orchestra.run.smithi067.stdout: "repo_digests": [ 2024-07-27T03:03:48.202 INFO:teuthology.orchestra.run.smithi067.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c" 2024-07-27T03:03:48.202 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-07-27T03:03:48.202 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:03:53.815 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-07-27T03:03:53.816 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_version": "ceph version 19.0.0-5358-gaf8fdbf0 (af8fdbf079811fe158ce62927bd145c35b74d4b8) squid (dev)", 2024-07-27T03:03:53.816 INFO:teuthology.orchestra.run.smithi149.stdout: "image_id": "b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3", 2024-07-27T03:03:53.816 INFO:teuthology.orchestra.run.smithi149.stdout: "repo_digests": [ 2024-07-27T03:03:53.816 INFO:teuthology.orchestra.run.smithi149.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c" 2024-07-27T03:03:53.816 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-07-27T03:03:53.816 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-07-27T03:03:53.847 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /etc/ceph 2024-07-27T03:03:53.880 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /etc/ceph 2024-07-27T03:03:53.909 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph 2024-07-27T03:03:53.942 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 777 /etc/ceph 2024-07-27T03:03:53.969 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 777 /etc/ceph 2024-07-27T03:03:53.994 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 777 /etc/ceph 2024-07-27T03:03:54.021 INFO:tasks.cephadm:Writing seed config... 2024-07-27T03:03:54.024 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-07-27T03:03:54.024 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-07-27T03:03:54.024 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-07-27T03:03:54.024 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-07-27T03:03:54.025 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-07-27T03:03:54.025 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-07-27T03:03:54.025 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-07-27T03:03:54.025 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-07-27T03:03:54.025 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-07-27T03:03:54.025 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-07-27T03:03:54.026 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-07-27T03:03:54.026 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-07-27T03:03:54.026 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-07-27T03:03:54.026 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-07-27T03:03:54.026 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-07-27T03:03:54.026 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-07-27T03:03:54.027 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-07-27T03:03:54.027 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-07-27T03:03:54.027 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-07-27T03:03:54.027 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-07-27T03:03:54.027 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-07-27T03:03:54.027 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-07-27T03:03:54.027 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-07-27T03:03:54.028 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-07-27T03:03:54.029 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-27T03:03:54.029 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-07-27T03:03:54.047 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c0782a00-4bc4-11ef-bcc7-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_bal_replicate_threshold = 1 2024-07-27T03:03:54.047 DEBUG:teuthology.orchestra.run.smithi067:mon.a> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mon.a.service 2024-07-27T03:03:54.090 DEBUG:teuthology.orchestra.run.smithi067:mgr.x> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mgr.x.service 2024-07-27T03:03:54.133 INFO:tasks.cephadm:Bootstrapping... 2024-07-27T03:03:54.133 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 -v bootstrap --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.67 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-07-27T03:03:54.349 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-07-27T03:03:54.349 INFO:teuthology.orchestra.run.smithi067.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', '-v', 'bootstrap', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.67', '--skip-admin-label'] 2024-07-27T03:03:54.349 INFO:teuthology.orchestra.run.smithi067.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-07-27T03:03:54.350 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying podman|docker is present... 2024-07-27T03:03:54.377 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout 5.1.2 2024-07-27T03:03:54.377 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying lvm2 is present... 2024-07-27T03:03:54.377 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying time synchronization is in place... 2024-07-27T03:03:54.385 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-07-27T03:03:54.386 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-07-27T03:03:54.391 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-07-27T03:03:54.391 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-07-27T03:03:54.398 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout enabled 2024-07-27T03:03:54.404 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout active 2024-07-27T03:03:54.404 INFO:teuthology.orchestra.run.smithi067.stdout:Unit chronyd.service is enabled and running 2024-07-27T03:03:54.405 INFO:teuthology.orchestra.run.smithi067.stdout:Repeating the final host check... 2024-07-27T03:03:54.429 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout 5.1.2 2024-07-27T03:03:54.430 INFO:teuthology.orchestra.run.smithi067.stdout:podman (/bin/podman) version 5.1.2 is present 2024-07-27T03:03:54.430 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl is present 2024-07-27T03:03:54.430 INFO:teuthology.orchestra.run.smithi067.stdout:lvcreate is present 2024-07-27T03:03:54.436 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-07-27T03:03:54.436 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-07-27T03:03:54.443 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-07-27T03:03:54.443 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-07-27T03:03:54.450 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout enabled 2024-07-27T03:03:54.457 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout active 2024-07-27T03:03:54.457 INFO:teuthology.orchestra.run.smithi067.stdout:Unit chronyd.service is enabled and running 2024-07-27T03:03:54.457 INFO:teuthology.orchestra.run.smithi067.stdout:Host looks OK 2024-07-27T03:03:54.457 INFO:teuthology.orchestra.run.smithi067.stdout:Cluster fsid: c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:03:54.458 INFO:teuthology.orchestra.run.smithi067.stdout:Acquiring lock 139827065746240 on /run/cephadm/c0782a00-4bc4-11ef-bcc7-c7b262605968.lock 2024-07-27T03:03:54.458 INFO:teuthology.orchestra.run.smithi067.stdout:Lock 139827065746240 acquired on /run/cephadm/c0782a00-4bc4-11ef-bcc7-c7b262605968.lock 2024-07-27T03:03:54.458 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying IP 172.21.15.67 port 3300 ... 2024-07-27T03:03:54.458 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying IP 172.21.15.67 port 6789 ... 2024-07-27T03:03:54.459 INFO:teuthology.orchestra.run.smithi067.stdout:Base mon IP(s) is [172.21.15.67:3300, 172.21.15.67:6789], mon addrv is [v2:172.21.15.67:3300,v1:172.21.15.67:6789] 2024-07-27T03:03:54.461 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.67 metric 100 2024-07-27T03:03:54.461 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.67 metric 100 2024-07-27T03:03:54.464 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-07-27T03:03:54.464 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-07-27T03:03:54.465 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1795sec hoplimit 64 pref medium 2024-07-27T03:03:54.468 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-07-27T03:03:54.468 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-07-27T03:03:54.468 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-07-27T03:03:54.468 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-07-27T03:03:54.468 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-07-27T03:03:54.469 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-07-27T03:03:54.469 INFO:teuthology.orchestra.run.smithi067.stdout:Mon IP `172.21.15.67` is in CIDR network `172.21.0.0/20` 2024-07-27T03:03:54.469 INFO:teuthology.orchestra.run.smithi067.stdout:Mon IP `172.21.15.67` is in CIDR network `172.21.0.0/20` 2024-07-27T03:03:54.469 INFO:teuthology.orchestra.run.smithi067.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-07-27T03:03:54.469 INFO:teuthology.orchestra.run.smithi067.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-07-27T03:03:54.470 INFO:teuthology.orchestra.run.smithi067.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8... 2024-07-27T03:03:55.206 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 2024-07-27T03:03:55.207 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8... 2024-07-27T03:03:55.207 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Getting image source signatures 2024-07-27T03:03:55.207 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Copying blob sha256:6dbd6856232c9e97971258c34a2b7a40b6b47bfb31749b9c9184817bbb48bfd8 2024-07-27T03:03:55.207 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Copying blob sha256:43735fbce9214f061a6894341d7822563f5aafe16fc7532189f4415cff6d2d91 2024-07-27T03:03:55.207 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Copying config sha256:b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 2024-07-27T03:03:55.207 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Writing manifest to image destination 2024-07-27T03:04:01.312 INFO:teuthology.orchestra.run.smithi067.stdout:ceph: stdout ceph version 19.0.0-5358-gaf8fdbf0 (af8fdbf079811fe158ce62927bd145c35b74d4b8) squid (dev) 2024-07-27T03:04:01.312 INFO:teuthology.orchestra.run.smithi067.stdout:Ceph version: ceph version 19.0.0-5358-gaf8fdbf0 (af8fdbf079811fe158ce62927bd145c35b74d4b8) squid (dev) 2024-07-27T03:04:01.312 INFO:teuthology.orchestra.run.smithi067.stdout:Extracting ceph user uid/gid from container image... 2024-07-27T03:04:02.342 INFO:teuthology.orchestra.run.smithi067.stdout:stat: stdout 167 167 2024-07-27T03:04:02.343 INFO:teuthology.orchestra.run.smithi067.stdout:Creating initial keys... 2024-07-27T03:04:03.173 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph-authtool: stdout AQCiY6Rmw7fgJRAAOpI13w0ZprAISgUu4ShqfQ== 2024-07-27T03:04:03.944 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph-authtool: stdout AQCjY6RmsbB1HBAAFihhAmxfH9ivguUCndlsDQ== 2024-07-27T03:04:04.832 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph-authtool: stdout AQCkY6Rmt2QwERAAmpdNzVwufg+HsbM1wPr5mA== 2024-07-27T03:04:04.833 INFO:teuthology.orchestra.run.smithi067.stdout:Creating initial monmap... 2024-07-27T03:04:05.781 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-07-27T03:04:05.781 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-07-27T03:04:05.781 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:04:05.781 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-07-27T03:04:05.781 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool for a [v2:172.21.15.67:3300,v1:172.21.15.67:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-07-27T03:04:05.781 INFO:teuthology.orchestra.run.smithi067.stdout:setting min_mon_release = quincy 2024-07-27T03:04:05.781 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: set fsid to c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:04:05.781 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-07-27T03:04:05.782 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:04:05.782 INFO:teuthology.orchestra.run.smithi067.stdout:Creating mon... 2024-07-27T03:04:06.669 INFO:teuthology.orchestra.run.smithi067.stdout:create mon.a on 2024-07-27T03:04:06.876 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-07-27T03:04:07.026 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-07-27T03:04:07.202 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968.target → /etc/systemd/system/ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968.target. 2024-07-27T03:04:07.203 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968.target → /etc/systemd/system/ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968.target. 2024-07-27T03:04:07.460 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mon.a 2024-07-27T03:04:07.460 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mon.a.service: Unit ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mon.a.service not loaded. 2024-07-27T03:04:07.630 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968.target.wants/ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mon.a.service → /etc/systemd/system/ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@.service. 2024-07-27T03:04:08.104 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:08 smithi067 ceph-mon[32450]: mon.a is new leader, mons a in quorum (ranks 0) 2024-07-27T03:04:08.135 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-27T03:04:08.135 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-07-27T03:04:08.142 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-27T03:04:08.142 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-07-27T03:04:08.142 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-07-27T03:04:08.142 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to enable service . firewalld.service is not available 2024-07-27T03:04:08.143 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mon to start... 2024-07-27T03:04:08.143 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mon... 2024-07-27T03:04:08.356 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:08 smithi067 bash[32431]: e1c40f3f5ab19a41c1923f12275ecb68a5015829cc4463fc3720e2c012aafd37 2024-07-27T03:04:08.356 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:08 smithi067 systemd[1]: Started Ceph mon.a for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:04:09.202 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:09 smithi067 ceph-mon[32450]: mon.a is new leader, mons a in quorum (ranks 0) 2024-07-27T03:04:09.203 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:09 smithi067 ceph-mon[32450]: monmap epoch 1 2024-07-27T03:04:09.203 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:09 smithi067 ceph-mon[32450]: fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:04:09.203 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:09 smithi067 ceph-mon[32450]: last_changed 2024-07-27T03:04:05.157867+0000 2024-07-27T03:04:09.203 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:09 smithi067 ceph-mon[32450]: created 2024-07-27T03:04:05.157867+0000 2024-07-27T03:04:09.203 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:09 smithi067 ceph-mon[32450]: min_mon_release 19 (squid) 2024-07-27T03:04:09.203 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:09 smithi067 ceph-mon[32450]: election_strategy: 1 2024-07-27T03:04:09.203 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:09 smithi067 ceph-mon[32450]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-07-27T03:04:09.203 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:09 smithi067 ceph-mon[32450]: fsmap 2024-07-27T03:04:09.203 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:09 smithi067 ceph-mon[32450]: osdmap e1: 0 total, 0 up, 0 in 2024-07-27T03:04:09.203 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:09 smithi067 ceph-mon[32450]: mgrmap e1: no daemons active 2024-07-27T03:04:09.203 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:09 smithi067 ceph-mon[32450]: from='client.? 172.21.15.67:0/2375074912' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-07-27T03:04:09.206 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout cluster: 2024-07-27T03:04:09.206 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout id: c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:04:09.206 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-07-27T03:04:09.206 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-27T03:04:09.206 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout services: 2024-07-27T03:04:09.206 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.518845s) 2024-07-27T03:04:09.206 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-07-27T03:04:09.206 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-07-27T03:04:09.206 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-27T03:04:09.207 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout data: 2024-07-27T03:04:09.207 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-07-27T03:04:09.207 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-07-27T03:04:09.207 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-07-27T03:04:09.207 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout pgs: 2024-07-27T03:04:09.207 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-27T03:04:09.207 INFO:teuthology.orchestra.run.smithi067.stdout:mon is available 2024-07-27T03:04:09.207 INFO:teuthology.orchestra.run.smithi067.stdout:Assimilating anything we can from ceph.conf... 2024-07-27T03:04:10.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:10 smithi067 ceph-mon[32450]: from='client.? 172.21.15.67:0/3471513502' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-07-27T03:04:10.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:10 smithi067 ceph-mon[32450]: from='client.? 172.21.15.67:0/3471513502' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-07-27T03:04:10.279 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-27T03:04:10.279 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [global] 2024-07-27T03:04:10.279 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout fsid = c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:04:10.279 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-07-27T03:04:10.279 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.67:3300,v1:172.21.15.67:6789] 2024-07-27T03:04:10.279 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-07-27T03:04:10.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-07-27T03:04:10.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-07-27T03:04:10.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-07-27T03:04:10.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-27T03:04:10.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [mgr] 2024-07-27T03:04:10.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-07-27T03:04:10.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-27T03:04:10.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [osd] 2024-07-27T03:04:10.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-07-27T03:04:10.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-07-27T03:04:10.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-07-27T03:04:10.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-07-27T03:04:10.280 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-07-27T03:04:10.280 INFO:teuthology.orchestra.run.smithi067.stdout:Generating new minimal ceph.conf... 2024-07-27T03:04:11.250 INFO:teuthology.orchestra.run.smithi067.stdout:Restarting the monitor... 2024-07-27T03:04:11.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:11 smithi067 ceph-mon[32450]: from='client.? 172.21.15.67:0/3531845363' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:04:11.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:11 smithi067 systemd[1]: Stopping Ceph mon.a for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:04:11.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:11 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-a[32446]: 2024-07-27T03:04:11.590+0000 7f61d4cc3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:04:11.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:11 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-a[32446]: 2024-07-27T03:04:11.590+0000 7f61d4cc3640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-07-27T03:04:11.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:11 smithi067 podman[32692]: 2024-07-27 03:04:11.815611572 +0000 UTC m=+0.400267244 container died e1c40f3f5ab19a41c1923f12275ecb68a5015829cc4463fc3720e2c012aafd37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, ceph=True, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-07-27T03:04:12.327 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:12 smithi067 podman[32692]: 2024-07-27 03:04:12.148413795 +0000 UTC m=+0.733069456 container cleanup e1c40f3f5ab19a41c1923f12275ecb68a5015829cc4463fc3720e2c012aafd37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-a, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-07-27T03:04:12.327 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:12 smithi067 bash[32692]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-a 2024-07-27T03:04:12.327 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:12 smithi067 podman[32703]: 2024-07-27 03:04:12.324244625 +0000 UTC m=+0.508272214 container remove e1c40f3f5ab19a41c1923f12275ecb68a5015829cc4463fc3720e2c012aafd37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-a, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-07-27T03:04:12.901 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:12 smithi067 systemd[1]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mon.a.service: Deactivated successfully. 2024-07-27T03:04:12.901 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:12 smithi067 systemd[1]: Stopped Ceph mon.a for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:04:12.902 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:12 smithi067 systemd[1]: Starting Ceph mon.a for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:04:12.902 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:12 smithi067 podman[32795]: 2024-07-27 03:04:12.791674678 +0000 UTC m=+0.108592049 container create 23a97e917213294314c4e6b62ead3c7165eb08c17308bf9c3d514b912dacad6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-27T03:04:12.902 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:12 smithi067 podman[32795]: 2024-07-27 03:04:12.700784158 +0000 UTC m=+0.017701534 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 2024-07-27T03:04:13.059 INFO:teuthology.orchestra.run.smithi067.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-07-27T03:04:13.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:12 smithi067 podman[32795]: 2024-07-27 03:04:12.985581805 +0000 UTC m=+0.302499194 container init 23a97e917213294314c4e6b62ead3c7165eb08c17308bf9c3d514b912dacad6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-a, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf) 2024-07-27T03:04:13.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:12 smithi067 podman[32795]: 2024-07-27 03:04:12.989214608 +0000 UTC m=+0.306131976 container start 23a97e917213294314c4e6b62ead3c7165eb08c17308bf9c3d514b912dacad6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-a, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-27T03:04:13.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: set uid:gid to 167:167 (ceph:ceph) 2024-07-27T03:04:13.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: ceph version 19.0.0-5358-gaf8fdbf0 (af8fdbf079811fe158ce62927bd145c35b74d4b8) squid (dev), process ceph-mon, pid 2 2024-07-27T03:04:13.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: pidfile_write: ignore empty --pid-file 2024-07-27T03:04:13.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: load: jerasure load: lrc 2024-07-27T03:04:13.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: RocksDB version: 7.9.2 2024-07-27T03:04:13.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Git sha 0 2024-07-27T03:04:13.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Compile date 2024-07-26 20:49:32 2024-07-27T03:04:13.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: DB SUMMARY 2024-07-27T03:04:13.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: DB Session ID: OOIIO8ZH7ZFM61AKR3KL 2024-07-27T03:04:13.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: CURRENT file: CURRENT 2024-07-27T03:04:13.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: IDENTITY file: IDENTITY 2024-07-27T03:04:13.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-07-27T03:04:13.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-07-27T03:04:13.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94413 ; 2024-07-27T03:04:13.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.error_if_exists: 0 2024-07-27T03:04:13.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.create_if_missing: 0 2024-07-27T03:04:13.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.paranoid_checks: 1 2024-07-27T03:04:13.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.flush_verify_memtable_count: 1 2024-07-27T03:04:13.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-07-27T03:04:13.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-07-27T03:04:13.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.env: 0x5597b8f00280 2024-07-27T03:04:13.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.fs: PosixFileSystem 2024-07-27T03:04:13.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.info_log: 0x5597bf3bf120 2024-07-27T03:04:13.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_file_opening_threads: 16 2024-07-27T03:04:13.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.statistics: (nil) 2024-07-27T03:04:13.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.use_fsync: 0 2024-07-27T03:04:13.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_log_file_size: 0 2024-07-27T03:04:13.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-07-27T03:04:13.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.log_file_time_to_roll: 0 2024-07-27T03:04:13.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.keep_log_file_num: 1000 2024-07-27T03:04:13.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.recycle_log_file_num: 0 2024-07-27T03:04:13.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.allow_fallocate: 1 2024-07-27T03:04:13.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.allow_mmap_reads: 0 2024-07-27T03:04:13.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.allow_mmap_writes: 0 2024-07-27T03:04:13.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.use_direct_reads: 0 2024-07-27T03:04:13.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-07-27T03:04:13.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.create_missing_column_families: 0 2024-07-27T03:04:13.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.db_log_dir: 2024-07-27T03:04:13.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.wal_dir: 2024-07-27T03:04:13.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.table_cache_numshardbits: 6 2024-07-27T03:04:13.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.WAL_ttl_seconds: 0 2024-07-27T03:04:13.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.WAL_size_limit_MB: 0 2024-07-27T03:04:13.184 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-07-27T03:04:13.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-07-27T03:04:13.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.is_fd_close_on_exec: 1 2024-07-27T03:04:13.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.advise_random_on_open: 1 2024-07-27T03:04:13.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.db_write_buffer_size: 0 2024-07-27T03:04:13.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.write_buffer_manager: 0x5597bf3c35e0 2024-07-27T03:04:13.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-07-27T03:04:13.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-07-27T03:04:13.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.use_adaptive_mutex: 0 2024-07-27T03:04:13.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.rate_limiter: (nil) 2024-07-27T03:04:13.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-07-27T03:04:13.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.wal_recovery_mode: 2 2024-07-27T03:04:13.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.enable_thread_tracking: 0 2024-07-27T03:04:13.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.enable_pipelined_write: 0 2024-07-27T03:04:13.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.unordered_write: 0 2024-07-27T03:04:13.186 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-07-27T03:04:13.186 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-07-27T03:04:13.186 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-07-27T03:04:13.186 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-07-27T03:04:13.186 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.row_cache: None 2024-07-27T03:04:13.186 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.wal_filter: None 2024-07-27T03:04:13.187 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-07-27T03:04:13.187 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.allow_ingest_behind: 0 2024-07-27T03:04:13.187 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.two_write_queues: 0 2024-07-27T03:04:13.187 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.manual_wal_flush: 0 2024-07-27T03:04:13.188 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.wal_compression: 0 2024-07-27T03:04:13.188 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.atomic_flush: 0 2024-07-27T03:04:13.188 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-07-27T03:04:13.188 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.persist_stats_to_disk: 0 2024-07-27T03:04:13.188 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.write_dbid_to_manifest: 0 2024-07-27T03:04:13.188 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.log_readahead_size: 0 2024-07-27T03:04:13.188 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-07-27T03:04:13.188 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.best_efforts_recovery: 0 2024-07-27T03:04:13.188 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-07-27T03:04:13.188 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-07-27T03:04:13.188 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.allow_data_in_errors: 0 2024-07-27T03:04:13.188 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.db_host_id: __hostname__ 2024-07-27T03:04:13.188 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.enforce_single_del_contracts: true 2024-07-27T03:04:13.188 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_background_jobs: 2 2024-07-27T03:04:13.189 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_background_compactions: -1 2024-07-27T03:04:13.189 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_subcompactions: 1 2024-07-27T03:04:13.189 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-07-27T03:04:13.189 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-07-27T03:04:13.189 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.delayed_write_rate : 16777216 2024-07-27T03:04:13.189 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_total_wal_size: 0 2024-07-27T03:04:13.189 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-07-27T03:04:13.189 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.stats_dump_period_sec: 600 2024-07-27T03:04:13.189 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.stats_persist_period_sec: 600 2024-07-27T03:04:13.189 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-07-27T03:04:13.189 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_open_files: -1 2024-07-27T03:04:13.189 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.bytes_per_sync: 0 2024-07-27T03:04:13.189 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.wal_bytes_per_sync: 0 2024-07-27T03:04:13.189 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.strict_bytes_per_sync: 0 2024-07-27T03:04:13.190 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compaction_readahead_size: 0 2024-07-27T03:04:13.190 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_background_flushes: -1 2024-07-27T03:04:13.190 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Compression algorithms supported: 2024-07-27T03:04:13.190 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: kZSTD supported: 0 2024-07-27T03:04:13.190 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: kXpressCompression supported: 0 2024-07-27T03:04:13.190 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: kBZip2Compression supported: 0 2024-07-27T03:04:13.190 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-07-27T03:04:13.190 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: kLZ4Compression supported: 1 2024-07-27T03:04:13.190 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: kZlibCompression supported: 1 2024-07-27T03:04:13.190 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: kLZ4HCCompression supported: 1 2024-07-27T03:04:13.190 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: kSnappyCompression supported: 1 2024-07-27T03:04:13.190 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Fast CRC32 supported: Supported on x86 2024-07-27T03:04:13.191 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: DMutex implementation: pthread_mutex_t 2024-07-27T03:04:13.191 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-07-27T03:04:13.191 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-07-27T03:04:13.191 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-07-27T03:04:13.191 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.merge_operator: 2024-07-27T03:04:13.191 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compaction_filter: None 2024-07-27T03:04:13.191 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compaction_filter_factory: None 2024-07-27T03:04:13.191 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.sst_partitioner_factory: None 2024-07-27T03:04:13.191 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.memtable_factory: SkipListFactory 2024-07-27T03:04:13.191 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.table_factory: BlockBasedTable 2024-07-27T03:04:13.191 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5597bf3bed40) 2024-07-27T03:04:13.191 INFO:journalctl@ceph.mon.a.smithi067.stdout: cache_index_and_filter_blocks: 1 2024-07-27T03:04:13.191 INFO:journalctl@ceph.mon.a.smithi067.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-07-27T03:04:13.191 INFO:journalctl@ceph.mon.a.smithi067.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-07-27T03:04:13.192 INFO:journalctl@ceph.mon.a.smithi067.stdout: pin_top_level_index_and_filter: 1 2024-07-27T03:04:13.192 INFO:journalctl@ceph.mon.a.smithi067.stdout: index_type: 0 2024-07-27T03:04:13.192 INFO:journalctl@ceph.mon.a.smithi067.stdout: data_block_index_type: 0 2024-07-27T03:04:13.192 INFO:journalctl@ceph.mon.a.smithi067.stdout: index_shortening: 1 2024-07-27T03:04:13.192 INFO:journalctl@ceph.mon.a.smithi067.stdout: data_block_hash_table_util_ratio: 0.750000 2024-07-27T03:04:13.192 INFO:journalctl@ceph.mon.a.smithi067.stdout: checksum: 4 2024-07-27T03:04:13.192 INFO:journalctl@ceph.mon.a.smithi067.stdout: no_block_cache: 0 2024-07-27T03:04:13.192 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache: 0x5597bb383610 2024-07-27T03:04:13.192 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache_name: BinnedLRUCache 2024-07-27T03:04:13.192 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache_options: 2024-07-27T03:04:13.192 INFO:journalctl@ceph.mon.a.smithi067.stdout: capacity : 536870912 2024-07-27T03:04:13.192 INFO:journalctl@ceph.mon.a.smithi067.stdout: num_shard_bits : 4 2024-07-27T03:04:13.192 INFO:journalctl@ceph.mon.a.smithi067.stdout: strict_capacity_limit : 0 2024-07-27T03:04:13.192 INFO:journalctl@ceph.mon.a.smithi067.stdout: high_pri_pool_ratio: 0.000 2024-07-27T03:04:13.192 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache_compressed: (nil) 2024-07-27T03:04:13.193 INFO:journalctl@ceph.mon.a.smithi067.stdout: persistent_cache: (nil) 2024-07-27T03:04:13.193 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_size: 4096 2024-07-27T03:04:13.193 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_size_deviation: 10 2024-07-27T03:04:13.193 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_restart_interval: 16 2024-07-27T03:04:13.193 INFO:journalctl@ceph.mon.a.smithi067.stdout: index_block_restart_interval: 1 2024-07-27T03:04:13.193 INFO:journalctl@ceph.mon.a.smithi067.stdout: metadata_block_size: 4096 2024-07-27T03:04:13.193 INFO:journalctl@ceph.mon.a.smithi067.stdout: partition_filters: 0 2024-07-27T03:04:13.193 INFO:journalctl@ceph.mon.a.smithi067.stdout: use_delta_encoding: 1 2024-07-27T03:04:13.193 INFO:journalctl@ceph.mon.a.smithi067.stdout: filter_policy: bloomfilter 2024-07-27T03:04:13.193 INFO:journalctl@ceph.mon.a.smithi067.stdout: whole_key_filtering: 1 2024-07-27T03:04:13.193 INFO:journalctl@ceph.mon.a.smithi067.stdout: verify_compression: 0 2024-07-27T03:04:13.193 INFO:journalctl@ceph.mon.a.smithi067.stdout: read_amp_bytes_per_bit: 0 2024-07-27T03:04:13.193 INFO:journalctl@ceph.mon.a.smithi067.stdout: format_version: 5 2024-07-27T03:04:13.193 INFO:journalctl@ceph.mon.a.smithi067.stdout: enable_index_compression: 1 2024-07-27T03:04:13.194 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_align: 0 2024-07-27T03:04:13.194 INFO:journalctl@ceph.mon.a.smithi067.stdout: max_auto_readahead_size: 262144 2024-07-27T03:04:13.194 INFO:journalctl@ceph.mon.a.smithi067.stdout: prepopulate_block_cache: 0 2024-07-27T03:04:13.194 INFO:journalctl@ceph.mon.a.smithi067.stdout: initial_auto_readahead_size: 8192 2024-07-27T03:04:13.194 INFO:journalctl@ceph.mon.a.smithi067.stdout: num_file_reads_for_auto_readahead: 2 2024-07-27T03:04:13.194 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.write_buffer_size: 33554432 2024-07-27T03:04:13.194 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_write_buffer_number: 2 2024-07-27T03:04:13.194 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compression: NoCompression 2024-07-27T03:04:13.194 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.bottommost_compression: Disabled 2024-07-27T03:04:13.195 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.prefix_extractor: nullptr 2024-07-27T03:04:13.195 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-07-27T03:04:13.195 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.num_levels: 7 2024-07-27T03:04:13.195 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-07-27T03:04:13.195 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-07-27T03:04:13.195 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-07-27T03:04:13.195 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-07-27T03:04:13.195 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-07-27T03:04:13.195 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-07-27T03:04:13.195 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-07-27T03:04:13.195 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-07-27T03:04:13.195 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-07-27T03:04:13.195 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-07-27T03:04:13.196 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-07-27T03:04:13.196 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-07-27T03:04:13.196 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compression_opts.window_bits: -14 2024-07-27T03:04:13.196 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compression_opts.level: 32767 2024-07-27T03:04:13.196 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compression_opts.strategy: 0 2024-07-27T03:04:13.196 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-07-27T03:04:13.196 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-07-27T03:04:13.196 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-07-27T03:04:13.196 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-07-27T03:04:13.196 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compression_opts.enabled: false 2024-07-27T03:04:13.196 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-07-27T03:04:13.196 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-07-27T03:04:13.196 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-07-27T03:04:13.196 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-07-27T03:04:13.197 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.target_file_size_base: 67108864 2024-07-27T03:04:13.197 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.target_file_size_multiplier: 1 2024-07-27T03:04:13.197 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-07-27T03:04:13.197 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-07-27T03:04:13.197 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-07-27T03:04:13.197 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-07-27T03:04:13.197 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-07-27T03:04:13.197 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-07-27T03:04:13.197 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-07-27T03:04:13.197 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-07-27T03:04:13.197 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-07-27T03:04:13.197 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-07-27T03:04:13.197 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-07-27T03:04:13.197 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-07-27T03:04:13.198 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-07-27T03:04:13.198 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.arena_block_size: 1048576 2024-07-27T03:04:13.198 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-07-27T03:04:13.198 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-07-27T03:04:13.198 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.disable_auto_compactions: 0 2024-07-27T03:04:13.198 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-07-27T03:04:13.198 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-07-27T03:04:13.198 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-07-27T03:04:13.198 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-07-27T03:04:13.198 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-07-27T03:04:13.198 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-07-27T03:04:13.198 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-07-27T03:04:13.198 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-07-27T03:04:13.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-07-27T03:04:13.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-07-27T03:04:13.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-07-27T03:04:13.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.inplace_update_support: 0 2024-07-27T03:04:13.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.inplace_update_num_locks: 10000 2024-07-27T03:04:13.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-07-27T03:04:13.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-07-27T03:04:13.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.memtable_huge_page_size: 0 2024-07-27T03:04:13.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.bloom_locality: 0 2024-07-27T03:04:13.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.max_successive_merges: 0 2024-07-27T03:04:13.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.optimize_filters_for_hits: 0 2024-07-27T03:04:13.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.paranoid_file_checks: 0 2024-07-27T03:04:13.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.force_consistency_checks: 1 2024-07-27T03:04:13.199 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.report_bg_io_stats: 0 2024-07-27T03:04:13.200 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.ttl: 2592000 2024-07-27T03:04:13.200 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.periodic_compaction_seconds: 0 2024-07-27T03:04:13.200 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-07-27T03:04:13.200 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-07-27T03:04:13.200 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.enable_blob_files: false 2024-07-27T03:04:13.200 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.min_blob_size: 0 2024-07-27T03:04:13.200 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.blob_file_size: 268435456 2024-07-27T03:04:13.200 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.blob_compression_type: NoCompression 2024-07-27T03:04:13.200 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.enable_blob_garbage_collection: false 2024-07-27T03:04:13.200 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-07-27T03:04:13.200 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-07-27T03:04:13.200 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-07-27T03:04:13.200 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.blob_file_starting_level: 0 2024-07-27T03:04:13.200 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-07-27T03:04:13.201 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-07-27T03:04:13.201 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-07-27T03:04:13.201 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9e39a5a2-f274-489a-a078-0feec11d8a18 2024-07-27T03:04:13.201 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722049453024524, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-07-27T03:04:13.201 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-07-27T03:04:13.201 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722049453025716, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 90643, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 280, "table_properties": {"data_size": 88684, "index_size": 279, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11488, "raw_average_key_size": 48, "raw_value_size": 82207, "raw_average_value_size": 343, "num_data_blocks": 12, "num_entries": 239, "num_filter_entries": 239, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1722049453, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9e39a5a2-f274-489a-a078-0feec11d8a18", "db_session_id": "OOIIO8ZH7ZFM61AKR3KL", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-07-27T03:04:13.201 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722049453025825, "job": 1, "event": "recovery_finished"} 2024-07-27T03:04:13.201 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-07-27T03:04:13.201 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-07-27T03:04:13.201 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5597bf3e0a80 2024-07-27T03:04:13.201 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: DB pointer 0x5597c0180000 2024-07-27T03:04:13.201 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-07-27T03:04:13.201 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-07-27T03:04:13.201 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** DB Stats ** 2024-07-27T03:04:13.202 INFO:journalctl@ceph.mon.a.smithi067.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-27T03:04:13.202 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-07-27T03:04:13.202 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-27T03:04:13.202 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-27T03:04:13.202 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-07-27T03:04:13.202 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-27T03:04:13.202 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-27T03:04:13.202 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-07-27T03:04:13.202 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** Compaction Stats [default] ** 2024-07-27T03:04:13.202 INFO:journalctl@ceph.mon.a.smithi067.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-27T03:04:13.202 INFO:journalctl@ceph.mon.a.smithi067.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-07-27T03:04:13.202 INFO:journalctl@ceph.mon.a.smithi067.stdout: L0 2/0 90.38 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 121.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-27T03:04:13.202 INFO:journalctl@ceph.mon.a.smithi067.stdout: Sum 2/0 90.38 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 121.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-27T03:04:13.203 INFO:journalctl@ceph.mon.a.smithi067.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 121.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-27T03:04:13.203 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-07-27T03:04:13.203 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** Compaction Stats [default] ** 2024-07-27T03:04:13.203 INFO:journalctl@ceph.mon.a.smithi067.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-27T03:04:13.203 INFO:journalctl@ceph.mon.a.smithi067.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-27T03:04:13.203 INFO:journalctl@ceph.mon.a.smithi067.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 121.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-27T03:04:13.203 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-07-27T03:04:13.203 INFO:journalctl@ceph.mon.a.smithi067.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-07-27T03:04:13.203 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-07-27T03:04:13.203 INFO:journalctl@ceph.mon.a.smithi067.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-27T03:04:13.203 INFO:journalctl@ceph.mon.a.smithi067.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-07-27T03:04:13.203 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-07-27T03:04:13.203 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-07-27T03:04:13.203 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-07-27T03:04:13.203 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(Keys): cumulative 0, interval 0 2024-07-27T03:04:13.204 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative compaction: 0.00 GB write, 24.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-27T03:04:13.204 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval compaction: 0.00 GB write, 24.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-27T03:04:13.204 INFO:journalctl@ceph.mon.a.smithi067.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-07-27T03:04:13.204 INFO:journalctl@ceph.mon.a.smithi067.stdout: Block cache BinnedLRUCache@0x5597bb383610#2 capacity: 512.00 MB usage: 1.31 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-07-27T03:04:13.204 INFO:journalctl@ceph.mon.a.smithi067.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-07-27T03:04:13.204 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-07-27T03:04:13.204 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** File Read Latency Histogram By Level [default] ** 2024-07-27T03:04:13.204 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: starting mon.a rank 0 at public addrs [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] at bind addrs [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:04:13.204 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: mon.a@-1(???) e1 preinit fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:04:13.204 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: mon.a@-1(???).mds e1 new map 2024-07-27T03:04:13.204 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: mon.a@-1(???).mds e1 print_map 2024-07-27T03:04:13.204 INFO:journalctl@ceph.mon.a.smithi067.stdout: e1 2024-07-27T03:04:13.204 INFO:journalctl@ceph.mon.a.smithi067.stdout: btime 2024-07-27T03:04:08:096587+0000 2024-07-27T03:04:13.204 INFO:journalctl@ceph.mon.a.smithi067.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-07-27T03:04:13.205 INFO:journalctl@ceph.mon.a.smithi067.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-27T03:04:13.205 INFO:journalctl@ceph.mon.a.smithi067.stdout: legacy client fscid: -1 2024-07-27T03:04:13.205 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-07-27T03:04:13.205 INFO:journalctl@ceph.mon.a.smithi067.stdout: No filesystems configured 2024-07-27T03:04:13.205 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-07-27T03:04:13.205 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-27T03:04:13.205 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-27T03:04:13.205 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-07-27T03:04:13.205 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-07-27T03:04:13.205 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 bash[32795]: 23a97e917213294314c4e6b62ead3c7165eb08c17308bf9c3d514b912dacad6f 2024-07-27T03:04:13.205 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 systemd[1]: Started Ceph mon.a for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:04:13.205 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: mon.a is new leader, mons a in quorum (ranks 0) 2024-07-27T03:04:13.205 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: monmap epoch 1 2024-07-27T03:04:13.205 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:04:13.206 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: last_changed 2024-07-27T03:04:05.157867+0000 2024-07-27T03:04:13.206 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: created 2024-07-27T03:04:05.157867+0000 2024-07-27T03:04:13.206 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: min_mon_release 19 (squid) 2024-07-27T03:04:13.206 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: election_strategy: 1 2024-07-27T03:04:13.206 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-07-27T03:04:13.206 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: fsmap 2024-07-27T03:04:13.206 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: osdmap e1: 0 total, 0 up, 0 in 2024-07-27T03:04:13.206 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:13 smithi067 ceph-mon[32814]: mgrmap e1: no daemons active 2024-07-27T03:04:14.149 INFO:teuthology.orchestra.run.smithi067.stdout:Wrote config to /etc/ceph/ceph.conf 2024-07-27T03:04:14.162 INFO:teuthology.orchestra.run.smithi067.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-07-27T03:04:14.162 INFO:teuthology.orchestra.run.smithi067.stdout:Creating mgr... 2024-07-27T03:04:14.163 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying port 0.0.0.0:9283 ... 2024-07-27T03:04:14.163 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying port 0.0.0.0:8765 ... 2024-07-27T03:04:14.428 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mgr.x 2024-07-27T03:04:14.428 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mgr.x.service: Unit ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mgr.x.service not loaded. 2024-07-27T03:04:14.587 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968.target.wants/ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@.service. 2024-07-27T03:04:14.948 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:14 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1413957611' entity='client.admin' 2024-07-27T03:04:15.148 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-27T03:04:15.148 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-07-27T03:04:15.157 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-27T03:04:15.157 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-07-27T03:04:15.157 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-07-27T03:04:15.157 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to enable service . firewalld.service is not available 2024-07-27T03:04:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-27T03:04:15.168 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-07-27T03:04:15.178 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-27T03:04:15.178 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-07-27T03:04:15.178 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-07-27T03:04:15.178 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-07-27T03:04:15.178 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr to start... 2024-07-27T03:04:15.178 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr... 2024-07-27T03:04:15.307 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:15 smithi067 systemd[1]: Started Ceph mgr.x for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:04:15.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:15 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1979913841' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-27T03:04:16.251 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-27T03:04:16.253 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-07-27T03:04:16.253 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "c0782a00-4bc4-11ef-bcc7-c7b262605968", 2024-07-27T03:04:16.254 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2024-07-27T03:04:16.254 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-27T03:04:16.254 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-27T03:04:16.254 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-27T03:04:16.254 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-27T03:04:16.254 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-27T03:04:16.254 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-27T03:04:16.254 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2024-07-27T03:04:16.254 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-07-27T03:04:16.254 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-27T03:04:16.254 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2024-07-27T03:04:16.254 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-07-27T03:04:16.254 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-07-27T03:04:16.255 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-27T03:04:16.255 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-27T03:04:16.255 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-07-27T03:04:16.255 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-27T03:04:16.255 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-27T03:04:16.255 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-27T03:04:16.255 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-27T03:04:16.255 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-27T03:04:16.255 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-27T03:04:16.255 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-27T03:04:16.255 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-27T03:04:16.255 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-27T03:04:16.255 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-27T03:04:16.259 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-27T03:04:16.259 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-27T03:04:16.261 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-27T03:04:16.262 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-27T03:04:16.262 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-27T03:04:16.262 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-27T03:04:16.262 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-27T03:04:16.262 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-27T03:04:16.262 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-27T03:04:16.262 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-27T03:04:16.262 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-27T03:04:16.262 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-27T03:04:16.262 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-27T03:04:16.262 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "btime": "2024-07-27T03:04:08:096587+0000", 2024-07-27T03:04:16.262 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-27T03:04:16.262 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-27T03:04:16.263 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-27T03:04:16.263 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-27T03:04:16.263 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": false, 2024-07-27T03:04:16.263 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-27T03:04:16.263 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-27T03:04:16.263 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-27T03:04:16.263 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-27T03:04:16.263 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2024-07-27T03:04:16.263 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-07-27T03:04:16.263 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-27T03:04:16.263 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-27T03:04:16.263 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-27T03:04:16.263 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-27T03:04:16.264 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2024-07-27T03:04:08.097588+0000", 2024-07-27T03:04:16.264 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-27T03:04:16.264 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-27T03:04:16.264 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-27T03:04:16.264 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-07-27T03:04:16.264 INFO:teuthology.orchestra.run.smithi067.stdout:mgr not available, waiting (1/15)... 2024-07-27T03:04:18.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:18 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/4074420680' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-27T03:04:19.555 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-27T03:04:19.555 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-07-27T03:04:19.555 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "c0782a00-4bc4-11ef-bcc7-c7b262605968", 2024-07-27T03:04:19.555 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2024-07-27T03:04:19.555 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-27T03:04:19.555 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-27T03:04:19.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-27T03:04:19.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-27T03:04:19.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-27T03:04:19.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-27T03:04:19.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2024-07-27T03:04:19.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-07-27T03:04:19.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-27T03:04:19.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2024-07-27T03:04:19.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-07-27T03:04:19.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-07-27T03:04:19.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-27T03:04:19.556 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-27T03:04:19.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-07-27T03:04:19.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-27T03:04:19.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-27T03:04:19.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-27T03:04:19.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-27T03:04:19.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-27T03:04:19.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-27T03:04:19.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-27T03:04:19.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-27T03:04:19.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-27T03:04:19.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-27T03:04:19.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-27T03:04:19.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-27T03:04:19.557 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-27T03:04:19.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-27T03:04:19.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-27T03:04:19.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-27T03:04:19.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-27T03:04:19.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-27T03:04:19.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-27T03:04:19.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-27T03:04:19.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-27T03:04:19.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-27T03:04:19.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-27T03:04:19.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "btime": "2024-07-27T03:04:08:096587+0000", 2024-07-27T03:04:19.558 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-27T03:04:19.559 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-27T03:04:19.559 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-27T03:04:19.559 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-27T03:04:19.559 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": false, 2024-07-27T03:04:19.559 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-27T03:04:19.559 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-27T03:04:19.559 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-27T03:04:19.559 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-27T03:04:19.559 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2024-07-27T03:04:19.559 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-07-27T03:04:19.559 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-27T03:04:19.559 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-27T03:04:19.559 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-27T03:04:19.559 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-27T03:04:19.560 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2024-07-27T03:04:08.097588+0000", 2024-07-27T03:04:19.560 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-27T03:04:19.560 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-27T03:04:19.560 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-27T03:04:19.560 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-07-27T03:04:19.560 INFO:teuthology.orchestra.run.smithi067.stdout:mgr not available, waiting (2/15)... 2024-07-27T03:04:20.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:20 smithi067 ceph-mon[32814]: Activating manager daemon x 2024-07-27T03:04:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:20 smithi067 ceph-mon[32814]: mgrmap e2: x(active, starting, since 0.00341717s) 2024-07-27T03:04:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:20 smithi067 ceph-mon[32814]: from='mgr.14100 172.21.15.67:0/1738184867' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-27T03:04:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:20 smithi067 ceph-mon[32814]: from='mgr.14100 172.21.15.67:0/1738184867' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-27T03:04:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:20 smithi067 ceph-mon[32814]: from='mgr.14100 172.21.15.67:0/1738184867' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-27T03:04:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:20 smithi067 ceph-mon[32814]: from='mgr.14100 172.21.15.67:0/1738184867' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-27T03:04:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:20 smithi067 ceph-mon[32814]: from='mgr.14100 172.21.15.67:0/1738184867' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-27T03:04:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:20 smithi067 ceph-mon[32814]: Manager daemon x is now available 2024-07-27T03:04:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:20 smithi067 ceph-mon[32814]: from='mgr.14100 172.21.15.67:0/1738184867' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-27T03:04:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:20 smithi067 ceph-mon[32814]: from='mgr.14100 172.21.15.67:0/1738184867' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-27T03:04:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:20 smithi067 ceph-mon[32814]: from='mgr.14100 172.21.15.67:0/1738184867' entity='mgr.x' 2024-07-27T03:04:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:20 smithi067 ceph-mon[32814]: from='mgr.14100 172.21.15.67:0/1738184867' entity='mgr.x' 2024-07-27T03:04:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:20 smithi067 ceph-mon[32814]: from='mgr.14100 172.21.15.67:0/1738184867' entity='mgr.x' 2024-07-27T03:04:22.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:22 smithi067 ceph-mon[32814]: mgrmap e3: x(active, since 1.00853s) 2024-07-27T03:04:22.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:22 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3780254116' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-07-27T03:04:22.798 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-27T03:04:22.798 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-07-27T03:04:22.798 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "c0782a00-4bc4-11ef-bcc7-c7b262605968", 2024-07-27T03:04:22.798 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2024-07-27T03:04:22.798 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-07-27T03:04:22.798 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-07-27T03:04:22.798 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-07-27T03:04:22.798 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-27T03:04:22.799 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-07-27T03:04:22.799 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-07-27T03:04:22.799 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2024-07-27T03:04:22.799 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-07-27T03:04:22.799 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-07-27T03:04:22.799 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2024-07-27T03:04:22.799 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-07-27T03:04:22.799 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-07-27T03:04:22.799 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2024-07-27T03:04:22.800 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-27T03:04:22.800 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-07-27T03:04:22.800 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-07-27T03:04:22.800 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-27T03:04:22.800 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-07-27T03:04:22.800 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-27T03:04:22.800 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-07-27T03:04:22.800 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-07-27T03:04:22.800 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-07-27T03:04:22.800 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-07-27T03:04:22.800 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-07-27T03:04:22.800 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-07-27T03:04:22.800 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-27T03:04:22.800 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-07-27T03:04:22.801 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-07-27T03:04:22.801 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-07-27T03:04:22.801 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-07-27T03:04:22.801 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-07-27T03:04:22.801 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-07-27T03:04:22.801 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-07-27T03:04:22.801 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-07-27T03:04:22.801 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-07-27T03:04:22.801 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-27T03:04:22.801 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-07-27T03:04:22.802 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-27T03:04:22.802 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "btime": "2024-07-27T03:04:08:096587+0000", 2024-07-27T03:04:22.802 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-07-27T03:04:22.802 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-07-27T03:04:22.802 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-27T03:04:22.802 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-07-27T03:04:22.802 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-27T03:04:22.802 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-07-27T03:04:22.802 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2024-07-27T03:04:22.802 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2024-07-27T03:04:22.802 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2024-07-27T03:04:22.802 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2024-07-27T03:04:22.802 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-07-27T03:04:22.803 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-27T03:04:22.803 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-27T03:04:22.803 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-07-27T03:04:22.803 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-07-27T03:04:22.803 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2024-07-27T03:04:08.097588+0000", 2024-07-27T03:04:22.803 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-07-27T03:04:22.803 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-07-27T03:04:22.803 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-07-27T03:04:22.803 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-07-27T03:04:22.803 INFO:teuthology.orchestra.run.smithi067.stdout:mgr is available 2024-07-27T03:04:23.364 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:23 smithi067 ceph-mon[32814]: mgrmap e4: x(active, since 2s) 2024-07-27T03:04:23.946 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-27T03:04:23.947 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [global] 2024-07-27T03:04:23.947 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout fsid = c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:04:23.947 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-07-27T03:04:23.947 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.67:3300,v1:172.21.15.67:6789] 2024-07-27T03:04:23.947 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-07-27T03:04:23.947 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-07-27T03:04:23.947 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-07-27T03:04:23.947 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-07-27T03:04:23.947 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-27T03:04:23.947 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [mgr] 2024-07-27T03:04:23.947 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-07-27T03:04:23.947 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-07-27T03:04:23.947 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [osd] 2024-07-27T03:04:23.948 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-07-27T03:04:23.948 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-07-27T03:04:23.948 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-07-27T03:04:23.948 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-07-27T03:04:23.948 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-07-27T03:04:23.948 INFO:teuthology.orchestra.run.smithi067.stdout:Enabling cephadm module... 2024-07-27T03:04:24.415 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:24 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2012117127' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-07-27T03:04:25.598 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:25 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2647522314' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-07-27T03:04:25.599 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:25 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: ignoring --setuser ceph since I am not root 2024-07-27T03:04:25.599 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:25 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: ignoring --setgroup ceph since I am not root 2024-07-27T03:04:26.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:26 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2647522314' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-07-27T03:04:26.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:26 smithi067 ceph-mon[32814]: mgrmap e5: x(active, since 5s) 2024-07-27T03:04:27.099 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-07-27T03:04:27.100 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-07-27T03:04:27.100 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-27T03:04:27.100 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-07-27T03:04:27.100 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-07-27T03:04:27.100 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-07-27T03:04:27.100 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for the mgr to restart... 2024-07-27T03:04:27.100 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr epoch 5... 2024-07-27T03:04:27.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:27 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/420890957' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-07-27T03:04:30.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:30 smithi067 ceph-mon[32814]: Active manager daemon x restarted 2024-07-27T03:04:30.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:30 smithi067 ceph-mon[32814]: Activating manager daemon x 2024-07-27T03:04:30.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:30 smithi067 ceph-mon[32814]: osdmap e2: 0 total, 0 up, 0 in 2024-07-27T03:04:30.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:30 smithi067 ceph-mon[32814]: mgrmap e6: x(active, starting, since 0.00370622s) 2024-07-27T03:04:30.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:30 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-27T03:04:30.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:30 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-27T03:04:30.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:30 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-27T03:04:30.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:30 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-27T03:04:30.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:30 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-27T03:04:30.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:30 smithi067 ceph-mon[32814]: Manager daemon x is now available 2024-07-27T03:04:30.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:30 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' 2024-07-27T03:04:30.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:30 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' 2024-07-27T03:04:30.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:30 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:04:30.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:30 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:04:30.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:30 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-27T03:04:30.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:30 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-27T03:04:31.919 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:31 smithi067 ceph-mon[32814]: Found migration_current of "None". Setting to last migration. 2024-07-27T03:04:31.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:31 smithi067 ceph-mon[32814]: mgrmap e7: x(active, since 1.00692s) 2024-07-27T03:04:32.164 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-07-27T03:04:32.164 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-07-27T03:04:32.164 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "initialized": true 2024-07-27T03:04:32.164 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-07-27T03:04:32.164 INFO:teuthology.orchestra.run.smithi067.stdout:mgr epoch 5 is available 2024-07-27T03:04:32.164 INFO:teuthology.orchestra.run.smithi067.stdout:Setting orchestrator backend to cephadm... 2024-07-27T03:04:33.150 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:32 smithi067 ceph-mon[32814]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-07-27T03:04:33.151 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:32 smithi067 ceph-mon[32814]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-07-27T03:04:33.151 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:32 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' 2024-07-27T03:04:33.151 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:32 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' 2024-07-27T03:04:33.151 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:32 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' 2024-07-27T03:04:33.151 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:32 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:04:33.410 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: 2024-07-27T03:04:33.269+0000 7ff9e1b93640 -1 log_channel(cephadm) log [ERR] : [27/Jul/2024:03:04:33] ENGINE Error in HTTPServer.serve 2024-07-27T03:04:33.410 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: Traceback (most recent call last): 2024-07-27T03:04:33.410 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-27T03:04:33.410 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: self._connections.run(self.expiration_interval) 2024-07-27T03:04:33.411 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-27T03:04:33.411 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: self._run(expiration_interval) 2024-07-27T03:04:33.411 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-27T03:04:33.411 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: new_conn = self._from_server_socket(self.server.socket) 2024-07-27T03:04:33.411 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-27T03:04:33.411 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-27T03:04:33.411 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-27T03:04:33.411 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: s = self.context.wrap_socket( 2024-07-27T03:04:33.411 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-27T03:04:33.411 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: return self.sslsocket_class._create( 2024-07-27T03:04:33.411 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-27T03:04:33.411 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: self.do_handshake() 2024-07-27T03:04:33.411 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-27T03:04:33.412 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: self._sslobj.do_handshake() 2024-07-27T03:04:33.412 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-27T03:04:33.412 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: 2024-07-27T03:04:33.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-mon[32814]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:04:33.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-mon[32814]: mgrmap e8: x(active, since 2s) 2024-07-27T03:04:33.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-mon[32814]: [27/Jul/2024:03:04:33] ENGINE Bus STARTING 2024-07-27T03:04:33.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-mon[32814]: [27/Jul/2024:03:04:33] ENGINE Serving on http://172.21.15.67:8765 2024-07-27T03:04:33.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-mon[32814]: [27/Jul/2024:03:04:33] ENGINE Serving on https://172.21.15.67:7150 2024-07-27T03:04:33.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-mon[32814]: [27/Jul/2024:03:04:33] ENGINE Bus STARTED 2024-07-27T03:04:33.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:04:33.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:33 smithi067 ceph-mon[32814]: [27/Jul/2024:03:04:33] ENGINE Error in HTTPServer.serve 2024-07-27T03:04:33.970 INFO:journalctl@ceph.mon.a.smithi067.stdout: Traceback (most recent call last): 2024-07-27T03:04:33.970 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-27T03:04:33.970 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._connections.run(self.expiration_interval) 2024-07-27T03:04:33.970 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-27T03:04:33.970 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._run(expiration_interval) 2024-07-27T03:04:33.970 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-27T03:04:33.971 INFO:journalctl@ceph.mon.a.smithi067.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-27T03:04:33.971 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-27T03:04:33.971 INFO:journalctl@ceph.mon.a.smithi067.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-27T03:04:33.971 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-27T03:04:33.971 INFO:journalctl@ceph.mon.a.smithi067.stdout: s = self.context.wrap_socket( 2024-07-27T03:04:33.971 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-27T03:04:33.971 INFO:journalctl@ceph.mon.a.smithi067.stdout: return self.sslsocket_class._create( 2024-07-27T03:04:33.971 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-27T03:04:33.971 INFO:journalctl@ceph.mon.a.smithi067.stdout: self.do_handshake() 2024-07-27T03:04:33.971 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-27T03:04:33.971 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._sslobj.do_handshake() 2024-07-27T03:04:33.971 INFO:journalctl@ceph.mon.a.smithi067.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-27T03:04:34.627 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout value unchanged 2024-07-27T03:04:34.627 INFO:teuthology.orchestra.run.smithi067.stdout:Generating ssh key... 2024-07-27T03:04:34.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:34 smithi067 ceph-mon[32814]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:04:35.719 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: Generating public/private rsa key pair. 2024-07-27T03:04:35.720 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: Your identification has been saved in /tmp/tmpui_c_nei/key 2024-07-27T03:04:35.720 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: Your public key has been saved in /tmp/tmpui_c_nei/key.pub 2024-07-27T03:04:35.720 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: The key fingerprint is: 2024-07-27T03:04:35.720 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: SHA256:OL8B2KZeYn8K4xxG5OhE9IQqDKTSxnsAAMJh3Di7SXE ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:04:35.720 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: The key's randomart image is: 2024-07-27T03:04:35.720 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: +---[RSA 3072]----+ 2024-07-27T03:04:35.720 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: |@+=.. | 2024-07-27T03:04:35.720 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: |=@.E | 2024-07-27T03:04:35.720 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: |=.@ o | 2024-07-27T03:04:35.720 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: |+* * o . | 2024-07-27T03:04:35.720 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: |o * = * S | 2024-07-27T03:04:35.721 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: | = o o + | 2024-07-27T03:04:35.721 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: | . O . o | 2024-07-27T03:04:35.721 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: | * B .o | 2024-07-27T03:04:35.721 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: | + oo. | 2024-07-27T03:04:35.721 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: +----[SHA256]-----+ 2024-07-27T03:04:36.097 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:35 smithi067 ceph-mon[32814]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:04:36.097 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:35 smithi067 ceph-mon[32814]: Generating ssh key... 2024-07-27T03:04:36.097 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:35 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' 2024-07-27T03:04:36.097 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:35 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' 2024-07-27T03:04:37.088 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjkiyZb2G8AUpbvgw3nkCiu9dhONayCphQo03gKUTIjfmjutJF0pM68NDkrCkSgEjNtu/WzvXkDXX3ZrJ7IGTB4p7/ZZaccB//2XHGO96SqweSxvooNhuZCflyv1/lzq9uXKez9myT66W9TVapwPJatYblhxN6v3uCAQRbupnzpmhrtuwZBAWU523P9gBQ4K2dYUr/YxmwqhXbQaFxZE4HsM95P+f/qiU80W4zVDkXEVNSpUGe8ASrdmmGFO5qoGuBS/VvgYiCkYzJG8FhaB4nwj2EyrGRtczWaeLNxcuZANUFxIuP2dbhYhsvdkabpvxYgvQdFvmX5EHc0q9k8S2YlHMMn2JpKDCqiVF2QQgRVilPRHN3nKd5HwH+CmWSAEZr8ne63dL7IFAjibo/viQ02p29osjGjSGPJhlvIeI8x1J115qSElbwSMT6rmAGvLLmxaszZZNgZ5pp0WwrMPGT4UOsMUFvlAuug3jxpbw4WlId41ZAnW0kHZBdrzI/QqM= ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:04:37.089 INFO:teuthology.orchestra.run.smithi067.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-07-27T03:04:37.089 INFO:teuthology.orchestra.run.smithi067.stdout:Adding key to root@localhost authorized_keys... 2024-07-27T03:04:37.089 INFO:teuthology.orchestra.run.smithi067.stdout:Adding host smithi067... 2024-07-27T03:04:37.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:37 smithi067 ceph-mon[32814]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:04:38.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:38 smithi067 ceph-mon[32814]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi067", "addr": "172.21.15.67", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:04:39.647 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:39 smithi067 ceph-mon[32814]: Deploying cephadm binary to smithi067 2024-07-27T03:04:42.154 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Added host 'smithi067' with addr '172.21.15.67' 2024-07-27T03:04:42.154 INFO:teuthology.orchestra.run.smithi067.stdout:Deploying unmanaged mon service... 2024-07-27T03:04:42.765 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:42 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' 2024-07-27T03:04:42.765 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:42 smithi067 ceph-mon[32814]: Added host smithi067 2024-07-27T03:04:42.765 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:42 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:04:43.479 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-07-27T03:04:43.479 INFO:teuthology.orchestra.run.smithi067.stdout:Deploying unmanaged mgr service... 2024-07-27T03:04:44.079 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:43 smithi067 ceph-mon[32814]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:04:44.079 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:43 smithi067 ceph-mon[32814]: Saving service mon spec with placement count:5 2024-07-27T03:04:44.079 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:43 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' 2024-07-27T03:04:44.645 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-07-27T03:04:45.364 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:45 smithi067 ceph-mon[32814]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:04:45.364 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:45 smithi067 ceph-mon[32814]: Saving service mgr spec with placement count:2 2024-07-27T03:04:45.364 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:45 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' 2024-07-27T03:04:46.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:46 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/749378714' entity='client.admin' 2024-07-27T03:04:46.529 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:46 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' 2024-07-27T03:04:47.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:47 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' 2024-07-27T03:04:47.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:47 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3537726762' entity='client.admin' 2024-07-27T03:04:48.107 INFO:teuthology.orchestra.run.smithi067.stdout:Enabling the dashboard module... 2024-07-27T03:04:49.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:49 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' 2024-07-27T03:04:49.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:49 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3489970803' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-07-27T03:04:49.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:49 smithi067 ceph-mon[32814]: from='mgr.14118 172.21.15.67:0/561293128' entity='mgr.x' 2024-07-27T03:04:49.970 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:49 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: ignoring --setuser ceph since I am not root 2024-07-27T03:04:49.970 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:49 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: ignoring --setgroup ceph since I am not root 2024-07-27T03:04:50.697 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:50 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3489970803' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-07-27T03:04:50.697 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:50 smithi067 ceph-mon[32814]: mgrmap e9: x(active, since 19s) 2024-07-27T03:04:51.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:51 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/309087080' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-07-27T03:04:52.344 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-07-27T03:04:52.344 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-07-27T03:04:52.344 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": true, 2024-07-27T03:04:52.344 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-07-27T03:04:52.344 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-07-27T03:04:52.344 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-07-27T03:04:52.344 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for the mgr to restart... 2024-07-27T03:04:52.344 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr epoch 9... 2024-07-27T03:04:55.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:55 smithi067 ceph-mon[32814]: Active manager daemon x restarted 2024-07-27T03:04:55.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:55 smithi067 ceph-mon[32814]: Activating manager daemon x 2024-07-27T03:04:55.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:55 smithi067 ceph-mon[32814]: osdmap e3: 0 total, 0 up, 0 in 2024-07-27T03:04:55.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:55 smithi067 ceph-mon[32814]: mgrmap e10: x(active, starting, since 0.00458364s) 2024-07-27T03:04:55.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-27T03:04:55.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-07-27T03:04:55.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-07-27T03:04:55.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-07-27T03:04:55.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-07-27T03:04:55.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:55 smithi067 ceph-mon[32814]: Manager daemon x is now available 2024-07-27T03:04:55.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:04:55.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-07-27T03:04:56.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:56 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-07-27T03:04:56.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:56 smithi067 ceph-mon[32814]: mgrmap e11: x(active, since 1.00727s) 2024-07-27T03:04:56.927 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-07-27T03:04:56.928 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-07-27T03:04:56.928 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "initialized": true 2024-07-27T03:04:56.928 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-07-27T03:04:56.928 INFO:teuthology.orchestra.run.smithi067.stdout:mgr epoch 9 is available 2024-07-27T03:04:56.928 INFO:teuthology.orchestra.run.smithi067.stdout:Generating a dashboard self-signed certificate... 2024-07-27T03:04:57.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-mon[32814]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-07-27T03:04:57.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-mon[32814]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-07-27T03:04:57.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:04:57.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:04:57.970 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: 2024-07-27T03:04:57.615+0000 7f448209a640 -1 log_channel(cephadm) log [ERR] : [27/Jul/2024:03:04:57] ENGINE Error in HTTPServer.serve 2024-07-27T03:04:57.970 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: Traceback (most recent call last): 2024-07-27T03:04:57.970 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-27T03:04:57.970 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: self._connections.run(self.expiration_interval) 2024-07-27T03:04:57.970 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-27T03:04:57.971 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: self._run(expiration_interval) 2024-07-27T03:04:57.971 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-27T03:04:57.971 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: new_conn = self._from_server_socket(self.server.socket) 2024-07-27T03:04:57.971 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-27T03:04:57.971 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-27T03:04:57.971 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-27T03:04:57.971 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: s = self.context.wrap_socket( 2024-07-27T03:04:57.971 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-27T03:04:57.971 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: return self.sslsocket_class._create( 2024-07-27T03:04:57.971 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-27T03:04:57.971 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: self.do_handshake() 2024-07-27T03:04:57.971 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-27T03:04:57.971 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: self._sslobj.do_handshake() 2024-07-27T03:04:57.972 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-27T03:04:57.972 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:04:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: 2024-07-27T03:04:58.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-07-27T03:04:58.234 INFO:teuthology.orchestra.run.smithi067.stdout:Creating initial admin user... 2024-07-27T03:04:58.634 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:58 smithi067 ceph-mon[32814]: [27/Jul/2024:03:04:57] ENGINE Bus STARTING 2024-07-27T03:04:58.635 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:58 smithi067 ceph-mon[32814]: mgrmap e12: x(active, since 2s) 2024-07-27T03:04:58.635 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:58 smithi067 ceph-mon[32814]: [27/Jul/2024:03:04:57] ENGINE Serving on https://172.21.15.67:7150 2024-07-27T03:04:58.635 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:58 smithi067 ceph-mon[32814]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:04:58.635 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:58 smithi067 ceph-mon[32814]: [27/Jul/2024:03:04:57] ENGINE Error in HTTPServer.serve 2024-07-27T03:04:58.635 INFO:journalctl@ceph.mon.a.smithi067.stdout: Traceback (most recent call last): 2024-07-27T03:04:58.635 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-07-27T03:04:58.635 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._connections.run(self.expiration_interval) 2024-07-27T03:04:58.635 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-07-27T03:04:58.636 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._run(expiration_interval) 2024-07-27T03:04:58.636 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-07-27T03:04:58.636 INFO:journalctl@ceph.mon.a.smithi067.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-07-27T03:04:58.636 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-07-27T03:04:58.636 INFO:journalctl@ceph.mon.a.smithi067.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-07-27T03:04:58.636 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-07-27T03:04:58.636 INFO:journalctl@ceph.mon.a.smithi067.stdout: s = self.context.wrap_socket( 2024-07-27T03:04:58.636 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-07-27T03:04:58.636 INFO:journalctl@ceph.mon.a.smithi067.stdout: return self.sslsocket_class._create( 2024-07-27T03:04:58.637 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-07-27T03:04:58.637 INFO:journalctl@ceph.mon.a.smithi067.stdout: self.do_handshake() 2024-07-27T03:04:58.637 INFO:journalctl@ceph.mon.a.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-07-27T03:04:58.637 INFO:journalctl@ceph.mon.a.smithi067.stdout: self._sslobj.do_handshake() 2024-07-27T03:04:58.637 INFO:journalctl@ceph.mon.a.smithi067.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-07-27T03:04:58.637 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:58 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:04:58.637 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:58 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:04:59.703 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$aO7RKZeprGaFjZyA3pN5IeScXC3RcYYvNkwknWNbtS/mZIATu6Daa", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1722049499, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-07-27T03:04:59.703 INFO:teuthology.orchestra.run.smithi067.stdout:Fetching dashboard port number... 2024-07-27T03:04:59.825 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:59 smithi067 ceph-mon[32814]: [27/Jul/2024:03:04:57] ENGINE Serving on http://172.21.15.67:8765 2024-07-27T03:04:59.825 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:59 smithi067 ceph-mon[32814]: [27/Jul/2024:03:04:57] ENGINE Bus STARTED 2024-07-27T03:04:59.826 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:04:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:00.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:00 smithi067 ceph-mon[32814]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:05:00.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:00 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1407898057' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-07-27T03:05:00.994 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 8443 2024-07-27T03:05:01.005 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-07-27T03:05:01.005 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-07-27T03:05:01.014 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-07-27T03:05:01.014 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-07-27T03:05:01.014 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-07-27T03:05:01.014 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-07-27T03:05:01.015 INFO:teuthology.orchestra.run.smithi067.stdout:Ceph Dashboard is now available at: 2024-07-27T03:05:01.016 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:05:01.016 INFO:teuthology.orchestra.run.smithi067.stdout: URL: https://smithi067.front.sepia.ceph.com:8443/ 2024-07-27T03:05:01.016 INFO:teuthology.orchestra.run.smithi067.stdout: User: admin 2024-07-27T03:05:01.016 INFO:teuthology.orchestra.run.smithi067.stdout: Password: hhapoybjwq 2024-07-27T03:05:01.016 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:05:01.016 INFO:teuthology.orchestra.run.smithi067.stdout:Saving cluster configuration to /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/config directory 2024-07-27T03:05:02.276 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-07-27T03:05:02.277 INFO:teuthology.orchestra.run.smithi067.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-07-27T03:05:02.277 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:05:02.277 INFO:teuthology.orchestra.run.smithi067.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-07-27T03:05:02.277 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:05:02.277 INFO:teuthology.orchestra.run.smithi067.stdout:Or, if you are only running a single cluster on this host: 2024-07-27T03:05:02.277 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:05:02.277 INFO:teuthology.orchestra.run.smithi067.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-07-27T03:05:02.278 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:05:02.278 INFO:teuthology.orchestra.run.smithi067.stdout:Please consider enabling telemetry to help improve Ceph: 2024-07-27T03:05:02.278 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:05:02.278 INFO:teuthology.orchestra.run.smithi067.stdout: ceph telemetry on 2024-07-27T03:05:02.278 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:05:02.278 INFO:teuthology.orchestra.run.smithi067.stdout:For more information see: 2024-07-27T03:05:02.278 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:05:02.279 INFO:teuthology.orchestra.run.smithi067.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-07-27T03:05:02.279 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:05:02.279 INFO:teuthology.orchestra.run.smithi067.stdout:Bootstrap complete. 2024-07-27T03:05:02.325 INFO:tasks.cephadm:Fetching config... 2024-07-27T03:05:02.325 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-27T03:05:02.325 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-07-27T03:05:02.343 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-07-27T03:05:02.343 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-27T03:05:02.344 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-07-27T03:05:02.399 INFO:tasks.cephadm:Fetching mon keyring... 2024-07-27T03:05:02.400 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-27T03:05:02.400 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/keyring of=/dev/stdout 2024-07-27T03:05:02.467 INFO:tasks.cephadm:Fetching pub ssh key... 2024-07-27T03:05:02.467 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-27T03:05:02.467 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-07-27T03:05:02.523 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-07-27T03:05:02.523 DEBUG:teuthology.orchestra.run.smithi067:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjkiyZb2G8AUpbvgw3nkCiu9dhONayCphQo03gKUTIjfmjutJF0pM68NDkrCkSgEjNtu/WzvXkDXX3ZrJ7IGTB4p7/ZZaccB//2XHGO96SqweSxvooNhuZCflyv1/lzq9uXKez9myT66W9TVapwPJatYblhxN6v3uCAQRbupnzpmhrtuwZBAWU523P9gBQ4K2dYUr/YxmwqhXbQaFxZE4HsM95P+f/qiU80W4zVDkXEVNSpUGe8ASrdmmGFO5qoGuBS/VvgYiCkYzJG8FhaB4nwj2EyrGRtczWaeLNxcuZANUFxIuP2dbhYhsvdkabpvxYgvQdFvmX5EHc0q9k8S2YlHMMn2JpKDCqiVF2QQgRVilPRHN3nKd5HwH+CmWSAEZr8ne63dL7IFAjibo/viQ02p29osjGjSGPJhlvIeI8x1J115qSElbwSMT6rmAGvLLmxaszZZNgZ5pp0WwrMPGT4UOsMUFvlAuug3jxpbw4WlId41ZAnW0kHZBdrzI/QqM= ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-07-27T03:05:02.599 INFO:teuthology.orchestra.run.smithi067.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjkiyZb2G8AUpbvgw3nkCiu9dhONayCphQo03gKUTIjfmjutJF0pM68NDkrCkSgEjNtu/WzvXkDXX3ZrJ7IGTB4p7/ZZaccB//2XHGO96SqweSxvooNhuZCflyv1/lzq9uXKez9myT66W9TVapwPJatYblhxN6v3uCAQRbupnzpmhrtuwZBAWU523P9gBQ4K2dYUr/YxmwqhXbQaFxZE4HsM95P+f/qiU80W4zVDkXEVNSpUGe8ASrdmmGFO5qoGuBS/VvgYiCkYzJG8FhaB4nwj2EyrGRtczWaeLNxcuZANUFxIuP2dbhYhsvdkabpvxYgvQdFvmX5EHc0q9k8S2YlHMMn2JpKDCqiVF2QQgRVilPRHN3nKd5HwH+CmWSAEZr8ne63dL7IFAjibo/viQ02p29osjGjSGPJhlvIeI8x1J115qSElbwSMT6rmAGvLLmxaszZZNgZ5pp0WwrMPGT4UOsMUFvlAuug3jxpbw4WlId41ZAnW0kHZBdrzI/QqM= ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:05:02.624 DEBUG:teuthology.orchestra.run.smithi149:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjkiyZb2G8AUpbvgw3nkCiu9dhONayCphQo03gKUTIjfmjutJF0pM68NDkrCkSgEjNtu/WzvXkDXX3ZrJ7IGTB4p7/ZZaccB//2XHGO96SqweSxvooNhuZCflyv1/lzq9uXKez9myT66W9TVapwPJatYblhxN6v3uCAQRbupnzpmhrtuwZBAWU523P9gBQ4K2dYUr/YxmwqhXbQaFxZE4HsM95P+f/qiU80W4zVDkXEVNSpUGe8ASrdmmGFO5qoGuBS/VvgYiCkYzJG8FhaB4nwj2EyrGRtczWaeLNxcuZANUFxIuP2dbhYhsvdkabpvxYgvQdFvmX5EHc0q9k8S2YlHMMn2JpKDCqiVF2QQgRVilPRHN3nKd5HwH+CmWSAEZr8ne63dL7IFAjibo/viQ02p29osjGjSGPJhlvIeI8x1J115qSElbwSMT6rmAGvLLmxaszZZNgZ5pp0WwrMPGT4UOsMUFvlAuug3jxpbw4WlId41ZAnW0kHZBdrzI/QqM= ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-07-27T03:05:02.663 INFO:teuthology.orchestra.run.smithi149.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjkiyZb2G8AUpbvgw3nkCiu9dhONayCphQo03gKUTIjfmjutJF0pM68NDkrCkSgEjNtu/WzvXkDXX3ZrJ7IGTB4p7/ZZaccB//2XHGO96SqweSxvooNhuZCflyv1/lzq9uXKez9myT66W9TVapwPJatYblhxN6v3uCAQRbupnzpmhrtuwZBAWU523P9gBQ4K2dYUr/YxmwqhXbQaFxZE4HsM95P+f/qiU80W4zVDkXEVNSpUGe8ASrdmmGFO5qoGuBS/VvgYiCkYzJG8FhaB4nwj2EyrGRtczWaeLNxcuZANUFxIuP2dbhYhsvdkabpvxYgvQdFvmX5EHc0q9k8S2YlHMMn2JpKDCqiVF2QQgRVilPRHN3nKd5HwH+CmWSAEZr8ne63dL7IFAjibo/viQ02p29osjGjSGPJhlvIeI8x1J115qSElbwSMT6rmAGvLLmxaszZZNgZ5pp0WwrMPGT4UOsMUFvlAuug3jxpbw4WlId41ZAnW0kHZBdrzI/QqM= ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:05:02.675 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjkiyZb2G8AUpbvgw3nkCiu9dhONayCphQo03gKUTIjfmjutJF0pM68NDkrCkSgEjNtu/WzvXkDXX3ZrJ7IGTB4p7/ZZaccB//2XHGO96SqweSxvooNhuZCflyv1/lzq9uXKez9myT66W9TVapwPJatYblhxN6v3uCAQRbupnzpmhrtuwZBAWU523P9gBQ4K2dYUr/YxmwqhXbQaFxZE4HsM95P+f/qiU80W4zVDkXEVNSpUGe8ASrdmmGFO5qoGuBS/VvgYiCkYzJG8FhaB4nwj2EyrGRtczWaeLNxcuZANUFxIuP2dbhYhsvdkabpvxYgvQdFvmX5EHc0q9k8S2YlHMMn2JpKDCqiVF2QQgRVilPRHN3nKd5HwH+CmWSAEZr8ne63dL7IFAjibo/viQ02p29osjGjSGPJhlvIeI8x1J115qSElbwSMT6rmAGvLLmxaszZZNgZ5pp0WwrMPGT4UOsMUFvlAuug3jxpbw4WlId41ZAnW0kHZBdrzI/QqM= ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-07-27T03:05:02.716 INFO:teuthology.orchestra.run.smithi175.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjkiyZb2G8AUpbvgw3nkCiu9dhONayCphQo03gKUTIjfmjutJF0pM68NDkrCkSgEjNtu/WzvXkDXX3ZrJ7IGTB4p7/ZZaccB//2XHGO96SqweSxvooNhuZCflyv1/lzq9uXKez9myT66W9TVapwPJatYblhxN6v3uCAQRbupnzpmhrtuwZBAWU523P9gBQ4K2dYUr/YxmwqhXbQaFxZE4HsM95P+f/qiU80W4zVDkXEVNSpUGe8ASrdmmGFO5qoGuBS/VvgYiCkYzJG8FhaB4nwj2EyrGRtczWaeLNxcuZANUFxIuP2dbhYhsvdkabpvxYgvQdFvmX5EHc0q9k8S2YlHMMn2JpKDCqiVF2QQgRVilPRHN3nKd5HwH+CmWSAEZr8ne63dL7IFAjibo/viQ02p29osjGjSGPJhlvIeI8x1J115qSElbwSMT6rmAGvLLmxaszZZNgZ5pp0WwrMPGT4UOsMUFvlAuug3jxpbw4WlId41ZAnW0kHZBdrzI/QqM= ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:05:02.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:02 smithi067 ceph-mon[32814]: mgrmap e13: x(active, since 6s) 2024-07-27T03:05:02.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:02 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1490427078' entity='client.admin' 2024-07-27T03:05:02.728 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-07-27T03:05:02.994 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:05:05.459 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:05 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:05.459 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:05 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:05.460 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:05 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:05:05.460 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:05 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:05.460 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:05 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:05:05.460 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:05 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:06.575 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:06 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/695242926' entity='client.admin' 2024-07-27T03:05:06.575 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:05:06.575 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:06.754 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-07-27T03:05:06.755 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-07-27T03:05:07.017 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:05:09.290 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi149 2024-07-27T03:05:09.290 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-07-27T03:05:09.290 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.conf 2024-07-27T03:05:09.308 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-07-27T03:05:09.308 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-27T03:05:09.366 INFO:tasks.cephadm:Adding host smithi149 to orchestrator... 2024-07-27T03:05:09.366 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph orch host add smithi149 2024-07-27T03:05:09.641 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:05:09.856 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:09 smithi067 ceph-mon[32814]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:05:09.856 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:09.856 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:05:09.856 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:09.856 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:05:09.856 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:09 smithi067 ceph-mon[32814]: Updating smithi067:/etc/ceph/ceph.conf 2024-07-27T03:05:09.856 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:09.856 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:09.856 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:10.876 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:10 smithi067 ceph-mon[32814]: Updating smithi067:/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/config/ceph.conf 2024-07-27T03:05:10.876 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:10 smithi067 ceph-mon[32814]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-07-27T03:05:10.876 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:10 smithi067 ceph-mon[32814]: Updating smithi067:/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/config/ceph.client.admin.keyring 2024-07-27T03:05:12.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:11 smithi067 ceph-mon[32814]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:05:13.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:12 smithi067 ceph-mon[32814]: Deploying cephadm binary to smithi149 2024-07-27T03:05:15.079 INFO:teuthology.orchestra.run.smithi067.stdout:Added host 'smithi149' with addr '172.21.15.149' 2024-07-27T03:05:15.684 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph orch host ls --format=json 2024-07-27T03:05:15.948 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:05:16.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:16 smithi067 ceph-mon[32814]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:16.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:16 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:16.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:16 smithi067 ceph-mon[32814]: Added host smithi149 2024-07-27T03:05:16.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:16 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:05:16.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:16 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:16.384 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:16 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:17.654 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:05:17.655 INFO:teuthology.orchestra.run.smithi067.stdout:[{"addr": "172.21.15.67", "hostname": "smithi067", "labels": [], "status": ""}, {"addr": "172.21.15.149", "hostname": "smithi149", "labels": [], "status": ""}] 2024-07-27T03:05:17.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:17 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:18.269 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi175 2024-07-27T03:05:18.269 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-27T03:05:18.269 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/etc/ceph/ceph.conf 2024-07-27T03:05:18.288 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-27T03:05:18.288 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-27T03:05:18.346 INFO:tasks.cephadm:Adding host smithi175 to orchestrator... 2024-07-27T03:05:18.346 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph orch host add smithi175 2024-07-27T03:05:18.615 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:05:18.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:18 smithi067 ceph-mon[32814]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:18.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:18 smithi067 ceph-mon[32814]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:05:20.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:20 smithi067 ceph-mon[32814]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:20 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:20 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:20 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:20 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:20 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:05:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:20 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:20 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:05:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:20 smithi067 ceph-mon[32814]: Updating smithi149:/etc/ceph/ceph.conf 2024-07-27T03:05:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:20 smithi067 ceph-mon[32814]: Updating smithi149:/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/config/ceph.conf 2024-07-27T03:05:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:20 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:20 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:20 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:21.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:21 smithi067 ceph-mon[32814]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-07-27T03:05:21.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:21 smithi067 ceph-mon[32814]: Updating smithi149:/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/config/ceph.client.admin.keyring 2024-07-27T03:05:21.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:21 smithi067 ceph-mon[32814]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi175", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:05:22.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:22 smithi067 ceph-mon[32814]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:22.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:22 smithi067 ceph-mon[32814]: Deploying cephadm binary to smithi175 2024-07-27T03:05:24.166 INFO:teuthology.orchestra.run.smithi067.stdout:Added host 'smithi175' with addr '172.21.15.175' 2024-07-27T03:05:24.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:24 smithi067 ceph-mon[32814]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:24.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:24 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:24.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:24 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:05:24.926 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph orch host ls --format=json 2024-07-27T03:05:25.185 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:05:25.600 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:25 smithi067 ceph-mon[32814]: Added host smithi175 2024-07-27T03:05:25.600 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:25 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:25.600 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:25 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:26.587 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:26 smithi067 ceph-mon[32814]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:26.587 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:26 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:27.073 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:05:27.074 INFO:teuthology.orchestra.run.smithi067.stdout:[{"addr": "172.21.15.67", "hostname": "smithi067", "labels": [], "status": ""}, {"addr": "172.21.15.149", "hostname": "smithi149", "labels": [], "status": ""}, {"addr": "172.21.15.175", "hostname": "smithi175", "labels": [], "status": ""}] 2024-07-27T03:05:27.748 INFO:tasks.cephadm:Setting crush tunables to default 2024-07-27T03:05:27.748 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd crush tunables default 2024-07-27T03:05:28.010 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:05:28.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:28 smithi067 ceph-mon[32814]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:28.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:28 smithi067 ceph-mon[32814]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:05:28.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:28 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:28.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:28 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:28.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:28 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:28.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:28 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:28.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:28 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi175", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:05:28.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:28 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:28.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:28 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:05:29.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:29 smithi067 ceph-mon[32814]: Updating smithi175:/etc/ceph/ceph.conf 2024-07-27T03:05:29.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:29 smithi067 ceph-mon[32814]: Updating smithi175:/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/config/ceph.conf 2024-07-27T03:05:29.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:29 smithi067 ceph-mon[32814]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-07-27T03:05:29.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:29 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:29.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:29 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:29.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:29 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:30.521 INFO:teuthology.orchestra.run.smithi067.stderr:adjusted tunables profile to default 2024-07-27T03:05:30.836 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:30 smithi067 ceph-mon[32814]: Updating smithi175:/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/config/ceph.client.admin.keyring 2024-07-27T03:05:30.836 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:30 smithi067 ceph-mon[32814]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:30.836 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:30 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/89488623' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-07-27T03:05:31.115 INFO:tasks.cephadm:Adding mon.a on smithi067 2024-07-27T03:05:31.115 INFO:tasks.cephadm:Adding mon.b on smithi149 2024-07-27T03:05:31.116 INFO:tasks.cephadm:Adding mon.c on smithi175 2024-07-27T03:05:31.116 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph orch apply mon '3;smithi067:172.21.15.67=a;smithi149:172.21.15.149=b;smithi175:172.21.15.175=c' 2024-07-27T03:05:31.365 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/config/ceph.conf 2024-07-27T03:05:31.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:31 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/89488623' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-07-27T03:05:31.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:31 smithi067 ceph-mon[32814]: osdmap e4: 0 total, 0 up, 0 in 2024-07-27T03:05:32.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:32 smithi067 ceph-mon[32814]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:33.026 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled mon update... 2024-07-27T03:05:33.641 DEBUG:teuthology.orchestra.run.smithi149:mon.b> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mon.b.service 2024-07-27T03:05:33.644 DEBUG:teuthology.orchestra.run.smithi175:mon.c> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mon.c.service 2024-07-27T03:05:33.646 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-07-27T03:05:33.646 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph mon dump -f json 2024-07-27T03:05:34.271 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/config/ceph.conf 2024-07-27T03:05:34.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:34 smithi067 ceph-mon[32814]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:34.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:34 smithi067 ceph-mon[32814]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi067:172.21.15.67=a;smithi149:172.21.15.149=b;smithi175:172.21.15.175=c", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:05:34.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:34 smithi067 ceph-mon[32814]: Saving service mon spec with placement smithi067:172.21.15.67=a;smithi149:172.21.15.149=b;smithi175:172.21.15.175=c;count:3 2024-07-27T03:05:34.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:34.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:05:34.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:34.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:05:34.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:34.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-27T03:05:34.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:34.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:34 smithi067 ceph-mon[32814]: Deploying daemon mon.c on smithi175 2024-07-27T03:05:36.212 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-27T03:05:36.212 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"c0782a00-4bc4-11ef-bcc7-c7b262605968","modified":"2024-07-27T03:04:05.157867Z","created":"2024-07-27T03:04:05.157867Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-07-27T03:05:36.213 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2024-07-27T03:05:36.471 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:36 smithi067 ceph-mon[32814]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:37.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:37 smithi067 ceph-mon[32814]: from='client.? 172.21.15.175:0/4255795118' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-27T03:05:38.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 podman[37105]: 2024-07-27 03:05:38.297744367 +0000 UTC m=+0.118919211 container create f9196a2808bcb3837bbb7580848001c39e79464a83850e0ebe482cfad09ac90b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_BRANCH=HEAD, ceph=True) 2024-07-27T03:05:38.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:38 smithi067 ceph-mon[32814]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:38.671 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-07-27T03:05:38.672 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph mon dump -f json 2024-07-27T03:05:38.695 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 podman[37105]: 2024-07-27 03:05:38.491470371 +0000 UTC m=+0.312645162 container init f9196a2808bcb3837bbb7580848001c39e79464a83850e0ebe482cfad09ac90b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-c, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-27T03:05:38.695 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 podman[37105]: 2024-07-27 03:05:38.49446624 +0000 UTC m=+0.315641016 container start f9196a2808bcb3837bbb7580848001c39e79464a83850e0ebe482cfad09ac90b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-07-27T03:05:38.695 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: set uid:gid to 167:167 (ceph:ceph) 2024-07-27T03:05:38.695 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: ceph version 19.0.0-5358-gaf8fdbf0 (af8fdbf079811fe158ce62927bd145c35b74d4b8) squid (dev), process ceph-mon, pid 2 2024-07-27T03:05:38.695 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: pidfile_write: ignore empty --pid-file 2024-07-27T03:05:38.695 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: load: jerasure load: lrc 2024-07-27T03:05:38.696 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: RocksDB version: 7.9.2 2024-07-27T03:05:38.696 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Git sha 0 2024-07-27T03:05:38.696 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Compile date 2024-07-26 20:49:32 2024-07-27T03:05:38.696 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: DB SUMMARY 2024-07-27T03:05:38.696 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: DB Session ID: MY9744V2TQT13Q7ECUTK 2024-07-27T03:05:38.696 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: CURRENT file: CURRENT 2024-07-27T03:05:38.696 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: IDENTITY file: IDENTITY 2024-07-27T03:05:38.696 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-07-27T03:05:38.696 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-07-27T03:05:38.696 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-07-27T03:05:38.696 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.error_if_exists: 0 2024-07-27T03:05:38.696 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.create_if_missing: 0 2024-07-27T03:05:38.696 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.paranoid_checks: 1 2024-07-27T03:05:38.697 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.flush_verify_memtable_count: 1 2024-07-27T03:05:38.697 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-07-27T03:05:38.697 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-07-27T03:05:38.697 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.env: 0x560d1ea9b280 2024-07-27T03:05:38.697 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.fs: PosixFileSystem 2024-07-27T03:05:38.697 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.info_log: 0x560d235df120 2024-07-27T03:05:38.697 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_file_opening_threads: 16 2024-07-27T03:05:38.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.statistics: (nil) 2024-07-27T03:05:38.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.use_fsync: 0 2024-07-27T03:05:38.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_log_file_size: 0 2024-07-27T03:05:38.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-07-27T03:05:38.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.log_file_time_to_roll: 0 2024-07-27T03:05:38.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.keep_log_file_num: 1000 2024-07-27T03:05:38.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.recycle_log_file_num: 0 2024-07-27T03:05:38.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.allow_fallocate: 1 2024-07-27T03:05:38.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.allow_mmap_reads: 0 2024-07-27T03:05:38.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.allow_mmap_writes: 0 2024-07-27T03:05:38.699 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.use_direct_reads: 0 2024-07-27T03:05:38.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-07-27T03:05:38.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.create_missing_column_families: 0 2024-07-27T03:05:38.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.db_log_dir: 2024-07-27T03:05:38.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.wal_dir: 2024-07-27T03:05:38.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.table_cache_numshardbits: 6 2024-07-27T03:05:38.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.WAL_ttl_seconds: 0 2024-07-27T03:05:38.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.WAL_size_limit_MB: 0 2024-07-27T03:05:38.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-07-27T03:05:38.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-07-27T03:05:38.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.is_fd_close_on_exec: 1 2024-07-27T03:05:38.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.advise_random_on_open: 1 2024-07-27T03:05:38.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.db_write_buffer_size: 0 2024-07-27T03:05:38.700 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.write_buffer_manager: 0x560d235e35e0 2024-07-27T03:05:38.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-07-27T03:05:38.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-07-27T03:05:38.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.use_adaptive_mutex: 0 2024-07-27T03:05:38.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.rate_limiter: (nil) 2024-07-27T03:05:38.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-07-27T03:05:38.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.wal_recovery_mode: 2 2024-07-27T03:05:38.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.enable_thread_tracking: 0 2024-07-27T03:05:38.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.enable_pipelined_write: 0 2024-07-27T03:05:38.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.unordered_write: 0 2024-07-27T03:05:38.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-07-27T03:05:38.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-07-27T03:05:38.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-07-27T03:05:38.701 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-07-27T03:05:38.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.row_cache: None 2024-07-27T03:05:38.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.wal_filter: None 2024-07-27T03:05:38.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-07-27T03:05:38.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.allow_ingest_behind: 0 2024-07-27T03:05:38.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.two_write_queues: 0 2024-07-27T03:05:38.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.manual_wal_flush: 0 2024-07-27T03:05:38.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.wal_compression: 0 2024-07-27T03:05:38.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.atomic_flush: 0 2024-07-27T03:05:38.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-07-27T03:05:38.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.persist_stats_to_disk: 0 2024-07-27T03:05:38.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.write_dbid_to_manifest: 0 2024-07-27T03:05:38.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.log_readahead_size: 0 2024-07-27T03:05:38.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-07-27T03:05:38.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.best_efforts_recovery: 0 2024-07-27T03:05:38.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-07-27T03:05:38.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-07-27T03:05:38.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.allow_data_in_errors: 0 2024-07-27T03:05:38.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.db_host_id: __hostname__ 2024-07-27T03:05:38.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.enforce_single_del_contracts: true 2024-07-27T03:05:38.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_background_jobs: 2 2024-07-27T03:05:38.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_background_compactions: -1 2024-07-27T03:05:38.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_subcompactions: 1 2024-07-27T03:05:38.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-07-27T03:05:38.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-07-27T03:05:38.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.delayed_write_rate : 16777216 2024-07-27T03:05:38.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_total_wal_size: 0 2024-07-27T03:05:38.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-07-27T03:05:38.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.stats_dump_period_sec: 600 2024-07-27T03:05:38.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.stats_persist_period_sec: 600 2024-07-27T03:05:38.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-07-27T03:05:38.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_open_files: -1 2024-07-27T03:05:38.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.bytes_per_sync: 0 2024-07-27T03:05:38.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.wal_bytes_per_sync: 0 2024-07-27T03:05:38.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.strict_bytes_per_sync: 0 2024-07-27T03:05:38.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compaction_readahead_size: 0 2024-07-27T03:05:38.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_background_flushes: -1 2024-07-27T03:05:38.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Compression algorithms supported: 2024-07-27T03:05:38.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: kZSTD supported: 0 2024-07-27T03:05:38.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: kXpressCompression supported: 0 2024-07-27T03:05:38.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: kBZip2Compression supported: 0 2024-07-27T03:05:38.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-07-27T03:05:38.704 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: kLZ4Compression supported: 1 2024-07-27T03:05:38.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: kZlibCompression supported: 1 2024-07-27T03:05:38.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: kLZ4HCCompression supported: 1 2024-07-27T03:05:38.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: kSnappyCompression supported: 1 2024-07-27T03:05:38.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Fast CRC32 supported: Supported on x86 2024-07-27T03:05:38.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: DMutex implementation: pthread_mutex_t 2024-07-27T03:05:38.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-07-27T03:05:38.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-07-27T03:05:38.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-07-27T03:05:38.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.merge_operator: 2024-07-27T03:05:38.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compaction_filter: None 2024-07-27T03:05:38.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compaction_filter_factory: None 2024-07-27T03:05:38.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.sst_partitioner_factory: None 2024-07-27T03:05:38.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.memtable_factory: SkipListFactory 2024-07-27T03:05:38.705 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.table_factory: BlockBasedTable 2024-07-27T03:05:38.706 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560d235ded40) 2024-07-27T03:05:38.706 INFO:journalctl@ceph.mon.c.smithi175.stdout: cache_index_and_filter_blocks: 1 2024-07-27T03:05:38.706 INFO:journalctl@ceph.mon.c.smithi175.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-07-27T03:05:38.706 INFO:journalctl@ceph.mon.c.smithi175.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-07-27T03:05:38.706 INFO:journalctl@ceph.mon.c.smithi175.stdout: pin_top_level_index_and_filter: 1 2024-07-27T03:05:38.706 INFO:journalctl@ceph.mon.c.smithi175.stdout: index_type: 0 2024-07-27T03:05:38.706 INFO:journalctl@ceph.mon.c.smithi175.stdout: data_block_index_type: 0 2024-07-27T03:05:38.706 INFO:journalctl@ceph.mon.c.smithi175.stdout: index_shortening: 1 2024-07-27T03:05:38.706 INFO:journalctl@ceph.mon.c.smithi175.stdout: data_block_hash_table_util_ratio: 0.750000 2024-07-27T03:05:38.706 INFO:journalctl@ceph.mon.c.smithi175.stdout: checksum: 4 2024-07-27T03:05:38.706 INFO:journalctl@ceph.mon.c.smithi175.stdout: no_block_cache: 0 2024-07-27T03:05:38.706 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_cache: 0x560d1f5a3610 2024-07-27T03:05:38.706 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_cache_name: BinnedLRUCache 2024-07-27T03:05:38.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_cache_options: 2024-07-27T03:05:38.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: capacity : 536870912 2024-07-27T03:05:38.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: num_shard_bits : 4 2024-07-27T03:05:38.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: strict_capacity_limit : 0 2024-07-27T03:05:38.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: high_pri_pool_ratio: 0.000 2024-07-27T03:05:38.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_cache_compressed: (nil) 2024-07-27T03:05:38.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: persistent_cache: (nil) 2024-07-27T03:05:38.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_size: 4096 2024-07-27T03:05:38.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_size_deviation: 10 2024-07-27T03:05:38.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_restart_interval: 16 2024-07-27T03:05:38.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: index_block_restart_interval: 1 2024-07-27T03:05:38.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: metadata_block_size: 4096 2024-07-27T03:05:38.707 INFO:journalctl@ceph.mon.c.smithi175.stdout: partition_filters: 0 2024-07-27T03:05:38.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: use_delta_encoding: 1 2024-07-27T03:05:38.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: filter_policy: bloomfilter 2024-07-27T03:05:38.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: whole_key_filtering: 1 2024-07-27T03:05:38.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: verify_compression: 0 2024-07-27T03:05:38.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: read_amp_bytes_per_bit: 0 2024-07-27T03:05:38.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: format_version: 5 2024-07-27T03:05:38.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: enable_index_compression: 1 2024-07-27T03:05:38.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: block_align: 0 2024-07-27T03:05:38.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: max_auto_readahead_size: 262144 2024-07-27T03:05:38.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: prepopulate_block_cache: 0 2024-07-27T03:05:38.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: initial_auto_readahead_size: 8192 2024-07-27T03:05:38.708 INFO:journalctl@ceph.mon.c.smithi175.stdout: num_file_reads_for_auto_readahead: 2 2024-07-27T03:05:38.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.write_buffer_size: 33554432 2024-07-27T03:05:38.708 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_write_buffer_number: 2 2024-07-27T03:05:38.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compression: NoCompression 2024-07-27T03:05:38.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.bottommost_compression: Disabled 2024-07-27T03:05:38.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.prefix_extractor: nullptr 2024-07-27T03:05:38.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-07-27T03:05:38.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.num_levels: 7 2024-07-27T03:05:38.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-07-27T03:05:38.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-07-27T03:05:38.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-07-27T03:05:38.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-07-27T03:05:38.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-07-27T03:05:38.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-07-27T03:05:38.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-07-27T03:05:38.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-07-27T03:05:38.709 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-07-27T03:05:38.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-07-27T03:05:38.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-07-27T03:05:38.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-07-27T03:05:38.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compression_opts.window_bits: -14 2024-07-27T03:05:38.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compression_opts.level: 32767 2024-07-27T03:05:38.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compression_opts.strategy: 0 2024-07-27T03:05:38.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-07-27T03:05:38.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-07-27T03:05:38.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-07-27T03:05:38.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-07-27T03:05:38.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compression_opts.enabled: false 2024-07-27T03:05:38.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-07-27T03:05:38.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-07-27T03:05:38.710 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-07-27T03:05:38.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-07-27T03:05:38.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.target_file_size_base: 67108864 2024-07-27T03:05:38.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.target_file_size_multiplier: 1 2024-07-27T03:05:38.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-07-27T03:05:38.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-07-27T03:05:38.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-07-27T03:05:38.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-07-27T03:05:38.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-07-27T03:05:38.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-07-27T03:05:38.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-07-27T03:05:38.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-07-27T03:05:38.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-07-27T03:05:38.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-07-27T03:05:38.711 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-07-27T03:05:38.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-07-27T03:05:38.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-07-27T03:05:38.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.arena_block_size: 1048576 2024-07-27T03:05:38.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-07-27T03:05:38.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-07-27T03:05:38.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.disable_auto_compactions: 0 2024-07-27T03:05:38.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-07-27T03:05:38.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-07-27T03:05:38.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-07-27T03:05:38.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-07-27T03:05:38.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-07-27T03:05:38.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-07-27T03:05:38.712 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-07-27T03:05:38.713 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-07-27T03:05:38.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-07-27T03:05:38.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-07-27T03:05:38.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-07-27T03:05:38.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.inplace_update_support: 0 2024-07-27T03:05:38.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.inplace_update_num_locks: 10000 2024-07-27T03:05:38.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-07-27T03:05:38.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-07-27T03:05:38.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.memtable_huge_page_size: 0 2024-07-27T03:05:38.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.bloom_locality: 0 2024-07-27T03:05:38.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.max_successive_merges: 0 2024-07-27T03:05:38.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.optimize_filters_for_hits: 0 2024-07-27T03:05:38.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.paranoid_file_checks: 0 2024-07-27T03:05:38.714 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.force_consistency_checks: 1 2024-07-27T03:05:38.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.report_bg_io_stats: 0 2024-07-27T03:05:38.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.ttl: 2592000 2024-07-27T03:05:38.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.periodic_compaction_seconds: 0 2024-07-27T03:05:38.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-07-27T03:05:38.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-07-27T03:05:38.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.enable_blob_files: false 2024-07-27T03:05:38.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.min_blob_size: 0 2024-07-27T03:05:38.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.blob_file_size: 268435456 2024-07-27T03:05:38.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.blob_compression_type: NoCompression 2024-07-27T03:05:38.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.enable_blob_garbage_collection: false 2024-07-27T03:05:38.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-07-27T03:05:38.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-07-27T03:05:38.715 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-07-27T03:05:38.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.blob_file_starting_level: 0 2024-07-27T03:05:38.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-07-27T03:05:38.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-07-27T03:05:38.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-07-27T03:05:38.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7329116c-f21d-4ddc-9dc5-4eaee8346efb 2024-07-27T03:05:38.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722049538531928, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-07-27T03:05:38.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-07-27T03:05:38.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722049538532450, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1722049538, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7329116c-f21d-4ddc-9dc5-4eaee8346efb", "db_session_id": "MY9744V2TQT13Q7ECUTK", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-07-27T03:05:38.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722049538532571, "job": 1, "event": "recovery_finished"} 2024-07-27T03:05:38.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-07-27T03:05:38.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-07-27T03:05:38.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560d23600a80 2024-07-27T03:05:38.716 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: DB pointer 0x560d243a0000 2024-07-27T03:05:38.717 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-07-27T03:05:38.717 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-07-27T03:05:38.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: ** DB Stats ** 2024-07-27T03:05:38.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-27T03:05:38.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-07-27T03:05:38.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-27T03:05:38.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-27T03:05:38.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-07-27T03:05:38.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-27T03:05:38.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-27T03:05:38.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-07-27T03:05:38.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: ** Compaction Stats [default] ** 2024-07-27T03:05:38.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-27T03:05:38.717 INFO:journalctl@ceph.mon.c.smithi175.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-07-27T03:05:38.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-27T03:05:38.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-27T03:05:38.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-27T03:05:38.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-07-27T03:05:38.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: ** Compaction Stats [default] ** 2024-07-27T03:05:38.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-27T03:05:38.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-27T03:05:38.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-07-27T03:05:38.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-07-27T03:05:38.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-07-27T03:05:38.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-07-27T03:05:38.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-27T03:05:38.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-07-27T03:05:38.718 INFO:journalctl@ceph.mon.c.smithi175.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-07-27T03:05:38.719 INFO:journalctl@ceph.mon.c.smithi175.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-07-27T03:05:38.719 INFO:journalctl@ceph.mon.c.smithi175.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-07-27T03:05:38.719 INFO:journalctl@ceph.mon.c.smithi175.stdout: AddFile(Keys): cumulative 0, interval 0 2024-07-27T03:05:38.719 INFO:journalctl@ceph.mon.c.smithi175.stdout: Cumulative compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-27T03:05:38.719 INFO:journalctl@ceph.mon.c.smithi175.stdout: Interval compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-27T03:05:38.719 INFO:journalctl@ceph.mon.c.smithi175.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-07-27T03:05:38.719 INFO:journalctl@ceph.mon.c.smithi175.stdout: Block cache BinnedLRUCache@0x560d1f5a3610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-07-27T03:05:38.719 INFO:journalctl@ceph.mon.c.smithi175.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-07-27T03:05:38.719 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-07-27T03:05:38.719 INFO:journalctl@ceph.mon.c.smithi175.stdout: ** File Read Latency Histogram By Level [default] ** 2024-07-27T03:05:38.719 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-07-27T03:05:38.719 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: using public_addr v2:172.21.15.175:0/0 -> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2024-07-27T03:05:38.719 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: starting mon.c rank -1 at public addrs [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] at bind addrs [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:05:38.719 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: mon.c@-1(???) e0 preinit fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:05:38.720 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: mon.c@-1(synchronizing).mds e1 new map 2024-07-27T03:05:38.720 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: mon.c@-1(synchronizing).mds e1 print_map 2024-07-27T03:05:38.720 INFO:journalctl@ceph.mon.c.smithi175.stdout: e1 2024-07-27T03:05:38.720 INFO:journalctl@ceph.mon.c.smithi175.stdout: btime 2024-07-27T03:04:08:096587+0000 2024-07-27T03:05:38.720 INFO:journalctl@ceph.mon.c.smithi175.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-07-27T03:05:38.720 INFO:journalctl@ceph.mon.c.smithi175.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-27T03:05:38.720 INFO:journalctl@ceph.mon.c.smithi175.stdout: legacy client fscid: -1 2024-07-27T03:05:38.720 INFO:journalctl@ceph.mon.c.smithi175.stdout: 2024-07-27T03:05:38.720 INFO:journalctl@ceph.mon.c.smithi175.stdout: No filesystems configured 2024-07-27T03:05:38.720 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-07-27T03:05:38.720 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-07-27T03:05:38.720 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-07-27T03:05:38.720 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-07-27T03:05:38.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-07-27T03:05:38.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-07-27T03:05:38.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-07-27T03:05:38.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-27T03:05:38.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-27T03:05:38.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-27T03:05:38.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:38.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: Deploying cephadm binary to smithi175 2024-07-27T03:05:38.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:38.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:38.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:05:38.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: Added host smithi175 2024-07-27T03:05:38.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:38.721 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:38.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:38.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:38.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:38.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:05:38.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:38.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:38.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:38.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:38.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi175", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:05:38.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:38.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:05:38.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: Updating smithi175:/etc/ceph/ceph.conf 2024-07-27T03:05:38.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: Updating smithi175:/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/config/ceph.conf 2024-07-27T03:05:38.722 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-07-27T03:05:38.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:38.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:38.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:38.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: Updating smithi175:/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/config/ceph.client.admin.keyring 2024-07-27T03:05:38.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:38.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/89488623' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-07-27T03:05:38.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/89488623' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-07-27T03:05:38.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: osdmap e4: 0 total, 0 up, 0 in 2024-07-27T03:05:38.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:38.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:38.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi067:172.21.15.67=a;smithi149:172.21.15.149=b;smithi175:172.21.15.175=c", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:05:38.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: Saving service mon spec with placement smithi067:172.21.15.67=a;smithi149:172.21.15.149=b;smithi175:172.21.15.175=c;count:3 2024-07-27T03:05:38.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:38.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:05:38.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:38.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:05:38.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:38.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-27T03:05:38.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:38.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: Deploying daemon mon.c on smithi175 2024-07-27T03:05:38.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:38.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: from='client.? 172.21.15.175:0/4255795118' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-27T03:05:38.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:38.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 ceph-mon[37125]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-07-27T03:05:38.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 bash[37105]: f9196a2808bcb3837bbb7580848001c39e79464a83850e0ebe482cfad09ac90b 2024-07-27T03:05:38.724 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:38 smithi175 systemd[1]: Started Ceph mon.c for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:05:38.920 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.c/config 2024-07-27T03:05:43.892 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-27T03:05:43.892 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:43.893 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: mon.a calling monitor election 2024-07-27T03:05:43.893 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:43.893 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:43.893 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:43.893 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: mon.c calling monitor election 2024-07-27T03:05:43.893 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:43.893 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:43.893 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:43.894 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:43.894 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-27T03:05:43.894 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: monmap epoch 2 2024-07-27T03:05:43.894 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:05:43.894 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: last_changed 2024-07-27T03:05:38.561211+0000 2024-07-27T03:05:43.894 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: created 2024-07-27T03:04:05.157867+0000 2024-07-27T03:05:43.894 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: min_mon_release 19 (squid) 2024-07-27T03:05:43.894 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: election_strategy: 1 2024-07-27T03:05:43.894 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-07-27T03:05:43.894 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-07-27T03:05:43.895 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: fsmap 2024-07-27T03:05:43.895 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: osdmap e4: 0 total, 0 up, 0 in 2024-07-27T03:05:43.895 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: mgrmap e13: x(active, since 48s) 2024-07-27T03:05:43.895 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: overall HEALTH_OK 2024-07-27T03:05:43.895 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:43.895 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:43.895 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:43.895 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-27T03:05:43.895 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:43 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:44.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-27T03:05:44.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:44.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: mon.a calling monitor election 2024-07-27T03:05:44.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:44.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:44.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:44.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: mon.c calling monitor election 2024-07-27T03:05:44.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:44.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:44.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:44.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:44.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-27T03:05:44.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: monmap epoch 2 2024-07-27T03:05:44.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:05:44.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: last_changed 2024-07-27T03:05:38.561211+0000 2024-07-27T03:05:44.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: created 2024-07-27T03:04:05.157867+0000 2024-07-27T03:05:44.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: min_mon_release 19 (squid) 2024-07-27T03:05:44.127 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: election_strategy: 1 2024-07-27T03:05:44.127 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-07-27T03:05:44.127 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-07-27T03:05:44.127 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: fsmap 2024-07-27T03:05:44.127 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: osdmap e4: 0 total, 0 up, 0 in 2024-07-27T03:05:44.127 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: mgrmap e13: x(active, since 48s) 2024-07-27T03:05:44.127 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: overall HEALTH_OK 2024-07-27T03:05:44.127 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:44.127 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:44.127 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:44.127 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-27T03:05:44.127 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:43 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:44.197 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-27T03:05:44.197 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":2,"fsid":"c0782a00-4bc4-11ef-bcc7-c7b262605968","modified":"2024-07-27T03:05:38.561211Z","created":"2024-07-27T03:04:05.157867Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:3300","nonce":0},{"type":"v1","addr":"172.21.15.175:6789","nonce":0}]},"addr":"172.21.15.175:6789/0","public_addr":"172.21.15.175:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-07-27T03:05:44.197 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 2 2024-07-27T03:05:44.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:44 smithi175 ceph-mon[37125]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:44.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:44 smithi175 ceph-mon[37125]: Deploying daemon mon.b on smithi149 2024-07-27T03:05:44.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:44 smithi175 ceph-mon[37125]: from='client.? 172.21.15.175:0/526190184' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-27T03:05:44.723 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:44 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:44.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:44 smithi067 ceph-mon[32814]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:44.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:44 smithi067 ceph-mon[32814]: Deploying daemon mon.b on smithi149 2024-07-27T03:05:44.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:44 smithi067 ceph-mon[32814]: from='client.? 172.21.15.175:0/526190184' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-27T03:05:44.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:44 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:45.751 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-07-27T03:05:45.752 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph mon dump -f json 2024-07-27T03:05:46.001 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.c/config 2024-07-27T03:05:46.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:46 smithi175 ceph-mon[37125]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:46.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:46 smithi067 ceph-mon[32814]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:47.587 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: load: jerasure load: lrc 2024-07-27T03:05:48.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: RocksDB version: 7.9.2 2024-07-27T03:05:48.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Git sha 0 2024-07-27T03:05:48.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Compile date 2024-07-26 20:49:32 2024-07-27T03:05:48.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: DB SUMMARY 2024-07-27T03:05:48.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: DB Session ID: B09Q1SU87DF0ZED990VZ 2024-07-27T03:05:48.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: CURRENT file: CURRENT 2024-07-27T03:05:48.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: IDENTITY file: IDENTITY 2024-07-27T03:05:48.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-07-27T03:05:48.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-07-27T03:05:48.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-07-27T03:05:48.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.error_if_exists: 0 2024-07-27T03:05:48.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.create_if_missing: 0 2024-07-27T03:05:48.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.paranoid_checks: 1 2024-07-27T03:05:48.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.flush_verify_memtable_count: 1 2024-07-27T03:05:48.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-07-27T03:05:48.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-07-27T03:05:48.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.env: 0x5570b5c7a280 2024-07-27T03:05:48.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.fs: PosixFileSystem 2024-07-27T03:05:48.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.info_log: 0x5570babaf120 2024-07-27T03:05:48.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_file_opening_threads: 16 2024-07-27T03:05:48.003 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.statistics: (nil) 2024-07-27T03:05:48.003 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.use_fsync: 0 2024-07-27T03:05:48.003 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_log_file_size: 0 2024-07-27T03:05:48.003 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-07-27T03:05:48.003 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.log_file_time_to_roll: 0 2024-07-27T03:05:48.003 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.keep_log_file_num: 1000 2024-07-27T03:05:48.003 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.recycle_log_file_num: 0 2024-07-27T03:05:48.003 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.allow_fallocate: 1 2024-07-27T03:05:48.003 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.allow_mmap_reads: 0 2024-07-27T03:05:48.003 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.allow_mmap_writes: 0 2024-07-27T03:05:48.003 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.use_direct_reads: 0 2024-07-27T03:05:48.003 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-07-27T03:05:48.004 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.create_missing_column_families: 0 2024-07-27T03:05:48.004 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.db_log_dir: 2024-07-27T03:05:48.004 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.wal_dir: 2024-07-27T03:05:48.004 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.table_cache_numshardbits: 6 2024-07-27T03:05:48.004 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.WAL_ttl_seconds: 0 2024-07-27T03:05:48.004 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.WAL_size_limit_MB: 0 2024-07-27T03:05:48.004 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-07-27T03:05:48.006 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-07-27T03:05:48.006 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.is_fd_close_on_exec: 1 2024-07-27T03:05:48.006 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.advise_random_on_open: 1 2024-07-27T03:05:48.006 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.db_write_buffer_size: 0 2024-07-27T03:05:48.006 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.write_buffer_manager: 0x5570babb35e0 2024-07-27T03:05:48.006 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-07-27T03:05:48.006 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-07-27T03:05:48.006 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.use_adaptive_mutex: 0 2024-07-27T03:05:48.007 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.rate_limiter: (nil) 2024-07-27T03:05:48.007 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-07-27T03:05:48.007 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.wal_recovery_mode: 2 2024-07-27T03:05:48.007 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.enable_thread_tracking: 0 2024-07-27T03:05:48.007 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.enable_pipelined_write: 0 2024-07-27T03:05:48.007 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.unordered_write: 0 2024-07-27T03:05:48.007 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-07-27T03:05:48.007 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-07-27T03:05:48.007 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-07-27T03:05:48.007 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-07-27T03:05:48.007 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.row_cache: None 2024-07-27T03:05:48.007 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.wal_filter: None 2024-07-27T03:05:48.007 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-07-27T03:05:48.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.allow_ingest_behind: 0 2024-07-27T03:05:48.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.two_write_queues: 0 2024-07-27T03:05:48.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.manual_wal_flush: 0 2024-07-27T03:05:48.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.wal_compression: 0 2024-07-27T03:05:48.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.atomic_flush: 0 2024-07-27T03:05:48.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-07-27T03:05:48.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.persist_stats_to_disk: 0 2024-07-27T03:05:48.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.write_dbid_to_manifest: 0 2024-07-27T03:05:48.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.log_readahead_size: 0 2024-07-27T03:05:48.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-07-27T03:05:48.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.best_efforts_recovery: 0 2024-07-27T03:05:48.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-07-27T03:05:48.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-07-27T03:05:48.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.allow_data_in_errors: 0 2024-07-27T03:05:48.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.db_host_id: __hostname__ 2024-07-27T03:05:48.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.enforce_single_del_contracts: true 2024-07-27T03:05:48.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_background_jobs: 2 2024-07-27T03:05:48.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_background_compactions: -1 2024-07-27T03:05:48.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_subcompactions: 1 2024-07-27T03:05:48.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-07-27T03:05:48.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-07-27T03:05:48.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.delayed_write_rate : 16777216 2024-07-27T03:05:48.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_total_wal_size: 0 2024-07-27T03:05:48.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-07-27T03:05:48.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.stats_dump_period_sec: 600 2024-07-27T03:05:48.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.stats_persist_period_sec: 600 2024-07-27T03:05:48.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-07-27T03:05:48.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_open_files: -1 2024-07-27T03:05:48.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.bytes_per_sync: 0 2024-07-27T03:05:48.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.wal_bytes_per_sync: 0 2024-07-27T03:05:48.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.strict_bytes_per_sync: 0 2024-07-27T03:05:48.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compaction_readahead_size: 0 2024-07-27T03:05:48.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_background_flushes: -1 2024-07-27T03:05:48.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Compression algorithms supported: 2024-07-27T03:05:48.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: kZSTD supported: 0 2024-07-27T03:05:48.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: kXpressCompression supported: 0 2024-07-27T03:05:48.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: kBZip2Compression supported: 0 2024-07-27T03:05:48.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-07-27T03:05:48.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: kLZ4Compression supported: 1 2024-07-27T03:05:48.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: kZlibCompression supported: 1 2024-07-27T03:05:48.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: kLZ4HCCompression supported: 1 2024-07-27T03:05:48.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: kSnappyCompression supported: 1 2024-07-27T03:05:48.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Fast CRC32 supported: Supported on x86 2024-07-27T03:05:48.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: DMutex implementation: pthread_mutex_t 2024-07-27T03:05:48.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-07-27T03:05:48.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-07-27T03:05:48.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-07-27T03:05:48.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.merge_operator: 2024-07-27T03:05:48.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compaction_filter: None 2024-07-27T03:05:48.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compaction_filter_factory: None 2024-07-27T03:05:48.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.sst_partitioner_factory: None 2024-07-27T03:05:48.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.memtable_factory: SkipListFactory 2024-07-27T03:05:48.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.table_factory: BlockBasedTable 2024-07-27T03:05:48.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5570babaed40) 2024-07-27T03:05:48.012 INFO:journalctl@ceph.mon.b.smithi149.stdout: cache_index_and_filter_blocks: 1 2024-07-27T03:05:48.012 INFO:journalctl@ceph.mon.b.smithi149.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-07-27T03:05:48.012 INFO:journalctl@ceph.mon.b.smithi149.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-07-27T03:05:48.012 INFO:journalctl@ceph.mon.b.smithi149.stdout: pin_top_level_index_and_filter: 1 2024-07-27T03:05:48.012 INFO:journalctl@ceph.mon.b.smithi149.stdout: index_type: 0 2024-07-27T03:05:48.012 INFO:journalctl@ceph.mon.b.smithi149.stdout: data_block_index_type: 0 2024-07-27T03:05:48.012 INFO:journalctl@ceph.mon.b.smithi149.stdout: index_shortening: 1 2024-07-27T03:05:48.012 INFO:journalctl@ceph.mon.b.smithi149.stdout: data_block_hash_table_util_ratio: 0.750000 2024-07-27T03:05:48.012 INFO:journalctl@ceph.mon.b.smithi149.stdout: checksum: 4 2024-07-27T03:05:48.012 INFO:journalctl@ceph.mon.b.smithi149.stdout: no_block_cache: 0 2024-07-27T03:05:48.012 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_cache: 0x5570b6b73610 2024-07-27T03:05:48.012 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_cache_name: BinnedLRUCache 2024-07-27T03:05:48.012 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_cache_options: 2024-07-27T03:05:48.013 INFO:journalctl@ceph.mon.b.smithi149.stdout: capacity : 536870912 2024-07-27T03:05:48.013 INFO:journalctl@ceph.mon.b.smithi149.stdout: num_shard_bits : 4 2024-07-27T03:05:48.013 INFO:journalctl@ceph.mon.b.smithi149.stdout: strict_capacity_limit : 0 2024-07-27T03:05:48.013 INFO:journalctl@ceph.mon.b.smithi149.stdout: high_pri_pool_ratio: 0.000 2024-07-27T03:05:48.013 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_cache_compressed: (nil) 2024-07-27T03:05:48.013 INFO:journalctl@ceph.mon.b.smithi149.stdout: persistent_cache: (nil) 2024-07-27T03:05:48.013 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_size: 4096 2024-07-27T03:05:48.013 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_size_deviation: 10 2024-07-27T03:05:48.013 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_restart_interval: 16 2024-07-27T03:05:48.013 INFO:journalctl@ceph.mon.b.smithi149.stdout: index_block_restart_interval: 1 2024-07-27T03:05:48.013 INFO:journalctl@ceph.mon.b.smithi149.stdout: metadata_block_size: 4096 2024-07-27T03:05:48.013 INFO:journalctl@ceph.mon.b.smithi149.stdout: partition_filters: 0 2024-07-27T03:05:48.013 INFO:journalctl@ceph.mon.b.smithi149.stdout: use_delta_encoding: 1 2024-07-27T03:05:48.013 INFO:journalctl@ceph.mon.b.smithi149.stdout: filter_policy: bloomfilter 2024-07-27T03:05:48.013 INFO:journalctl@ceph.mon.b.smithi149.stdout: whole_key_filtering: 1 2024-07-27T03:05:48.014 INFO:journalctl@ceph.mon.b.smithi149.stdout: verify_compression: 0 2024-07-27T03:05:48.014 INFO:journalctl@ceph.mon.b.smithi149.stdout: read_amp_bytes_per_bit: 0 2024-07-27T03:05:48.014 INFO:journalctl@ceph.mon.b.smithi149.stdout: format_version: 5 2024-07-27T03:05:48.014 INFO:journalctl@ceph.mon.b.smithi149.stdout: enable_index_compression: 1 2024-07-27T03:05:48.014 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_align: 0 2024-07-27T03:05:48.014 INFO:journalctl@ceph.mon.b.smithi149.stdout: max_auto_readahead_size: 262144 2024-07-27T03:05:48.014 INFO:journalctl@ceph.mon.b.smithi149.stdout: prepopulate_block_cache: 0 2024-07-27T03:05:48.014 INFO:journalctl@ceph.mon.b.smithi149.stdout: initial_auto_readahead_size: 8192 2024-07-27T03:05:48.014 INFO:journalctl@ceph.mon.b.smithi149.stdout: num_file_reads_for_auto_readahead: 2 2024-07-27T03:05:48.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.write_buffer_size: 33554432 2024-07-27T03:05:48.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_write_buffer_number: 2 2024-07-27T03:05:48.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compression: NoCompression 2024-07-27T03:05:48.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.bottommost_compression: Disabled 2024-07-27T03:05:48.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.prefix_extractor: nullptr 2024-07-27T03:05:48.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-07-27T03:05:48.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.num_levels: 7 2024-07-27T03:05:48.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-07-27T03:05:48.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-07-27T03:05:48.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-07-27T03:05:48.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-07-27T03:05:48.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-07-27T03:05:48.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-07-27T03:05:48.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-07-27T03:05:48.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-07-27T03:05:48.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-07-27T03:05:48.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-07-27T03:05:48.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-07-27T03:05:48.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-07-27T03:05:48.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compression_opts.window_bits: -14 2024-07-27T03:05:48.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compression_opts.level: 32767 2024-07-27T03:05:48.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compression_opts.strategy: 0 2024-07-27T03:05:48.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-07-27T03:05:48.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-07-27T03:05:48.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-07-27T03:05:48.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-07-27T03:05:48.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compression_opts.enabled: false 2024-07-27T03:05:48.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-07-27T03:05:48.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-07-27T03:05:48.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-07-27T03:05:48.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-07-27T03:05:48.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.target_file_size_base: 67108864 2024-07-27T03:05:48.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.target_file_size_multiplier: 1 2024-07-27T03:05:48.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-07-27T03:05:48.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-07-27T03:05:48.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-07-27T03:05:48.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-07-27T03:05:48.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-07-27T03:05:48.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-07-27T03:05:48.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-07-27T03:05:48.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-07-27T03:05:48.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-07-27T03:05:48.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-07-27T03:05:48.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-07-27T03:05:48.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-07-27T03:05:48.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-07-27T03:05:48.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.arena_block_size: 1048576 2024-07-27T03:05:48.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-07-27T03:05:48.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-07-27T03:05:48.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.disable_auto_compactions: 0 2024-07-27T03:05:48.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-07-27T03:05:48.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-07-27T03:05:48.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-07-27T03:05:48.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-07-27T03:05:48.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-07-27T03:05:48.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-07-27T03:05:48.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-07-27T03:05:48.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-07-27T03:05:48.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-07-27T03:05:48.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-07-27T03:05:48.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-07-27T03:05:48.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.inplace_update_support: 0 2024-07-27T03:05:48.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.inplace_update_num_locks: 10000 2024-07-27T03:05:48.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-07-27T03:05:48.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-07-27T03:05:48.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.memtable_huge_page_size: 0 2024-07-27T03:05:48.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.bloom_locality: 0 2024-07-27T03:05:48.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.max_successive_merges: 0 2024-07-27T03:05:48.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.optimize_filters_for_hits: 0 2024-07-27T03:05:48.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.paranoid_file_checks: 0 2024-07-27T03:05:48.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.force_consistency_checks: 1 2024-07-27T03:05:48.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.report_bg_io_stats: 0 2024-07-27T03:05:48.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.ttl: 2592000 2024-07-27T03:05:48.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.periodic_compaction_seconds: 0 2024-07-27T03:05:48.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-07-27T03:05:48.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-07-27T03:05:48.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.enable_blob_files: false 2024-07-27T03:05:48.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.min_blob_size: 0 2024-07-27T03:05:48.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.blob_file_size: 268435456 2024-07-27T03:05:48.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.blob_compression_type: NoCompression 2024-07-27T03:05:48.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.enable_blob_garbage_collection: false 2024-07-27T03:05:48.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-07-27T03:05:48.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-07-27T03:05:48.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-07-27T03:05:48.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.blob_file_starting_level: 0 2024-07-27T03:05:48.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-07-27T03:05:48.021 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-07-27T03:05:48.021 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-07-27T03:05:48.021 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 60564d21-3efe-4551-bd9d-03ac49cf304c 2024-07-27T03:05:48.021 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722049547586919, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-07-27T03:05:48.021 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-07-27T03:05:48.021 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722049547587391, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1722049547, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "60564d21-3efe-4551-bd9d-03ac49cf304c", "db_session_id": "B09Q1SU87DF0ZED990VZ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-07-27T03:05:48.021 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722049547587488, "job": 1, "event": "recovery_finished"} 2024-07-27T03:05:48.021 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-07-27T03:05:48.021 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-07-27T03:05:48.021 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5570babd0a80 2024-07-27T03:05:48.021 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: DB pointer 0x5570bb970000 2024-07-27T03:05:48.021 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-07-27T03:05:48.021 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-07-27T03:05:48.021 INFO:journalctl@ceph.mon.b.smithi149.stdout: ** DB Stats ** 2024-07-27T03:05:48.022 INFO:journalctl@ceph.mon.b.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-27T03:05:48.022 INFO:journalctl@ceph.mon.b.smithi149.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-07-27T03:05:48.022 INFO:journalctl@ceph.mon.b.smithi149.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-27T03:05:48.022 INFO:journalctl@ceph.mon.b.smithi149.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-27T03:05:48.022 INFO:journalctl@ceph.mon.b.smithi149.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-07-27T03:05:48.022 INFO:journalctl@ceph.mon.b.smithi149.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-07-27T03:05:48.022 INFO:journalctl@ceph.mon.b.smithi149.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-07-27T03:05:48.022 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2024-07-27T03:05:48.022 INFO:journalctl@ceph.mon.b.smithi149.stdout: ** Compaction Stats [default] ** 2024-07-27T03:05:48.022 INFO:journalctl@ceph.mon.b.smithi149.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-27T03:05:48.022 INFO:journalctl@ceph.mon.b.smithi149.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-07-27T03:05:48.022 INFO:journalctl@ceph.mon.b.smithi149.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-07-27T03:05:48.022 INFO:journalctl@ceph.mon.b.smithi149.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-07-27T03:05:48.023 INFO:journalctl@ceph.mon.b.smithi149.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-07-27T03:05:48.023 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2024-07-27T03:05:48.023 INFO:journalctl@ceph.mon.b.smithi149.stdout: ** Compaction Stats [default] ** 2024-07-27T03:05:48.023 INFO:journalctl@ceph.mon.b.smithi149.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-07-27T03:05:48.023 INFO:journalctl@ceph.mon.b.smithi149.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-07-27T03:05:48.023 INFO:journalctl@ceph.mon.b.smithi149.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-07-27T03:05:48.023 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2024-07-27T03:05:48.023 INFO:journalctl@ceph.mon.b.smithi149.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-07-27T03:05:48.023 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2024-07-27T03:05:48.023 INFO:journalctl@ceph.mon.b.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-07-27T03:05:48.023 INFO:journalctl@ceph.mon.b.smithi149.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-07-27T03:05:48.024 INFO:journalctl@ceph.mon.b.smithi149.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-07-27T03:05:48.024 INFO:journalctl@ceph.mon.b.smithi149.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-07-27T03:05:48.024 INFO:journalctl@ceph.mon.b.smithi149.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-07-27T03:05:48.024 INFO:journalctl@ceph.mon.b.smithi149.stdout: AddFile(Keys): cumulative 0, interval 0 2024-07-27T03:05:48.024 INFO:journalctl@ceph.mon.b.smithi149.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-27T03:05:48.024 INFO:journalctl@ceph.mon.b.smithi149.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-07-27T03:05:48.024 INFO:journalctl@ceph.mon.b.smithi149.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-07-27T03:05:48.024 INFO:journalctl@ceph.mon.b.smithi149.stdout: Block cache BinnedLRUCache@0x5570b6b73610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-07-27T03:05:48.024 INFO:journalctl@ceph.mon.b.smithi149.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-07-27T03:05:48.024 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2024-07-27T03:05:48.024 INFO:journalctl@ceph.mon.b.smithi149.stdout: ** File Read Latency Histogram By Level [default] ** 2024-07-27T03:05:48.024 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-07-27T03:05:48.024 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: using public_addr v2:172.21.15.149:0/0 -> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] 2024-07-27T03:05:48.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: starting mon.b rank -1 at public addrs [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] at bind addrs [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:05:48.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: mon.b@-1(???) e0 preinit fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:05:48.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: mon.b@-1(synchronizing).mds e1 new map 2024-07-27T03:05:48.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: mon.b@-1(synchronizing).mds e1 print_map 2024-07-27T03:05:48.025 INFO:journalctl@ceph.mon.b.smithi149.stdout: e1 2024-07-27T03:05:48.025 INFO:journalctl@ceph.mon.b.smithi149.stdout: btime 2024-07-27T03:04:08:096587+0000 2024-07-27T03:05:48.025 INFO:journalctl@ceph.mon.b.smithi149.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-07-27T03:05:48.025 INFO:journalctl@ceph.mon.b.smithi149.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-27T03:05:48.025 INFO:journalctl@ceph.mon.b.smithi149.stdout: legacy client fscid: -1 2024-07-27T03:05:48.025 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2024-07-27T03:05:48.025 INFO:journalctl@ceph.mon.b.smithi149.stdout: No filesystems configured 2024-07-27T03:05:48.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-07-27T03:05:48.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-07-27T03:05:48.026 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-07-27T03:05:48.026 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-07-27T03:05:48.026 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-07-27T03:05:48.026 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-07-27T03:05:48.026 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-07-27T03:05:48.026 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-27T03:05:48.026 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-27T03:05:48.026 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-07-27T03:05:48.026 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:48.026 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: Deploying cephadm binary to smithi175 2024-07-27T03:05:48.026 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:48.027 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:48.027 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:05:48.027 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: Added host smithi175 2024-07-27T03:05:48.027 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:48.027 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:48.027 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:48.027 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:48.027 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:48.027 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:05:48.027 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:48.027 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:48.028 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:48.028 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:48.028 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi175", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:05:48.028 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:48.028 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:05:48.028 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: Updating smithi175:/etc/ceph/ceph.conf 2024-07-27T03:05:48.028 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: Updating smithi175:/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/config/ceph.conf 2024-07-27T03:05:48.028 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-07-27T03:05:48.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:48.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:48.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:48.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: Updating smithi175:/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/config/ceph.client.admin.keyring 2024-07-27T03:05:48.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:48.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/89488623' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-07-27T03:05:48.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/89488623' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-07-27T03:05:48.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: osdmap e4: 0 total, 0 up, 0 in 2024-07-27T03:05:48.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:48.029 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:48.030 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi067:172.21.15.67=a;smithi149:172.21.15.149=b;smithi175:172.21.15.175=c", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:05:48.030 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: Saving service mon spec with placement smithi067:172.21.15.67=a;smithi149:172.21.15.149=b;smithi175:172.21.15.175=c;count:3 2024-07-27T03:05:48.030 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:48.030 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:05:48.030 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:48.030 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:05:48.030 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:48.030 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-27T03:05:48.030 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:48.031 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: Deploying daemon mon.c on smithi175 2024-07-27T03:05:48.031 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:48.031 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='client.? 172.21.15.175:0/4255795118' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-27T03:05:48.031 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:48.031 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-27T03:05:48.032 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:48.032 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: mon.a calling monitor election 2024-07-27T03:05:48.032 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:48.032 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:48.032 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:48.032 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: mon.c calling monitor election 2024-07-27T03:05:48.033 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:48.033 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:48.033 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:48.033 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:48.033 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-07-27T03:05:48.034 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: monmap epoch 2 2024-07-27T03:05:48.034 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:05:48.034 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: last_changed 2024-07-27T03:05:38.561211+0000 2024-07-27T03:05:48.034 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: created 2024-07-27T03:04:05.157867+0000 2024-07-27T03:05:48.034 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: min_mon_release 19 (squid) 2024-07-27T03:05:48.034 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: election_strategy: 1 2024-07-27T03:05:48.034 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-07-27T03:05:48.034 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-07-27T03:05:48.034 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: fsmap 2024-07-27T03:05:48.034 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: osdmap e4: 0 total, 0 up, 0 in 2024-07-27T03:05:48.035 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: mgrmap e13: x(active, since 48s) 2024-07-27T03:05:48.035 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: overall HEALTH_OK 2024-07-27T03:05:48.035 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:48.035 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:48.035 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:48.035 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-27T03:05:48.035 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:48.035 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:48.035 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: Deploying daemon mon.b on smithi149 2024-07-27T03:05:48.035 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='client.? 172.21.15.175:0/526190184' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-27T03:05:48.035 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:48.035 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:48.035 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 ceph-mon[37173]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-07-27T03:05:48.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 bash[37146]: caccb2bcb96e35cf409d8bd047e3a0dcc030005a4207d39be2ba59cbb6a719fb 2024-07-27T03:05:48.036 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:47 smithi149 systemd[1]: Started Ceph mon.b for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:05:52.636 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-07-27T03:05:52.636 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":3,"fsid":"c0782a00-4bc4-11ef-bcc7-c7b262605968","modified":"2024-07-27T03:05:47.614387Z","created":"2024-07-27T03:04:05.157867Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:3300","nonce":0},{"type":"v1","addr":"172.21.15.175:6789","nonce":0}]},"addr":"172.21.15.175:6789/0","public_addr":"172.21.15.175:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-07-27T03:05:52.637 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 3 2024-07-27T03:05:52.935 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:52.935 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-27T03:05:52.935 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:05:52.935 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: mon.a calling monitor election 2024-07-27T03:05:52.935 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: mon.c calling monitor election 2024-07-27T03:05:52.935 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:52.935 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: from='client.? 172.21.15.175:0/687804799' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-27T03:05:52.935 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:05:52.935 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:52.935 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:05:52.935 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: mon.b calling monitor election 2024-07-27T03:05:52.936 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:05:52.936 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:52.936 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:05:52.936 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:05:52.936 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-07-27T03:05:52.936 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: monmap epoch 3 2024-07-27T03:05:52.936 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:05:52.936 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: last_changed 2024-07-27T03:05:47.614387+0000 2024-07-27T03:05:52.936 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: created 2024-07-27T03:04:05.157867+0000 2024-07-27T03:05:52.936 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: min_mon_release 19 (squid) 2024-07-27T03:05:52.936 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: election_strategy: 1 2024-07-27T03:05:52.936 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-07-27T03:05:52.936 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-07-27T03:05:52.937 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: 2: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.b 2024-07-27T03:05:52.937 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: fsmap 2024-07-27T03:05:52.937 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: osdmap e4: 0 total, 0 up, 0 in 2024-07-27T03:05:52.937 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: mgrmap e13: x(active, since 57s) 2024-07-27T03:05:52.937 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: overall HEALTH_OK 2024-07-27T03:05:52.937 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:52.937 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:52.937 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:52.937 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:52.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:52.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-27T03:05:52.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:05:52.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: mon.a calling monitor election 2024-07-27T03:05:52.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: mon.c calling monitor election 2024-07-27T03:05:52.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:52.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: from='client.? 172.21.15.175:0/687804799' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-27T03:05:52.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:05:52.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:52.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:05:52.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: mon.b calling monitor election 2024-07-27T03:05:52.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:05:52.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:52.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:05:52.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:05:52.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-07-27T03:05:52.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: monmap epoch 3 2024-07-27T03:05:52.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:05:52.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: last_changed 2024-07-27T03:05:47.614387+0000 2024-07-27T03:05:52.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: created 2024-07-27T03:04:05.157867+0000 2024-07-27T03:05:52.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: min_mon_release 19 (squid) 2024-07-27T03:05:52.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: election_strategy: 1 2024-07-27T03:05:52.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-07-27T03:05:52.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-07-27T03:05:52.972 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: 2: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.b 2024-07-27T03:05:52.972 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: fsmap 2024-07-27T03:05:52.972 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: osdmap e4: 0 total, 0 up, 0 in 2024-07-27T03:05:52.972 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: mgrmap e13: x(active, since 57s) 2024-07-27T03:05:52.972 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: overall HEALTH_OK 2024-07-27T03:05:52.972 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:52.972 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:52.972 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:52.972 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:53.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:53.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-27T03:05:53.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:05:53.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: mon.a calling monitor election 2024-07-27T03:05:53.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: mon.c calling monitor election 2024-07-27T03:05:53.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:05:53.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: from='client.? 172.21.15.175:0/687804799' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-07-27T03:05:53.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:05:53.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:53.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:05:53.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: mon.b calling monitor election 2024-07-27T03:05:53.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:05:53.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:53.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:05:53.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:05:53.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-07-27T03:05:53.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: monmap epoch 3 2024-07-27T03:05:53.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:05:53.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: last_changed 2024-07-27T03:05:47.614387+0000 2024-07-27T03:05:53.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: created 2024-07-27T03:04:05.157867+0000 2024-07-27T03:05:53.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: min_mon_release 19 (squid) 2024-07-27T03:05:53.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: election_strategy: 1 2024-07-27T03:05:53.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-07-27T03:05:53.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.c 2024-07-27T03:05:53.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: 2: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.b 2024-07-27T03:05:53.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: fsmap 2024-07-27T03:05:53.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: osdmap e4: 0 total, 0 up, 0 in 2024-07-27T03:05:53.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: mgrmap e13: x(active, since 57s) 2024-07-27T03:05:53.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: overall HEALTH_OK 2024-07-27T03:05:53.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:53.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:53.003 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:53.003 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:53.205 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-07-27T03:05:53.206 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config generate-minimal-conf 2024-07-27T03:05:53.481 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:05:53.779 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:53 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:05:53.780 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:53 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:05:53.938 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:53 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:05:53.938 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:53 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:05:54.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:53 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:05:54.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:53 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:05:54.966 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:05:54 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: 2024-07-27T03:05:54.613+0000 7f44b5a95640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-07-27T03:05:54.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:54 smithi067 ceph-mon[32814]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:54.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:54.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:54.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:54.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:54.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:54.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:05:54.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:54.967 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:54.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:54.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:54.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:54.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:54.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:54.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:54.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:54.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:54.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:54.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-27T03:05:54.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-27T03:05:54.968 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:55.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:54 smithi149 ceph-mon[37173]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:55.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:55.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:05:55.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-27T03:05:55.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-27T03:05:55.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:55.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:54 smithi175 ceph-mon[37125]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:55.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:55.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:05:55.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-27T03:05:55.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-27T03:05:55.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:55.391 INFO:teuthology.orchestra.run.smithi067.stdout:# minimal ceph.conf for c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:05:55.391 INFO:teuthology.orchestra.run.smithi067.stdout:[global] 2024-07-27T03:05:55.391 INFO:teuthology.orchestra.run.smithi067.stdout: fsid = c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:05:55.391 INFO:teuthology.orchestra.run.smithi067.stdout: mon_host = [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] 2024-07-27T03:05:55.833 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:55 smithi067 ceph-mon[32814]: Updating smithi067:/etc/ceph/ceph.conf 2024-07-27T03:05:55.833 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:55 smithi067 ceph-mon[32814]: Updating smithi149:/etc/ceph/ceph.conf 2024-07-27T03:05:55.833 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:55 smithi067 ceph-mon[32814]: Updating smithi175:/etc/ceph/ceph.conf 2024-07-27T03:05:55.833 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:55 smithi067 ceph-mon[32814]: Updating smithi175:/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/config/ceph.conf 2024-07-27T03:05:55.834 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:55 smithi067 ceph-mon[32814]: Updating smithi149:/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/config/ceph.conf 2024-07-27T03:05:55.834 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:55 smithi067 ceph-mon[32814]: Updating smithi067:/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/config/ceph.conf 2024-07-27T03:05:55.834 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:55 smithi067 ceph-mon[32814]: Reconfiguring mon.a (unknown last config time)... 2024-07-27T03:05:55.834 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:55 smithi067 ceph-mon[32814]: Reconfiguring daemon mon.a on smithi067 2024-07-27T03:05:55.834 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:55.834 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:55 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/955789717' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:56.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:55 smithi149 ceph-mon[37173]: Updating smithi067:/etc/ceph/ceph.conf 2024-07-27T03:05:56.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:55 smithi149 ceph-mon[37173]: Updating smithi149:/etc/ceph/ceph.conf 2024-07-27T03:05:56.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:55 smithi149 ceph-mon[37173]: Updating smithi175:/etc/ceph/ceph.conf 2024-07-27T03:05:56.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:55 smithi149 ceph-mon[37173]: Updating smithi175:/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/config/ceph.conf 2024-07-27T03:05:56.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:55 smithi149 ceph-mon[37173]: Updating smithi149:/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/config/ceph.conf 2024-07-27T03:05:56.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:55 smithi149 ceph-mon[37173]: Updating smithi067:/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/config/ceph.conf 2024-07-27T03:05:56.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:55 smithi149 ceph-mon[37173]: Reconfiguring mon.a (unknown last config time)... 2024-07-27T03:05:56.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:55 smithi149 ceph-mon[37173]: Reconfiguring daemon mon.a on smithi067 2024-07-27T03:05:56.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:55 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:56.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:55 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/955789717' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:56.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:55 smithi175 ceph-mon[37125]: Updating smithi067:/etc/ceph/ceph.conf 2024-07-27T03:05:56.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:55 smithi175 ceph-mon[37125]: Updating smithi149:/etc/ceph/ceph.conf 2024-07-27T03:05:56.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:55 smithi175 ceph-mon[37125]: Updating smithi175:/etc/ceph/ceph.conf 2024-07-27T03:05:56.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:55 smithi175 ceph-mon[37125]: Updating smithi175:/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/config/ceph.conf 2024-07-27T03:05:56.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:55 smithi175 ceph-mon[37125]: Updating smithi149:/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/config/ceph.conf 2024-07-27T03:05:56.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:55 smithi175 ceph-mon[37125]: Updating smithi067:/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/config/ceph.conf 2024-07-27T03:05:56.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:55 smithi175 ceph-mon[37125]: Reconfiguring mon.a (unknown last config time)... 2024-07-27T03:05:56.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:55 smithi175 ceph-mon[37125]: Reconfiguring daemon mon.a on smithi067 2024-07-27T03:05:56.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:55 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:56.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:55 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/955789717' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:56.447 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-07-27T03:05:56.448 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-27T03:05:56.448 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.conf 2024-07-27T03:05:56.476 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-27T03:05:56.477 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-27T03:05:56.543 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-07-27T03:05:56.543 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.conf 2024-07-27T03:05:56.574 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-07-27T03:05:56.574 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-27T03:05:56.640 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-27T03:05:56.640 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.conf 2024-07-27T03:05:56.672 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-27T03:05:56.672 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-07-27T03:05:56.740 INFO:tasks.cephadm:Adding mgr.x on smithi067 2024-07-27T03:05:56.740 INFO:tasks.cephadm:Adding mgr.y on smithi149 2024-07-27T03:05:56.740 INFO:tasks.cephadm:Adding mgr.z on smithi175 2024-07-27T03:05:56.741 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph orch apply mgr '3;smithi067=x;smithi149=y;smithi175=z' 2024-07-27T03:05:56.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:56 smithi067 ceph-mon[32814]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:57.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:56 smithi149 ceph-mon[37173]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:57.029 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.c/config 2024-07-27T03:05:57.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:56 smithi175 ceph-mon[37125]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:58.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:57 smithi149 ceph-mon[37173]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:58.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:58.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:58.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:57 smithi149 ceph-mon[37173]: Reconfiguring mon.b (monmap changed)... 2024-07-27T03:05:58.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-27T03:05:58.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-27T03:05:58.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:58.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:57 smithi149 ceph-mon[37173]: Reconfiguring daemon mon.b on smithi149 2024-07-27T03:05:58.255 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:57 smithi175 ceph-mon[37125]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:58.256 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:58.256 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:58.256 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:57 smithi175 ceph-mon[37125]: Reconfiguring mon.b (monmap changed)... 2024-07-27T03:05:58.256 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-27T03:05:58.256 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-27T03:05:58.256 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:58.256 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:57 smithi175 ceph-mon[37125]: Reconfiguring daemon mon.b on smithi149 2024-07-27T03:05:58.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:57 smithi067 ceph-mon[32814]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:05:58.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:58.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:58.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:57 smithi067 ceph-mon[32814]: Reconfiguring mon.b (monmap changed)... 2024-07-27T03:05:58.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-27T03:05:58.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-27T03:05:58.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:58.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:57 smithi067 ceph-mon[32814]: Reconfiguring daemon mon.b on smithi149 2024-07-27T03:05:58.660 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled mgr update... 2024-07-27T03:05:59.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:59.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:59.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:59 smithi175 ceph-mon[37125]: Reconfiguring mon.c (monmap changed)... 2024-07-27T03:05:59.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-27T03:05:59.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-27T03:05:59.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:59.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:59 smithi175 ceph-mon[37125]: Reconfiguring daemon mon.c on smithi175 2024-07-27T03:05:59.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:59 smithi175 ceph-mon[37125]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi067=x;smithi149=y;smithi175=z", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:05:59.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:59 smithi175 ceph-mon[37125]: Saving service mgr spec with placement smithi067=x;smithi149=y;smithi175=z;count:3 2024-07-27T03:05:59.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:05:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:59.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:59.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:59.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:59 smithi067 ceph-mon[32814]: Reconfiguring mon.c (monmap changed)... 2024-07-27T03:05:59.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-27T03:05:59.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-27T03:05:59.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:59.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:59 smithi067 ceph-mon[32814]: Reconfiguring daemon mon.c on smithi175 2024-07-27T03:05:59.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:59 smithi067 ceph-mon[32814]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi067=x;smithi149=y;smithi175=z", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:05:59.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:59 smithi067 ceph-mon[32814]: Saving service mgr spec with placement smithi067=x;smithi149=y;smithi175=z;count:3 2024-07-27T03:05:59.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:05:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:59.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:59.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:05:59.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:59 smithi149 ceph-mon[37173]: Reconfiguring mon.c (monmap changed)... 2024-07-27T03:05:59.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-07-27T03:05:59.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-07-27T03:05:59.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:05:59.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:59 smithi149 ceph-mon[37173]: Reconfiguring daemon mon.c on smithi175 2024-07-27T03:05:59.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:59 smithi149 ceph-mon[37173]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi067=x;smithi149=y;smithi175=z", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:05:59.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:59 smithi149 ceph-mon[37173]: Saving service mgr spec with placement smithi067=x;smithi149=y;smithi175=z;count:3 2024-07-27T03:05:59.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:05:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:00.193 DEBUG:teuthology.orchestra.run.smithi149:mgr.y> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mgr.y.service 2024-07-27T03:06:00.196 DEBUG:teuthology.orchestra.run.smithi175:mgr.z> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mgr.z.service 2024-07-27T03:06:00.199 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-07-27T03:06:00.199 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set client client_mount_timeout 600 2024-07-27T03:06:00.201 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-07-27T03:06:00.201 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set client debug_client 20 2024-07-27T03:06:00.244 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-07-27T03:06:00.244 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set client debug_ms 1 2024-07-27T03:06:00.287 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-07-27T03:06:00.288 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set client rados_mon_op_timeout 900 2024-07-27T03:06:00.332 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-07-27T03:06:00.333 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set client rados_osd_op_timeout 900 2024-07-27T03:06:00.375 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-07-27T03:06:00.376 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set mds debug_mds 20 2024-07-27T03:06:00.418 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-07-27T03:06:00.419 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set mds debug_mds_balancer 20 2024-07-27T03:06:00.461 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-07-27T03:06:00.462 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set mds debug_ms 2 2024-07-27T03:06:00.506 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-07-27T03:06:00.507 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-07-27T03:06:00.521 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:00.522 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-07-27T03:06:00.522 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set mds mds_bal_merge_size 5 2024-07-27T03:06:00.576 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-07-27T03:06:00.577 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set mds mds_bal_split_bits 3 2024-07-27T03:06:00.620 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-07-27T03:06:00.621 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set mds mds_bal_split_size 100 2024-07-27T03:06:00.669 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-07-27T03:06:00.670 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set mds mds_debug_frag True 2024-07-27T03:06:00.722 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-07-27T03:06:00.722 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set mds mds_debug_scatterstat True 2024-07-27T03:06:00.766 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-07-27T03:06:00.767 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set mds mds_op_complaint_time 180 2024-07-27T03:06:00.772 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:00 smithi067 ceph-mon[32814]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:00.772 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:00.773 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:00.773 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:06:00.773 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:06:00.773 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:06:00.773 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:00.773 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-27T03:06:00.773 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-27T03:06:00.773 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-27T03:06:00.773 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:06:00.775 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-07-27T03:06:00.775 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set mds mds_verify_scatter True 2024-07-27T03:06:00.818 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-07-27T03:06:00.818 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set mds osd_op_complaint_time 180 2024-07-27T03:06:00.833 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:00.842 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-07-27T03:06:00.842 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set mds rados_mon_op_timeout 900 2024-07-27T03:06:00.887 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-07-27T03:06:00.888 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set mds rados_osd_op_timeout 900 2024-07-27T03:06:00.903 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:00 smithi175 ceph-mon[37125]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:00.903 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:00.903 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:00.903 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:06:00.903 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:06:00.903 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:06:00.903 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:00.903 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-27T03:06:00.903 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-27T03:06:00.903 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-27T03:06:00.903 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:06:00.942 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-07-27T03:06:00.942 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set mon mon_op_complaint_time 120 2024-07-27T03:06:00.993 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-07-27T03:06:00.993 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set mon mon_warn_on_pool_no_app False 2024-07-27T03:06:01.003 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:00 smithi149 ceph-mon[37173]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:01.003 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:01.003 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:01.004 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:06:01.004 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:06:01.004 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:06:01.004 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:01.004 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-27T03:06:01.004 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-27T03:06:01.004 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-27T03:06:01.004 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:06:01.050 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-07-27T03:06:01.051 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config set osd osd_op_complaint_time 180 2024-07-27T03:06:01.097 DEBUG:tasks.cephadm:set 22 configs 2024-07-27T03:06:01.097 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi067') 2024-07-27T03:06:01.830 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:01 smithi067 ceph-mon[32814]: Deploying daemon mgr.z on smithi175 2024-07-27T03:06:01.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:01 smithi175 ceph-mon[37125]: Deploying daemon mgr.z on smithi175 2024-07-27T03:06:02.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:01 smithi149 ceph-mon[37173]: Deploying daemon mgr.z on smithi175 2024-07-27T03:06:02.625 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jul 27 03:06:02 smithi175 systemd[1]: Started Ceph mgr.z for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:06:02.672 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:02.674 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:02.925 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:02 smithi149 ceph-mon[37173]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:02.925 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:02 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:02.925 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:02 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:02.925 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:02 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:02.925 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:02 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-27T03:06:02.925 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:02 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-27T03:06:02.925 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:02 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-27T03:06:02.926 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:02 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:06:02.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:02 smithi067 ceph-mon[32814]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:02.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:02 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:02.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:02 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:02.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:02 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:02.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:02 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-27T03:06:02.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:02 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-27T03:06:02.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:02 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-27T03:06:02.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:02 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:06:03.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:02 smithi175 ceph-mon[37125]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:03.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:02 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:03.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:02 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:03.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:02 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:03.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:02 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-27T03:06:03.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:02 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-07-27T03:06:03.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:02 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-27T03:06:03.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:02 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:06:03.926 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:03 smithi067 ceph-mon[32814]: Deploying daemon mgr.y on smithi149 2024-07-27T03:06:03.927 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:03 smithi149 ceph-mon[37173]: Deploying daemon mgr.y on smithi149 2024-07-27T03:06:04.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:03 smithi175 ceph-mon[37125]: Deploying daemon mgr.y on smithi149 2024-07-27T03:06:04.562 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:04.562 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:04.563 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:04.563 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:04.564 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:04.801 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:04.804 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:04.805 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:04.951 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:04 smithi067 ceph-mon[32814]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:04.951 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:04 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3611784466' entity='client.admin' 2024-07-27T03:06:05.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:04 smithi149 ceph-mon[37173]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:05.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:04 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3611784466' entity='client.admin' 2024-07-27T03:06:05.000 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Jul 27 03:06:04 smithi149 systemd[1]: Started Ceph mgr.y for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:06:05.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:04 smithi175 ceph-mon[37125]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:05.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:04 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3611784466' entity='client.admin' 2024-07-27T03:06:06.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:05 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1724147799' entity='client.admin' 2024-07-27T03:06:06.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:05 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:06.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:05 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:06.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:05 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:06.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:05 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:06.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:05 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:06:06.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:05 smithi149 ceph-mon[37173]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:06.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:05 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:06.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:05 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1724147799' entity='client.admin' 2024-07-27T03:06:06.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:05 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:06.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:05 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:06.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:05 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:06.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:05 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:06.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:05 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:06:06.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:05 smithi175 ceph-mon[37125]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:06.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:05 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:06.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:05 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1724147799' entity='client.admin' 2024-07-27T03:06:06.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:05 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:06.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:05 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:06.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:05 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:06.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:05 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:06.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:05 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:06:06.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:05 smithi067 ceph-mon[32814]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:06.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:05 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:07.095 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:07.096 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:07.096 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:07.096 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:07.096 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:06:07.096 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:06:07.096 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:07.096 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:06 smithi067 ceph-mon[32814]: Reconfiguring mgr.x (unknown last config time)... 2024-07-27T03:06:07.096 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-27T03:06:07.096 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-27T03:06:07.096 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:06:07.096 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:06 smithi067 ceph-mon[32814]: Reconfiguring daemon mgr.x on smithi067 2024-07-27T03:06:07.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:06 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:07.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:06 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:07.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:06 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:07.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:06 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:07.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:06 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:06:07.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:06 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:06:07.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:06 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:07.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:06 smithi149 ceph-mon[37173]: Reconfiguring mgr.x (unknown last config time)... 2024-07-27T03:06:07.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:06 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-27T03:06:07.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:06 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-27T03:06:07.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:06 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:06:07.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:06 smithi149 ceph-mon[37173]: Reconfiguring daemon mgr.x on smithi067 2024-07-27T03:06:07.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:06 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:07.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:06 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:07.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:06 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:07.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:06 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:07.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:06 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:06:07.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:06 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:06:07.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:06 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:07.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:06 smithi175 ceph-mon[37125]: Reconfiguring mgr.x (unknown last config time)... 2024-07-27T03:06:07.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:06 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-07-27T03:06:07.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:06 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-07-27T03:06:07.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:06 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:06:07.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:06 smithi175 ceph-mon[37125]: Reconfiguring daemon mgr.x on smithi067 2024-07-27T03:06:08.164 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:08 smithi067 ceph-mon[32814]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:08.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:08 smithi067 ceph-mon[32814]: Standby manager daemon z started 2024-07-27T03:06:08.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:08 smithi067 ceph-mon[32814]: from='mgr.? 172.21.15.175:0/2491713146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-07-27T03:06:08.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:08 smithi067 ceph-mon[32814]: from='mgr.? 172.21.15.175:0/2491713146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-27T03:06:08.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:08 smithi067 ceph-mon[32814]: from='mgr.? 172.21.15.175:0/2491713146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-07-27T03:06:08.165 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:08 smithi067 ceph-mon[32814]: from='mgr.? 172.21.15.175:0/2491713146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-27T03:06:08.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:08 smithi175 ceph-mon[37125]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:08.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:08 smithi175 ceph-mon[37125]: Standby manager daemon z started 2024-07-27T03:06:08.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:08 smithi175 ceph-mon[37125]: from='mgr.? 172.21.15.175:0/2491713146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-07-27T03:06:08.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:08 smithi175 ceph-mon[37125]: from='mgr.? 172.21.15.175:0/2491713146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-27T03:06:08.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:08 smithi175 ceph-mon[37125]: from='mgr.? 172.21.15.175:0/2491713146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-07-27T03:06:08.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:08 smithi175 ceph-mon[37125]: from='mgr.? 172.21.15.175:0/2491713146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-27T03:06:08.498 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:08.498 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:08.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:08 smithi149 ceph-mon[37173]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:08.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:08 smithi149 ceph-mon[37173]: Standby manager daemon z started 2024-07-27T03:06:08.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:08 smithi149 ceph-mon[37173]: from='mgr.? 172.21.15.175:0/2491713146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-07-27T03:06:08.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:08 smithi149 ceph-mon[37173]: from='mgr.? 172.21.15.175:0/2491713146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-27T03:06:08.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:08 smithi149 ceph-mon[37173]: from='mgr.? 172.21.15.175:0/2491713146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-07-27T03:06:08.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:08 smithi149 ceph-mon[37173]: from='mgr.? 172.21.15.175:0/2491713146' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-27T03:06:09.087 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:09 smithi067 ceph-mon[32814]: mgrmap e14: x(active, since 73s), standbys: z 2024-07-27T03:06:09.087 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-07-27T03:06:09.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:09 smithi175 ceph-mon[37125]: mgrmap e14: x(active, since 73s), standbys: z 2024-07-27T03:06:09.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-07-27T03:06:09.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:09 smithi149 ceph-mon[37173]: mgrmap e14: x(active, since 73s), standbys: z 2024-07-27T03:06:09.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-07-27T03:06:09.627 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi067') 2024-07-27T03:06:09.729 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:09.729 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:10.145 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:10 smithi067 ceph-mon[32814]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:10.145 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:10 smithi067 ceph-mon[32814]: Standby manager daemon y started 2024-07-27T03:06:10.145 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:10 smithi067 ceph-mon[32814]: from='mgr.? 172.21.15.149:0/1890737139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-07-27T03:06:10.145 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:10 smithi067 ceph-mon[32814]: from='mgr.? 172.21.15.149:0/1890737139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-27T03:06:10.145 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:10 smithi067 ceph-mon[32814]: from='mgr.? 172.21.15.149:0/1890737139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-07-27T03:06:10.145 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:10 smithi067 ceph-mon[32814]: from='mgr.? 172.21.15.149:0/1890737139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-27T03:06:10.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:10 smithi175 ceph-mon[37125]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:10.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:10 smithi175 ceph-mon[37125]: Standby manager daemon y started 2024-07-27T03:06:10.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:10 smithi175 ceph-mon[37125]: from='mgr.? 172.21.15.149:0/1890737139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-07-27T03:06:10.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:10 smithi175 ceph-mon[37125]: from='mgr.? 172.21.15.149:0/1890737139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-27T03:06:10.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:10 smithi175 ceph-mon[37125]: from='mgr.? 172.21.15.149:0/1890737139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-07-27T03:06:10.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:10 smithi175 ceph-mon[37125]: from='mgr.? 172.21.15.149:0/1890737139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-27T03:06:10.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:10 smithi149 ceph-mon[37173]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:10.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:10 smithi149 ceph-mon[37173]: Standby manager daemon y started 2024-07-27T03:06:10.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:10 smithi149 ceph-mon[37173]: from='mgr.? 172.21.15.149:0/1890737139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-07-27T03:06:10.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:10 smithi149 ceph-mon[37173]: from='mgr.? 172.21.15.149:0/1890737139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-07-27T03:06:10.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:10 smithi149 ceph-mon[37173]: from='mgr.? 172.21.15.149:0/1890737139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-07-27T03:06:10.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:10 smithi149 ceph-mon[37173]: from='mgr.? 172.21.15.149:0/1890737139' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-07-27T03:06:11.276 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:11 smithi067 ceph-mon[32814]: mgrmap e15: x(active, since 75s), standbys: z, y 2024-07-27T03:06:11.276 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:11 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-07-27T03:06:11.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:11 smithi175 ceph-mon[37125]: mgrmap e15: x(active, since 75s), standbys: z, y 2024-07-27T03:06:11.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:11 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-07-27T03:06:11.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:11 smithi149 ceph-mon[37173]: mgrmap e15: x(active, since 75s), standbys: z, y 2024-07-27T03:06:11.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:11 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-07-27T03:06:12.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:12 smithi067 ceph-mon[32814]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:12.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:12 smithi175 ceph-mon[37125]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:12.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:12 smithi149 ceph-mon[37173]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:14.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:14 smithi067 ceph-mon[32814]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:14.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:14 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/775396469' entity='client.admin' 2024-07-27T03:06:14.254 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:14 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3181092575' entity='client.admin' 2024-07-27T03:06:14.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:14 smithi175 ceph-mon[37125]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:14.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:14 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/775396469' entity='client.admin' 2024-07-27T03:06:14.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:14 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3181092575' entity='client.admin' 2024-07-27T03:06:14.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:14 smithi149 ceph-mon[37173]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:14.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:14 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/775396469' entity='client.admin' 2024-07-27T03:06:14.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:14 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3181092575' entity='client.admin' 2024-07-27T03:06:15.708 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:15 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2839965001' entity='client.admin' 2024-07-27T03:06:15.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:15 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2839965001' entity='client.admin' 2024-07-27T03:06:16.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:15 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2839965001' entity='client.admin' 2024-07-27T03:06:16.716 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:16.717 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:16.717 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:16.717 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:16.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:16 smithi149 ceph-mon[37173]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:16.757 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi067') 2024-07-27T03:06:16.803 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:16 smithi067 ceph-mon[32814]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:16.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:16 smithi175 ceph-mon[37125]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:18.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:18 smithi067 ceph-mon[32814]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:18.669 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:18.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:18 smithi175 ceph-mon[37125]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:19.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:18 smithi149 ceph-mon[37173]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:20.853 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:20 smithi067 ceph-mon[32814]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:20.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:20 smithi175 ceph-mon[37125]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:21.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:20 smithi149 ceph-mon[37173]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:22.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:21 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3951566657' entity='client.admin' 2024-07-27T03:06:22.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:21 smithi067 ceph-mon[32814]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:22.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:21 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3951566657' entity='client.admin' 2024-07-27T03:06:22.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:21 smithi149 ceph-mon[37173]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:22.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:21 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3951566657' entity='client.admin' 2024-07-27T03:06:22.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:21 smithi175 ceph-mon[37125]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:24.049 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:23 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2739156304' entity='client.admin' 2024-07-27T03:06:24.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:23 smithi067 ceph-mon[32814]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:24.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:23 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2832865705' entity='client.admin' 2024-07-27T03:06:24.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:23 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/727000843' entity='client.admin' 2024-07-27T03:06:24.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:23 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2739156304' entity='client.admin' 2024-07-27T03:06:24.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:23 smithi175 ceph-mon[37125]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:24.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:23 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2832865705' entity='client.admin' 2024-07-27T03:06:24.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:23 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/727000843' entity='client.admin' 2024-07-27T03:06:24.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:23 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2739156304' entity='client.admin' 2024-07-27T03:06:24.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:23 smithi149 ceph-mon[37173]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:24.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:23 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2832865705' entity='client.admin' 2024-07-27T03:06:24.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:23 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/727000843' entity='client.admin' 2024-07-27T03:06:26.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:25 smithi067 ceph-mon[32814]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:26.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:25 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1607264876' entity='client.admin' 2024-07-27T03:06:26.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:25 smithi149 ceph-mon[37173]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:26.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:25 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1607264876' entity='client.admin' 2024-07-27T03:06:26.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:25 smithi175 ceph-mon[37125]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:26.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:25 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1607264876' entity='client.admin' 2024-07-27T03:06:27.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:27 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3795073505' entity='client.admin' 2024-07-27T03:06:27.628 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:27 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3795073505' entity='client.admin' 2024-07-27T03:06:27.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:27 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3795073505' entity='client.admin' 2024-07-27T03:06:28.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:28 smithi067 ceph-mon[32814]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:28.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:28 smithi175 ceph-mon[37125]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:28.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:28 smithi149 ceph-mon[37173]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:30.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:30 smithi067 ceph-mon[32814]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:30.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:30.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:30.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:06:30.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:06:30.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:06:30.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:30.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:30 smithi175 ceph-mon[37125]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:30.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:30.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:30.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:06:30.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:06:30.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:06:30.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:30.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:30 smithi149 ceph-mon[37173]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:30.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:30.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:30.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:06:30.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:06:30.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:06:30.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:06:31.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:31.254+0000 7f809aca9640 1 -- 172.21.15.67:0/1774962717 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f80800b3bc0 msgr2=0x7f80800b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:31.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:31.254+0000 7f809aca9640 1 --2- 172.21.15.67:0/1774962717 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f80800b3bc0 0x7f80800b60d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f807c005d00 tx=0x7f807c00b410 comp rx=0 tx=0).stop 2024-07-27T03:06:31.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:31.254+0000 7f809aca9640 1 -- 172.21.15.67:0/1774962717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8094076ad0 msgr2=0x7f809419c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:31.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:31.254+0000 7f809aca9640 1 --2- 172.21.15.67:0/1774962717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8094076ad0 0x7f809419c1d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f808402ade0 tx=0x7f8084003c20 comp rx=0 tx=0).stop 2024-07-27T03:06:31.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:31.254+0000 7f809aca9640 1 -- 172.21.15.67:0/1774962717 shutdown_connections 2024-07-27T03:06:31.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:31.254+0000 7f809aca9640 1 --2- 172.21.15.67:0/1774962717 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f80800b3bc0 0x7f80800b60d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:31.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:31.254+0000 7f809aca9640 1 --2- 172.21.15.67:0/1774962717 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f809410b1e0 0x7f809419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:31.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:31.254+0000 7f809aca9640 1 --2- 172.21.15.67:0/1774962717 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8094076ad0 0x7f809419c1d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:31.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:31.254+0000 7f809aca9640 1 --2- 172.21.15.67:0/1774962717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80940756e0 0x7f809419bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:31.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:31.254+0000 7f809aca9640 1 -- 172.21.15.67:0/1774962717 >> 172.21.15.67:0/1774962717 conn(0x7f80940feb30 msgr2=0x7f80941071c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:06:31.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:31.254+0000 7f809aca9640 1 -- 172.21.15.67:0/1774962717 shutdown_connections 2024-07-27T03:06:31.258 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:31.254+0000 7f809aca9640 1 -- 172.21.15.67:0/1774962717 wait complete. 2024-07-27T03:06:32.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.464+0000 7f006cd37640 1 -- 172.21.15.67:0/4061018750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0068103140 msgr2=0x7f0068103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:32.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.464+0000 7f006cd37640 1 --2- 172.21.15.67:0/4061018750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0068103140 0x7f0068103550 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0050009900 tx=0x7f005002a480 comp rx=0 tx=0).stop 2024-07-27T03:06:32.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.464+0000 7f006cd37640 1 -- 172.21.15.67:0/4061018750 shutdown_connections 2024-07-27T03:06:32.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.464+0000 7f006cd37640 1 --2- 172.21.15.67:0/4061018750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0068103ae0 0x7f006810a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:32.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.464+0000 7f006cd37640 1 --2- 172.21.15.67:0/4061018750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0068103140 0x7f0068103550 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:32.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.464+0000 7f006cd37640 1 --2- 172.21.15.67:0/4061018750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00681020b0 0x7f0068102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:32.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.464+0000 7f006cd37640 1 -- 172.21.15.67:0/4061018750 >> 172.21.15.67:0/4061018750 conn(0x7f00680fda20 msgr2=0x7f00680ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:06:32.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.464+0000 7f006cd37640 1 -- 172.21.15.67:0/4061018750 shutdown_connections 2024-07-27T03:06:32.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.464+0000 7f006cd37640 1 -- 172.21.15.67:0/4061018750 wait complete. 2024-07-27T03:06:32.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.465+0000 7f006cd37640 1 Processor -- start 2024-07-27T03:06:32.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.465+0000 7f006cd37640 1 -- start start 2024-07-27T03:06:32.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.465+0000 7f006cd37640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00681020b0 0x7f0068199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:32.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.465+0000 7f006cd37640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0068103140 0x7f006819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:32.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.465+0000 7f006cd37640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0068103ae0 0x7f006819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:32.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.465+0000 7f006cd37640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f006810dd10 con 0x7f0068103ae0 2024-07-27T03:06:32.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.465+0000 7f006cd37640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f006810db90 con 0x7f0068103140 2024-07-27T03:06:32.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.465+0000 7f006cd37640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f006810de90 con 0x7f00681020b0 2024-07-27T03:06:32.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.465+0000 7f0066d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0068103ae0 0x7f006819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:32.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.465+0000 7f0066d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0068103ae0 0x7f006819ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32878/0 (socket says 172.21.15.67:32878) 2024-07-27T03:06:32.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.465+0000 7f0066d9d640 1 -- 172.21.15.67:0/1550832266 learned_addr learned my addr 172.21.15.67:0/1550832266 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:06:32.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.465+0000 7f006659c640 1 --2- 172.21.15.67:0/1550832266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00681020b0 0x7f0068199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:32.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.465+0000 7f005ffff640 1 --2- 172.21.15.67:0/1550832266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0068103140 0x7f006819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:32.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.466+0000 7f0066d9d640 1 -- 172.21.15.67:0/1550832266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00681020b0 msgr2=0x7f0068199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:32.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.466+0000 7f0066d9d640 1 --2- 172.21.15.67:0/1550832266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00681020b0 0x7f0068199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:32.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.466+0000 7f0066d9d640 1 -- 172.21.15.67:0/1550832266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0068103140 msgr2=0x7f006819a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:32.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.466+0000 7f0066d9d640 1 --2- 172.21.15.67:0/1550832266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0068103140 0x7f006819a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:32.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.466+0000 7f0066d9d640 1 -- 172.21.15.67:0/1550832266 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006819b3e0 con 0x7f0068103ae0 2024-07-27T03:06:32.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.466+0000 7f005ffff640 1 --2- 172.21.15.67:0/1550832266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0068103140 0x7f006819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:06:32.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.466+0000 7f0066d9d640 1 --2- 172.21.15.67:0/1550832266 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0068103ae0 0x7f006819ac50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f006000cad0 tx=0x7f006000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:06:32.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.466+0000 7f005f7fe640 1 -- 172.21.15.67:0/1550832266 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006000ad60 con 0x7f0068103ae0 2024-07-27T03:06:32.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.466+0000 7f006cd37640 1 -- 172.21.15.67:0/1550832266 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006819b6d0 con 0x7f0068103ae0 2024-07-27T03:06:32.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.466+0000 7f005f7fe640 1 -- 172.21.15.67:0/1550832266 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0060005020 con 0x7f0068103ae0 2024-07-27T03:06:32.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.466+0000 7f005f7fe640 1 -- 172.21.15.67:0/1550832266 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006000f6b0 con 0x7f0068103ae0 2024-07-27T03:06:32.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.466+0000 7f006cd37640 1 -- 172.21.15.67:0/1550832266 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00681a1e60 con 0x7f0068103ae0 2024-07-27T03:06:32.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.467+0000 7f006cd37640 1 -- 172.21.15.67:0/1550832266 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0068102520 con 0x7f0068103ae0 2024-07-27T03:06:32.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.469+0000 7f005f7fe640 1 -- 172.21.15.67:0/1550832266 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 151003+0+0 (secure 0 0 0) 0x7f00600051d0 con 0x7f0068103ae0 2024-07-27T03:06:32.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.470+0000 7f005f7fe640 1 --2- 172.21.15.67:0/1550832266 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f00540b3b40 0x7f00540b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:32.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.470+0000 7f006659c640 1 --2- 172.21.15.67:0/1550832266 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f00540b3b40 0x7f00540b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:32.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.470+0000 7f005f7fe640 1 -- 172.21.15.67:0/1550832266 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f00600e1ea0 con 0x7f0068103ae0 2024-07-27T03:06:32.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.470+0000 7f006659c640 1 --2- 172.21.15.67:0/1550832266 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f00540b3b40 0x7f00540b6050 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f005800abe0 tx=0x7f005800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:06:32.477 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.474+0000 7f005f7fe640 1 -- 172.21.15.67:0/1550832266 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f0060014030 con 0x7f0068103ae0 2024-07-27T03:06:32.541 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:32 smithi067 ceph-mon[32814]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:32.541 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:32 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1774962717' entity='client.admin' 2024-07-27T03:06:32.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:32 smithi175 ceph-mon[37125]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:32 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1774962717' entity='client.admin' 2024-07-27T03:06:32.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.631+0000 7ff7b655f640 1 -- 172.21.15.67:0/3643373625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7b01031e0 msgr2=0x7ff7b0103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:32.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.631+0000 7ff7b655f640 1 --2- 172.21.15.67:0/3643373625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7b01031e0 0x7ff7b0103650 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009900 tx=0x7ff7a002a480 comp rx=0 tx=0).stop 2024-07-27T03:06:32.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.631+0000 7ff7b655f640 1 -- 172.21.15.67:0/3643373625 shutdown_connections 2024-07-27T03:06:32.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.631+0000 7ff7b655f640 1 --2- 172.21.15.67:0/3643373625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7b0104c10 0x7ff7b010b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:32.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.631+0000 7ff7b655f640 1 --2- 172.21.15.67:0/3643373625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7b0104270 0x7ff7b0104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:32.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.631+0000 7ff7b655f640 1 --2- 172.21.15.67:0/3643373625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7b01031e0 0x7ff7b0103650 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:32.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.631+0000 7ff7b655f640 1 -- 172.21.15.67:0/3643373625 >> 172.21.15.67:0/3643373625 conn(0x7ff7b00feb50 msgr2=0x7ff7b0101000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:06:32.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.631+0000 7ff7b655f640 1 -- 172.21.15.67:0/3643373625 shutdown_connections 2024-07-27T03:06:32.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.631+0000 7ff7b655f640 1 -- 172.21.15.67:0/3643373625 wait complete. 2024-07-27T03:06:32.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.632+0000 7ff7b655f640 1 Processor -- start 2024-07-27T03:06:32.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.632+0000 7ff7b655f640 1 -- start start 2024-07-27T03:06:32.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.633+0000 7ff7b655f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7b01031e0 0x7ff7b0199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:32.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.633+0000 7ff7b655f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7b0104270 0x7ff7b019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:32.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.633+0000 7ff7b655f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7b0104c10 0x7ff7b019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:32.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.633+0000 7ff7b655f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b010ee90 con 0x7ff7b01031e0 2024-07-27T03:06:32.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.633+0000 7ff7b655f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b010ed10 con 0x7ff7b0104270 2024-07-27T03:06:32.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.633+0000 7ff7b655f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b010f010 con 0x7ff7b0104c10 2024-07-27T03:06:32.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.633+0000 7ff7affff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7b01031e0 0x7ff7b0199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:32.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.633+0000 7ff7affff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7b01031e0 0x7ff7b0199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32902/0 (socket says 172.21.15.67:32902) 2024-07-27T03:06:32.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.633+0000 7ff7affff640 1 -- 172.21.15.67:0/3662159728 learned_addr learned my addr 172.21.15.67:0/3662159728 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:06:32.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.633+0000 7ff7b4afc640 1 --2- 172.21.15.67:0/3662159728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7b0104c10 0x7ff7b019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:32.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.633+0000 7ff7af7fe640 1 --2- 172.21.15.67:0/3662159728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7b0104270 0x7ff7b019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:32.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.633+0000 7ff7affff640 1 -- 172.21.15.67:0/3662159728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7b0104c10 msgr2=0x7ff7b019ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:32.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.633+0000 7ff7affff640 1 --2- 172.21.15.67:0/3662159728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7b0104c10 0x7ff7b019ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:32.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.633+0000 7ff7affff640 1 -- 172.21.15.67:0/3662159728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7b0104270 msgr2=0x7ff7b019a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:32.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.633+0000 7ff7affff640 1 --2- 172.21.15.67:0/3662159728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7b0104270 0x7ff7b019a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:32.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.633+0000 7ff7affff640 1 -- 172.21.15.67:0/3662159728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b019b3f0 con 0x7ff7b01031e0 2024-07-27T03:06:32.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.634+0000 7ff7af7fe640 1 --2- 172.21.15.67:0/3662159728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7b0104270 0x7ff7b019a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:06:32.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.634+0000 7ff7affff640 1 --2- 172.21.15.67:0/3662159728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7b01031e0 0x7ff7b0199a50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0031440 tx=0x7ff7a0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:06:32.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.634+0000 7ff7ad7fa640 1 -- 172.21.15.67:0/3662159728 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a003f070 con 0x7ff7b01031e0 2024-07-27T03:06:32.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.634+0000 7ff7ad7fa640 1 -- 172.21.15.67:0/3662159728 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7a0031c50 con 0x7ff7b01031e0 2024-07-27T03:06:32.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.634+0000 7ff7b655f640 1 -- 172.21.15.67:0/3662159728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b019b680 con 0x7ff7b01031e0 2024-07-27T03:06:32.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.634+0000 7ff7ad7fa640 1 -- 172.21.15.67:0/3662159728 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0035030 con 0x7ff7b01031e0 2024-07-27T03:06:32.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.634+0000 7ff7b655f640 1 -- 172.21.15.67:0/3662159728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b01a1ca0 con 0x7ff7b01031e0 2024-07-27T03:06:32.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.635+0000 7ff7b655f640 1 -- 172.21.15.67:0/3662159728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b010a3c0 con 0x7ff7b01031e0 2024-07-27T03:06:32.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.637+0000 7ff7ad7fa640 1 -- 172.21.15.67:0/3662159728 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 151003+0+0 (secure 0 0 0) 0x7ff7a0042070 con 0x7ff7b01031e0 2024-07-27T03:06:32.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.638+0000 7ff7ad7fa640 1 --2- 172.21.15.67:0/3662159728 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff7980b3bc0 0x7ff7980b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:32.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.638+0000 7ff7af7fe640 1 --2- 172.21.15.67:0/3662159728 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff7980b3bc0 0x7ff7980b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:32.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.639+0000 7ff7ad7fa640 1 -- 172.21.15.67:0/3662159728 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff7a0102f00 con 0x7ff7b01031e0 2024-07-27T03:06:32.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.639+0000 7ff7af7fe640 1 --2- 172.21.15.67:0/3662159728 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff7980b3bc0 0x7ff7980b60d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400abe0 tx=0x7ff7a400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:06:32.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.641+0000 7ff7ad7fa640 1 -- 172.21.15.67:0/3662159728 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7ff7a00cc180 con 0x7ff7b01031e0 2024-07-27T03:06:32.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.664+0000 7f006cd37640 1 -- 172.21.15.67:0/1550832266 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f00681a2030 con 0x7f0068103ae0 2024-07-27T03:06:32.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.670+0000 7f005f7fe640 1 -- 172.21.15.67:0/1550832266 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v22)=0 v22) ==== 120+0+0 (secure 0 0 0) 0x7f00600ab1c0 con 0x7f0068103ae0 2024-07-27T03:06:32.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.672+0000 7f006cd37640 1 -- 172.21.15.67:0/1550832266 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f00540b3b40 msgr2=0x7f00540b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:32.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.672+0000 7f006cd37640 1 --2- 172.21.15.67:0/1550832266 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f00540b3b40 0x7f00540b6050 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f005800abe0 tx=0x7f005800a2d0 comp rx=0 tx=0).stop 2024-07-27T03:06:32.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.672+0000 7f006cd37640 1 -- 172.21.15.67:0/1550832266 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0068103ae0 msgr2=0x7f006819ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:32.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.672+0000 7f006cd37640 1 --2- 172.21.15.67:0/1550832266 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0068103ae0 0x7f006819ac50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f006000cad0 tx=0x7f006000ced0 comp rx=0 tx=0).stop 2024-07-27T03:06:32.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.672+0000 7f006cd37640 1 -- 172.21.15.67:0/1550832266 shutdown_connections 2024-07-27T03:06:32.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.672+0000 7f006cd37640 1 --2- 172.21.15.67:0/1550832266 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f00540b3b40 0x7f00540b6050 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:32.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.672+0000 7f006cd37640 1 --2- 172.21.15.67:0/1550832266 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0068103ae0 0x7f006819ac50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:32.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.672+0000 7f006cd37640 1 --2- 172.21.15.67:0/1550832266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0068103140 0x7f006819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:32.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.672+0000 7f006cd37640 1 --2- 172.21.15.67:0/1550832266 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00681020b0 0x7f0068199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:32.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.672+0000 7f006cd37640 1 -- 172.21.15.67:0/1550832266 >> 172.21.15.67:0/1550832266 conn(0x7f00680fda20 msgr2=0x7f00680ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:06:32.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.672+0000 7f006cd37640 1 -- 172.21.15.67:0/1550832266 shutdown_connections 2024-07-27T03:06:32.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.672+0000 7f006cd37640 1 -- 172.21.15.67:0/1550832266 wait complete. 2024-07-27T03:06:32.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:32 smithi149 ceph-mon[37173]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:32.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:32 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1774962717' entity='client.admin' 2024-07-27T03:06:32.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.831+0000 7ff7b655f640 1 -- 172.21.15.67:0/3662159728 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7ff7b002e100 con 0x7ff7b01031e0 2024-07-27T03:06:32.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.835+0000 7ff7ad7fa640 1 -- 172.21.15.67:0/3662159728 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v23)=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7ff7a00cba80 con 0x7ff7b01031e0 2024-07-27T03:06:32.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.838+0000 7ff7b655f640 1 -- 172.21.15.67:0/3662159728 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff7980b3bc0 msgr2=0x7ff7980b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:32.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.838+0000 7ff7b655f640 1 --2- 172.21.15.67:0/3662159728 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff7980b3bc0 0x7ff7980b60d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400abe0 tx=0x7ff7a400a5c0 comp rx=0 tx=0).stop 2024-07-27T03:06:32.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.838+0000 7ff7b655f640 1 -- 172.21.15.67:0/3662159728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7b01031e0 msgr2=0x7ff7b0199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:32.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.838+0000 7ff7b655f640 1 --2- 172.21.15.67:0/3662159728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7b01031e0 0x7ff7b0199a50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0031440 tx=0x7ff7a0003c10 comp rx=0 tx=0).stop 2024-07-27T03:06:32.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.838+0000 7ff7b655f640 1 -- 172.21.15.67:0/3662159728 shutdown_connections 2024-07-27T03:06:32.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.838+0000 7ff7b655f640 1 --2- 172.21.15.67:0/3662159728 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff7980b3bc0 0x7ff7980b60d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:32.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.838+0000 7ff7b655f640 1 --2- 172.21.15.67:0/3662159728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7b0104c10 0x7ff7b019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:32.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.838+0000 7ff7b655f640 1 --2- 172.21.15.67:0/3662159728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7b0104270 0x7ff7b019a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:32.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.838+0000 7ff7b655f640 1 --2- 172.21.15.67:0/3662159728 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7b01031e0 0x7ff7b0199a50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:32.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.838+0000 7ff7b655f640 1 -- 172.21.15.67:0/3662159728 >> 172.21.15.67:0/3662159728 conn(0x7ff7b00feb50 msgr2=0x7ff7b0107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:06:32.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.838+0000 7ff7b655f640 1 -- 172.21.15.67:0/3662159728 shutdown_connections 2024-07-27T03:06:32.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:32.838+0000 7ff7b655f640 1 -- 172.21.15.67:0/3662159728 wait complete. 2024-07-27T03:06:33.771 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:33 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1550832266' entity='client.admin' 2024-07-27T03:06:33.771 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:33 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3662159728' entity='client.admin' 2024-07-27T03:06:33.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.767+0000 7fe9a8ff8640 1 -- 172.21.15.67:0/629569714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a4104250 msgr2=0x7fe9a4104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:33.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.767+0000 7fe9a8ff8640 1 --2- 172.21.15.67:0/629569714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a4104250 0x7fe9a4104660 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe998009900 tx=0x7fe99802a480 comp rx=0 tx=0).stop 2024-07-27T03:06:33.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.768+0000 7fe9a8ff8640 1 -- 172.21.15.67:0/629569714 shutdown_connections 2024-07-27T03:06:33.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.768+0000 7fe9a8ff8640 1 --2- 172.21.15.67:0/629569714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9a4104bf0 0x7fe9a410b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:33.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.768+0000 7fe9a8ff8640 1 --2- 172.21.15.67:0/629569714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a4104250 0x7fe9a4104660 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:33.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.768+0000 7fe9a8ff8640 1 --2- 172.21.15.67:0/629569714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9a41031c0 0x7fe9a4103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:33.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.768+0000 7fe9a8ff8640 1 -- 172.21.15.67:0/629569714 >> 172.21.15.67:0/629569714 conn(0x7fe9a40feb30 msgr2=0x7fe9a4100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:06:33.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.768+0000 7fe9a8ff8640 1 -- 172.21.15.67:0/629569714 shutdown_connections 2024-07-27T03:06:33.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.768+0000 7fe9a8ff8640 1 -- 172.21.15.67:0/629569714 wait complete. 2024-07-27T03:06:33.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.769+0000 7fe9a8ff8640 1 Processor -- start 2024-07-27T03:06:33.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.769+0000 7fe9a8ff8640 1 -- start start 2024-07-27T03:06:33.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.769+0000 7fe9a8ff8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9a41031c0 0x7fe9a4199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:33.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.769+0000 7fe9a8ff8640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9a4104250 0x7fe9a419a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:33.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.769+0000 7fe9a8ff8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a4104bf0 0x7fe9a419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:33.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.769+0000 7fe9a8ff8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a410ee20 con 0x7fe9a4104bf0 2024-07-27T03:06:33.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.769+0000 7fe9a8ff8640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a410eca0 con 0x7fe9a4104250 2024-07-27T03:06:33.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.769+0000 7fe9a8ff8640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a410efa0 con 0x7fe9a41031c0 2024-07-27T03:06:33.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.769+0000 7fe9a2d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a4104bf0 0x7fe9a419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:33.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.769+0000 7fe9a2d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a4104bf0 0x7fe9a419ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32916/0 (socket says 172.21.15.67:32916) 2024-07-27T03:06:33.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.769+0000 7fe9a2d9d640 1 -- 172.21.15.67:0/1054939188 learned_addr learned my addr 172.21.15.67:0/1054939188 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:06:33.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.769+0000 7fe9a259c640 1 --2- 172.21.15.67:0/1054939188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9a41031c0 0x7fe9a4199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:33.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.769+0000 7fe9a1d9b640 1 --2- 172.21.15.67:0/1054939188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9a4104250 0x7fe9a419a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:33.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.770+0000 7fe9a2d9d640 1 -- 172.21.15.67:0/1054939188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9a41031c0 msgr2=0x7fe9a4199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:33.806 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.770+0000 7fe9a2d9d640 1 --2- 172.21.15.67:0/1054939188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9a41031c0 0x7fe9a4199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:33.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.770+0000 7fe9a2d9d640 1 -- 172.21.15.67:0/1054939188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9a4104250 msgr2=0x7fe9a419a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:33.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.770+0000 7fe9a2d9d640 1 --2- 172.21.15.67:0/1054939188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9a4104250 0x7fe9a419a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:33.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.770+0000 7fe9a2d9d640 1 -- 172.21.15.67:0/1054939188 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a419b3d0 con 0x7fe9a4104bf0 2024-07-27T03:06:33.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.770+0000 7fe9a1d9b640 1 --2- 172.21.15.67:0/1054939188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9a4104250 0x7fe9a419a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:06:33.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.770+0000 7fe9a2d9d640 1 --2- 172.21.15.67:0/1054939188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a4104bf0 0x7fe9a419ac40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00cce0 tx=0x7fe99c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:06:33.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.770+0000 7fe9937fe640 1 -- 172.21.15.67:0/1054939188 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c00f450 con 0x7fe9a4104bf0 2024-07-27T03:06:33.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.770+0000 7fe9937fe640 1 -- 172.21.15.67:0/1054939188 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe99c00faa0 con 0x7fe9a4104bf0 2024-07-27T03:06:33.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.770+0000 7fe9a8ff8640 1 -- 172.21.15.67:0/1054939188 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a419b6c0 con 0x7fe9a4104bf0 2024-07-27T03:06:33.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.770+0000 7fe9937fe640 1 -- 172.21.15.67:0/1054939188 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c018b70 con 0x7fe9a4104bf0 2024-07-27T03:06:33.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.770+0000 7fe9a8ff8640 1 -- 172.21.15.67:0/1054939188 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a41a1c80 con 0x7fe9a4104bf0 2024-07-27T03:06:33.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.771+0000 7fe9a8ff8640 1 -- 172.21.15.67:0/1054939188 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a410a380 con 0x7fe9a4104bf0 2024-07-27T03:06:33.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.773+0000 7fe9937fe640 1 -- 172.21.15.67:0/1054939188 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 151003+0+0 (secure 0 0 0) 0x7fe99c018d10 con 0x7fe9a4104bf0 2024-07-27T03:06:33.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.774+0000 7fe9937fe640 1 --2- 172.21.15.67:0/1054939188 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe98c0b3b40 0x7fe98c0b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:33.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.774+0000 7fe9a259c640 1 --2- 172.21.15.67:0/1054939188 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe98c0b3b40 0x7fe98c0b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:33.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.774+0000 7fe9937fe640 1 -- 172.21.15.67:0/1054939188 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe99c0e2380 con 0x7fe9a4104bf0 2024-07-27T03:06:33.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.774+0000 7fe9a259c640 1 --2- 172.21.15.67:0/1054939188 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe98c0b3b40 0x7fe98c0b6050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe994005d00 tx=0x7fe99400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:06:33.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.777+0000 7fe9937fe640 1 -- 172.21.15.67:0/1054939188 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fe99c014030 con 0x7fe9a4104bf0 2024-07-27T03:06:33.966 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.963+0000 7fe9a8ff8640 1 -- 172.21.15.67:0/1054939188 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7fe9a402e100 con 0x7fe9a4104bf0 2024-07-27T03:06:33.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.967+0000 7fe9937fe640 1 -- 172.21.15.67:0/1054939188 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v24)=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7fe99c0ab730 con 0x7fe9a4104bf0 2024-07-27T03:06:33.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.969+0000 7fe9a8ff8640 1 -- 172.21.15.67:0/1054939188 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe98c0b3b40 msgr2=0x7fe98c0b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:33.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.969+0000 7fe9a8ff8640 1 --2- 172.21.15.67:0/1054939188 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe98c0b3b40 0x7fe98c0b6050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe994005d00 tx=0x7fe99400b410 comp rx=0 tx=0).stop 2024-07-27T03:06:33.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.969+0000 7fe9a8ff8640 1 -- 172.21.15.67:0/1054939188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a4104bf0 msgr2=0x7fe9a419ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:33.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.969+0000 7fe9a8ff8640 1 --2- 172.21.15.67:0/1054939188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a4104bf0 0x7fe9a419ac40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00cce0 tx=0x7fe99c00a620 comp rx=0 tx=0).stop 2024-07-27T03:06:33.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.969+0000 7fe9a8ff8640 1 -- 172.21.15.67:0/1054939188 shutdown_connections 2024-07-27T03:06:33.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.969+0000 7fe9a8ff8640 1 --2- 172.21.15.67:0/1054939188 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe98c0b3b40 0x7fe98c0b6050 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:33.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.969+0000 7fe9a8ff8640 1 --2- 172.21.15.67:0/1054939188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe9a4104bf0 0x7fe9a419ac40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:33.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.969+0000 7fe9a8ff8640 1 --2- 172.21.15.67:0/1054939188 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9a4104250 0x7fe9a419a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:33.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.969+0000 7fe9a8ff8640 1 --2- 172.21.15.67:0/1054939188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9a41031c0 0x7fe9a4199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:33.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.969+0000 7fe9a8ff8640 1 -- 172.21.15.67:0/1054939188 >> 172.21.15.67:0/1054939188 conn(0x7fe9a40feb30 msgr2=0x7fe9a4107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:06:33.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.970+0000 7fe9a8ff8640 1 -- 172.21.15.67:0/1054939188 shutdown_connections 2024-07-27T03:06:33.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:33.970+0000 7fe9a8ff8640 1 -- 172.21.15.67:0/1054939188 wait complete. 2024-07-27T03:06:34.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:33 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1550832266' entity='client.admin' 2024-07-27T03:06:34.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:33 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3662159728' entity='client.admin' 2024-07-27T03:06:34.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:33 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1550832266' entity='client.admin' 2024-07-27T03:06:34.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:33 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3662159728' entity='client.admin' 2024-07-27T03:06:34.894 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:34 smithi067 ceph-mon[32814]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:34.894 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:34 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1054939188' entity='client.admin' 2024-07-27T03:06:35.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:34 smithi149 ceph-mon[37173]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:35.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:34 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1054939188' entity='client.admin' 2024-07-27T03:06:35.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:34 smithi175 ceph-mon[37125]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:35.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:34 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1054939188' entity='client.admin' 2024-07-27T03:06:35.277 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi067') 2024-07-27T03:06:35.277 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi067') 2024-07-27T03:06:35.278 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi067') 2024-07-27T03:06:35.278 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi067') 2024-07-27T03:06:36.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.020+0000 7f3343732640 1 -- 172.21.15.67:0/997892779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f333c103af0 msgr2=0x7f333c10a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:36.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.020+0000 7f3343732640 1 --2- 172.21.15.67:0/997892779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f333c103af0 0x7f333c10a490 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3334009ca0 tx=0x7f333402a950 comp rx=0 tx=0).stop 2024-07-27T03:06:36.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.020+0000 7f3343732640 1 -- 172.21.15.67:0/997892779 shutdown_connections 2024-07-27T03:06:36.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.020+0000 7f3343732640 1 --2- 172.21.15.67:0/997892779 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f333c103af0 0x7f333c10a490 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:36.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.020+0000 7f3343732640 1 --2- 172.21.15.67:0/997892779 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f333c103150 0x7f333c103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:36.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.020+0000 7f3343732640 1 --2- 172.21.15.67:0/997892779 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f333c1020c0 0x7f333c102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:36.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.020+0000 7f3343732640 1 -- 172.21.15.67:0/997892779 >> 172.21.15.67:0/997892779 conn(0x7f333c0fda50 msgr2=0x7f333c0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:06:36.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.020+0000 7f3343732640 1 -- 172.21.15.67:0/997892779 shutdown_connections 2024-07-27T03:06:36.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.020+0000 7f3343732640 1 -- 172.21.15.67:0/997892779 wait complete. 2024-07-27T03:06:36.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.021+0000 7f3343732640 1 Processor -- start 2024-07-27T03:06:36.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.021+0000 7f3343732640 1 -- start start 2024-07-27T03:06:36.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.021+0000 7f3343732640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f333c1020c0 0x7f333c19bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:36.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.021+0000 7f3343732640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f333c103150 0x7f333c19c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:36.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.021+0000 7f3343732640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f333c103af0 0x7f333c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:36.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.021+0000 7f3343732640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f333c10ee90 con 0x7f333c1020c0 2024-07-27T03:06:36.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.021+0000 7f3343732640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f333c10ed10 con 0x7f333c103af0 2024-07-27T03:06:36.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.021+0000 7f3343732640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f333c10f010 con 0x7f333c103150 2024-07-27T03:06:36.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.021+0000 7f33414ce640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f333c1020c0 0x7f333c19bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:36.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.021+0000 7f33414ce640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f333c1020c0 0x7f333c19bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51706/0 (socket says 172.21.15.67:51706) 2024-07-27T03:06:36.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.021+0000 7f33414ce640 1 -- 172.21.15.67:0/78998981 learned_addr learned my addr 172.21.15.67:0/78998981 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:06:36.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.022+0000 7f3340ccd640 1 --2- 172.21.15.67:0/78998981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f333c103150 0x7f333c19c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:36.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.022+0000 7f3341ccf640 1 --2- 172.21.15.67:0/78998981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f333c103af0 0x7f333c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:36.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.022+0000 7f33414ce640 1 -- 172.21.15.67:0/78998981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f333c103150 msgr2=0x7f333c19c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:36.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.022+0000 7f33414ce640 1 --2- 172.21.15.67:0/78998981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f333c103150 0x7f333c19c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:36.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.022+0000 7f33414ce640 1 -- 172.21.15.67:0/78998981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f333c103af0 msgr2=0x7f333c19ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:36.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.022+0000 7f33414ce640 1 --2- 172.21.15.67:0/78998981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f333c103af0 0x7f333c19ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:36.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.022+0000 7f33414ce640 1 -- 172.21.15.67:0/78998981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333c19d620 con 0x7f333c1020c0 2024-07-27T03:06:36.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.022+0000 7f33414ce640 1 --2- 172.21.15.67:0/78998981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f333c1020c0 0x7f333c19bbf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f333800c890 tx=0x7f333800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:06:36.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.022+0000 7f33327fc640 1 -- 172.21.15.67:0/78998981 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3338004220 con 0x7f333c1020c0 2024-07-27T03:06:36.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.022+0000 7f3343732640 1 -- 172.21.15.67:0/78998981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f333c19d910 con 0x7f333c1020c0 2024-07-27T03:06:36.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.023+0000 7f33327fc640 1 -- 172.21.15.67:0/78998981 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33380043c0 con 0x7f333c1020c0 2024-07-27T03:06:36.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.023+0000 7f33327fc640 1 -- 172.21.15.67:0/78998981 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3338004a00 con 0x7f333c1020c0 2024-07-27T03:06:36.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.023+0000 7f3343732640 1 -- 172.21.15.67:0/78998981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f333c1a4010 con 0x7f333c1020c0 2024-07-27T03:06:36.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.024+0000 7f3343732640 1 -- 172.21.15.67:0/78998981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f333c1a41e0 con 0x7f333c1020c0 2024-07-27T03:06:36.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.026+0000 7f33327fc640 1 -- 172.21.15.67:0/78998981 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 151003+0+0 (secure 0 0 0) 0x7f33380079b0 con 0x7f333c1020c0 2024-07-27T03:06:36.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.027+0000 7f33327fc640 1 --2- 172.21.15.67:0/78998981 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f33280b3bc0 0x7f33280b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:36.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.027+0000 7f3340ccd640 1 --2- 172.21.15.67:0/78998981 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f33280b3bc0 0x7f33280b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:36.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.027+0000 7f33327fc640 1 -- 172.21.15.67:0/78998981 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f33380e1f50 con 0x7f333c1020c0 2024-07-27T03:06:36.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.027+0000 7f3340ccd640 1 --2- 172.21.15.67:0/78998981 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f33280b3bc0 0x7f33280b60d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f332c005d00 tx=0x7f332c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:06:36.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.029+0000 7f33327fc640 1 -- 172.21.15.67:0/78998981 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f3338014030 con 0x7f333c1020c0 2024-07-27T03:06:36.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.214+0000 7f3343732640 1 -- 172.21.15.67:0/78998981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7f333c1a43b0 con 0x7f333c1020c0 2024-07-27T03:06:36.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.218+0000 7f33327fc640 1 -- 172.21.15.67:0/78998981 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v25)=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f33380aaca0 con 0x7f333c1020c0 2024-07-27T03:06:36.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.220+0000 7f3343732640 1 -- 172.21.15.67:0/78998981 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f33280b3bc0 msgr2=0x7f33280b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:36.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.220+0000 7f3343732640 1 --2- 172.21.15.67:0/78998981 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f33280b3bc0 0x7f33280b60d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f332c005d00 tx=0x7f332c00b410 comp rx=0 tx=0).stop 2024-07-27T03:06:36.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.220+0000 7f3343732640 1 -- 172.21.15.67:0/78998981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f333c1020c0 msgr2=0x7f333c19bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:36.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.220+0000 7f3343732640 1 --2- 172.21.15.67:0/78998981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f333c1020c0 0x7f333c19bbf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f333800c890 tx=0x7f333800cd50 comp rx=0 tx=0).stop 2024-07-27T03:06:36.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.220+0000 7f3343732640 1 -- 172.21.15.67:0/78998981 shutdown_connections 2024-07-27T03:06:36.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.220+0000 7f3343732640 1 --2- 172.21.15.67:0/78998981 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f33280b3bc0 0x7f33280b60d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:36.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.220+0000 7f3343732640 1 --2- 172.21.15.67:0/78998981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f333c103af0 0x7f333c19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:36.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.220+0000 7f3343732640 1 --2- 172.21.15.67:0/78998981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f333c103150 0x7f333c19c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:36.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.220+0000 7f3343732640 1 --2- 172.21.15.67:0/78998981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f333c1020c0 0x7f333c19bbf0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:36.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.220+0000 7f3343732640 1 -- 172.21.15.67:0/78998981 >> 172.21.15.67:0/78998981 conn(0x7f333c0fda50 msgr2=0x7f333c10afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:06:36.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.220+0000 7f3343732640 1 -- 172.21.15.67:0/78998981 shutdown_connections 2024-07-27T03:06:36.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:36.220+0000 7f3343732640 1 -- 172.21.15.67:0/78998981 wait complete. 2024-07-27T03:06:36.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:36 smithi067 ceph-mon[32814]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:36.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:36 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/78998981' entity='client.admin' 2024-07-27T03:06:37.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:36 smithi149 ceph-mon[37173]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:37.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:36 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/78998981' entity='client.admin' 2024-07-27T03:06:37.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:36 smithi175 ceph-mon[37125]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:37.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:36 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/78998981' entity='client.admin' 2024-07-27T03:06:38.964 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:38 smithi067 ceph-mon[32814]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:38.969 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:38.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.977+0000 7f9290b2e640 1 -- 172.21.15.67:0/3437052720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f928c103170 msgr2=0x7f928c103580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:38.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.977+0000 7f9290b2e640 1 --2- 172.21.15.67:0/3437052720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f928c103170 0x7f928c103580 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9274009900 tx=0x7f927402a480 comp rx=0 tx=0).stop 2024-07-27T03:06:38.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.977+0000 7f9290b2e640 1 -- 172.21.15.67:0/3437052720 shutdown_connections 2024-07-27T03:06:38.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.977+0000 7f9290b2e640 1 --2- 172.21.15.67:0/3437052720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f928c103b10 0x7f928c10a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:38.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.977+0000 7f9290b2e640 1 --2- 172.21.15.67:0/3437052720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f928c103170 0x7f928c103580 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:38.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.977+0000 7f9290b2e640 1 --2- 172.21.15.67:0/3437052720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f928c1020e0 0x7f928c102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:38.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.977+0000 7f9290b2e640 1 -- 172.21.15.67:0/3437052720 >> 172.21.15.67:0/3437052720 conn(0x7f928c0fda50 msgr2=0x7f928c0fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:06:38.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.977+0000 7f9290b2e640 1 -- 172.21.15.67:0/3437052720 shutdown_connections 2024-07-27T03:06:38.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.977+0000 7f9290b2e640 1 -- 172.21.15.67:0/3437052720 wait complete. 2024-07-27T03:06:38.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.978+0000 7f9290b2e640 1 Processor -- start 2024-07-27T03:06:38.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.978+0000 7f9290b2e640 1 -- start start 2024-07-27T03:06:38.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.978+0000 7f9290b2e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f928c1020e0 0x7f928c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:38.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.978+0000 7f9290b2e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f928c103170 0x7f928c198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:38.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.978+0000 7f9290b2e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f928c103b10 0x7f928c199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:38.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.979+0000 7f928a59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f928c1020e0 0x7f928c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:38.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.979+0000 7f928a59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f928c1020e0 0x7f928c198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51752/0 (socket says 172.21.15.67:51752) 2024-07-27T03:06:38.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.979+0000 7f928a59c640 1 -- 172.21.15.67:0/2831330090 learned_addr learned my addr 172.21.15.67:0/2831330090 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:06:38.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.979+0000 7f9290b2e640 1 -- 172.21.15.67:0/2831330090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f928c10dd40 con 0x7f928c1020e0 2024-07-27T03:06:38.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.979+0000 7f9290b2e640 1 -- 172.21.15.67:0/2831330090 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f928c10dbc0 con 0x7f928c103b10 2024-07-27T03:06:38.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.979+0000 7f9290b2e640 1 -- 172.21.15.67:0/2831330090 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f928c10dec0 con 0x7f928c103170 2024-07-27T03:06:38.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.979+0000 7f9289d9b640 1 --2- 172.21.15.67:0/2831330090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f928c103170 0x7f928c198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:38.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.979+0000 7f928ad9d640 1 --2- 172.21.15.67:0/2831330090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f928c103b10 0x7f928c199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:38.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.979+0000 7f928a59c640 1 -- 172.21.15.67:0/2831330090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f928c103170 msgr2=0x7f928c198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:38.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.979+0000 7f928a59c640 1 --2- 172.21.15.67:0/2831330090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f928c103170 0x7f928c198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:38.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.979+0000 7f928a59c640 1 -- 172.21.15.67:0/2831330090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f928c103b10 msgr2=0x7f928c199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:38.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.979+0000 7f928a59c640 1 --2- 172.21.15.67:0/2831330090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f928c103b10 0x7f928c199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:38.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.979+0000 7f928a59c640 1 -- 172.21.15.67:0/2831330090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928c19a360 con 0x7f928c1020e0 2024-07-27T03:06:38.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.979+0000 7f928a59c640 1 --2- 172.21.15.67:0/2831330090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f928c1020e0 0x7f928c198930 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f927c00eab0 tx=0x7f927c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:06:38.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.980+0000 7f92837fe640 1 -- 172.21.15.67:0/2831330090 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c00ce40 con 0x7f928c1020e0 2024-07-27T03:06:38.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.980+0000 7f92837fe640 1 -- 172.21.15.67:0/2831330090 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f927c011920 con 0x7f928c1020e0 2024-07-27T03:06:38.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.980+0000 7f9290b2e640 1 -- 172.21.15.67:0/2831330090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f928c19a650 con 0x7f928c1020e0 2024-07-27T03:06:38.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.980+0000 7f92837fe640 1 -- 172.21.15.67:0/2831330090 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c010a30 con 0x7f928c1020e0 2024-07-27T03:06:38.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.980+0000 7f9290b2e640 1 -- 172.21.15.67:0/2831330090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f928c06b290 con 0x7f928c1020e0 2024-07-27T03:06:38.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.981+0000 7f9290b2e640 1 -- 172.21.15.67:0/2831330090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f928c102550 con 0x7f928c1020e0 2024-07-27T03:06:38.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.982+0000 7f92837fe640 1 -- 172.21.15.67:0/2831330090 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 151003+0+0 (secure 0 0 0) 0x7f927c010bd0 con 0x7f928c1020e0 2024-07-27T03:06:38.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.983+0000 7f92837fe640 1 --2- 172.21.15.67:0/2831330090 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f92780b3bc0 0x7f92780b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:38.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.984+0000 7f9289d9b640 1 --2- 172.21.15.67:0/2831330090 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f92780b3bc0 0x7f92780b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:38.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.984+0000 7f92837fe640 1 -- 172.21.15.67:0/2831330090 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f927c0e24a0 con 0x7f928c1020e0 2024-07-27T03:06:38.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.984+0000 7f9289d9b640 1 --2- 172.21.15.67:0/2831330090 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f92780b3bc0 0x7f92780b60d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9274000c00 tx=0x7f9274031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:06:38.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.988+0000 7f92837fe640 1 -- 172.21.15.67:0/2831330090 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f927c016030 con 0x7f928c1020e0 2024-07-27T03:06:39.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:38 smithi149 ceph-mon[37173]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:39.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:38 smithi175 ceph-mon[37125]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:39.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.176+0000 7f9290b2e640 1 -- 172.21.15.67:0/2831330090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f928c108d30 con 0x7f928c1020e0 2024-07-27T03:06:39.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.182+0000 7f92837fe640 1 -- 172.21.15.67:0/2831330090 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v27)=0 v27) ==== 116+0+0 (secure 0 0 0) 0x7f927c0ab7c0 con 0x7f928c1020e0 2024-07-27T03:06:39.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.185+0000 7f9290b2e640 1 -- 172.21.15.67:0/2831330090 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f92780b3bc0 msgr2=0x7f92780b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:39.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.185+0000 7f9290b2e640 1 --2- 172.21.15.67:0/2831330090 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f92780b3bc0 0x7f92780b60d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9274000c00 tx=0x7f9274031400 comp rx=0 tx=0).stop 2024-07-27T03:06:39.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.185+0000 7f9290b2e640 1 -- 172.21.15.67:0/2831330090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f928c1020e0 msgr2=0x7f928c198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:39.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.185+0000 7f9290b2e640 1 --2- 172.21.15.67:0/2831330090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f928c1020e0 0x7f928c198930 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f927c00eab0 tx=0x7f927c00ef70 comp rx=0 tx=0).stop 2024-07-27T03:06:39.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.185+0000 7f9290b2e640 1 -- 172.21.15.67:0/2831330090 shutdown_connections 2024-07-27T03:06:39.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.185+0000 7f9290b2e640 1 --2- 172.21.15.67:0/2831330090 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f92780b3bc0 0x7f92780b60d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:39.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.185+0000 7f9290b2e640 1 --2- 172.21.15.67:0/2831330090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f928c103b10 0x7f928c199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:39.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.185+0000 7f9290b2e640 1 --2- 172.21.15.67:0/2831330090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f928c103170 0x7f928c198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:39.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.185+0000 7f9290b2e640 1 --2- 172.21.15.67:0/2831330090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f928c1020e0 0x7f928c198930 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:39.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.185+0000 7f9290b2e640 1 -- 172.21.15.67:0/2831330090 >> 172.21.15.67:0/2831330090 conn(0x7f928c0fda50 msgr2=0x7f928c106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:06:39.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.185+0000 7f9290b2e640 1 -- 172.21.15.67:0/2831330090 shutdown_connections 2024-07-27T03:06:39.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.185+0000 7f9290b2e640 1 -- 172.21.15.67:0/2831330090 wait complete. 2024-07-27T03:06:39.189 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.579+0000 7fb49cab1640 1 -- 172.21.15.67:0/1399013975 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb498103140 msgr2=0x7fb498103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:39.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.579+0000 7fb49cab1640 1 --2- 172.21.15.67:0/1399013975 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb498103140 0x7fb498103550 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb488009900 tx=0x7fb48802a480 comp rx=0 tx=0).stop 2024-07-27T03:06:39.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.579+0000 7fb49cab1640 1 -- 172.21.15.67:0/1399013975 shutdown_connections 2024-07-27T03:06:39.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.579+0000 7fb49cab1640 1 --2- 172.21.15.67:0/1399013975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb498103ae0 0x7fb49810a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:39.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.579+0000 7fb49cab1640 1 --2- 172.21.15.67:0/1399013975 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb498103140 0x7fb498103550 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:39.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.579+0000 7fb49cab1640 1 --2- 172.21.15.67:0/1399013975 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4981020b0 0x7fb498102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:39.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.579+0000 7fb49cab1640 1 -- 172.21.15.67:0/1399013975 >> 172.21.15.67:0/1399013975 conn(0x7fb4980fda20 msgr2=0x7fb4980ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:06:39.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.579+0000 7fb49cab1640 1 -- 172.21.15.67:0/1399013975 shutdown_connections 2024-07-27T03:06:39.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.579+0000 7fb49cab1640 1 -- 172.21.15.67:0/1399013975 wait complete. 2024-07-27T03:06:39.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.580+0000 7fb49cab1640 1 Processor -- start 2024-07-27T03:06:39.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.580+0000 7fb49cab1640 1 -- start start 2024-07-27T03:06:39.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.580+0000 7fb49cab1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4981020b0 0x7fb498198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:39.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.580+0000 7fb49cab1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb498103140 0x7fb498199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:39.190 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.580+0000 7fb49cab1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb498103ae0 0x7fb498199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.580+0000 7fb49cab1640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb49810dda0 con 0x7fb4981020b0 2024-07-27T03:06:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.580+0000 7fb49cab1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb49810dc20 con 0x7fb498103ae0 2024-07-27T03:06:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.580+0000 7fb49cab1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb49810df20 con 0x7fb498103140 2024-07-27T03:06:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.580+0000 7fb49659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4981020b0 0x7fb498198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.581+0000 7fb49659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4981020b0 0x7fb498198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51738/0 (socket says 172.21.15.67:51738) 2024-07-27T03:06:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.581+0000 7fb49659c640 1 -- 172.21.15.67:0/1673267592 learned_addr learned my addr 172.21.15.67:0/1673267592 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:06:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.581+0000 7fb495d9b640 1 --2- 172.21.15.67:0/1673267592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb498103140 0x7fb498199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.581+0000 7fb496d9d640 1 --2- 172.21.15.67:0/1673267592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb498103ae0 0x7fb498199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.581+0000 7fb49659c640 1 -- 172.21.15.67:0/1673267592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb498103140 msgr2=0x7fb498199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.581+0000 7fb49659c640 1 --2- 172.21.15.67:0/1673267592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb498103140 0x7fb498199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.581+0000 7fb49659c640 1 -- 172.21.15.67:0/1673267592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb498103ae0 msgr2=0x7fb498199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.581+0000 7fb49659c640 1 --2- 172.21.15.67:0/1673267592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb498103ae0 0x7fb498199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:39.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.581+0000 7fb49659c640 1 -- 172.21.15.67:0/1673267592 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb49819a3c0 con 0x7fb4981020b0 2024-07-27T03:06:39.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.581+0000 7fb49659c640 1 --2- 172.21.15.67:0/1673267592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4981020b0 0x7fb498198990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb48000b4a0 tx=0x7fb48000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:06:39.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.581+0000 7fb48f7fe640 1 -- 172.21.15.67:0/1673267592 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480014070 con 0x7fb4981020b0 2024-07-27T03:06:39.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.581+0000 7fb48f7fe640 1 -- 172.21.15.67:0/1673267592 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb480010ac0 con 0x7fb4981020b0 2024-07-27T03:06:39.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.581+0000 7fb48f7fe640 1 -- 172.21.15.67:0/1673267592 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480017c30 con 0x7fb4981020b0 2024-07-27T03:06:39.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.581+0000 7fb49cab1640 1 -- 172.21.15.67:0/1673267592 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb49819a6b0 con 0x7fb4981020b0 2024-07-27T03:06:39.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.581+0000 7fb49cab1640 1 -- 172.21.15.67:0/1673267592 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4981a0db0 con 0x7fb4981020b0 2024-07-27T03:06:39.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.582+0000 7fb49cab1640 1 -- 172.21.15.67:0/1673267592 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb498109290 con 0x7fb4981020b0 2024-07-27T03:06:39.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.584+0000 7fb48f7fe640 1 -- 172.21.15.67:0/1673267592 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 151003+0+0 (secure 0 0 0) 0x7fb480017070 con 0x7fb4981020b0 2024-07-27T03:06:39.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.585+0000 7fb48f7fe640 1 --2- 172.21.15.67:0/1673267592 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb4840b3b40 0x7fb4840b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:39.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.585+0000 7fb495d9b640 1 --2- 172.21.15.67:0/1673267592 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb4840b3b40 0x7fb4840b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:39.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.585+0000 7fb48f7fe640 1 -- 172.21.15.67:0/1673267592 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb4800e1620 con 0x7fb4981020b0 2024-07-27T03:06:39.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.585+0000 7fb495d9b640 1 --2- 172.21.15.67:0/1673267592 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb4840b3b40 0x7fb4840b6050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb488004770 tx=0x7fb4880314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:06:39.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.588+0000 7fb48f7fe640 1 -- 172.21.15.67:0/1673267592 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fb4800aa2f0 con 0x7fb4981020b0 2024-07-27T03:06:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.774+0000 7fb49cab1640 1 -- 172.21.15.67:0/1673267592 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7fb49802e100 con 0x7fb4981020b0 2024-07-27T03:06:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.780+0000 7fb48f7fe640 1 -- 172.21.15.67:0/1673267592 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v26)=0 v26) ==== 117+0+0 (secure 0 0 0) 0x7fb4800ae840 con 0x7fb4981020b0 2024-07-27T03:06:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.783+0000 7fb49cab1640 1 -- 172.21.15.67:0/1673267592 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb4840b3b40 msgr2=0x7fb4840b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.783+0000 7fb49cab1640 1 --2- 172.21.15.67:0/1673267592 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb4840b3b40 0x7fb4840b6050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb488004770 tx=0x7fb4880314a0 comp rx=0 tx=0).stop 2024-07-27T03:06:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.783+0000 7fb49cab1640 1 -- 172.21.15.67:0/1673267592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4981020b0 msgr2=0x7fb498198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.783+0000 7fb49cab1640 1 --2- 172.21.15.67:0/1673267592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4981020b0 0x7fb498198990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb48000b4a0 tx=0x7fb48000b960 comp rx=0 tx=0).stop 2024-07-27T03:06:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.783+0000 7fb49cab1640 1 -- 172.21.15.67:0/1673267592 shutdown_connections 2024-07-27T03:06:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.783+0000 7fb49cab1640 1 --2- 172.21.15.67:0/1673267592 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb4840b3b40 0x7fb4840b6050 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.783+0000 7fb49cab1640 1 --2- 172.21.15.67:0/1673267592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb498103ae0 0x7fb498199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.783+0000 7fb49cab1640 1 --2- 172.21.15.67:0/1673267592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb498103140 0x7fb498199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.783+0000 7fb49cab1640 1 --2- 172.21.15.67:0/1673267592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4981020b0 0x7fb498198990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.783+0000 7fb49cab1640 1 -- 172.21.15.67:0/1673267592 >> 172.21.15.67:0/1673267592 conn(0x7fb4980fda20 msgr2=0x7fb498106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:06:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.783+0000 7fb49cab1640 1 -- 172.21.15.67:0/1673267592 shutdown_connections 2024-07-27T03:06:39.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:38.783+0000 7fb49cab1640 1 -- 172.21.15.67:0/1673267592 wait complete. 2024-07-27T03:06:39.962 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:39 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1673267592' entity='client.admin' 2024-07-27T03:06:39.962 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:39 smithi067 ceph-mon[32814]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:39.962 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:39 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2831330090' entity='client.admin' 2024-07-27T03:06:39.962 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:39 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/955489053' entity='client.admin' 2024-07-27T03:06:39.997 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi067') 2024-07-27T03:06:39.997 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi067') 2024-07-27T03:06:40.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:39 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1673267592' entity='client.admin' 2024-07-27T03:06:40.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:39 smithi175 ceph-mon[37125]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:40.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:39 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2831330090' entity='client.admin' 2024-07-27T03:06:40.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:39 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/955489053' entity='client.admin' 2024-07-27T03:06:40.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:39 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1673267592' entity='client.admin' 2024-07-27T03:06:40.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:39 smithi149 ceph-mon[37173]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:40.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:39 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2831330090' entity='client.admin' 2024-07-27T03:06:40.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:39 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/955489053' entity='client.admin' 2024-07-27T03:06:40.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.201+0000 7f4d52b52640 1 -- 172.21.15.67:0/673607434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d4c103ae0 msgr2=0x7f4d4c10a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:40.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.201+0000 7f4d52b52640 1 --2- 172.21.15.67:0/673607434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d4c103ae0 0x7f4d4c10a480 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009ca0 tx=0x7f4d4402a950 comp rx=0 tx=0).stop 2024-07-27T03:06:40.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.201+0000 7f4d52b52640 1 -- 172.21.15.67:0/673607434 shutdown_connections 2024-07-27T03:06:40.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.201+0000 7f4d52b52640 1 --2- 172.21.15.67:0/673607434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d4c103ae0 0x7f4d4c10a480 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:40.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.201+0000 7f4d52b52640 1 --2- 172.21.15.67:0/673607434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d4c103140 0x7f4d4c103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:40.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.201+0000 7f4d52b52640 1 --2- 172.21.15.67:0/673607434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d4c1020b0 0x7f4d4c102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:40.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.201+0000 7f4d52b52640 1 -- 172.21.15.67:0/673607434 >> 172.21.15.67:0/673607434 conn(0x7f4d4c0fda20 msgr2=0x7f4d4c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:06:40.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.201+0000 7f4d52b52640 1 -- 172.21.15.67:0/673607434 shutdown_connections 2024-07-27T03:06:40.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.201+0000 7f4d52b52640 1 -- 172.21.15.67:0/673607434 wait complete. 2024-07-27T03:06:40.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.202+0000 7f4d52b52640 1 Processor -- start 2024-07-27T03:06:40.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.202+0000 7f4d52b52640 1 -- start start 2024-07-27T03:06:40.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.202+0000 7f4d52b52640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d4c1020b0 0x7f4d4c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:40.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.202+0000 7f4d52b52640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d4c103140 0x7f4d4c198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:40.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.202+0000 7f4d52b52640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d4c103ae0 0x7f4d4c199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:40.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.202+0000 7f4d52b52640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c10dd10 con 0x7f4d4c103140 2024-07-27T03:06:40.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.202+0000 7f4d52b52640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c10db90 con 0x7f4d4c103ae0 2024-07-27T03:06:40.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.202+0000 7f4d52b52640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c10de90 con 0x7f4d4c1020b0 2024-07-27T03:06:40.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.203+0000 7f4d4bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d4c103140 0x7f4d4c198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:40.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.203+0000 7f4d4bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d4c103140 0x7f4d4c198f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51780/0 (socket says 172.21.15.67:51780) 2024-07-27T03:06:40.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.203+0000 7f4d4bfff640 1 -- 172.21.15.67:0/955489053 learned_addr learned my addr 172.21.15.67:0/955489053 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:06:40.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.203+0000 7f4d510ef640 1 --2- 172.21.15.67:0/955489053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d4c103ae0 0x7f4d4c199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:40.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.203+0000 7f4d508ee640 1 --2- 172.21.15.67:0/955489053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d4c1020b0 0x7f4d4c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:40.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.203+0000 7f4d4bfff640 1 -- 172.21.15.67:0/955489053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d4c1020b0 msgr2=0x7f4d4c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:40.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.203+0000 7f4d4bfff640 1 --2- 172.21.15.67:0/955489053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d4c1020b0 0x7f4d4c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:40.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.203+0000 7f4d4bfff640 1 -- 172.21.15.67:0/955489053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d4c103ae0 msgr2=0x7f4d4c199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:40.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.203+0000 7f4d4bfff640 1 --2- 172.21.15.67:0/955489053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d4c103ae0 0x7f4d4c199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:40.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.203+0000 7f4d4bfff640 1 -- 172.21.15.67:0/955489053 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4c19a320 con 0x7f4d4c103140 2024-07-27T03:06:40.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.203+0000 7f4d4bfff640 1 --2- 172.21.15.67:0/955489053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d4c103140 0x7f4d4c198f60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009970 tx=0x7f4d38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:06:40.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.203+0000 7f4d49ffb640 1 -- 172.21.15.67:0/955489053 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3800d020 con 0x7f4d4c103140 2024-07-27T03:06:40.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.203+0000 7f4d49ffb640 1 -- 172.21.15.67:0/955489053 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d3800d1c0 con 0x7f4d4c103140 2024-07-27T03:06:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.203+0000 7f4d52b52640 1 -- 172.21.15.67:0/955489053 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4c19a610 con 0x7f4d4c103140 2024-07-27T03:06:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.203+0000 7f4d49ffb640 1 -- 172.21.15.67:0/955489053 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3800cc50 con 0x7f4d4c103140 2024-07-27T03:06:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.204+0000 7f4d52b52640 1 -- 172.21.15.67:0/955489053 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d4c06aaa0 con 0x7f4d4c103140 2024-07-27T03:06:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.205+0000 7f4d52b52640 1 -- 172.21.15.67:0/955489053 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d4c102520 con 0x7f4d4c103140 2024-07-27T03:06:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.207+0000 7f4d49ffb640 1 -- 172.21.15.67:0/955489053 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 151003+0+0 (secure 0 0 0) 0x7f4d38007860 con 0x7f4d4c103140 2024-07-27T03:06:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.208+0000 7f4d49ffb640 1 --2- 172.21.15.67:0/955489053 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4d3c0b3b40 0x7f4d3c0b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.208+0000 7f4d508ee640 1 --2- 172.21.15.67:0/955489053 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4d3c0b3b40 0x7f4d3c0b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.208+0000 7f4d49ffb640 1 -- 172.21.15.67:0/955489053 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4d380e37a0 con 0x7f4d4c103140 2024-07-27T03:06:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.209+0000 7f4d508ee640 1 --2- 172.21.15.67:0/955489053 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4d3c0b3b40 0x7f4d3c0b6050 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4d40005fd0 tx=0x7f4d40005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:06:40.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.210+0000 7f4d49ffb640 1 -- 172.21.15.67:0/955489053 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f4d380acac0 con 0x7f4d4c103140 2024-07-27T03:06:40.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.397+0000 7f4d52b52640 1 -- 172.21.15.67:0/955489053 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f4d4c06b680 con 0x7f4d4c103140 2024-07-27T03:06:40.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.401+0000 7f4d49ffb640 1 -- 172.21.15.67:0/955489053 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v28)=0 v28) ==== 120+0+0 (secure 0 0 0) 0x7f4d380ac3c0 con 0x7f4d4c103140 2024-07-27T03:06:40.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.403+0000 7f4d52b52640 1 -- 172.21.15.67:0/955489053 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4d3c0b3b40 msgr2=0x7f4d3c0b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:40.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.403+0000 7f4d52b52640 1 --2- 172.21.15.67:0/955489053 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4d3c0b3b40 0x7f4d3c0b6050 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4d40005fd0 tx=0x7f4d40005850 comp rx=0 tx=0).stop 2024-07-27T03:06:40.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.403+0000 7f4d52b52640 1 -- 172.21.15.67:0/955489053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d4c103140 msgr2=0x7f4d4c198f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:40.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.403+0000 7f4d52b52640 1 --2- 172.21.15.67:0/955489053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d4c103140 0x7f4d4c198f60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009970 tx=0x7f4d38009e30 comp rx=0 tx=0).stop 2024-07-27T03:06:40.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.403+0000 7f4d52b52640 1 -- 172.21.15.67:0/955489053 shutdown_connections 2024-07-27T03:06:40.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.403+0000 7f4d52b52640 1 --2- 172.21.15.67:0/955489053 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4d3c0b3b40 0x7f4d3c0b6050 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:40.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.403+0000 7f4d52b52640 1 --2- 172.21.15.67:0/955489053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d4c103ae0 0x7f4d4c199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:40.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.403+0000 7f4d52b52640 1 --2- 172.21.15.67:0/955489053 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d4c103140 0x7f4d4c198f60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:40.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.403+0000 7f4d52b52640 1 --2- 172.21.15.67:0/955489053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d4c1020b0 0x7f4d4c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:40.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.403+0000 7f4d52b52640 1 -- 172.21.15.67:0/955489053 >> 172.21.15.67:0/955489053 conn(0x7f4d4c0fda20 msgr2=0x7f4d4c106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:06:40.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.403+0000 7f4d52b52640 1 -- 172.21.15.67:0/955489053 shutdown_connections 2024-07-27T03:06:40.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:39.403+0000 7f4d52b52640 1 -- 172.21.15.67:0/955489053 wait complete. 2024-07-27T03:06:41.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.198+0000 7f98b92c8640 1 -- 172.21.15.67:0/3091028871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98b41031c0 msgr2=0x7f98b4103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:41.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.198+0000 7f98b92c8640 1 --2- 172.21.15.67:0/3091028871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98b41031c0 0x7f98b4103630 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f98a4009900 tx=0x7f98a402a480 comp rx=0 tx=0).stop 2024-07-27T03:06:41.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.198+0000 7f98b92c8640 1 -- 172.21.15.67:0/3091028871 shutdown_connections 2024-07-27T03:06:41.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.198+0000 7f98b92c8640 1 --2- 172.21.15.67:0/3091028871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b4104bf0 0x7f98b410b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:41.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.198+0000 7f98b92c8640 1 --2- 172.21.15.67:0/3091028871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98b4104250 0x7f98b4104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:41.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.198+0000 7f98b92c8640 1 --2- 172.21.15.67:0/3091028871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98b41031c0 0x7f98b4103630 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:41.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.198+0000 7f98b92c8640 1 -- 172.21.15.67:0/3091028871 >> 172.21.15.67:0/3091028871 conn(0x7f98b40feb30 msgr2=0x7f98b4100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:06:41.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.198+0000 7f98b92c8640 1 -- 172.21.15.67:0/3091028871 shutdown_connections 2024-07-27T03:06:41.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.198+0000 7f98b92c8640 1 -- 172.21.15.67:0/3091028871 wait complete. 2024-07-27T03:06:41.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.199+0000 7f98b92c8640 1 Processor -- start 2024-07-27T03:06:41.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.199+0000 7f98b92c8640 1 -- start start 2024-07-27T03:06:41.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.199+0000 7f98b92c8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b41031c0 0x7f98b4199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:41.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.199+0000 7f98b92c8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98b4104250 0x7f98b419a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:41.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.199+0000 7f98b92c8640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98b4104bf0 0x7f98b419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:41.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.200+0000 7f98b92c8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f98b410ee20 con 0x7f98b4104250 2024-07-27T03:06:41.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.200+0000 7f98b92c8640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f98b410eca0 con 0x7f98b4104bf0 2024-07-27T03:06:41.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.200+0000 7f98b92c8640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98b410efa0 con 0x7f98b41031c0 2024-07-27T03:06:41.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.200+0000 7f98b27fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98b4104250 0x7f98b419a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:41.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.200+0000 7f98b27fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98b4104250 0x7f98b419a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51790/0 (socket says 172.21.15.67:51790) 2024-07-27T03:06:41.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.200+0000 7f98b27fc640 1 -- 172.21.15.67:0/875379389 learned_addr learned my addr 172.21.15.67:0/875379389 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:06:41.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.200+0000 7f98b2ffd640 1 --2- 172.21.15.67:0/875379389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b41031c0 0x7f98b4199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:41.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.200+0000 7f98b37fe640 1 --2- 172.21.15.67:0/875379389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98b4104bf0 0x7f98b419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:41.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.200+0000 7f98b27fc640 1 -- 172.21.15.67:0/875379389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b41031c0 msgr2=0x7f98b4199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:41.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.200+0000 7f98b27fc640 1 --2- 172.21.15.67:0/875379389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b41031c0 0x7f98b4199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:41.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.200+0000 7f98b27fc640 1 -- 172.21.15.67:0/875379389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98b4104bf0 msgr2=0x7f98b419ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:41.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.200+0000 7f98b27fc640 1 --2- 172.21.15.67:0/875379389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98b4104bf0 0x7f98b419ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:41.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.200+0000 7f98b27fc640 1 -- 172.21.15.67:0/875379389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b419b3b0 con 0x7f98b4104250 2024-07-27T03:06:41.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.200+0000 7f98b27fc640 1 --2- 172.21.15.67:0/875379389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98b4104250 0x7f98b419a0f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f989c00c920 tx=0x7f989c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:06:41.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.200+0000 7f98abfff640 1 -- 172.21.15.67:0/875379389 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c007c70 con 0x7f98b4104250 2024-07-27T03:06:41.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.201+0000 7f98b92c8640 1 -- 172.21.15.67:0/875379389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b419b6a0 con 0x7f98b4104250 2024-07-27T03:06:41.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.201+0000 7f98abfff640 1 -- 172.21.15.67:0/875379389 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f989c007e10 con 0x7f98b4104250 2024-07-27T03:06:41.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.201+0000 7f98abfff640 1 -- 172.21.15.67:0/875379389 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c018820 con 0x7f98b4104250 2024-07-27T03:06:41.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.201+0000 7f98b92c8640 1 -- 172.21.15.67:0/875379389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b41a1e30 con 0x7f98b4104250 2024-07-27T03:06:41.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.202+0000 7f98b92c8640 1 -- 172.21.15.67:0/875379389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98b410a3a0 con 0x7f98b4104250 2024-07-27T03:06:41.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.203+0000 7f98abfff640 1 -- 172.21.15.67:0/875379389 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 151003+0+0 (secure 0 0 0) 0x7f989c00f450 con 0x7f98b4104250 2024-07-27T03:06:41.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.204+0000 7f98abfff640 1 --2- 172.21.15.67:0/875379389 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f98a00b3c40 0x7f98a00b6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:41.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.204+0000 7f98b2ffd640 1 --2- 172.21.15.67:0/875379389 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f98a00b3c40 0x7f98a00b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:41.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.204+0000 7f98abfff640 1 -- 172.21.15.67:0/875379389 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f989c0e2270 con 0x7f98b4104250 2024-07-27T03:06:41.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.205+0000 7f98b2ffd640 1 --2- 172.21.15.67:0/875379389 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f98a00b3c40 0x7f98a00b6150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f98a4000c00 tx=0x7f98a4031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:06:41.209 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.207+0000 7f98abfff640 1 -- 172.21.15.67:0/875379389 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f989c014030 con 0x7f98b4104250 2024-07-27T03:06:41.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.392+0000 7f98b92c8640 1 -- 172.21.15.67:0/875379389 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f98b402e100 con 0x7f98b4104250 2024-07-27T03:06:41.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.396+0000 7f98abfff640 1 -- 172.21.15.67:0/875379389 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v29)=0 v29) ==== 127+0+0 (secure 0 0 0) 0x7f989c0ab600 con 0x7f98b4104250 2024-07-27T03:06:41.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.398+0000 7f98b92c8640 1 -- 172.21.15.67:0/875379389 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f98a00b3c40 msgr2=0x7f98a00b6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:41.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.398+0000 7f98b92c8640 1 --2- 172.21.15.67:0/875379389 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f98a00b3c40 0x7f98a00b6150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f98a4000c00 tx=0x7f98a4031400 comp rx=0 tx=0).stop 2024-07-27T03:06:41.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.398+0000 7f98b92c8640 1 -- 172.21.15.67:0/875379389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98b4104250 msgr2=0x7f98b419a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:41.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.398+0000 7f98b92c8640 1 --2- 172.21.15.67:0/875379389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98b4104250 0x7f98b419a0f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f989c00c920 tx=0x7f989c00cde0 comp rx=0 tx=0).stop 2024-07-27T03:06:41.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.398+0000 7f98b92c8640 1 -- 172.21.15.67:0/875379389 shutdown_connections 2024-07-27T03:06:41.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.398+0000 7f98b92c8640 1 --2- 172.21.15.67:0/875379389 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f98a00b3c40 0x7f98a00b6150 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:41.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.398+0000 7f98b92c8640 1 --2- 172.21.15.67:0/875379389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98b4104bf0 0x7f98b419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:41.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.398+0000 7f98b92c8640 1 --2- 172.21.15.67:0/875379389 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f98b4104250 0x7f98b419a0f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:41.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.398+0000 7f98b92c8640 1 --2- 172.21.15.67:0/875379389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98b41031c0 0x7f98b4199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:41.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.398+0000 7f98b92c8640 1 -- 172.21.15.67:0/875379389 >> 172.21.15.67:0/875379389 conn(0x7f98b40feb30 msgr2=0x7f98b4107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:06:41.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.399+0000 7f98b92c8640 1 -- 172.21.15.67:0/875379389 shutdown_connections 2024-07-27T03:06:41.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.399+0000 7f98b92c8640 1 -- 172.21.15.67:0/875379389 wait complete. 2024-07-27T03:06:41.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.435+0000 7fbe150e4640 1 -- 172.21.15.67:0/3937775376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe101031e0 msgr2=0x7fbe10103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.435+0000 7fbe150e4640 1 --2- 172.21.15.67:0/3937775376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe101031e0 0x7fbe10103650 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009900 tx=0x7fbe0002a480 comp rx=0 tx=0).stop 2024-07-27T03:06:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.435+0000 7fbe150e4640 1 -- 172.21.15.67:0/3937775376 shutdown_connections 2024-07-27T03:06:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.435+0000 7fbe150e4640 1 --2- 172.21.15.67:0/3937775376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe10104c10 0x7fbe1010b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.435+0000 7fbe150e4640 1 --2- 172.21.15.67:0/3937775376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe10104270 0x7fbe10104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.435+0000 7fbe150e4640 1 --2- 172.21.15.67:0/3937775376 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe101031e0 0x7fbe10103650 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.435+0000 7fbe150e4640 1 -- 172.21.15.67:0/3937775376 >> 172.21.15.67:0/3937775376 conn(0x7fbe100feb50 msgr2=0x7fbe10101000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:06:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.435+0000 7fbe150e4640 1 -- 172.21.15.67:0/3937775376 shutdown_connections 2024-07-27T03:06:41.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.435+0000 7fbe150e4640 1 -- 172.21.15.67:0/3937775376 wait complete. 2024-07-27T03:06:41.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.435+0000 7fbe150e4640 1 Processor -- start 2024-07-27T03:06:41.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.436+0000 7fbe150e4640 1 -- start start 2024-07-27T03:06:41.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.436+0000 7fbe150e4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe101031e0 0x7fbe10199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:41.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.436+0000 7fbe150e4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe10104270 0x7fbe1019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:41.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.436+0000 7fbe150e4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe10104c10 0x7fbe1019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:41.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.436+0000 7fbe150e4640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1010ee40 con 0x7fbe10104270 2024-07-27T03:06:41.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.436+0000 7fbe150e4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1010ecc0 con 0x7fbe101031e0 2024-07-27T03:06:41.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.436+0000 7fbe150e4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1010efc0 con 0x7fbe10104c10 2024-07-27T03:06:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.436+0000 7fbe0e59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe10104270 0x7fbe1019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.436+0000 7fbe0e59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe10104270 0x7fbe1019a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51810/0 (socket says 172.21.15.67:51810) 2024-07-27T03:06:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.436+0000 7fbe0e59c640 1 -- 172.21.15.67:0/3059972778 learned_addr learned my addr 172.21.15.67:0/3059972778 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:06:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.436+0000 7fbe0f59e640 1 --2- 172.21.15.67:0/3059972778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe10104c10 0x7fbe1019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.436+0000 7fbe0ed9d640 1 --2- 172.21.15.67:0/3059972778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe101031e0 0x7fbe10199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.436+0000 7fbe0e59c640 1 -- 172.21.15.67:0/3059972778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe10104c10 msgr2=0x7fbe1019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.436+0000 7fbe0e59c640 1 --2- 172.21.15.67:0/3059972778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe10104c10 0x7fbe1019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.436+0000 7fbe0e59c640 1 -- 172.21.15.67:0/3059972778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe101031e0 msgr2=0x7fbe10199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.436+0000 7fbe0e59c640 1 --2- 172.21.15.67:0/3059972778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe101031e0 0x7fbe10199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.436+0000 7fbe0e59c640 1 -- 172.21.15.67:0/3059972778 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1019b450 con 0x7fbe10104270 2024-07-27T03:06:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.437+0000 7fbe0e59c640 1 --2- 172.21.15.67:0/3059972778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe10104270 0x7fbe1019a090 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400e9f0 tx=0x7fbe0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:06:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.437+0000 7fbdf7fff640 1 -- 172.21.15.67:0/3059972778 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0400cd80 con 0x7fbe10104270 2024-07-27T03:06:41.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.437+0000 7fbdf7fff640 1 -- 172.21.15.67:0/3059972778 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe040108e0 con 0x7fbe10104270 2024-07-27T03:06:41.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.437+0000 7fbe150e4640 1 -- 172.21.15.67:0/3059972778 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe1019b740 con 0x7fbe10104270 2024-07-27T03:06:41.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.437+0000 7fbdf7fff640 1 -- 172.21.15.67:0/3059972778 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe04018980 con 0x7fbe10104270 2024-07-27T03:06:41.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.437+0000 7fbe150e4640 1 -- 172.21.15.67:0/3059972778 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe101a1e40 con 0x7fbe10104270 2024-07-27T03:06:41.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.438+0000 7fbe150e4640 1 -- 172.21.15.67:0/3059972778 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe10103650 con 0x7fbe10104270 2024-07-27T03:06:41.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.440+0000 7fbdf7fff640 1 -- 172.21.15.67:0/3059972778 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 151003+0+0 (secure 0 0 0) 0x7fbe04010430 con 0x7fbe10104270 2024-07-27T03:06:41.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.440+0000 7fbdf7fff640 1 --2- 172.21.15.67:0/3059972778 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbdf80b3b40 0x7fbdf80b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:41.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.441+0000 7fbe0ed9d640 1 --2- 172.21.15.67:0/3059972778 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbdf80b3b40 0x7fbdf80b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:41.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.441+0000 7fbdf7fff640 1 -- 172.21.15.67:0/3059972778 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbe040e2280 con 0x7fbe10104270 2024-07-27T03:06:41.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.441+0000 7fbe0ed9d640 1 --2- 172.21.15.67:0/3059972778 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbdf80b3b40 0x7fbdf80b6050 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbe00000c00 tx=0x7fbe00031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:06:41.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.445+0000 7fbdf7fff640 1 -- 172.21.15.67:0/3059972778 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fbe040aaeb0 con 0x7fbe10104270 2024-07-27T03:06:41.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.629+0000 7fbe150e4640 1 -- 172.21.15.67:0/3059972778 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7fbe1010a3c0 con 0x7fbe10104270 2024-07-27T03:06:41.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.633+0000 7fbdf7fff640 1 -- 172.21.15.67:0/3059972778 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v30)=0 v30) ==== 121+0+0 (secure 0 0 0) 0x7fbe040af400 con 0x7fbe10104270 2024-07-27T03:06:41.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.635+0000 7fbe150e4640 1 -- 172.21.15.67:0/3059972778 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbdf80b3b40 msgr2=0x7fbdf80b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:41.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.635+0000 7fbe150e4640 1 --2- 172.21.15.67:0/3059972778 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbdf80b3b40 0x7fbdf80b6050 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbe00000c00 tx=0x7fbe00031400 comp rx=0 tx=0).stop 2024-07-27T03:06:41.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.635+0000 7fbe150e4640 1 -- 172.21.15.67:0/3059972778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe10104270 msgr2=0x7fbe1019a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:41.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.635+0000 7fbe150e4640 1 --2- 172.21.15.67:0/3059972778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe10104270 0x7fbe1019a090 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbe0400e9f0 tx=0x7fbe0400eeb0 comp rx=0 tx=0).stop 2024-07-27T03:06:41.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.635+0000 7fbe150e4640 1 -- 172.21.15.67:0/3059972778 shutdown_connections 2024-07-27T03:06:41.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.635+0000 7fbe150e4640 1 --2- 172.21.15.67:0/3059972778 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbdf80b3b40 0x7fbdf80b6050 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:41.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.635+0000 7fbe150e4640 1 --2- 172.21.15.67:0/3059972778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe10104c10 0x7fbe1019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:41.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.635+0000 7fbe150e4640 1 --2- 172.21.15.67:0/3059972778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe10104270 0x7fbe1019a090 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:41.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.635+0000 7fbe150e4640 1 --2- 172.21.15.67:0/3059972778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe101031e0 0x7fbe10199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:41.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.635+0000 7fbe150e4640 1 -- 172.21.15.67:0/3059972778 >> 172.21.15.67:0/3059972778 conn(0x7fbe100feb50 msgr2=0x7fbe10107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:06:41.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.635+0000 7fbe150e4640 1 -- 172.21.15.67:0/3059972778 shutdown_connections 2024-07-27T03:06:41.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:41.635+0000 7fbe150e4640 1 -- 172.21.15.67:0/3059972778 wait complete. 2024-07-27T03:06:42.038 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi067') 2024-07-27T03:06:42.038 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi067') 2024-07-27T03:06:42.678 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:42 smithi067 ceph-mon[32814]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:42.678 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:42 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/875379389' entity='client.admin' 2024-07-27T03:06:42.678 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:42 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3059972778' entity='client.admin' 2024-07-27T03:06:42.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:42 smithi149 ceph-mon[37173]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:42.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:42 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/875379389' entity='client.admin' 2024-07-27T03:06:42.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:42 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3059972778' entity='client.admin' 2024-07-27T03:06:42.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:42 smithi175 ceph-mon[37125]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:42.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:42 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/875379389' entity='client.admin' 2024-07-27T03:06:42.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:42 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3059972778' entity='client.admin' 2024-07-27T03:06:43.640 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi067') 2024-07-27T03:06:43.641 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi067') 2024-07-27T03:06:43.641 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi067') 2024-07-27T03:06:43.641 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi067') 2024-07-27T03:06:43.641 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi067') 2024-07-27T03:06:43.641 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi067') 2024-07-27T03:06:43.641 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi067') 2024-07-27T03:06:43.641 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi067') 2024-07-27T03:06:43.641 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi067') 2024-07-27T03:06:44.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.105+0000 7f15ad766640 1 -- 172.21.15.67:0/1532945707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15a81020d0 msgr2=0x7f15a8102540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:44.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.105+0000 7f15ad766640 1 --2- 172.21.15.67:0/1532945707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15a81020d0 0x7f15a8102540 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1598009900 tx=0x7f159802a480 comp rx=0 tx=0).stop 2024-07-27T03:06:44.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.105+0000 7f15ad766640 1 -- 172.21.15.67:0/1532945707 shutdown_connections 2024-07-27T03:06:44.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.105+0000 7f15ad766640 1 --2- 172.21.15.67:0/1532945707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15a8103b00 0x7f15a810a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:44.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.105+0000 7f15ad766640 1 --2- 172.21.15.67:0/1532945707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15a8103160 0x7f15a8103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:44.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.105+0000 7f15ad766640 1 --2- 172.21.15.67:0/1532945707 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15a81020d0 0x7f15a8102540 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:44.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.105+0000 7f15ad766640 1 -- 172.21.15.67:0/1532945707 >> 172.21.15.67:0/1532945707 conn(0x7f15a80fda40 msgr2=0x7f15a80ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:06:44.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.105+0000 7f15ad766640 1 -- 172.21.15.67:0/1532945707 shutdown_connections 2024-07-27T03:06:44.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.105+0000 7f15ad766640 1 -- 172.21.15.67:0/1532945707 wait complete. 2024-07-27T03:06:44.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.106+0000 7f15ad766640 1 Processor -- start 2024-07-27T03:06:44.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.106+0000 7f15ad766640 1 -- start start 2024-07-27T03:06:44.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.106+0000 7f15ad766640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15a81020d0 0x7f15a8199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:44.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.106+0000 7f15ad766640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15a8103160 0x7f15a819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:44.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.106+0000 7f15ad766640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15a8103b00 0x7f15a819aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:44.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.106+0000 7f15ad766640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f15a810dd30 con 0x7f15a81020d0 2024-07-27T03:06:44.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.106+0000 7f15ad766640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f15a810dbb0 con 0x7f15a8103160 2024-07-27T03:06:44.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.106+0000 7f15ad766640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f15a810deb0 con 0x7f15a8103b00 2024-07-27T03:06:44.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.106+0000 7f15a6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15a81020d0 0x7f15a8199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:44.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.106+0000 7f15a6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15a81020d0 0x7f15a8199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51828/0 (socket says 172.21.15.67:51828) 2024-07-27T03:06:44.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.107+0000 7f15a6ffd640 1 -- 172.21.15.67:0/2768481769 learned_addr learned my addr 172.21.15.67:0/2768481769 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:06:44.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.107+0000 7f15a67fc640 1 --2- 172.21.15.67:0/2768481769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15a8103160 0x7f15a819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:44.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.107+0000 7f15a77fe640 1 --2- 172.21.15.67:0/2768481769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15a8103b00 0x7f15a819aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:44.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.107+0000 7f15a6ffd640 1 -- 172.21.15.67:0/2768481769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15a8103b00 msgr2=0x7f15a819aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:44.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.107+0000 7f15a6ffd640 1 --2- 172.21.15.67:0/2768481769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15a8103b00 0x7f15a819aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:44.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.107+0000 7f15a6ffd640 1 -- 172.21.15.67:0/2768481769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15a8103160 msgr2=0x7f15a819a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:44.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.107+0000 7f15a6ffd640 1 --2- 172.21.15.67:0/2768481769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15a8103160 0x7f15a819a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:44.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.107+0000 7f15a6ffd640 1 -- 172.21.15.67:0/2768481769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a819b4c0 con 0x7f15a81020d0 2024-07-27T03:06:44.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.107+0000 7f15a77fe640 1 --2- 172.21.15.67:0/2768481769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15a8103b00 0x7f15a819aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:06:44.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.107+0000 7f15a6ffd640 1 --2- 172.21.15.67:0/2768481769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15a81020d0 0x7f15a8199a90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1598006010 tx=0x7f1598004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:06:44.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.107+0000 7f158ffff640 1 -- 172.21.15.67:0/2768481769 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159803f030 con 0x7f15a81020d0 2024-07-27T03:06:44.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.107+0000 7f158ffff640 1 -- 172.21.15.67:0/2768481769 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1598003cb0 con 0x7f15a81020d0 2024-07-27T03:06:44.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.107+0000 7f15ad766640 1 -- 172.21.15.67:0/2768481769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a819b750 con 0x7f15a81020d0 2024-07-27T03:06:44.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.107+0000 7f158ffff640 1 -- 172.21.15.67:0/2768481769 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598035070 con 0x7f15a81020d0 2024-07-27T03:06:44.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.107+0000 7f15ad766640 1 -- 172.21.15.67:0/2768481769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a81a1eb0 con 0x7f15a81020d0 2024-07-27T03:06:44.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.109+0000 7f15ad766640 1 -- 172.21.15.67:0/2768481769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a8102540 con 0x7f15a81020d0 2024-07-27T03:06:44.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.110+0000 7f158ffff640 1 -- 172.21.15.67:0/2768481769 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 151003+0+0 (secure 0 0 0) 0x7f1598042030 con 0x7f15a81020d0 2024-07-27T03:06:44.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.111+0000 7f158ffff640 1 --2- 172.21.15.67:0/2768481769 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f15900b3bc0 0x7f15900b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:44.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.111+0000 7f15a67fc640 1 --2- 172.21.15.67:0/2768481769 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f15900b3bc0 0x7f15900b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:44.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.111+0000 7f158ffff640 1 -- 172.21.15.67:0/2768481769 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1598102af0 con 0x7f15a81020d0 2024-07-27T03:06:44.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.111+0000 7f15a67fc640 1 --2- 172.21.15.67:0/2768481769 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f15900b3bc0 0x7f15900b60d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f159c005d00 tx=0x7f159c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:06:44.116 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.114+0000 7f158ffff640 1 -- 172.21.15.67:0/2768481769 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f15980cbe10 con 0x7f15a81020d0 2024-07-27T03:06:44.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.300+0000 7f15ad766640 1 -- 172.21.15.67:0/2768481769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f15a81a2080 con 0x7f15a81020d0 2024-07-27T03:06:44.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.304+0000 7f158ffff640 1 -- 172.21.15.67:0/2768481769 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v31)=0 v31) ==== 121+0+0 (secure 0 0 0) 0x7f15980cb710 con 0x7f15a81020d0 2024-07-27T03:06:44.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.306+0000 7f15ad766640 1 -- 172.21.15.67:0/2768481769 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f15900b3bc0 msgr2=0x7f15900b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:44.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.306+0000 7f15ad766640 1 --2- 172.21.15.67:0/2768481769 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f15900b3bc0 0x7f15900b60d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f159c005d00 tx=0x7f159c00b410 comp rx=0 tx=0).stop 2024-07-27T03:06:44.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.306+0000 7f15ad766640 1 -- 172.21.15.67:0/2768481769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15a81020d0 msgr2=0x7f15a8199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:44.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.306+0000 7f15ad766640 1 --2- 172.21.15.67:0/2768481769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15a81020d0 0x7f15a8199a90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1598006010 tx=0x7f1598004340 comp rx=0 tx=0).stop 2024-07-27T03:06:44.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.306+0000 7f15ad766640 1 -- 172.21.15.67:0/2768481769 shutdown_connections 2024-07-27T03:06:44.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.306+0000 7f15ad766640 1 --2- 172.21.15.67:0/2768481769 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f15900b3bc0 0x7f15900b60d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:44.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.306+0000 7f15ad766640 1 --2- 172.21.15.67:0/2768481769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15a8103b00 0x7f15a819aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.306+0000 7f15ad766640 1 --2- 172.21.15.67:0/2768481769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15a8103160 0x7f15a819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.306+0000 7f15ad766640 1 --2- 172.21.15.67:0/2768481769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15a81020d0 0x7f15a8199a90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.306+0000 7f15ad766640 1 -- 172.21.15.67:0/2768481769 >> 172.21.15.67:0/2768481769 conn(0x7f15a80fda40 msgr2=0x7f15a80ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:06:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.307+0000 7f15ad766640 1 -- 172.21.15.67:0/2768481769 shutdown_connections 2024-07-27T03:06:44.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:44.307+0000 7f15ad766640 1 -- 172.21.15.67:0/2768481769 wait complete. 2024-07-27T03:06:44.506 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi067') 2024-07-27T03:06:44.506 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c0782a00-4bc4-11ef-bcc7-c7b262605968', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi067') 2024-07-27T03:06:44.608 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:44 smithi067 ceph-mon[32814]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:44.608 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:44 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2768481769' entity='client.admin' 2024-07-27T03:06:44.699 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:44 smithi149 ceph-mon[37173]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:44.699 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:44 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2768481769' entity='client.admin' 2024-07-27T03:06:44.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:44 smithi175 ceph-mon[37125]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:44.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:44 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2768481769' entity='client.admin' 2024-07-27T03:06:45.162 INFO:tasks.cephadm:Deploying OSDs... 2024-07-27T03:06:45.163 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-27T03:06:45.163 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/scratch_devs of=/dev/stdout 2024-07-27T03:06:45.180 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-27T03:06:45.180 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_1 2024-07-27T03:06:45.237 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-27T03:06:45.237 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-27T03:06:45.237 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-07-27T03:06:45.237 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-27T03:06:45.237 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-07-27T03:06:45.237 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-07-27 03:05:05.758915094 +0000 2024-07-27T03:06:45.237 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-07-27 03:00:32.173177090 +0000 2024-07-27T03:06:45.237 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-07-27 03:00:32.173177090 +0000 2024-07-27T03:06:45.237 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-07-27 03:00:32.173177090 +0000 2024-07-27T03:06:45.238 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-27T03:06:45.303 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-07-27T03:06:45.304 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-07-27T03:06:45.304 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000196112 s, 2.6 MB/s 2024-07-27T03:06:45.305 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-27T03:06:45.367 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_2 2024-07-27T03:06:45.424 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-27T03:06:45.424 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-27T03:06:45.424 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-07-27T03:06:45.424 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-27T03:06:45.424 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-07-27T03:06:45.424 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-07-27 03:05:05.792914340 +0000 2024-07-27T03:06:45.424 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-07-27 03:00:32.533168922 +0000 2024-07-27T03:06:45.424 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-07-27 03:00:32.533168922 +0000 2024-07-27T03:06:45.424 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-07-27 03:00:32.533168922 +0000 2024-07-27T03:06:45.425 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-27T03:06:45.488 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-07-27T03:06:45.489 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-07-27T03:06:45.489 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000238056 s, 2.2 MB/s 2024-07-27T03:06:45.490 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-27T03:06:45.547 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_3 2024-07-27T03:06:45.605 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-27T03:06:45.605 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-27T03:06:45.605 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-07-27T03:06:45.605 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-27T03:06:45.605 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-07-27T03:06:45.606 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-07-27 03:05:05.826913587 +0000 2024-07-27T03:06:45.606 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-07-27 03:00:32.881161027 +0000 2024-07-27T03:06:45.606 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-07-27 03:00:32.881161027 +0000 2024-07-27T03:06:45.606 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-07-27 03:00:32.881161027 +0000 2024-07-27T03:06:45.606 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-27T03:06:45.671 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-07-27T03:06:45.671 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-07-27T03:06:45.671 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000191347 s, 2.7 MB/s 2024-07-27T03:06:45.672 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-27T03:06:45.732 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_4 2024-07-27T03:06:45.790 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-27T03:06:45.790 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-27T03:06:45.790 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-07-27T03:06:45.791 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-27T03:06:45.791 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-07-27T03:06:45.791 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-07-27 03:05:05.859912856 +0000 2024-07-27T03:06:45.791 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-07-27 03:00:33.225153223 +0000 2024-07-27T03:06:45.791 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-07-27 03:00:33.225153223 +0000 2024-07-27T03:06:45.791 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-07-27 03:00:33.225153223 +0000 2024-07-27T03:06:45.792 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-27T03:06:45.861 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-07-27T03:06:45.861 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-07-27T03:06:45.861 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000192364 s, 2.7 MB/s 2024-07-27T03:06:45.863 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-27T03:06:45.923 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-07-27T03:06:45.924 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/scratch_devs of=/dev/stdout 2024-07-27T03:06:45.945 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-27T03:06:45.945 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_1 2024-07-27T03:06:46.004 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-27T03:06:46.004 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-27T03:06:46.004 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-07-27T03:06:46.004 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-27T03:06:46.004 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-07-27T03:06:46.004 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-07-27 03:06:04.219652570 +0000 2024-07-27T03:06:46.004 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-07-27 03:00:41.092348431 +0000 2024-07-27T03:06:46.004 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-07-27 03:00:41.092348431 +0000 2024-07-27T03:06:46.004 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-07-27 03:00:41.092348431 +0000 2024-07-27T03:06:46.005 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-27T03:06:46.078 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-07-27T03:06:46.079 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-07-27T03:06:46.079 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000295924 s, 1.7 MB/s 2024-07-27T03:06:46.080 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-27T03:06:46.140 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_2 2024-07-27T03:06:46.200 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-27T03:06:46.200 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-27T03:06:46.200 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-07-27T03:06:46.200 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-27T03:06:46.200 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-07-27T03:06:46.200 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-07-27 03:06:04.220652550 +0000 2024-07-27T03:06:46.200 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-07-27 03:00:41.417341866 +0000 2024-07-27T03:06:46.201 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-07-27 03:00:41.417341866 +0000 2024-07-27T03:06:46.201 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-07-27 03:00:41.417341866 +0000 2024-07-27T03:06:46.201 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-27T03:06:46.267 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-07-27T03:06:46.267 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-07-27T03:06:46.267 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000240286 s, 2.1 MB/s 2024-07-27T03:06:46.268 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-27T03:06:46.327 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_3 2024-07-27T03:06:46.387 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-27T03:06:46.387 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-27T03:06:46.387 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-07-27T03:06:46.387 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-27T03:06:46.387 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-07-27T03:06:46.387 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-07-27 03:06:04.220652550 +0000 2024-07-27T03:06:46.387 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-07-27 03:00:41.746335221 +0000 2024-07-27T03:06:46.387 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-07-27 03:00:41.746335221 +0000 2024-07-27T03:06:46.388 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-07-27 03:00:41.746335221 +0000 2024-07-27T03:06:46.388 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-27T03:06:46.453 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-07-27T03:06:46.453 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-07-27T03:06:46.453 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000206822 s, 2.5 MB/s 2024-07-27T03:06:46.454 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-27T03:06:46.513 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_4 2024-07-27T03:06:46.569 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-27T03:06:46.569 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-27T03:06:46.569 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-07-27T03:06:46.569 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-27T03:06:46.569 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-07-27T03:06:46.569 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-07-27 03:06:04.220652550 +0000 2024-07-27T03:06:46.569 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-07-27 03:00:42.063328818 +0000 2024-07-27T03:06:46.569 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-07-27 03:00:42.063328818 +0000 2024-07-27T03:06:46.570 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-07-27 03:00:42.063328818 +0000 2024-07-27T03:06:46.570 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-27T03:06:46.633 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-07-27T03:06:46.633 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-07-27T03:06:46.633 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000218531 s, 2.3 MB/s 2024-07-27T03:06:46.634 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-27T03:06:46.692 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-07-27T03:06:46.692 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2024-07-27T03:06:46.710 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-07-27T03:06:46.711 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_1 2024-07-27T03:06:46.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:46 smithi067 ceph-mon[32814]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:46.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:46 smithi149 ceph-mon[37173]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:46.768 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-07-27T03:06:46.768 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-27T03:06:46.768 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-07-27T03:06:46.768 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-27T03:06:46.768 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-07-27T03:06:46.768 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-07-27 03:06:01.868500913 +0000 2024-07-27T03:06:46.768 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-07-27 03:00:41.879293172 +0000 2024-07-27T03:06:46.768 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-07-27 03:00:41.879293172 +0000 2024-07-27T03:06:46.768 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-07-27 03:00:41.879293172 +0000 2024-07-27T03:06:46.769 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-07-27T03:06:46.832 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:46 smithi175 ceph-mon[37125]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:46.836 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-07-27T03:06:46.836 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-07-27T03:06:46.836 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000243896 s, 2.1 MB/s 2024-07-27T03:06:46.837 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-07-27T03:06:46.895 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_2 2024-07-27T03:06:46.951 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-07-27T03:06:46.951 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-27T03:06:46.951 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-07-27T03:06:46.951 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-27T03:06:46.951 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-07-27T03:06:46.951 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-07-27 03:06:01.869500897 +0000 2024-07-27T03:06:46.952 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-07-27 03:00:42.205287739 +0000 2024-07-27T03:06:46.952 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-07-27 03:00:42.205287739 +0000 2024-07-27T03:06:46.952 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-07-27 03:00:42.205287739 +0000 2024-07-27T03:06:46.952 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-07-27T03:06:47.015 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-07-27T03:06:47.015 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-07-27T03:06:47.015 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000212728 s, 2.4 MB/s 2024-07-27T03:06:47.017 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-07-27T03:06:47.073 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_3 2024-07-27T03:06:47.128 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-07-27T03:06:47.129 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-27T03:06:47.129 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-07-27T03:06:47.129 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-27T03:06:47.129 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-07-27T03:06:47.129 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-07-27 03:06:01.869500897 +0000 2024-07-27T03:06:47.129 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-07-27 03:00:42.531282306 +0000 2024-07-27T03:06:47.129 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-07-27 03:00:42.531282306 +0000 2024-07-27T03:06:47.129 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-07-27 03:00:42.531282306 +0000 2024-07-27T03:06:47.129 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-07-27T03:06:47.191 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-07-27T03:06:47.192 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-07-27T03:06:47.192 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000240865 s, 2.1 MB/s 2024-07-27T03:06:47.193 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-07-27T03:06:47.249 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_4 2024-07-27T03:06:47.304 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-07-27T03:06:47.304 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-07-27T03:06:47.304 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-07-27T03:06:47.304 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-27T03:06:47.304 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-07-27T03:06:47.304 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-07-27 03:06:01.869500897 +0000 2024-07-27T03:06:47.304 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-07-27 03:00:42.862276788 +0000 2024-07-27T03:06:47.304 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-07-27 03:00:42.862276788 +0000 2024-07-27T03:06:47.304 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-07-27 03:00:42.862276788 +0000 2024-07-27T03:06:47.305 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-07-27T03:06:47.366 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-07-27T03:06:47.366 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-07-27T03:06:47.366 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000228499 s, 2.2 MB/s 2024-07-27T03:06:47.367 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-07-27T03:06:47.423 INFO:tasks.cephadm:Deploying osd.0 on smithi067 with /dev/vg_nvme/lv_4... 2024-07-27T03:06:47.423 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-07-27T03:06:47.697 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:48.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:48 smithi067 ceph-mon[32814]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:48.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:48 smithi149 ceph-mon[37173]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:48.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:48 smithi175 ceph-mon[37125]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-07-27T03:06:49.452 INFO:teuthology.orchestra.run.smithi067.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-07-27T03:06:49.465 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10+0 records in 2024-07-27T03:06:49.465 INFO:teuthology.orchestra.run.smithi067.stderr:10+0 records out 2024-07-27T03:06:49.466 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112846 s, 929 MB/s 2024-07-27T03:06:49.466 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping successful for: 2024-07-27T03:06:50.104 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph orch daemon add osd smithi067:vg_nvme/lv_4 2024-07-27T03:06:50.369 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:06:50.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:50 smithi067 ceph-mon[32814]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:50.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:50 smithi149 ceph-mon[37173]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:50.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:50 smithi175 ceph-mon[37125]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:51.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.916+0000 7fa52eb63640 1 -- 172.21.15.67:0/2554537839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa528103150 msgr2=0x7fa528103560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:51.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.916+0000 7fa52eb63640 1 --2- 172.21.15.67:0/2554537839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa528103150 0x7fa528103560 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa514009900 tx=0x7fa51402a480 comp rx=0 tx=0).stop 2024-07-27T03:06:51.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.916+0000 7fa52eb63640 1 -- 172.21.15.67:0/2554537839 shutdown_connections 2024-07-27T03:06:51.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.916+0000 7fa52eb63640 1 --2- 172.21.15.67:0/2554537839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa528103af0 0x7fa52810a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:51.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.916+0000 7fa52eb63640 1 --2- 172.21.15.67:0/2554537839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa528103150 0x7fa528103560 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:51.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.916+0000 7fa52eb63640 1 --2- 172.21.15.67:0/2554537839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa5281020c0 0x7fa528102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:51.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.916+0000 7fa52eb63640 1 -- 172.21.15.67:0/2554537839 >> 172.21.15.67:0/2554537839 conn(0x7fa5280fda50 msgr2=0x7fa5280ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:06:51.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.916+0000 7fa52eb63640 1 -- 172.21.15.67:0/2554537839 shutdown_connections 2024-07-27T03:06:51.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.916+0000 7fa52eb63640 1 -- 172.21.15.67:0/2554537839 wait complete. 2024-07-27T03:06:51.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.916+0000 7fa52eb63640 1 Processor -- start 2024-07-27T03:06:51.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.917+0000 7fa52eb63640 1 -- start start 2024-07-27T03:06:51.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.917+0000 7fa52eb63640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5281020c0 0x7fa528199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:51.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.917+0000 7fa52eb63640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa528103150 0x7fa52819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:51.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.917+0000 7fa52eb63640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa528103af0 0x7fa52819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:51.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.917+0000 7fa52eb63640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa52810dd20 con 0x7fa528103150 2024-07-27T03:06:51.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.917+0000 7fa52eb63640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa52810dba0 con 0x7fa528103af0 2024-07-27T03:06:51.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.917+0000 7fa52eb63640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa52810dea0 con 0x7fa5281020c0 2024-07-27T03:06:51.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.917+0000 7fa527fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa528103150 0x7fa52819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:51.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.917+0000 7fa527fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa528103150 0x7fa52819a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56898/0 (socket says 172.21.15.67:56898) 2024-07-27T03:06:51.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.917+0000 7fa527fff640 1 -- 172.21.15.67:0/1353307591 learned_addr learned my addr 172.21.15.67:0/1353307591 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:06:51.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.917+0000 7fa52c8ff640 1 --2- 172.21.15.67:0/1353307591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5281020c0 0x7fa528199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:51.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.917+0000 7fa52d100640 1 --2- 172.21.15.67:0/1353307591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa528103af0 0x7fa52819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:51.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.917+0000 7fa527fff640 1 -- 172.21.15.67:0/1353307591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5281020c0 msgr2=0x7fa528199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:51.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.917+0000 7fa527fff640 1 --2- 172.21.15.67:0/1353307591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5281020c0 0x7fa528199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:51.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.917+0000 7fa527fff640 1 -- 172.21.15.67:0/1353307591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa528103af0 msgr2=0x7fa52819ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:06:51.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.917+0000 7fa527fff640 1 --2- 172.21.15.67:0/1353307591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa528103af0 0x7fa52819ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:06:51.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.917+0000 7fa527fff640 1 -- 172.21.15.67:0/1353307591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52819b3e0 con 0x7fa528103150 2024-07-27T03:06:51.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.917+0000 7fa52d100640 1 --2- 172.21.15.67:0/1353307591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa528103af0 0x7fa52819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:06:51.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.917+0000 7fa52c8ff640 1 --2- 172.21.15.67:0/1353307591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5281020c0 0x7fa528199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:06:51.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.918+0000 7fa527fff640 1 --2- 172.21.15.67:0/1353307591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa528103150 0x7fa52819a0b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa514006010 tx=0x7fa514004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:06:51.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.918+0000 7fa525ffb640 1 -- 172.21.15.67:0/1353307591 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51403f030 con 0x7fa528103150 2024-07-27T03:06:51.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.918+0000 7fa525ffb640 1 -- 172.21.15.67:0/1353307591 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa514003cb0 con 0x7fa528103150 2024-07-27T03:06:51.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.918+0000 7fa52eb63640 1 -- 172.21.15.67:0/1353307591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52819b670 con 0x7fa528103150 2024-07-27T03:06:51.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.918+0000 7fa525ffb640 1 -- 172.21.15.67:0/1353307591 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa514035070 con 0x7fa528103150 2024-07-27T03:06:51.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.918+0000 7fa52eb63640 1 -- 172.21.15.67:0/1353307591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5281a1c90 con 0x7fa528103150 2024-07-27T03:06:51.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.919+0000 7fa52eb63640 1 -- 172.21.15.67:0/1353307591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5281a1ec0 con 0x7fa528103150 2024-07-27T03:06:51.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.921+0000 7fa525ffb640 1 -- 172.21.15.67:0/1353307591 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 151003+0+0 (secure 0 0 0) 0x7fa514042030 con 0x7fa528103150 2024-07-27T03:06:51.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.921+0000 7fa525ffb640 1 --2- 172.21.15.67:0/1353307591 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa5180b3bc0 0x7fa5180b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:06:51.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.921+0000 7fa52c8ff640 1 --2- 172.21.15.67:0/1353307591 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa5180b3bc0 0x7fa5180b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:06:51.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.922+0000 7fa525ffb640 1 -- 172.21.15.67:0/1353307591 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa514102ca0 con 0x7fa528103150 2024-07-27T03:06:51.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.922+0000 7fa52c8ff640 1 --2- 172.21.15.67:0/1353307591 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa5180b3bc0 0x7fa5180b60d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0058c0 tx=0x7fa51c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:06:51.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:51.925+0000 7fa525ffb640 1 -- 172.21.15.67:0/1353307591 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fa5140cbfc0 con 0x7fa528103150 2024-07-27T03:06:52.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:06:52.117+0000 7fa52eb63640 1 -- 172.21.15.67:0/1353307591 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa5280ffaa0 con 0x7fa5180b3bc0 2024-07-27T03:06:52.713 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:52 smithi067 ceph-mon[32814]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:52.713 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:06:52.713 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:06:52.713 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:06:52.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:52 smithi149 ceph-mon[37173]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:52.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:06:52.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:06:52.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:06:52.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:52 smithi175 ceph-mon[37125]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:52.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:06:52.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:06:52.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:06:53.684 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:53 smithi067 ceph-mon[32814]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:06:53.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:53 smithi149 ceph-mon[37173]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:06:53.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:53 smithi175 ceph-mon[37125]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:06:54.688 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:54 smithi067 ceph-mon[32814]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:54.688 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:54 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2792072990' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f614272-18e8-4562-812c-4f43e40efd2b"}]: dispatch 2024-07-27T03:06:54.688 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:54 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2792072990' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f614272-18e8-4562-812c-4f43e40efd2b"}]': finished 2024-07-27T03:06:54.688 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:54 smithi067 ceph-mon[32814]: osdmap e5: 1 total, 0 up, 1 in 2024-07-27T03:06:54.688 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-27T03:06:54.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:54 smithi149 ceph-mon[37173]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:54.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:54 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2792072990' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f614272-18e8-4562-812c-4f43e40efd2b"}]: dispatch 2024-07-27T03:06:54.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:54 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2792072990' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f614272-18e8-4562-812c-4f43e40efd2b"}]': finished 2024-07-27T03:06:54.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:54 smithi149 ceph-mon[37173]: osdmap e5: 1 total, 0 up, 1 in 2024-07-27T03:06:54.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-27T03:06:54.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:54 smithi175 ceph-mon[37125]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:54.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:54 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2792072990' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f614272-18e8-4562-812c-4f43e40efd2b"}]: dispatch 2024-07-27T03:06:54.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:54 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2792072990' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f614272-18e8-4562-812c-4f43e40efd2b"}]': finished 2024-07-27T03:06:54.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:54 smithi175 ceph-mon[37125]: osdmap e5: 1 total, 0 up, 1 in 2024-07-27T03:06:54.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-27T03:06:55.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:55 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2091519784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:06:55.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:55 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2091519784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:06:55.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:55 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2091519784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:06:56.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:56 smithi067 ceph-mon[32814]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:56.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:56 smithi149 ceph-mon[37173]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:56.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:56 smithi175 ceph-mon[37125]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:58.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:06:58 smithi067 ceph-mon[32814]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:58.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:06:58 smithi149 ceph-mon[37173]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:06:58.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:06:58 smithi175 ceph-mon[37125]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:00.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:00 smithi067 ceph-mon[32814]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:00.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:00 smithi149 ceph-mon[37173]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:00.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:00 smithi175 ceph-mon[37125]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:01.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:01 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-27T03:07:01.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:01 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:01.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:01 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-27T03:07:01.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:01 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:01.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:01 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-07-27T03:07:01.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:01 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:02.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:02 smithi067 ceph-mon[32814]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:02.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:02 smithi067 ceph-mon[32814]: Deploying daemon osd.0 on smithi067 2024-07-27T03:07:02.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:02 smithi149 ceph-mon[37173]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:02.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:02 smithi149 ceph-mon[37173]: Deploying daemon osd.0 on smithi067 2024-07-27T03:07:02.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:02 smithi175 ceph-mon[37125]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:02.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:02 smithi175 ceph-mon[37125]: Deploying daemon osd.0 on smithi067 2024-07-27T03:07:04.696 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:04 smithi067 ceph-mon[32814]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:04.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:04 smithi149 ceph-mon[37173]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:04.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:04 smithi175 ceph-mon[37125]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:06.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:06 smithi067 ceph-mon[32814]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:06.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:07:06.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:06.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:06.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:06 smithi149 ceph-mon[37173]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:06.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:06 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:07:06.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:06 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:06.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:06 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:06.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:06 smithi175 ceph-mon[37125]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:06.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:06 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:07:06.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:06 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:06.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:06 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:08.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:08 smithi067 ceph-mon[32814]: pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:08.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:08 smithi149 ceph-mon[37173]: pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:08.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:08 smithi175 ceph-mon[37125]: pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:09.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:07:09.691+0000 7fa525ffb640 1 -- 172.21.15.67:0/1353307591 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa5280ffaa0 con 0x7fa5180b3bc0 2024-07-27T03:07:09.694 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 0 on host 'smithi067' 2024-07-27T03:07:09.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:07:09.694+0000 7fa52eb63640 1 -- 172.21.15.67:0/1353307591 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa5180b3bc0 msgr2=0x7fa5180b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:07:09.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:07:09.694+0000 7fa52eb63640 1 --2- 172.21.15.67:0/1353307591 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa5180b3bc0 0x7fa5180b60d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0058c0 tx=0x7fa51c005850 comp rx=0 tx=0).stop 2024-07-27T03:07:09.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:07:09.694+0000 7fa52eb63640 1 -- 172.21.15.67:0/1353307591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa528103150 msgr2=0x7fa52819a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:07:09.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:07:09.694+0000 7fa52eb63640 1 --2- 172.21.15.67:0/1353307591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa528103150 0x7fa52819a0b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa514006010 tx=0x7fa514004340 comp rx=0 tx=0).stop 2024-07-27T03:07:09.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:07:09.694+0000 7fa52eb63640 1 -- 172.21.15.67:0/1353307591 shutdown_connections 2024-07-27T03:07:09.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:07:09.694+0000 7fa52eb63640 1 --2- 172.21.15.67:0/1353307591 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa5180b3bc0 0x7fa5180b60d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:07:09.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:07:09.694+0000 7fa52eb63640 1 --2- 172.21.15.67:0/1353307591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa528103af0 0x7fa52819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:07:09.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:07:09.694+0000 7fa52eb63640 1 --2- 172.21.15.67:0/1353307591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa528103150 0x7fa52819a0b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:07:09.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:07:09.694+0000 7fa52eb63640 1 --2- 172.21.15.67:0/1353307591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5281020c0 0x7fa528199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:07:09.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:07:09.694+0000 7fa52eb63640 1 -- 172.21.15.67:0/1353307591 >> 172.21.15.67:0/1353307591 conn(0x7fa5280fda50 msgr2=0x7fa5280ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:07:09.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:07:09.694+0000 7fa52eb63640 1 -- 172.21.15.67:0/1353307591 shutdown_connections 2024-07-27T03:07:09.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:07:09.694+0000 7fa52eb63640 1 -- 172.21.15.67:0/1353307591 wait complete. 2024-07-27T03:07:09.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:09.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:09.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:09 smithi067 ceph-mon[32814]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:09.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:09.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:07:09.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:09.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:07:09.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:09.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:10.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:10.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:10.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:09 smithi175 ceph-mon[37125]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:10.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:10.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:07:10.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:10.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:07:10.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:10.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:10.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:10.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:10.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:09 smithi149 ceph-mon[37173]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:10.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:10.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:07:10.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:10.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:07:10.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:10.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:10.300 DEBUG:teuthology.orchestra.run.smithi067:osd.0> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.0.service 2024-07-27T03:07:10.303 INFO:tasks.cephadm:Deploying osd.1 on smithi149 with /dev/vg_nvme/lv_4... 2024-07-27T03:07:10.303 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-07-27T03:07:10.557 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:07:10 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-0[50496]: 2024-07-27T03:07:10.390+0000 7fd6ea495640 -1 osd.0 0 log_to_monitors true 2024-07-27T03:07:10.557 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.b/config 2024-07-27T03:07:10.953 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:10 smithi149 ceph-mon[37173]: from='osd.0 [v2:172.21.15.67:6802/2648129674,v1:172.21.15.67:6803/2648129674]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-27T03:07:10.953 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:10 smithi149 ceph-mon[37173]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-27T03:07:11.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:10 smithi067 ceph-mon[32814]: from='osd.0 [v2:172.21.15.67:6802/2648129674,v1:172.21.15.67:6803/2648129674]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-27T03:07:11.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:10 smithi067 ceph-mon[32814]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-27T03:07:11.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:10 smithi175 ceph-mon[37125]: from='osd.0 [v2:172.21.15.67:6802/2648129674,v1:172.21.15.67:6803/2648129674]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-27T03:07:11.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:10 smithi175 ceph-mon[37125]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-07-27T03:07:12.110 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-07-27T03:07:12.110 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:11 smithi149 ceph-mon[37173]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-27T03:07:12.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:11 smithi149 ceph-mon[37173]: osdmap e6: 1 total, 0 up, 1 in 2024-07-27T03:07:12.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:11 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-27T03:07:12.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:11 smithi149 ceph-mon[37173]: from='osd.0 [v2:172.21.15.67:6802/2648129674,v1:172.21.15.67:6803/2648129674]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-27T03:07:12.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:11 smithi149 ceph-mon[37173]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-27T03:07:12.111 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:11 smithi149 ceph-mon[37173]: pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:12.202 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-07-27T03:07:12.219 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-07-27T03:07:12.219 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-07-27T03:07:12.219 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0149288 s, 702 MB/s 2024-07-27T03:07:12.219 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-07-27T03:07:12.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:11 smithi067 ceph-mon[32814]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-27T03:07:12.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:11 smithi067 ceph-mon[32814]: osdmap e6: 1 total, 0 up, 1 in 2024-07-27T03:07:12.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:11 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-27T03:07:12.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:11 smithi067 ceph-mon[32814]: from='osd.0 [v2:172.21.15.67:6802/2648129674,v1:172.21.15.67:6803/2648129674]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-27T03:07:12.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:11 smithi067 ceph-mon[32814]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-27T03:07:12.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:11 smithi067 ceph-mon[32814]: pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:12.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:11 smithi175 ceph-mon[37125]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-07-27T03:07:12.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:11 smithi175 ceph-mon[37125]: osdmap e6: 1 total, 0 up, 1 in 2024-07-27T03:07:12.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:11 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-27T03:07:12.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:11 smithi175 ceph-mon[37125]: from='osd.0 [v2:172.21.15.67:6802/2648129674,v1:172.21.15.67:6803/2648129674]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-27T03:07:12.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:11 smithi175 ceph-mon[37125]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-27T03:07:12.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:11 smithi175 ceph-mon[37125]: pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:12.833 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_4 2024-07-27T03:07:12.892 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:07:12 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-0[50496]: 2024-07-27T03:07:12.657+0000 7fd6e6c2c640 -1 osd.0 0 waiting for initial osdmap 2024-07-27T03:07:12.892 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:07:12 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-0[50496]: 2024-07-27T03:07:12.661+0000 7fd6de20d640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-27T03:07:13.088 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.b/config 2024-07-27T03:07:13.196 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:12 smithi149 ceph-mon[37173]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-27T03:07:13.196 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:12 smithi149 ceph-mon[37173]: osdmap e7: 1 total, 0 up, 1 in 2024-07-27T03:07:13.196 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:12 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-27T03:07:13.196 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:12 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-27T03:07:13.196 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:12 smithi149 ceph-mon[37173]: from='osd.0 ' entity='osd.0' 2024-07-27T03:07:13.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:12 smithi067 ceph-mon[32814]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-27T03:07:13.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:12 smithi067 ceph-mon[32814]: osdmap e7: 1 total, 0 up, 1 in 2024-07-27T03:07:13.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:12 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-27T03:07:13.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:12 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-27T03:07:13.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:12 smithi067 ceph-mon[32814]: from='osd.0 ' entity='osd.0' 2024-07-27T03:07:13.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:12 smithi175 ceph-mon[37125]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-27T03:07:13.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:12 smithi175 ceph-mon[37125]: osdmap e7: 1 total, 0 up, 1 in 2024-07-27T03:07:13.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:12 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-27T03:07:13.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:12 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-27T03:07:13.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:12 smithi175 ceph-mon[37125]: from='osd.0 ' entity='osd.0' 2024-07-27T03:07:14.199 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:13 smithi149 ceph-mon[37173]: purged_snaps scrub starts 2024-07-27T03:07:14.200 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:13 smithi149 ceph-mon[37173]: purged_snaps scrub ok 2024-07-27T03:07:14.200 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:13 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-27T03:07:14.200 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:13 smithi149 ceph-mon[37173]: pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:14.200 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:13 smithi149 ceph-mon[37173]: Detected new or changed devices on smithi067 2024-07-27T03:07:14.200 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:13 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:14.200 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:13 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:14.200 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:13 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:07:14.200 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:13 smithi149 ceph-mon[37173]: Adjusting osd_memory_target on smithi067 to 17088M 2024-07-27T03:07:14.200 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:13 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:14.200 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:13 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:14.200 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:13 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:07:14.201 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:13 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:14.201 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:13 smithi149 ceph-mon[37173]: osd.0 [v2:172.21.15.67:6802/2648129674,v1:172.21.15.67:6803/2648129674] boot 2024-07-27T03:07:14.201 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:13 smithi149 ceph-mon[37173]: osdmap e8: 1 total, 1 up, 1 in 2024-07-27T03:07:14.201 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:13 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-27T03:07:14.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:13 smithi067 ceph-mon[32814]: purged_snaps scrub starts 2024-07-27T03:07:14.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:13 smithi067 ceph-mon[32814]: purged_snaps scrub ok 2024-07-27T03:07:14.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:13 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-27T03:07:14.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:13 smithi067 ceph-mon[32814]: pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:14.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:13 smithi067 ceph-mon[32814]: Detected new or changed devices on smithi067 2024-07-27T03:07:14.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:13 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:14.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:13 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:14.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:13 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:07:14.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:13 smithi067 ceph-mon[32814]: Adjusting osd_memory_target on smithi067 to 17088M 2024-07-27T03:07:14.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:13 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:14.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:13 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:14.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:13 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:07:14.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:13 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:14.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:13 smithi067 ceph-mon[32814]: osd.0 [v2:172.21.15.67:6802/2648129674,v1:172.21.15.67:6803/2648129674] boot 2024-07-27T03:07:14.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:13 smithi067 ceph-mon[32814]: osdmap e8: 1 total, 1 up, 1 in 2024-07-27T03:07:14.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:13 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-27T03:07:14.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:13 smithi175 ceph-mon[37125]: purged_snaps scrub starts 2024-07-27T03:07:14.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:13 smithi175 ceph-mon[37125]: purged_snaps scrub ok 2024-07-27T03:07:14.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:13 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-27T03:07:14.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:13 smithi175 ceph-mon[37125]: pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-07-27T03:07:14.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:13 smithi175 ceph-mon[37125]: Detected new or changed devices on smithi067 2024-07-27T03:07:14.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:13 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:14.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:13 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:14.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:13 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:07:14.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:13 smithi175 ceph-mon[37125]: Adjusting osd_memory_target on smithi067 to 17088M 2024-07-27T03:07:14.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:13 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:14.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:13 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:14.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:13 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:07:14.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:13 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:14.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:13 smithi175 ceph-mon[37125]: osd.0 [v2:172.21.15.67:6802/2648129674,v1:172.21.15.67:6803/2648129674] boot 2024-07-27T03:07:14.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:13 smithi175 ceph-mon[37125]: osdmap e8: 1 total, 1 up, 1 in 2024-07-27T03:07:14.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:13 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-07-27T03:07:14.617 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.614+0000 7fd35dae1640 1 -- 172.21.15.149:0/2922530684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd358103c80 msgr2=0x7fd35810a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:07:14.617 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.614+0000 7fd35dae1640 1 --2- 172.21.15.149:0/2922530684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd358103c80 0x7fd35810a720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd350009ca0 tx=0x7fd35002a8c0 comp rx=0 tx=0).stop 2024-07-27T03:07:14.617 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.614+0000 7fd35dae1640 1 -- 172.21.15.149:0/2922530684 shutdown_connections 2024-07-27T03:07:14.617 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.614+0000 7fd35dae1640 1 --2- 172.21.15.149:0/2922530684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd358103c80 0x7fd35810a720 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:07:14.617 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.614+0000 7fd35dae1640 1 --2- 172.21.15.149:0/2922530684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd358103070 0x7fd358103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:07:14.618 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.614+0000 7fd35dae1640 1 --2- 172.21.15.149:0/2922530684 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd358102690 0x7fd358102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:07:14.618 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.614+0000 7fd35dae1640 1 -- 172.21.15.149:0/2922530684 >> 172.21.15.149:0/2922530684 conn(0x7fd3580fd270 msgr2=0x7fd3580ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:07:14.618 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.614+0000 7fd35dae1640 1 -- 172.21.15.149:0/2922530684 shutdown_connections 2024-07-27T03:07:14.618 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.614+0000 7fd35dae1640 1 -- 172.21.15.149:0/2922530684 wait complete. 2024-07-27T03:07:14.618 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.615+0000 7fd35dae1640 1 Processor -- start 2024-07-27T03:07:14.618 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.615+0000 7fd35dae1640 1 -- start start 2024-07-27T03:07:14.621 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.615+0000 7fd35dae1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd358102690 0x7fd358105f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:07:14.621 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.615+0000 7fd35dae1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd358103070 0x7fd358104810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:07:14.622 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.615+0000 7fd35dae1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd358103c80 0x7fd358104d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:07:14.622 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.615+0000 7fd35dae1640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd35810dd30 con 0x7fd358103c80 2024-07-27T03:07:14.622 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.615+0000 7fd35dae1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd35810dbb0 con 0x7fd358102690 2024-07-27T03:07:14.622 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.615+0000 7fd35dae1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd35810deb0 con 0x7fd358103070 2024-07-27T03:07:14.622 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.615+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd358102690 0x7fd358105f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:07:14.622 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.615+0000 7fd3577fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd358102690 0x7fd358105f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:48332/0 (socket says 172.21.15.149:48332) 2024-07-27T03:07:14.622 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.615+0000 7fd3577fe640 1 -- 172.21.15.149:0/3430562610 learned_addr learned my addr 172.21.15.149:0/3430562610 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-07-27T03:07:14.623 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.615+0000 7fd357fff640 1 --2- 172.21.15.149:0/3430562610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd358103c80 0x7fd358104d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:07:14.623 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.615+0000 7fd356ffd640 1 --2- 172.21.15.149:0/3430562610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd358103070 0x7fd358104810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:07:14.623 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.615+0000 7fd3577fe640 1 -- 172.21.15.149:0/3430562610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd358103070 msgr2=0x7fd358104810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:07:14.623 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.615+0000 7fd3577fe640 1 --2- 172.21.15.149:0/3430562610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd358103070 0x7fd358104810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:07:14.623 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.615+0000 7fd3577fe640 1 -- 172.21.15.149:0/3430562610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd358103c80 msgr2=0x7fd358104d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:07:14.623 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.615+0000 7fd3577fe640 1 --2- 172.21.15.149:0/3430562610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd358103c80 0x7fd358104d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:07:14.623 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.615+0000 7fd3577fe640 1 -- 172.21.15.149:0/3430562610 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd358105450 con 0x7fd358102690 2024-07-27T03:07:14.623 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.616+0000 7fd3577fe640 1 --2- 172.21.15.149:0/3430562610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd358102690 0x7fd358105f40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd34800e9f0 tx=0x7fd34800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:07:14.623 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.616+0000 7fd354ff9640 1 -- 172.21.15.149:0/3430562610 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34800cd80 con 0x7fd358102690 2024-07-27T03:07:14.624 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.616+0000 7fd354ff9640 1 -- 172.21.15.149:0/3430562610 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3480108e0 con 0x7fd358102690 2024-07-27T03:07:14.624 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.616+0000 7fd35dae1640 1 -- 172.21.15.149:0/3430562610 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3581a3a20 con 0x7fd358102690 2024-07-27T03:07:14.624 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.616+0000 7fd354ff9640 1 -- 172.21.15.149:0/3430562610 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348018980 con 0x7fd358102690 2024-07-27T03:07:14.624 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.616+0000 7fd35dae1640 1 -- 172.21.15.149:0/3430562610 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3581a3ed0 con 0x7fd358102690 2024-07-27T03:07:14.624 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.617+0000 7fd35dae1640 1 -- 172.21.15.149:0/3430562610 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd358102aa0 con 0x7fd358102690 2024-07-27T03:07:14.624 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.618+0000 7fd354ff9640 1 -- 172.21.15.149:0/3430562610 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 15) ==== 151003+0+0 (secure 0 0 0) 0x7fd348010430 con 0x7fd358102690 2024-07-27T03:07:14.624 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.619+0000 7fd354ff9640 1 --2- 172.21.15.149:0/3430562610 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd3400b3bc0 0x7fd3400b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:07:14.624 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.619+0000 7fd354ff9640 1 -- 172.21.15.149:0/3430562610 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7fd3480e2950 con 0x7fd358102690 2024-07-27T03:07:14.625 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.619+0000 7fd356ffd640 1 --2- 172.21.15.149:0/3430562610 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd3400b3bc0 0x7fd3400b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:07:14.625 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.620+0000 7fd356ffd640 1 --2- 172.21.15.149:0/3430562610 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd3400b3bc0 0x7fd3400b60d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd34c005fd0 tx=0x7fd34c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:07:14.626 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.622+0000 7fd354ff9640 1 -- 172.21.15.149:0/3430562610 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fd3480aba10 con 0x7fd358102690 2024-07-27T03:07:14.807 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:14.804+0000 7fd35dae1640 1 -- 172.21.15.149:0/3430562610 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd3581a41c0 con 0x7fd3400b3bc0 2024-07-27T03:07:15.213 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:07:15.214 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:07:15.214 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:15.222 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:07:15.223 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:07:15.223 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:15.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:07:15.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:07:15.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:15.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:15 smithi067 ceph-mon[32814]: from='client.24245 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:07:15.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:15 smithi067 ceph-mon[32814]: osdmap e9: 1 total, 1 up, 1 in 2024-07-27T03:07:15.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:15 smithi067 ceph-mon[32814]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:16.217 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:15 smithi149 ceph-mon[37173]: from='client.24245 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:07:16.217 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:15 smithi149 ceph-mon[37173]: osdmap e9: 1 total, 1 up, 1 in 2024-07-27T03:07:16.217 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:15 smithi149 ceph-mon[37173]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:16.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:15 smithi175 ceph-mon[37125]: from='client.24245 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:07:16.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:15 smithi175 ceph-mon[37125]: osdmap e9: 1 total, 1 up, 1 in 2024-07-27T03:07:16.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:15 smithi175 ceph-mon[37125]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:18.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:18 smithi175 ceph-mon[37125]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:18.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:18 smithi175 ceph-mon[37125]: from='client.? 172.21.15.149:0/2601705454' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f59f5879-163a-4269-bbe7-1200eea65341"}]: dispatch 2024-07-27T03:07:18.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:18 smithi175 ceph-mon[37125]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f59f5879-163a-4269-bbe7-1200eea65341"}]: dispatch 2024-07-27T03:07:18.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:18 smithi175 ceph-mon[37125]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f59f5879-163a-4269-bbe7-1200eea65341"}]': finished 2024-07-27T03:07:18.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:18 smithi175 ceph-mon[37125]: osdmap e10: 2 total, 1 up, 2 in 2024-07-27T03:07:18.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:18 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-27T03:07:18.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:18 smithi175 ceph-mon[37125]: from='client.? 172.21.15.149:0/2582281438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:07:18.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:18 smithi067 ceph-mon[32814]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:18.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:18 smithi067 ceph-mon[32814]: from='client.? 172.21.15.149:0/2601705454' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f59f5879-163a-4269-bbe7-1200eea65341"}]: dispatch 2024-07-27T03:07:18.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:18 smithi067 ceph-mon[32814]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f59f5879-163a-4269-bbe7-1200eea65341"}]: dispatch 2024-07-27T03:07:18.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:18 smithi067 ceph-mon[32814]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f59f5879-163a-4269-bbe7-1200eea65341"}]': finished 2024-07-27T03:07:18.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:18 smithi067 ceph-mon[32814]: osdmap e10: 2 total, 1 up, 2 in 2024-07-27T03:07:18.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:18 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-27T03:07:18.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:18 smithi067 ceph-mon[32814]: from='client.? 172.21.15.149:0/2582281438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:07:18.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:18 smithi149 ceph-mon[37173]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:18.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:18 smithi149 ceph-mon[37173]: from='client.? 172.21.15.149:0/2601705454' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f59f5879-163a-4269-bbe7-1200eea65341"}]: dispatch 2024-07-27T03:07:18.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:18 smithi149 ceph-mon[37173]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f59f5879-163a-4269-bbe7-1200eea65341"}]: dispatch 2024-07-27T03:07:18.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:18 smithi149 ceph-mon[37173]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f59f5879-163a-4269-bbe7-1200eea65341"}]': finished 2024-07-27T03:07:18.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:18 smithi149 ceph-mon[37173]: osdmap e10: 2 total, 1 up, 2 in 2024-07-27T03:07:18.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:18 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-27T03:07:18.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:18 smithi149 ceph-mon[37173]: from='client.? 172.21.15.149:0/2582281438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:07:20.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:20 smithi175 ceph-mon[37125]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:20.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:20 smithi067 ceph-mon[32814]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:20.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:20 smithi149 ceph-mon[37173]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:22.340 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:22 smithi149 ceph-mon[37173]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:22.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:22 smithi175 ceph-mon[37125]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:22.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:22 smithi067 ceph-mon[32814]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:24.295 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:24 smithi149 ceph-mon[37173]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:24.295 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:24 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-27T03:07:24.295 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:24 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:24.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:24 smithi175 ceph-mon[37125]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:24.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:24 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-27T03:07:24.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:24 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:24.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:24 smithi067 ceph-mon[32814]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:24.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:24 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-07-27T03:07:24.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:24 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:25.311 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:25 smithi149 ceph-mon[37173]: Deploying daemon osd.1 on smithi149 2024-07-27T03:07:25.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:25 smithi175 ceph-mon[37125]: Deploying daemon osd.1 on smithi149 2024-07-27T03:07:25.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:25 smithi067 ceph-mon[32814]: Deploying daemon osd.1 on smithi149 2024-07-27T03:07:26.192 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:26 smithi149 ceph-mon[37173]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:26.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:26 smithi175 ceph-mon[37125]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:26.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:26 smithi067 ceph-mon[32814]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:28.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:28 smithi149 ceph-mon[37173]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:28.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:28 smithi175 ceph-mon[37125]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:28.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:28 smithi067 ceph-mon[32814]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:29.317 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:29 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:07:29.317 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:29 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:29.317 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:29 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:29.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:29 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:07:29.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:29 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:29.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:29 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:29.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:29 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:07:29.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:29 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:29.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:29 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:30.242 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:30 smithi149 ceph-mon[37173]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:30.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:30 smithi175 ceph-mon[37125]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:30.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:30 smithi067 ceph-mon[32814]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:32.388 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:32.385+0000 7fd354ff9640 1 -- 172.21.15.149:0/3430562610 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd3581a41c0 con 0x7fd3400b3bc0 2024-07-27T03:07:32.389 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 1 on host 'smithi149' 2024-07-27T03:07:32.391 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:32.387+0000 7fd35dae1640 1 -- 172.21.15.149:0/3430562610 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd3400b3bc0 msgr2=0x7fd3400b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:07:32.391 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:32.387+0000 7fd35dae1640 1 --2- 172.21.15.149:0/3430562610 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd3400b3bc0 0x7fd3400b60d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd34c005fd0 tx=0x7fd34c005850 comp rx=0 tx=0).stop 2024-07-27T03:07:32.391 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:32.387+0000 7fd35dae1640 1 -- 172.21.15.149:0/3430562610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd358102690 msgr2=0x7fd358105f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:07:32.391 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:32.387+0000 7fd35dae1640 1 --2- 172.21.15.149:0/3430562610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd358102690 0x7fd358105f40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd34800e9f0 tx=0x7fd34800eeb0 comp rx=0 tx=0).stop 2024-07-27T03:07:32.391 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:32.387+0000 7fd35dae1640 1 -- 172.21.15.149:0/3430562610 shutdown_connections 2024-07-27T03:07:32.391 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:32.387+0000 7fd35dae1640 1 --2- 172.21.15.149:0/3430562610 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd3400b3bc0 0x7fd3400b60d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:07:32.391 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:32.387+0000 7fd35dae1640 1 --2- 172.21.15.149:0/3430562610 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd358103c80 0x7fd358104d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:07:32.391 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:32.387+0000 7fd35dae1640 1 --2- 172.21.15.149:0/3430562610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd358103070 0x7fd358104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:07:32.392 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:32.387+0000 7fd35dae1640 1 --2- 172.21.15.149:0/3430562610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd358102690 0x7fd358105f40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:07:32.392 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:32.387+0000 7fd35dae1640 1 -- 172.21.15.149:0/3430562610 >> 172.21.15.149:0/3430562610 conn(0x7fd3580fd270 msgr2=0x7fd3580fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:07:32.392 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:32.387+0000 7fd35dae1640 1 -- 172.21.15.149:0/3430562610 shutdown_connections 2024-07-27T03:07:32.392 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:07:32.388+0000 7fd35dae1640 1 -- 172.21.15.149:0/3430562610 wait complete. 2024-07-27T03:07:32.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:32 smithi175 ceph-mon[37125]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:32 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:32 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:32 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:32 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:07:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:32 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:32.697 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:32 smithi149 ceph-mon[37173]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:32.697 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:32 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:32.697 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:32 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:32.697 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:32 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:32.697 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:32 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:07:32.697 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:32 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:32.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:32 smithi067 ceph-mon[32814]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:32.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:32 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:32.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:32 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:32.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:32 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:32.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:32 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:07:32.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:32 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:32.984 DEBUG:teuthology.orchestra.run.smithi149:osd.1> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.1.service 2024-07-27T03:07:32.986 INFO:tasks.cephadm:Deploying osd.2 on smithi175 with /dev/vg_nvme/lv_4... 2024-07-27T03:07:32.986 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-07-27T03:07:33.228 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.c/config 2024-07-27T03:07:33.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:33 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:07:33.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:33 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:33.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:33 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:33.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:33 smithi149 ceph-mon[37173]: from='osd.1 [v2:172.21.15.149:6800/2561753152,v1:172.21.15.149:6801/2561753152]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-27T03:07:33.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:33 smithi149 ceph-mon[37173]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-27T03:07:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:33 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:07:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:33 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:33 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:33 smithi175 ceph-mon[37125]: from='osd.1 [v2:172.21.15.149:6800/2561753152,v1:172.21.15.149:6801/2561753152]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-27T03:07:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:33 smithi175 ceph-mon[37125]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-27T03:07:33.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:33 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:07:33.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:33 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:33.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:33 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:33.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:33 smithi067 ceph-mon[32814]: from='osd.1 [v2:172.21.15.149:6800/2561753152,v1:172.21.15.149:6801/2561753152]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-27T03:07:33.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:33 smithi067 ceph-mon[32814]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-07-27T03:07:34.680 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:34 smithi175 ceph-mon[37125]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:34.680 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:34 smithi175 ceph-mon[37125]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-27T03:07:34.680 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:34 smithi175 ceph-mon[37125]: from='osd.1 [v2:172.21.15.149:6800/2561753152,v1:172.21.15.149:6801/2561753152]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:07:34.680 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:34 smithi175 ceph-mon[37125]: osdmap e11: 2 total, 1 up, 2 in 2024-07-27T03:07:34.680 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-27T03:07:34.680 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:34 smithi175 ceph-mon[37125]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:07:34.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:34 smithi067 ceph-mon[32814]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:34.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:34 smithi067 ceph-mon[32814]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-27T03:07:34.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:34 smithi067 ceph-mon[32814]: from='osd.1 [v2:172.21.15.149:6800/2561753152,v1:172.21.15.149:6801/2561753152]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:07:34.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:34 smithi067 ceph-mon[32814]: osdmap e11: 2 total, 1 up, 2 in 2024-07-27T03:07:34.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-27T03:07:34.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:34 smithi067 ceph-mon[32814]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:07:34.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:34 smithi149 ceph-mon[37173]: pgmap v79: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:34.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:34 smithi149 ceph-mon[37173]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-07-27T03:07:34.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:34 smithi149 ceph-mon[37173]: from='osd.1 [v2:172.21.15.149:6800/2561753152,v1:172.21.15.149:6801/2561753152]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:07:34.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:34 smithi149 ceph-mon[37173]: osdmap e11: 2 total, 1 up, 2 in 2024-07-27T03:07:34.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-27T03:07:34.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:34 smithi149 ceph-mon[37173]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:07:34.930 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-07-27T03:07:35.014 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-07-27T03:07:35.027 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-07-27T03:07:35.028 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-07-27T03:07:35.028 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114839 s, 913 MB/s 2024-07-27T03:07:35.028 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-07-27T03:07:35.411 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:07:35 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-1[41680]: 2024-07-27T03:07:35.142+0000 7fce2e7da640 -1 osd.1 0 waiting for initial osdmap 2024-07-27T03:07:35.412 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:07:35 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-1[41680]: 2024-07-27T03:07:35.147+0000 7fce265d0640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-27T03:07:35.583 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:35 smithi175 ceph-mon[37125]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-07-27T03:07:35.584 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:35 smithi175 ceph-mon[37125]: osdmap e12: 2 total, 1 up, 2 in 2024-07-27T03:07:35.584 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:35 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-27T03:07:35.584 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:35 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-27T03:07:35.584 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:35 smithi175 ceph-mon[37125]: from='osd.1 ' entity='osd.1' 2024-07-27T03:07:35.619 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_4 2024-07-27T03:07:35.661 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:35 smithi149 ceph-mon[37173]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-07-27T03:07:35.662 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:35 smithi149 ceph-mon[37173]: osdmap e12: 2 total, 1 up, 2 in 2024-07-27T03:07:35.662 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:35 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-27T03:07:35.662 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:35 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-27T03:07:35.662 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:35 smithi149 ceph-mon[37173]: from='osd.1 ' entity='osd.1' 2024-07-27T03:07:35.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:35 smithi067 ceph-mon[32814]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-07-27T03:07:35.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:35 smithi067 ceph-mon[32814]: osdmap e12: 2 total, 1 up, 2 in 2024-07-27T03:07:35.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:35 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-27T03:07:35.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:35 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-27T03:07:35.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:35 smithi067 ceph-mon[32814]: from='osd.1 ' entity='osd.1' 2024-07-27T03:07:35.876 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.c/config 2024-07-27T03:07:36.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:36 smithi175 ceph-mon[37125]: purged_snaps scrub starts 2024-07-27T03:07:36.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:36 smithi175 ceph-mon[37125]: purged_snaps scrub ok 2024-07-27T03:07:36.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:36 smithi175 ceph-mon[37125]: pgmap v82: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:36.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:36 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-27T03:07:36.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:36 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:36.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:36 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:36.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:36 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:07:36.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:36 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:36.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:36 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:36.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:36 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:07:36.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:36 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:36.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:36 smithi175 ceph-mon[37125]: osd.1 [v2:172.21.15.149:6800/2561753152,v1:172.21.15.149:6801/2561753152] boot 2024-07-27T03:07:36.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:36 smithi175 ceph-mon[37125]: osdmap e13: 2 total, 2 up, 2 in 2024-07-27T03:07:36.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:36 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-27T03:07:36.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:36 smithi067 ceph-mon[32814]: purged_snaps scrub starts 2024-07-27T03:07:36.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:36 smithi067 ceph-mon[32814]: purged_snaps scrub ok 2024-07-27T03:07:36.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:36 smithi067 ceph-mon[32814]: pgmap v82: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:36.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:36 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-27T03:07:36.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:36 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:36.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:36 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:36.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:36 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:07:36.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:36 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:36.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:36 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:36.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:36 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:07:36.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:36 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:36.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:36 smithi067 ceph-mon[32814]: osd.1 [v2:172.21.15.149:6800/2561753152,v1:172.21.15.149:6801/2561753152] boot 2024-07-27T03:07:36.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:36 smithi067 ceph-mon[32814]: osdmap e13: 2 total, 2 up, 2 in 2024-07-27T03:07:36.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:36 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-27T03:07:36.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:36 smithi149 ceph-mon[37173]: purged_snaps scrub starts 2024-07-27T03:07:36.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:36 smithi149 ceph-mon[37173]: purged_snaps scrub ok 2024-07-27T03:07:36.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:36 smithi149 ceph-mon[37173]: pgmap v82: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-07-27T03:07:36.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:36 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-27T03:07:36.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:36 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:36.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:36 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:36.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:36 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:07:36.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:36 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:36.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:36 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:36.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:36 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:07:36.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:36 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:36.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:36 smithi149 ceph-mon[37173]: osd.1 [v2:172.21.15.149:6800/2561753152,v1:172.21.15.149:6801/2561753152] boot 2024-07-27T03:07:36.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:36 smithi149 ceph-mon[37173]: osdmap e13: 2 total, 2 up, 2 in 2024-07-27T03:07:36.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:36 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-07-27T03:07:37.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.407+0000 7f6aa1cf9640 1 -- 172.21.15.175:0/147202473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a9c104250 msgr2=0x7f6a9c104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:07:37.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.407+0000 7f6aa1cf9640 1 --2- 172.21.15.175:0/147202473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a9c104250 0x7f6a9c104660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009900 tx=0x7f6a8402a480 comp rx=0 tx=0).stop 2024-07-27T03:07:37.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.407+0000 7f6aa1cf9640 1 -- 172.21.15.175:0/147202473 shutdown_connections 2024-07-27T03:07:37.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.407+0000 7f6aa1cf9640 1 --2- 172.21.15.175:0/147202473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a9c104bf0 0x7f6a9c10b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:07:37.414 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.407+0000 7f6aa1cf9640 1 --2- 172.21.15.175:0/147202473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a9c104250 0x7f6a9c104660 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:07:37.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.407+0000 7f6aa1cf9640 1 --2- 172.21.15.175:0/147202473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a9c1031c0 0x7f6a9c103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:07:37.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.407+0000 7f6aa1cf9640 1 -- 172.21.15.175:0/147202473 >> 172.21.15.175:0/147202473 conn(0x7f6a9c0feb30 msgr2=0x7f6a9c100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:07:37.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.407+0000 7f6aa1cf9640 1 -- 172.21.15.175:0/147202473 shutdown_connections 2024-07-27T03:07:37.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.408+0000 7f6aa1cf9640 1 -- 172.21.15.175:0/147202473 wait complete. 2024-07-27T03:07:37.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.408+0000 7f6aa1cf9640 1 Processor -- start 2024-07-27T03:07:37.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.408+0000 7f6aa1cf9640 1 -- start start 2024-07-27T03:07:37.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.409+0000 7f6aa1cf9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a9c1031c0 0x7f6a9c199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:07:37.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.409+0000 7f6aa1cf9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a9c104250 0x7f6a9c19a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:07:37.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.409+0000 7f6aa1cf9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a9c104bf0 0x7f6a9c19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:07:37.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.409+0000 7f6aa1cf9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c10ee60 con 0x7f6a9c1031c0 2024-07-27T03:07:37.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.409+0000 7f6aa1cf9640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c10ece0 con 0x7f6a9c104250 2024-07-27T03:07:37.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.409+0000 7f6aa1cf9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c10efe0 con 0x7f6a9c104bf0 2024-07-27T03:07:37.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.409+0000 7f6a9bfff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a9c104bf0 0x7f6a9c19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:07:37.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.409+0000 7f6a9b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a9c1031c0 0x7f6a9c199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:07:37.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.409+0000 7f6a9b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a9c1031c0 0x7f6a9c199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.175:41920/0 (socket says 172.21.15.175:41920) 2024-07-27T03:07:37.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.409+0000 7f6a9bfff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a9c104bf0 0x7f6a9c19ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:46910/0 (socket says 172.21.15.175:46910) 2024-07-27T03:07:37.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.409+0000 7f6a9b7fe640 1 -- 172.21.15.175:0/1085421448 learned_addr learned my addr 172.21.15.175:0/1085421448 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-27T03:07:37.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.409+0000 7f6a93fff640 1 --2- 172.21.15.175:0/1085421448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a9c104250 0x7f6a9c19a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:07:37.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.409+0000 7f6a9b7fe640 1 -- 172.21.15.175:0/1085421448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a9c104bf0 msgr2=0x7f6a9c19ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:07:37.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.409+0000 7f6a9b7fe640 1 --2- 172.21.15.175:0/1085421448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a9c104bf0 0x7f6a9c19ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:07:37.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.409+0000 7f6a9b7fe640 1 -- 172.21.15.175:0/1085421448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a9c104250 msgr2=0x7f6a9c19a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:07:37.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.409+0000 7f6a9b7fe640 1 --2- 172.21.15.175:0/1085421448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a9c104250 0x7f6a9c19a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:07:37.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.409+0000 7f6a9b7fe640 1 -- 172.21.15.175:0/1085421448 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9c19b4b0 con 0x7f6a9c1031c0 2024-07-27T03:07:37.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.409+0000 7f6a9bfff640 1 --2- 172.21.15.175:0/1085421448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a9c104bf0 0x7f6a9c19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:07:37.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.409+0000 7f6a9b7fe640 1 --2- 172.21.15.175:0/1085421448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a9c1031c0 0x7f6a9c199a80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00d950 tx=0x7f6a8c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:07:37.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.410+0000 7f6a997fa640 1 -- 172.21.15.175:0/1085421448 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c017070 con 0x7f6a9c1031c0 2024-07-27T03:07:37.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.410+0000 7f6aa1cf9640 1 -- 172.21.15.175:0/1085421448 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a9c19b7a0 con 0x7f6a9c1031c0 2024-07-27T03:07:37.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.410+0000 7f6a997fa640 1 -- 172.21.15.175:0/1085421448 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a8c009c30 con 0x7f6a9c1031c0 2024-07-27T03:07:37.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.410+0000 7f6a997fa640 1 -- 172.21.15.175:0/1085421448 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c00f620 con 0x7f6a9c1031c0 2024-07-27T03:07:37.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.410+0000 7f6aa1cf9640 1 -- 172.21.15.175:0/1085421448 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a9c1a1ea0 con 0x7f6a9c1031c0 2024-07-27T03:07:37.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.411+0000 7f6aa1cf9640 1 -- 172.21.15.175:0/1085421448 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a9c103630 con 0x7f6a9c1031c0 2024-07-27T03:07:37.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.413+0000 7f6a997fa640 1 -- 172.21.15.175:0/1085421448 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 151003+0+0 (secure 0 0 0) 0x7f6a8c010460 con 0x7f6a9c1031c0 2024-07-27T03:07:37.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.413+0000 7f6a997fa640 1 --2- 172.21.15.175:0/1085421448 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f6a880b3b40 0x7f6a880b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:07:37.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.413+0000 7f6a997fa640 1 -- 172.21.15.175:0/1085421448 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f6a8c0e1f90 con 0x7f6a9c1031c0 2024-07-27T03:07:37.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.413+0000 7f6a93fff640 1 --2- 172.21.15.175:0/1085421448 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f6a880b3b40 0x7f6a880b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:07:37.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.414+0000 7f6a93fff640 1 --2- 172.21.15.175:0/1085421448 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f6a880b3b40 0x7f6a880b6050 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6a84000c00 tx=0x7f6a84031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:07:37.423 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.420+0000 7f6a997fa640 1 -- 172.21.15.175:0/1085421448 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f6a8c009de0 con 0x7f6a9c1031c0 2024-07-27T03:07:37.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:37.603+0000 7f6aa1cf9640 1 -- 172.21.15.175:0/1085421448 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6a9c063330 con 0x7f6a880b3b40 2024-07-27T03:07:37.607 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:37 smithi175 ceph-mon[37125]: Detected new or changed devices on smithi149 2024-07-27T03:07:37.607 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:37 smithi175 ceph-mon[37125]: Adjusting osd_memory_target on smithi149 to 5816M 2024-07-27T03:07:37.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:37 smithi067 ceph-mon[32814]: Detected new or changed devices on smithi149 2024-07-27T03:07:37.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:37 smithi067 ceph-mon[32814]: Adjusting osd_memory_target on smithi149 to 5816M 2024-07-27T03:07:37.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:37 smithi149 ceph-mon[37173]: Detected new or changed devices on smithi149 2024-07-27T03:07:37.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:37 smithi149 ceph-mon[37173]: Adjusting osd_memory_target on smithi149 to 5816M 2024-07-27T03:07:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:38 smithi175 ceph-mon[37125]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:38 smithi175 ceph-mon[37125]: osdmap e14: 2 total, 2 up, 2 in 2024-07-27T03:07:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:38 smithi175 ceph-mon[37125]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:07:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:07:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:07:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:38.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:38 smithi067 ceph-mon[32814]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:38.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:38 smithi067 ceph-mon[32814]: osdmap e14: 2 total, 2 up, 2 in 2024-07-27T03:07:38.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:38 smithi067 ceph-mon[32814]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:07:38.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:38 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:07:38.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:38 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:07:38.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:38 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:38.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:38 smithi149 ceph-mon[37173]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:38.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:38 smithi149 ceph-mon[37173]: osdmap e14: 2 total, 2 up, 2 in 2024-07-27T03:07:38.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:38 smithi149 ceph-mon[37173]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:07:38.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:38 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:07:38.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:38 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:07:38.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:38 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:40.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:40 smithi067 ceph-mon[32814]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:40.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:40 smithi067 ceph-mon[32814]: from='client.? 172.21.15.175:0/8752601' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "444e0ebe-7ca3-4be9-8bd5-db859f824445"}]: dispatch 2024-07-27T03:07:40.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:40 smithi067 ceph-mon[32814]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "444e0ebe-7ca3-4be9-8bd5-db859f824445"}]: dispatch 2024-07-27T03:07:40.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:40 smithi067 ceph-mon[32814]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "444e0ebe-7ca3-4be9-8bd5-db859f824445"}]': finished 2024-07-27T03:07:40.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:40 smithi067 ceph-mon[32814]: osdmap e15: 3 total, 2 up, 3 in 2024-07-27T03:07:40.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:40 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-27T03:07:40.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:40 smithi149 ceph-mon[37173]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:40.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:40 smithi149 ceph-mon[37173]: from='client.? 172.21.15.175:0/8752601' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "444e0ebe-7ca3-4be9-8bd5-db859f824445"}]: dispatch 2024-07-27T03:07:40.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:40 smithi149 ceph-mon[37173]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "444e0ebe-7ca3-4be9-8bd5-db859f824445"}]: dispatch 2024-07-27T03:07:40.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:40 smithi149 ceph-mon[37173]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "444e0ebe-7ca3-4be9-8bd5-db859f824445"}]': finished 2024-07-27T03:07:40.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:40 smithi149 ceph-mon[37173]: osdmap e15: 3 total, 2 up, 3 in 2024-07-27T03:07:40.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:40 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-27T03:07:40.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:40 smithi175 ceph-mon[37125]: pgmap v86: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:40.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:40 smithi175 ceph-mon[37125]: from='client.? 172.21.15.175:0/8752601' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "444e0ebe-7ca3-4be9-8bd5-db859f824445"}]: dispatch 2024-07-27T03:07:40.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:40 smithi175 ceph-mon[37125]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "444e0ebe-7ca3-4be9-8bd5-db859f824445"}]: dispatch 2024-07-27T03:07:40.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:40 smithi175 ceph-mon[37125]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "444e0ebe-7ca3-4be9-8bd5-db859f824445"}]': finished 2024-07-27T03:07:40.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:40 smithi175 ceph-mon[37125]: osdmap e15: 3 total, 2 up, 3 in 2024-07-27T03:07:40.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:40 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-27T03:07:41.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:41 smithi067 ceph-mon[32814]: from='client.? 172.21.15.175:0/3309229753' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:07:41.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:41 smithi149 ceph-mon[37173]: from='client.? 172.21.15.175:0/3309229753' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:07:41.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:41 smithi175 ceph-mon[37125]: from='client.? 172.21.15.175:0/3309229753' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:07:42.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:42 smithi067 ceph-mon[32814]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:42.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:42 smithi149 ceph-mon[37173]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:42.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:42 smithi175 ceph-mon[37125]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:44.717 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:44 smithi175 ceph-mon[37125]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:44.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:44 smithi067 ceph-mon[32814]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:44.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:44 smithi149 ceph-mon[37173]: pgmap v89: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:46.560 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:46 smithi175 ceph-mon[37125]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:46.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:46 smithi067 ceph-mon[32814]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:46.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:46 smithi149 ceph-mon[37173]: pgmap v90: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:47.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:47 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-27T03:07:47.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:47 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:47.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:47 smithi175 ceph-mon[37125]: Deploying daemon osd.2 on smithi175 2024-07-27T03:07:47.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:47 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-27T03:07:47.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:47 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:47.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:47 smithi067 ceph-mon[32814]: Deploying daemon osd.2 on smithi175 2024-07-27T03:07:47.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-07-27T03:07:47.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:47.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:47 smithi149 ceph-mon[37173]: Deploying daemon osd.2 on smithi175 2024-07-27T03:07:48.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:48 smithi067 ceph-mon[32814]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:48.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:48 smithi149 ceph-mon[37173]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:48.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:48 smithi175 ceph-mon[37125]: pgmap v91: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:50.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:50 smithi175 ceph-mon[37125]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:50.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:50 smithi067 ceph-mon[32814]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:50.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:50 smithi149 ceph-mon[37173]: pgmap v92: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:52.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:52 smithi175 ceph-mon[37125]: pgmap v93: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:52.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:07:52.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:52.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:52.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:52 smithi067 ceph-mon[32814]: pgmap v93: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:52.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:07:52.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:52.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:52.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:52 smithi149 ceph-mon[37173]: pgmap v93: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:52.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:07:52.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:52.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:54.318 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:54.314+0000 7f6a997fa640 1 -- 172.21.15.175:0/1085421448 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6a9c063330 con 0x7f6a880b3b40 2024-07-27T03:07:54.318 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 2 on host 'smithi175' 2024-07-27T03:07:54.319 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:54.316+0000 7f6aa1cf9640 1 -- 172.21.15.175:0/1085421448 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f6a880b3b40 msgr2=0x7f6a880b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:07:54.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:54.317+0000 7f6aa1cf9640 1 --2- 172.21.15.175:0/1085421448 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f6a880b3b40 0x7f6a880b6050 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6a84000c00 tx=0x7f6a84031400 comp rx=0 tx=0).stop 2024-07-27T03:07:54.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:54.317+0000 7f6aa1cf9640 1 -- 172.21.15.175:0/1085421448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a9c1031c0 msgr2=0x7f6a9c199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:07:54.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:54.317+0000 7f6aa1cf9640 1 --2- 172.21.15.175:0/1085421448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a9c1031c0 0x7f6a9c199a80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00d950 tx=0x7f6a8c00de10 comp rx=0 tx=0).stop 2024-07-27T03:07:54.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:54.317+0000 7f6aa1cf9640 1 -- 172.21.15.175:0/1085421448 shutdown_connections 2024-07-27T03:07:54.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:54.317+0000 7f6aa1cf9640 1 --2- 172.21.15.175:0/1085421448 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f6a880b3b40 0x7f6a880b6050 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:07:54.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:54.317+0000 7f6aa1cf9640 1 --2- 172.21.15.175:0/1085421448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a9c104bf0 0x7f6a9c19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:07:54.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:54.317+0000 7f6aa1cf9640 1 --2- 172.21.15.175:0/1085421448 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a9c104250 0x7f6a9c19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:07:54.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:54.317+0000 7f6aa1cf9640 1 --2- 172.21.15.175:0/1085421448 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6a9c1031c0 0x7f6a9c199a80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:07:54.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:54.317+0000 7f6aa1cf9640 1 -- 172.21.15.175:0/1085421448 >> 172.21.15.175:0/1085421448 conn(0x7f6a9c0feb30 msgr2=0x7f6a9c107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:07:54.320 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:54.317+0000 7f6aa1cf9640 1 -- 172.21.15.175:0/1085421448 shutdown_connections 2024-07-27T03:07:54.321 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:07:54.317+0000 7f6aa1cf9640 1 -- 172.21.15.175:0/1085421448 wait complete. 2024-07-27T03:07:54.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:54 smithi175 ceph-mon[37125]: pgmap v94: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:54.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:54.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:54.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:54 smithi067 ceph-mon[32814]: pgmap v94: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:54.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:54.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:54.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:54 smithi149 ceph-mon[37173]: pgmap v94: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:54.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:54.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:55.283 DEBUG:teuthology.orchestra.run.smithi175:osd.2> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.2.service 2024-07-27T03:07:55.286 INFO:tasks.cephadm:Deploying osd.3 on smithi067 with /dev/vg_nvme/lv_3... 2024-07-27T03:07:55.286 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-07-27T03:07:55.561 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:07:55.624 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:07:55 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-2[42269]: 2024-07-27T03:07:55.434+0000 7f09ba255640 -1 osd.2 0 log_to_monitors true 2024-07-27T03:07:56.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:56 smithi175 ceph-mon[37125]: pgmap v95: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:56.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:56 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:56.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:56 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:56.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:56 smithi175 ceph-mon[37125]: from='osd.2 [v2:172.21.15.175:6800/3770940078,v1:172.21.15.175:6801/3770940078]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-27T03:07:56.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:56 smithi175 ceph-mon[37125]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-27T03:07:56.705 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:56 smithi067 ceph-mon[32814]: pgmap v95: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:56.705 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:56 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:56.706 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:56 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:56.706 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:56 smithi067 ceph-mon[32814]: from='osd.2 [v2:172.21.15.175:6800/3770940078,v1:172.21.15.175:6801/3770940078]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-27T03:07:56.706 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:56 smithi067 ceph-mon[32814]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-27T03:07:56.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:56 smithi149 ceph-mon[37173]: pgmap v95: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:56.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:56 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:56.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:56 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:56.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:56 smithi149 ceph-mon[37173]: from='osd.2 [v2:172.21.15.175:6800/3770940078,v1:172.21.15.175:6801/3770940078]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-27T03:07:56.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:56 smithi149 ceph-mon[37173]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-07-27T03:07:57.471 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-07-27T03:07:57.553 INFO:teuthology.orchestra.run.smithi067.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-07-27T03:07:57.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:57 smithi067 ceph-mon[32814]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-27T03:07:57.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:57 smithi067 ceph-mon[32814]: osdmap e16: 3 total, 2 up, 3 in 2024-07-27T03:07:57.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-27T03:07:57.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:57 smithi067 ceph-mon[32814]: from='osd.2 [v2:172.21.15.175:6800/3770940078,v1:172.21.15.175:6801/3770940078]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:07:57.554 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:57 smithi067 ceph-mon[32814]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:07:57.568 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10+0 records in 2024-07-27T03:07:57.568 INFO:teuthology.orchestra.run.smithi067.stderr:10+0 records out 2024-07-27T03:07:57.568 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128356 s, 817 MB/s 2024-07-27T03:07:57.568 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping successful for: 2024-07-27T03:07:57.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:57 smithi175 ceph-mon[37125]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-27T03:07:57.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:57 smithi175 ceph-mon[37125]: osdmap e16: 3 total, 2 up, 3 in 2024-07-27T03:07:57.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-27T03:07:57.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:57 smithi175 ceph-mon[37125]: from='osd.2 [v2:172.21.15.175:6800/3770940078,v1:172.21.15.175:6801/3770940078]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:07:57.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:57 smithi175 ceph-mon[37125]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:07:57.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:57 smithi149 ceph-mon[37173]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-07-27T03:07:57.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:57 smithi149 ceph-mon[37173]: osdmap e16: 3 total, 2 up, 3 in 2024-07-27T03:07:57.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-27T03:07:57.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:57 smithi149 ceph-mon[37173]: from='osd.2 [v2:172.21.15.175:6800/3770940078,v1:172.21.15.175:6801/3770940078]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:07:57.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:57 smithi149 ceph-mon[37173]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:07:58.125 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:07:58 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-2[42269]: 2024-07-27T03:07:58.070+0000 7f09b61d7640 -1 osd.2 0 waiting for initial osdmap 2024-07-27T03:07:58.125 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:07:58 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-2[42269]: 2024-07-27T03:07:58.075+0000 7f09adfcd640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-27T03:07:58.263 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph orch daemon add osd smithi067:vg_nvme/lv_3 2024-07-27T03:07:58.511 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:58 smithi175 ceph-mon[37125]: pgmap v97: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:58.511 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:58 smithi175 ceph-mon[37125]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-27T03:07:58.511 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:58 smithi175 ceph-mon[37125]: osdmap e17: 3 total, 2 up, 3 in 2024-07-27T03:07:58.511 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:58 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-27T03:07:58.511 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:58 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-27T03:07:58.512 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:58 smithi175 ceph-mon[37125]: from='osd.2 ' entity='osd.2' 2024-07-27T03:07:58.534 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:07:58.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:58 smithi067 ceph-mon[32814]: pgmap v97: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:58.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:58 smithi067 ceph-mon[32814]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-27T03:07:58.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:58 smithi067 ceph-mon[32814]: osdmap e17: 3 total, 2 up, 3 in 2024-07-27T03:07:58.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:58 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-27T03:07:58.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:58 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-27T03:07:58.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:58 smithi067 ceph-mon[32814]: from='osd.2 ' entity='osd.2' 2024-07-27T03:07:58.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:58 smithi149 ceph-mon[37173]: pgmap v97: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:07:58.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:58 smithi149 ceph-mon[37173]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-27T03:07:58.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:58 smithi149 ceph-mon[37173]: osdmap e17: 3 total, 2 up, 3 in 2024-07-27T03:07:58.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:58 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-27T03:07:58.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:58 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-27T03:07:58.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:58 smithi149 ceph-mon[37173]: from='osd.2 ' entity='osd.2' 2024-07-27T03:07:59.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:59 smithi175 ceph-mon[37125]: purged_snaps scrub starts 2024-07-27T03:07:59.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:59 smithi175 ceph-mon[37125]: purged_snaps scrub ok 2024-07-27T03:07:59.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-27T03:07:59.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:59 smithi175 ceph-mon[37125]: Detected new or changed devices on smithi175 2024-07-27T03:07:59.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:59.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:59.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:07:59.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:59 smithi175 ceph-mon[37125]: Adjusting osd_memory_target on smithi175 to 17088M 2024-07-27T03:07:59.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:59.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:59.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:07:59.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:59.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:07:59.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:59.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:07:59.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:59.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:59 smithi175 ceph-mon[37125]: osd.2 [v2:172.21.15.175:6800/3770940078,v1:172.21.15.175:6801/3770940078] boot 2024-07-27T03:07:59.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:59 smithi175 ceph-mon[37125]: osdmap e18: 3 total, 3 up, 3 in 2024-07-27T03:07:59.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:07:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-27T03:07:59.638 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:59 smithi067 ceph-mon[32814]: purged_snaps scrub starts 2024-07-27T03:07:59.638 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:59 smithi067 ceph-mon[32814]: purged_snaps scrub ok 2024-07-27T03:07:59.638 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-27T03:07:59.639 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:59 smithi067 ceph-mon[32814]: Detected new or changed devices on smithi175 2024-07-27T03:07:59.639 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:59.639 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:59.639 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:07:59.639 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:59 smithi067 ceph-mon[32814]: Adjusting osd_memory_target on smithi175 to 17088M 2024-07-27T03:07:59.639 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:59.639 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:59.639 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:07:59.639 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:59.639 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:07:59.639 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:59.639 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:07:59.639 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:59.640 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:59 smithi067 ceph-mon[32814]: osd.2 [v2:172.21.15.175:6800/3770940078,v1:172.21.15.175:6801/3770940078] boot 2024-07-27T03:07:59.640 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:59 smithi067 ceph-mon[32814]: osdmap e18: 3 total, 3 up, 3 in 2024-07-27T03:07:59.640 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:07:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-27T03:07:59.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:59 smithi149 ceph-mon[37173]: purged_snaps scrub starts 2024-07-27T03:07:59.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:59 smithi149 ceph-mon[37173]: purged_snaps scrub ok 2024-07-27T03:07:59.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-27T03:07:59.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:59 smithi149 ceph-mon[37173]: Detected new or changed devices on smithi175 2024-07-27T03:07:59.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:59.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:59.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:07:59.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:59 smithi149 ceph-mon[37173]: Adjusting osd_memory_target on smithi175 to 17088M 2024-07-27T03:07:59.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:59.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:59.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:07:59.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:59.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:07:59.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:07:59.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:07:59.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:07:59.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:59 smithi149 ceph-mon[37173]: osd.2 [v2:172.21.15.175:6800/3770940078,v1:172.21.15.175:6801/3770940078] boot 2024-07-27T03:07:59.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:59 smithi149 ceph-mon[37173]: osdmap e18: 3 total, 3 up, 3 in 2024-07-27T03:07:59.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:07:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-07-27T03:08:00.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.158+0000 7f51b4449640 1 -- 172.21.15.67:0/1439759254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51ac06c0d0 msgr2=0x7f51ac105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:08:00.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.158+0000 7f51b4449640 1 --2- 172.21.15.67:0/1439759254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51ac06c0d0 0x7f51ac105e30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f51a4009900 tx=0x7f51a402a410 comp rx=0 tx=0).stop 2024-07-27T03:08:00.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.159+0000 7f51b4449640 1 -- 172.21.15.67:0/1439759254 shutdown_connections 2024-07-27T03:08:00.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.159+0000 7f51b4449640 1 --2- 172.21.15.67:0/1439759254 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51ac108f10 0x7f51ac10b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:00.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.159+0000 7f51b4449640 1 --2- 172.21.15.67:0/1439759254 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51ac106370 0x7f51ac1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:00.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.159+0000 7f51b4449640 1 --2- 172.21.15.67:0/1439759254 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51ac06c0d0 0x7f51ac105e30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:00.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.159+0000 7f51b4449640 1 -- 172.21.15.67:0/1439759254 >> 172.21.15.67:0/1439759254 conn(0x7f51ac0fda50 msgr2=0x7f51ac0fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:08:00.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.159+0000 7f51b4449640 1 -- 172.21.15.67:0/1439759254 shutdown_connections 2024-07-27T03:08:00.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.159+0000 7f51b4449640 1 -- 172.21.15.67:0/1439759254 wait complete. 2024-07-27T03:08:00.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.159+0000 7f51b4449640 1 Processor -- start 2024-07-27T03:08:00.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.159+0000 7f51b4449640 1 -- start start 2024-07-27T03:08:00.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.160+0000 7f51b4449640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51ac06c0d0 0x7f51ac199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:08:00.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.160+0000 7f51b4449640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51ac106370 0x7f51ac19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:08:00.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.160+0000 7f51b4449640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51ac108f10 0x7f51ac19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:08:00.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.160+0000 7f51b4449640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac10ee30 con 0x7f51ac06c0d0 2024-07-27T03:08:00.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.160+0000 7f51b4449640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac10ecb0 con 0x7f51ac108f10 2024-07-27T03:08:00.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.160+0000 7f51b4449640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac10efb0 con 0x7f51ac106370 2024-07-27T03:08:00.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.160+0000 7f51b21e5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51ac06c0d0 0x7f51ac199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:08:00.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.160+0000 7f51b21e5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51ac06c0d0 0x7f51ac199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55892/0 (socket says 172.21.15.67:55892) 2024-07-27T03:08:00.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.160+0000 7f51b21e5640 1 -- 172.21.15.67:0/384574579 learned_addr learned my addr 172.21.15.67:0/384574579 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:08:00.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.160+0000 7f51b19e4640 1 --2- 172.21.15.67:0/384574579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51ac106370 0x7f51ac19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:08:00.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.160+0000 7f51b29e6640 1 --2- 172.21.15.67:0/384574579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51ac108f10 0x7f51ac19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:08:00.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.160+0000 7f51b21e5640 1 -- 172.21.15.67:0/384574579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51ac106370 msgr2=0x7f51ac19a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:08:00.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.160+0000 7f51b21e5640 1 --2- 172.21.15.67:0/384574579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51ac106370 0x7f51ac19a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:00.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.160+0000 7f51b21e5640 1 -- 172.21.15.67:0/384574579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51ac108f10 msgr2=0x7f51ac19ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:08:00.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.160+0000 7f51b21e5640 1 --2- 172.21.15.67:0/384574579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51ac108f10 0x7f51ac19ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:00.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.160+0000 7f51b21e5640 1 -- 172.21.15.67:0/384574579 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ac19b440 con 0x7f51ac06c0d0 2024-07-27T03:08:00.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.160+0000 7f51b29e6640 1 --2- 172.21.15.67:0/384574579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51ac108f10 0x7f51ac19ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:08:00.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.160+0000 7f51b21e5640 1 --2- 172.21.15.67:0/384574579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51ac06c0d0 0x7f51ac199a10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f51a4006010 tx=0x7f51a4004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:08:00.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.161+0000 7f51ab7fe640 1 -- 172.21.15.67:0/384574579 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a403f030 con 0x7f51ac06c0d0 2024-07-27T03:08:00.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.161+0000 7f51ab7fe640 1 -- 172.21.15.67:0/384574579 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51a4003cb0 con 0x7f51ac06c0d0 2024-07-27T03:08:00.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.161+0000 7f51b4449640 1 -- 172.21.15.67:0/384574579 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51ac19b6d0 con 0x7f51ac06c0d0 2024-07-27T03:08:00.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.161+0000 7f51ab7fe640 1 -- 172.21.15.67:0/384574579 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a4035070 con 0x7f51ac06c0d0 2024-07-27T03:08:00.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.161+0000 7f51b4449640 1 -- 172.21.15.67:0/384574579 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51ac1a1e30 con 0x7f51ac06c0d0 2024-07-27T03:08:00.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.162+0000 7f51b4449640 1 -- 172.21.15.67:0/384574579 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51ac1067c0 con 0x7f51ac06c0d0 2024-07-27T03:08:00.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.163+0000 7f51ab7fe640 1 -- 172.21.15.67:0/384574579 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 15) ==== 151003+0+0 (secure 0 0 0) 0x7f51a4042030 con 0x7f51ac06c0d0 2024-07-27T03:08:00.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.164+0000 7f51ab7fe640 1 --2- 172.21.15.67:0/384574579 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f51940b3bc0 0x7f51940b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:08:00.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.165+0000 7f51b19e4640 1 --2- 172.21.15.67:0/384574579 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f51940b3bc0 0x7f51940b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:08:00.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.165+0000 7f51ab7fe640 1 -- 172.21.15.67:0/384574579 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7f51a4103220 con 0x7f51ac06c0d0 2024-07-27T03:08:00.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.165+0000 7f51b19e4640 1 --2- 172.21.15.67:0/384574579 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f51940b3bc0 0x7f51940b60d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f51a000bbe0 tx=0x7f51a000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:08:00.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.168+0000 7f51ab7fe640 1 -- 172.21.15.67:0/384574579 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f51a40cbea0 con 0x7f51ac06c0d0 2024-07-27T03:08:00.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:00.361+0000 7f51b4449640 1 -- 172.21.15.67:0/384574579 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f51ac02e4e0 con 0x7f51940b3bc0 2024-07-27T03:08:00.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:00 smithi067 ceph-mon[32814]: pgmap v99: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:08:00.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:08:00.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:08:00.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:00.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:00 smithi067 ceph-mon[32814]: osdmap e19: 3 total, 3 up, 3 in 2024-07-27T03:08:01.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:00 smithi149 ceph-mon[37173]: pgmap v99: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:08:01.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:08:01.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:08:01.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:01.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:00 smithi149 ceph-mon[37173]: osdmap e19: 3 total, 3 up, 3 in 2024-07-27T03:08:01.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:00 smithi175 ceph-mon[37125]: pgmap v99: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-07-27T03:08:01.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:08:01.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:08:01.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:01.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:00 smithi175 ceph-mon[37125]: osdmap e19: 3 total, 3 up, 3 in 2024-07-27T03:08:01.957 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:01 smithi067 ceph-mon[32814]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:08:01.958 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:01 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-07-27T03:08:02.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:01 smithi149 ceph-mon[37173]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:08:02.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:01 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-07-27T03:08:02.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:01 smithi175 ceph-mon[37125]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:08:02.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:01 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-07-27T03:08:02.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:02 smithi067 ceph-mon[32814]: pgmap v102: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:02.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:02 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-07-27T03:08:02.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:02 smithi067 ceph-mon[32814]: osdmap e20: 3 total, 3 up, 3 in 2024-07-27T03:08:02.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:02 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-07-27T03:08:02.970 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:08:02 smithi067 sudo[55373]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-27T03:08:02.970 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:08:02 smithi067 sudo[55373]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-27T03:08:02.970 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:08:02 smithi067 sudo[55373]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-27T03:08:02.970 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:08:02 smithi067 sudo[55373]: pam_unix(sudo:session): session closed for user root 2024-07-27T03:08:02.970 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:08:02 smithi067 sudo[55378]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-27T03:08:02.970 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:08:02 smithi067 sudo[55378]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-27T03:08:02.971 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:08:02 smithi067 sudo[55378]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-27T03:08:02.971 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:08:02 smithi067 sudo[55378]: pam_unix(sudo:session): session closed for user root 2024-07-27T03:08:03.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:02 smithi149 ceph-mon[37173]: pgmap v102: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:03.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:02 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-07-27T03:08:03.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:02 smithi149 ceph-mon[37173]: osdmap e20: 3 total, 3 up, 3 in 2024-07-27T03:08:03.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:02 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-07-27T03:08:03.000 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:08:02 smithi149 sudo[45901]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-27T03:08:03.000 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:08:02 smithi149 sudo[45901]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-27T03:08:03.000 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:08:02 smithi149 sudo[45901]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-27T03:08:03.001 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:08:02 smithi149 sudo[45901]: pam_unix(sudo:session): session closed for user root 2024-07-27T03:08:03.001 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:08:02 smithi149 sudo[45904]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-27T03:08:03.001 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:08:02 smithi149 sudo[45904]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-27T03:08:03.001 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:08:02 smithi149 sudo[45904]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-27T03:08:03.001 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:08:02 smithi149 sudo[45904]: pam_unix(sudo:session): session closed for user root 2024-07-27T03:08:03.019 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:08:02 smithi175 sudo[46289]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-27T03:08:03.020 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:08:02 smithi175 sudo[46289]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-27T03:08:03.020 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:08:02 smithi175 sudo[46289]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-27T03:08:03.020 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:08:02 smithi175 sudo[46289]: pam_unix(sudo:session): session closed for user root 2024-07-27T03:08:03.020 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:08:02 smithi175 sudo[46292]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-27T03:08:03.020 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:08:02 smithi175 sudo[46292]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-27T03:08:03.020 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:08:02 smithi175 sudo[46292]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-27T03:08:03.020 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:08:02 smithi175 sudo[46292]: pam_unix(sudo:session): session closed for user root 2024-07-27T03:08:03.020 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:02 smithi175 ceph-mon[37125]: pgmap v102: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:03.020 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:02 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-07-27T03:08:03.020 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:02 smithi175 ceph-mon[37125]: osdmap e20: 3 total, 3 up, 3 in 2024-07-27T03:08:03.020 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:02 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-07-27T03:08:03.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:02 smithi067 sudo[55387]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-27T03:08:03.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:02 smithi067 sudo[55387]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-27T03:08:03.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:02 smithi067 sudo[55387]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-27T03:08:03.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:02 smithi067 sudo[55387]: pam_unix(sudo:session): session closed for user root 2024-07-27T03:08:03.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 sudo[55390]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-27T03:08:03.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 sudo[55390]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-27T03:08:03.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 sudo[55390]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-27T03:08:03.310 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 sudo[55390]: pam_unix(sudo:session): session closed for user root 2024-07-27T03:08:03.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 sudo[46295]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-27T03:08:03.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 sudo[46295]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-27T03:08:03.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 sudo[46295]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-27T03:08:03.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 sudo[46295]: pam_unix(sudo:session): session closed for user root 2024-07-27T03:08:03.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 sudo[46298]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-27T03:08:03.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 sudo[46298]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-27T03:08:03.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 sudo[46298]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-27T03:08:03.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 sudo[46298]: pam_unix(sudo:session): session closed for user root 2024-07-27T03:08:03.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 sudo[45907]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-07-27T03:08:03.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 sudo[45907]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-27T03:08:03.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 sudo[45907]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-27T03:08:03.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 sudo[45907]: pam_unix(sudo:session): session closed for user root 2024-07-27T03:08:03.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 sudo[45910]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-07-27T03:08:03.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 sudo[45910]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-07-27T03:08:03.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 sudo[45910]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-07-27T03:08:03.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 sudo[45910]: pam_unix(sudo:session): session closed for user root 2024-07-27T03:08:03.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:03.975+0000 7f51ab7fe640 1 -- 172.21.15.67:0/384574579 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f51a40c7a30 con 0x7f51ac06c0d0 2024-07-27T03:08:03.980 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/794693336' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d554db3d-9604-427c-8736-b1bb379f8d5f"}]: dispatch 2024-07-27T03:08:03.980 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-07-27T03:08:03.980 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: osdmap e21: 3 total, 3 up, 3 in 2024-07-27T03:08:03.980 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/794693336' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d554db3d-9604-427c-8736-b1bb379f8d5f"}]': finished 2024-07-27T03:08:03.980 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: osdmap e22: 4 total, 3 up, 4 in 2024-07-27T03:08:03.980 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-27T03:08:03.980 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:03.980 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:03.981 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:03.981 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-27T03:08:03.981 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-27T03:08:03.981 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-27T03:08:03.981 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:08:03.981 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:08:03.981 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-27T03:08:03.981 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-27T03:08:03.981 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:08:03.981 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:08:03.981 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-27T03:08:03.981 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-27T03:08:03.981 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-27T03:08:03.981 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:08:03.981 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:08:03.982 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-27T03:08:03.982 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:03 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1443342035' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:08:04.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/794693336' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d554db3d-9604-427c-8736-b1bb379f8d5f"}]: dispatch 2024-07-27T03:08:04.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-07-27T03:08:04.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: osdmap e21: 3 total, 3 up, 3 in 2024-07-27T03:08:04.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/794693336' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d554db3d-9604-427c-8736-b1bb379f8d5f"}]': finished 2024-07-27T03:08:04.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: osdmap e22: 4 total, 3 up, 4 in 2024-07-27T03:08:04.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-27T03:08:04.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:04.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:04.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:04.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-27T03:08:04.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-27T03:08:04.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-27T03:08:04.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:08:04.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:08:04.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-27T03:08:04.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-27T03:08:04.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:08:04.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:08:04.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-27T03:08:04.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-27T03:08:04.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-27T03:08:04.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:08:04.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:08:04.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-27T03:08:04.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:03 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1443342035' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:08:04.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/794693336' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d554db3d-9604-427c-8736-b1bb379f8d5f"}]: dispatch 2024-07-27T03:08:04.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-07-27T03:08:04.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: osdmap e21: 3 total, 3 up, 3 in 2024-07-27T03:08:04.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/794693336' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d554db3d-9604-427c-8736-b1bb379f8d5f"}]': finished 2024-07-27T03:08:04.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: osdmap e22: 4 total, 3 up, 4 in 2024-07-27T03:08:04.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-27T03:08:04.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:04.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:04.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:04.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-27T03:08:04.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-27T03:08:04.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-27T03:08:04.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:08:04.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:08:04.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-27T03:08:04.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-27T03:08:04.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:08:04.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:08:04.127 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-27T03:08:04.127 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-07-27T03:08:04.127 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-07-27T03:08:04.127 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-07-27T03:08:04.127 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-07-27T03:08:04.127 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-07-27T03:08:04.127 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:03 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1443342035' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:08:05.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:04 smithi149 ceph-mon[37173]: pgmap v106: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:05.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:04 smithi149 ceph-mon[37173]: osdmap e23: 4 total, 3 up, 4 in 2024-07-27T03:08:05.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:04 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-27T03:08:05.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:04 smithi149 ceph-mon[37173]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-07-27T03:08:05.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:04 smithi175 ceph-mon[37125]: pgmap v106: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:05.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:04 smithi175 ceph-mon[37125]: osdmap e23: 4 total, 3 up, 4 in 2024-07-27T03:08:05.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:04 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-27T03:08:05.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:04 smithi175 ceph-mon[37125]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-07-27T03:08:05.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:04 smithi067 ceph-mon[32814]: pgmap v106: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:05.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:04 smithi067 ceph-mon[32814]: osdmap e23: 4 total, 3 up, 4 in 2024-07-27T03:08:05.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:04 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-27T03:08:05.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:04 smithi067 ceph-mon[32814]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-07-27T03:08:06.965 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:06 smithi067 ceph-mon[32814]: pgmap v108: 1 pgs: 1 unknown; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:07.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:06 smithi149 ceph-mon[37173]: pgmap v108: 1 pgs: 1 unknown; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:07.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:06 smithi175 ceph-mon[37125]: pgmap v108: 1 pgs: 1 unknown; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:09.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:08 smithi149 ceph-mon[37173]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:09.027 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:08 smithi067 ceph-mon[32814]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:09.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:08 smithi175 ceph-mon[37125]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:09.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-27T03:08:09.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:10.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-27T03:08:10.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:10.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-07-27T03:08:10.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:11.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:10 smithi149 ceph-mon[37173]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:11.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:10 smithi149 ceph-mon[37173]: Deploying daemon osd.3 on smithi067 2024-07-27T03:08:11.034 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:10 smithi067 ceph-mon[32814]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:11.034 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:10 smithi067 ceph-mon[32814]: Deploying daemon osd.3 on smithi067 2024-07-27T03:08:11.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:10 smithi175 ceph-mon[37125]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:11.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:10 smithi175 ceph-mon[37125]: Deploying daemon osd.3 on smithi067 2024-07-27T03:08:11.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:11 smithi067 ceph-mon[32814]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:12.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:11 smithi149 ceph-mon[37173]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:12.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:11 smithi175 ceph-mon[37125]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:14.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:14 smithi175 ceph-mon[37125]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:14.426 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:14 smithi067 ceph-mon[32814]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:14.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:14 smithi149 ceph-mon[37173]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:15.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:15 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:08:15.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:15 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:15.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:15 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:15.315 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:15 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:08:15.315 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:15 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:15.315 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:15 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:15.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:15 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:08:15.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:15 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:15.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:15 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:16.325 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:16 smithi067 ceph-mon[32814]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:16.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:16 smithi175 ceph-mon[37125]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:16.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:16 smithi149 ceph-mon[37173]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:18.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:18.507+0000 7f51ab7fe640 1 -- 172.21.15.67:0/384574579 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f51ac02e4e0 con 0x7f51940b3bc0 2024-07-27T03:08:18.510 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 3 on host 'smithi067' 2024-07-27T03:08:18.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:18.510+0000 7f51b4449640 1 -- 172.21.15.67:0/384574579 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f51940b3bc0 msgr2=0x7f51940b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:08:18.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:18.510+0000 7f51b4449640 1 --2- 172.21.15.67:0/384574579 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f51940b3bc0 0x7f51940b60d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f51a000bbe0 tx=0x7f51a000b390 comp rx=0 tx=0).stop 2024-07-27T03:08:18.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:18.510+0000 7f51b4449640 1 -- 172.21.15.67:0/384574579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51ac06c0d0 msgr2=0x7f51ac199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:08:18.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:18.510+0000 7f51b4449640 1 --2- 172.21.15.67:0/384574579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51ac06c0d0 0x7f51ac199a10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f51a4006010 tx=0x7f51a4004340 comp rx=0 tx=0).stop 2024-07-27T03:08:18.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:18.510+0000 7f51b4449640 1 -- 172.21.15.67:0/384574579 shutdown_connections 2024-07-27T03:08:18.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:18.510+0000 7f51b4449640 1 --2- 172.21.15.67:0/384574579 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f51940b3bc0 0x7f51940b60d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:18.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:18.510+0000 7f51b4449640 1 --2- 172.21.15.67:0/384574579 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51ac108f10 0x7f51ac19ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:18.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:18.510+0000 7f51b4449640 1 --2- 172.21.15.67:0/384574579 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51ac106370 0x7f51ac19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:18.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:18.510+0000 7f51b4449640 1 --2- 172.21.15.67:0/384574579 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f51ac06c0d0 0x7f51ac199a10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:18.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:18.510+0000 7f51b4449640 1 -- 172.21.15.67:0/384574579 >> 172.21.15.67:0/384574579 conn(0x7f51ac0fda50 msgr2=0x7f51ac1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:08:18.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:18.510+0000 7f51b4449640 1 -- 172.21.15.67:0/384574579 shutdown_connections 2024-07-27T03:08:18.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:08:18.510+0000 7f51b4449640 1 -- 172.21.15.67:0/384574579 wait complete. 2024-07-27T03:08:18.831 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:18 smithi067 ceph-mon[32814]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:18.831 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:18 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:18.831 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:18 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:18.831 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:18 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:18.832 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:18 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:08:18.832 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:18 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:18.832 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:18 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:08:18.832 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:18 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:18.832 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:18 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:19.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:18 smithi149 ceph-mon[37173]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:19.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:18 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:19.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:18 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:19.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:18 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:19.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:18 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:08:19.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:18 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:19.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:18 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:08:19.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:18 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:19.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:18 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:19.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:18 smithi175 ceph-mon[37125]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:19.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:18 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:19.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:18 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:19.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:18 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:19.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:18 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:08:19.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:18 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:19.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:18 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:08:19.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:18 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:19.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:18 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:19.153 DEBUG:teuthology.orchestra.run.smithi067:osd.3> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.3.service 2024-07-27T03:08:19.156 INFO:tasks.cephadm:Deploying osd.4 on smithi149 with /dev/vg_nvme/lv_3... 2024-07-27T03:08:19.156 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-07-27T03:08:19.421 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.b/config 2024-07-27T03:08:19.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:19 smithi067 ceph-mon[32814]: from='osd.3 [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-27T03:08:19.827 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:19 smithi149 ceph-mon[37173]: from='osd.3 [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-27T03:08:20.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:19 smithi175 ceph-mon[37125]: from='osd.3 [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-07-27T03:08:20.942 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:20 smithi149 ceph-mon[37173]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:20.943 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:20 smithi149 ceph-mon[37173]: from='osd.3 [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-27T03:08:20.943 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:20 smithi149 ceph-mon[37173]: osdmap e24: 4 total, 3 up, 4 in 2024-07-27T03:08:20.943 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:20 smithi149 ceph-mon[37173]: from='osd.3 [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-27T03:08:20.943 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:20 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-27T03:08:20.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:20 smithi067 ceph-mon[32814]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:20.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:20 smithi067 ceph-mon[32814]: from='osd.3 [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-27T03:08:20.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:20 smithi067 ceph-mon[32814]: osdmap e24: 4 total, 3 up, 4 in 2024-07-27T03:08:20.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:20 smithi067 ceph-mon[32814]: from='osd.3 [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-27T03:08:20.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:20 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-27T03:08:20.970 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 27 03:08:20 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-3[57396]: 2024-07-27T03:08:20.689+0000 7f9e6a663640 -1 osd.3 0 waiting for initial osdmap 2024-07-27T03:08:20.970 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 27 03:08:20 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-3[57396]: 2024-07-27T03:08:20.699+0000 7f9e66461640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-27T03:08:21.094 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-07-27T03:08:21.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:20 smithi175 ceph-mon[37125]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:21.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:20 smithi175 ceph-mon[37125]: from='osd.3 [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-07-27T03:08:21.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:20 smithi175 ceph-mon[37125]: osdmap e24: 4 total, 3 up, 4 in 2024-07-27T03:08:21.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:20 smithi175 ceph-mon[37125]: from='osd.3 [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-27T03:08:21.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:20 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-27T03:08:21.179 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-07-27T03:08:21.195 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-07-27T03:08:21.195 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-07-27T03:08:21.195 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133422 s, 786 MB/s 2024-07-27T03:08:21.195 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-07-27T03:08:21.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:21 smithi149 ceph-mon[37173]: from='osd.3 [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-27T03:08:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:21 smithi149 ceph-mon[37173]: osdmap e25: 4 total, 3 up, 4 in 2024-07-27T03:08:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-27T03:08:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-27T03:08:21.834 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_3 2024-07-27T03:08:21.955 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:21 smithi067 ceph-mon[32814]: from='osd.3 [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-27T03:08:21.956 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:21 smithi067 ceph-mon[32814]: osdmap e25: 4 total, 3 up, 4 in 2024-07-27T03:08:21.956 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-27T03:08:21.956 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-27T03:08:22.094 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.b/config 2024-07-27T03:08:22.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:21 smithi175 ceph-mon[37125]: from='osd.3 [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-27T03:08:22.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:21 smithi175 ceph-mon[37125]: osdmap e25: 4 total, 3 up, 4 in 2024-07-27T03:08:22.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-27T03:08:22.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-27T03:08:22.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:22 smithi149 ceph-mon[37173]: purged_snaps scrub starts 2024-07-27T03:08:22.886 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:22 smithi149 ceph-mon[37173]: purged_snaps scrub ok 2024-07-27T03:08:22.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:22 smithi149 ceph-mon[37173]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:22.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:22 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-27T03:08:22.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:22 smithi149 ceph-mon[37173]: osd.3 [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693] boot 2024-07-27T03:08:22.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:22 smithi149 ceph-mon[37173]: osdmap e26: 4 total, 4 up, 4 in 2024-07-27T03:08:22.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:22 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-27T03:08:22.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:22 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:22.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:22 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:22.887 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:22 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:08:22.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:22 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:08:22.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:22 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:22.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:22 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:22.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:22 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:08:22.888 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:22 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:22.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:22 smithi067 ceph-mon[32814]: purged_snaps scrub starts 2024-07-27T03:08:22.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:22 smithi067 ceph-mon[32814]: purged_snaps scrub ok 2024-07-27T03:08:22.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:22 smithi067 ceph-mon[32814]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:22.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:22 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-27T03:08:22.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:22 smithi067 ceph-mon[32814]: osd.3 [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693] boot 2024-07-27T03:08:22.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:22 smithi067 ceph-mon[32814]: osdmap e26: 4 total, 4 up, 4 in 2024-07-27T03:08:22.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:22 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-27T03:08:22.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:22 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:22.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:22 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:22.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:22 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:08:22.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:22 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:08:22.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:22 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:22.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:22 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:22.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:22 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:08:22.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:22 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:23.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:22 smithi175 ceph-mon[37125]: purged_snaps scrub starts 2024-07-27T03:08:23.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:22 smithi175 ceph-mon[37125]: purged_snaps scrub ok 2024-07-27T03:08:23.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:22 smithi175 ceph-mon[37125]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-07-27T03:08:23.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:22 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-27T03:08:23.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:22 smithi175 ceph-mon[37125]: osd.3 [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693] boot 2024-07-27T03:08:23.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:22 smithi175 ceph-mon[37125]: osdmap e26: 4 total, 4 up, 4 in 2024-07-27T03:08:23.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:22 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-07-27T03:08:23.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:22 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:23.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:22 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:23.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:22 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:08:23.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:22 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:08:23.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:22 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:23.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:22 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:23.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:22 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:08:23.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:22 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:23.606 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.576+0000 7f10408a7640 1 -- 172.21.15.149:0/3713928583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f103c1031c0 msgr2=0x7f103c103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:08:23.606 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.576+0000 7f10408a7640 1 --2- 172.21.15.149:0/3713928583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f103c1031c0 0x7f103c103630 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f102c009900 tx=0x7f102c02a480 comp rx=0 tx=0).stop 2024-07-27T03:08:23.606 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.576+0000 7f10408a7640 1 -- 172.21.15.149:0/3713928583 shutdown_connections 2024-07-27T03:08:23.606 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.576+0000 7f10408a7640 1 --2- 172.21.15.149:0/3713928583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f103c104bf0 0x7f103c10b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:23.606 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.576+0000 7f10408a7640 1 --2- 172.21.15.149:0/3713928583 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f103c104250 0x7f103c104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:23.607 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.576+0000 7f10408a7640 1 --2- 172.21.15.149:0/3713928583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f103c1031c0 0x7f103c103630 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:23.607 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.576+0000 7f10408a7640 1 -- 172.21.15.149:0/3713928583 >> 172.21.15.149:0/3713928583 conn(0x7f103c0feb30 msgr2=0x7f103c100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:08:23.607 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.576+0000 7f10408a7640 1 -- 172.21.15.149:0/3713928583 shutdown_connections 2024-07-27T03:08:23.607 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.576+0000 7f10408a7640 1 -- 172.21.15.149:0/3713928583 wait complete. 2024-07-27T03:08:23.607 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.577+0000 7f10408a7640 1 Processor -- start 2024-07-27T03:08:23.607 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.577+0000 7f10408a7640 1 -- start start 2024-07-27T03:08:23.607 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.577+0000 7f10408a7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f103c1031c0 0x7f103c199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:08:23.607 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.577+0000 7f10408a7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f103c104250 0x7f103c19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:08:23.607 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.577+0000 7f10408a7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f103c104bf0 0x7f103c19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:08:23.607 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.577+0000 7f10408a7640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f103c10ee60 con 0x7f103c104250 2024-07-27T03:08:23.607 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.577+0000 7f10408a7640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f103c10ece0 con 0x7f103c104bf0 2024-07-27T03:08:23.608 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.577+0000 7f10408a7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f103c10efe0 con 0x7f103c1031c0 2024-07-27T03:08:23.608 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.577+0000 7f103ad9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f103c104bf0 0x7f103c19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:08:23.608 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.577+0000 7f103ad9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f103c104bf0 0x7f103c19aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:52664/0 (socket says 172.21.15.149:52664) 2024-07-27T03:08:23.608 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.577+0000 7f103ad9d640 1 -- 172.21.15.149:0/2104390685 learned_addr learned my addr 172.21.15.149:0/2104390685 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-07-27T03:08:23.608 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.577+0000 7f103a59c640 1 --2- 172.21.15.149:0/2104390685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f103c1031c0 0x7f103c199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:08:23.608 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.577+0000 7f1033fff640 1 --2- 172.21.15.149:0/2104390685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f103c104250 0x7f103c19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:08:23.608 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.578+0000 7f103ad9d640 1 -- 172.21.15.149:0/2104390685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f103c1031c0 msgr2=0x7f103c199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:08:23.608 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.578+0000 7f103ad9d640 1 --2- 172.21.15.149:0/2104390685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f103c1031c0 0x7f103c199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:23.608 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.578+0000 7f103ad9d640 1 -- 172.21.15.149:0/2104390685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f103c104250 msgr2=0x7f103c19a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:08:23.608 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.578+0000 7f103ad9d640 1 --2- 172.21.15.149:0/2104390685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f103c104250 0x7f103c19a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:23.608 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.578+0000 7f103ad9d640 1 -- 172.21.15.149:0/2104390685 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103c19b4c0 con 0x7f103c104bf0 2024-07-27T03:08:23.608 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.578+0000 7f103ad9d640 1 --2- 172.21.15.149:0/2104390685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f103c104bf0 0x7f103c19aca0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f103400cad0 tx=0x7f103400cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:08:23.609 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.578+0000 7f10337fe640 1 -- 172.21.15.149:0/2104390685 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103400ae20 con 0x7f103c104bf0 2024-07-27T03:08:23.609 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.578+0000 7f10337fe640 1 -- 172.21.15.149:0/2104390685 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1034005020 con 0x7f103c104bf0 2024-07-27T03:08:23.609 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.578+0000 7f10408a7640 1 -- 172.21.15.149:0/2104390685 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f103c19b7b0 con 0x7f103c104bf0 2024-07-27T03:08:23.609 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.578+0000 7f10337fe640 1 -- 172.21.15.149:0/2104390685 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103400f6b0 con 0x7f103c104bf0 2024-07-27T03:08:23.609 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.578+0000 7f10408a7640 1 -- 172.21.15.149:0/2104390685 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f103c1a1eb0 con 0x7f103c104bf0 2024-07-27T03:08:23.609 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.579+0000 7f10408a7640 1 -- 172.21.15.149:0/2104390685 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f103c10a3a0 con 0x7f103c104bf0 2024-07-27T03:08:23.609 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.580+0000 7f10337fe640 1 -- 172.21.15.149:0/2104390685 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f10340051d0 con 0x7f103c104bf0 2024-07-27T03:08:23.609 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.581+0000 7f10337fe640 1 --2- 172.21.15.149:0/2104390685 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f10280b3bd0 0x7f10280b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:08:23.609 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.581+0000 7f10337fe640 1 -- 172.21.15.149:0/2104390685 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7f10340e2e20 con 0x7f103c104bf0 2024-07-27T03:08:23.609 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.581+0000 7f103a59c640 1 --2- 172.21.15.149:0/2104390685 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f10280b3bd0 0x7f10280b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:08:23.609 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.582+0000 7f103a59c640 1 --2- 172.21.15.149:0/2104390685 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f10280b3bd0 0x7f10280b60e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f102c000c00 tx=0x7f102c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:08:23.609 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.584+0000 7f10337fe640 1 -- 172.21.15.149:0/2104390685 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f1034014030 con 0x7f103c104bf0 2024-07-27T03:08:23.772 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:23.769+0000 7f10408a7640 1 -- 172.21.15.149:0/2104390685 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f103c063270 con 0x7f10280b3bd0 2024-07-27T03:08:23.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:23 smithi067 ceph-mon[32814]: Detected new or changed devices on smithi067 2024-07-27T03:08:23.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:23 smithi067 ceph-mon[32814]: Adjusting osd_memory_target on smithi067 to 8544M 2024-07-27T03:08:23.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:23 smithi067 ceph-mon[32814]: osdmap e27: 4 total, 4 up, 4 in 2024-07-27T03:08:24.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:23 smithi149 ceph-mon[37173]: Detected new or changed devices on smithi067 2024-07-27T03:08:24.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:23 smithi149 ceph-mon[37173]: Adjusting osd_memory_target on smithi067 to 8544M 2024-07-27T03:08:24.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:23 smithi149 ceph-mon[37173]: osdmap e27: 4 total, 4 up, 4 in 2024-07-27T03:08:24.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:23 smithi175 ceph-mon[37125]: Detected new or changed devices on smithi067 2024-07-27T03:08:24.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:23 smithi175 ceph-mon[37125]: Adjusting osd_memory_target on smithi067 to 8544M 2024-07-27T03:08:24.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:23 smithi175 ceph-mon[37125]: osdmap e27: 4 total, 4 up, 4 in 2024-07-27T03:08:24.820 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:24 smithi149 ceph-mon[37173]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-07-27T03:08:24.820 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:24 smithi149 ceph-mon[37173]: osdmap e28: 4 total, 4 up, 4 in 2024-07-27T03:08:24.820 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:24 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:08:24.820 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:24 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:08:24.820 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:24 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:24.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:24 smithi067 ceph-mon[32814]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-07-27T03:08:24.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:24 smithi067 ceph-mon[32814]: osdmap e28: 4 total, 4 up, 4 in 2024-07-27T03:08:24.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:24 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:08:24.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:24 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:08:24.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:24 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:25.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:24 smithi175 ceph-mon[37125]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-07-27T03:08:25.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:24 smithi175 ceph-mon[37125]: osdmap e28: 4 total, 4 up, 4 in 2024-07-27T03:08:25.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:24 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:08:25.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:24 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:08:25.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:24 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:25.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:25 smithi067 ceph-mon[32814]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:08:26.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:25 smithi149 ceph-mon[37173]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:08:26.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:25 smithi175 ceph-mon[37125]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:08:26.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:26 smithi067 ceph-mon[32814]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-07-27T03:08:26.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:26 smithi067 ceph-mon[32814]: from='client.? 172.21.15.149:0/56755257' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c317dbe5-3b49-47a6-8408-7042a3dac345"}]: dispatch 2024-07-27T03:08:26.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:26 smithi067 ceph-mon[32814]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c317dbe5-3b49-47a6-8408-7042a3dac345"}]: dispatch 2024-07-27T03:08:26.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:26 smithi067 ceph-mon[32814]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c317dbe5-3b49-47a6-8408-7042a3dac345"}]': finished 2024-07-27T03:08:26.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:26 smithi067 ceph-mon[32814]: osdmap e29: 5 total, 4 up, 5 in 2024-07-27T03:08:26.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:26 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-27T03:08:26.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:26 smithi067 ceph-mon[32814]: from='client.? 172.21.15.149:0/3442547942' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:08:27.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:26 smithi149 ceph-mon[37173]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-07-27T03:08:27.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:26 smithi149 ceph-mon[37173]: from='client.? 172.21.15.149:0/56755257' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c317dbe5-3b49-47a6-8408-7042a3dac345"}]: dispatch 2024-07-27T03:08:27.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:26 smithi149 ceph-mon[37173]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c317dbe5-3b49-47a6-8408-7042a3dac345"}]: dispatch 2024-07-27T03:08:27.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:26 smithi149 ceph-mon[37173]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c317dbe5-3b49-47a6-8408-7042a3dac345"}]': finished 2024-07-27T03:08:27.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:26 smithi149 ceph-mon[37173]: osdmap e29: 5 total, 4 up, 5 in 2024-07-27T03:08:27.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:26 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-27T03:08:27.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:26 smithi149 ceph-mon[37173]: from='client.? 172.21.15.149:0/3442547942' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:08:27.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:26 smithi175 ceph-mon[37125]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-07-27T03:08:27.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:26 smithi175 ceph-mon[37125]: from='client.? 172.21.15.149:0/56755257' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c317dbe5-3b49-47a6-8408-7042a3dac345"}]: dispatch 2024-07-27T03:08:27.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:26 smithi175 ceph-mon[37125]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c317dbe5-3b49-47a6-8408-7042a3dac345"}]: dispatch 2024-07-27T03:08:27.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:26 smithi175 ceph-mon[37125]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c317dbe5-3b49-47a6-8408-7042a3dac345"}]': finished 2024-07-27T03:08:27.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:26 smithi175 ceph-mon[37125]: osdmap e29: 5 total, 4 up, 5 in 2024-07-27T03:08:27.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:26 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-27T03:08:27.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:26 smithi175 ceph-mon[37125]: from='client.? 172.21.15.149:0/3442547942' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:08:29.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:28 smithi149 ceph-mon[37173]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-07-27T03:08:29.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:28 smithi175 ceph-mon[37125]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-07-27T03:08:29.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:28 smithi067 ceph-mon[32814]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-07-27T03:08:30.974 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:30 smithi149 ceph-mon[37173]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-07-27T03:08:31.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:30 smithi175 ceph-mon[37125]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-07-27T03:08:31.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:30 smithi067 ceph-mon[32814]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-07-27T03:08:32.984 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:32 smithi149 ceph-mon[37173]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-27T03:08:33.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:32 smithi175 ceph-mon[37125]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-27T03:08:33.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:32 smithi067 ceph-mon[32814]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-27T03:08:33.984 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:33 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-27T03:08:33.984 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:33 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:34.026 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:33 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-27T03:08:34.026 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:33 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:34.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:33 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-07-27T03:08:34.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:33 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:34.838 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:34 smithi149 ceph-mon[37173]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-07-27T03:08:34.838 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:34 smithi149 ceph-mon[37173]: Deploying daemon osd.4 on smithi149 2024-07-27T03:08:35.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:34 smithi175 ceph-mon[37125]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-07-27T03:08:35.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:34 smithi175 ceph-mon[37125]: Deploying daemon osd.4 on smithi149 2024-07-27T03:08:35.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:34 smithi067 ceph-mon[32814]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-07-27T03:08:35.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:34 smithi067 ceph-mon[32814]: Deploying daemon osd.4 on smithi149 2024-07-27T03:08:36.866 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:36 smithi149 ceph-mon[37173]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-27T03:08:37.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:36 smithi175 ceph-mon[37125]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-27T03:08:37.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:36 smithi067 ceph-mon[32814]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-27T03:08:38.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:38 smithi149 ceph-mon[37173]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-07-27T03:08:38.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:38 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:08:38.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:38 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:38.786 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:38 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:39.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:38 smithi175 ceph-mon[37125]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-07-27T03:08:39.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:08:39.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:39.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:39.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:38 smithi067 ceph-mon[32814]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-07-27T03:08:39.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:38 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:08:39.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:38 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:39.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:38 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:40.934 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:40 smithi149 ceph-mon[37173]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-07-27T03:08:41.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:40 smithi175 ceph-mon[37125]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-07-27T03:08:41.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:40 smithi067 ceph-mon[32814]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-07-27T03:08:41.677 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 4 on host 'smithi149' 2024-07-27T03:08:41.677 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:41.674+0000 7f10337fe640 1 -- 172.21.15.149:0/2104390685 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f103c063270 con 0x7f10280b3bd0 2024-07-27T03:08:41.679 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:41.677+0000 7f10408a7640 1 -- 172.21.15.149:0/2104390685 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f10280b3bd0 msgr2=0x7f10280b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:08:41.679 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:41.677+0000 7f10408a7640 1 --2- 172.21.15.149:0/2104390685 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f10280b3bd0 0x7f10280b60e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f102c000c00 tx=0x7f102c031400 comp rx=0 tx=0).stop 2024-07-27T03:08:41.679 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:41.677+0000 7f10408a7640 1 -- 172.21.15.149:0/2104390685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f103c104bf0 msgr2=0x7f103c19aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:08:41.680 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:41.677+0000 7f10408a7640 1 --2- 172.21.15.149:0/2104390685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f103c104bf0 0x7f103c19aca0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f103400cad0 tx=0x7f103400cf90 comp rx=0 tx=0).stop 2024-07-27T03:08:41.680 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:41.677+0000 7f10408a7640 1 -- 172.21.15.149:0/2104390685 shutdown_connections 2024-07-27T03:08:41.680 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:41.677+0000 7f10408a7640 1 --2- 172.21.15.149:0/2104390685 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f10280b3bd0 0x7f10280b60e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:41.680 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:41.677+0000 7f10408a7640 1 --2- 172.21.15.149:0/2104390685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f103c104bf0 0x7f103c19aca0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:41.680 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:41.677+0000 7f10408a7640 1 --2- 172.21.15.149:0/2104390685 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f103c104250 0x7f103c19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:41.680 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:41.677+0000 7f10408a7640 1 --2- 172.21.15.149:0/2104390685 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f103c1031c0 0x7f103c199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:41.680 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:41.677+0000 7f10408a7640 1 -- 172.21.15.149:0/2104390685 >> 172.21.15.149:0/2104390685 conn(0x7f103c0feb30 msgr2=0x7f103c107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:08:41.680 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:41.677+0000 7f10408a7640 1 -- 172.21.15.149:0/2104390685 shutdown_connections 2024-07-27T03:08:41.680 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:08:41.677+0000 7f10408a7640 1 -- 172.21.15.149:0/2104390685 wait complete. 2024-07-27T03:08:41.972 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:41 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:41.972 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:41 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:41.972 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:41 smithi149 ceph-mon[37173]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-07-27T03:08:41.972 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:41 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:41.972 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:41 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:08:41.972 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:41 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:41.972 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:41 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:08:41.972 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:41 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:41.972 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:41 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:42.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:41 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:42.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:41 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:42.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:41 smithi175 ceph-mon[37125]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-07-27T03:08:42.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:41 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:42.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:41 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:08:42.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:41 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:42.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:41 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:08:42.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:41 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:42.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:41 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:42.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:41 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:42.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:41 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:42.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:41 smithi067 ceph-mon[32814]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-07-27T03:08:42.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:41 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:42.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:41 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:08:42.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:41 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:42.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:41 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:08:42.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:41 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:42.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:41 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:42.315 DEBUG:teuthology.orchestra.run.smithi149:osd.4> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.4.service 2024-07-27T03:08:42.318 INFO:tasks.cephadm:Deploying osd.5 on smithi175 with /dev/vg_nvme/lv_3... 2024-07-27T03:08:42.318 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-07-27T03:08:42.573 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.c/config 2024-07-27T03:08:43.026 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:42 smithi175 ceph-mon[37125]: from='osd.4 [v2:172.21.15.149:6808/3393503532,v1:172.21.15.149:6809/3393503532]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-27T03:08:43.026 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:42 smithi175 ceph-mon[37125]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-27T03:08:43.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:42 smithi067 ceph-mon[32814]: from='osd.4 [v2:172.21.15.149:6808/3393503532,v1:172.21.15.149:6809/3393503532]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-27T03:08:43.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:42 smithi067 ceph-mon[32814]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-27T03:08:43.222 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:42 smithi149 ceph-mon[37173]: from='osd.4 [v2:172.21.15.149:6808/3393503532,v1:172.21.15.149:6809/3393503532]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-27T03:08:43.222 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:42 smithi149 ceph-mon[37173]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-07-27T03:08:43.979 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:43 smithi175 ceph-mon[37125]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-27T03:08:43.979 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:43 smithi175 ceph-mon[37125]: osdmap e30: 5 total, 4 up, 5 in 2024-07-27T03:08:43.980 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:43 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-27T03:08:43.980 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:43 smithi175 ceph-mon[37125]: from='osd.4 [v2:172.21.15.149:6808/3393503532,v1:172.21.15.149:6809/3393503532]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:08:43.980 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:43 smithi175 ceph-mon[37125]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:08:43.980 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:43 smithi175 ceph-mon[37125]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-07-27T03:08:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:43 smithi067 ceph-mon[32814]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-27T03:08:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:43 smithi067 ceph-mon[32814]: osdmap e30: 5 total, 4 up, 5 in 2024-07-27T03:08:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:43 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-27T03:08:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:43 smithi067 ceph-mon[32814]: from='osd.4 [v2:172.21.15.149:6808/3393503532,v1:172.21.15.149:6809/3393503532]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:08:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:43 smithi067 ceph-mon[32814]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:08:44.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:43 smithi067 ceph-mon[32814]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-07-27T03:08:44.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:43 smithi149 ceph-mon[37173]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-07-27T03:08:44.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:43 smithi149 ceph-mon[37173]: osdmap e30: 5 total, 4 up, 5 in 2024-07-27T03:08:44.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:43 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-27T03:08:44.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:43 smithi149 ceph-mon[37173]: from='osd.4 [v2:172.21.15.149:6808/3393503532,v1:172.21.15.149:6809/3393503532]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:08:44.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:43 smithi149 ceph-mon[37173]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:08:44.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:43 smithi149 ceph-mon[37173]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 128 MiB used, 357 GiB / 358 GiB avail 2024-07-27T03:08:44.483 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-07-27T03:08:44.584 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-07-27T03:08:44.597 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-07-27T03:08:44.597 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-07-27T03:08:44.597 INFO:teuthology.orchestra.run.smithi175.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112143 s, 935 MB/s 2024-07-27T03:08:44.597 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-07-27T03:08:44.858 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:44 smithi175 ceph-mon[37125]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-07-27T03:08:44.861 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:44 smithi175 ceph-mon[37125]: osdmap e31: 5 total, 4 up, 5 in 2024-07-27T03:08:44.862 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:44 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-27T03:08:44.862 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:44 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-27T03:08:44.872 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:44 smithi149 ceph-mon[37173]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-07-27T03:08:44.872 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:44 smithi149 ceph-mon[37173]: osdmap e31: 5 total, 4 up, 5 in 2024-07-27T03:08:44.872 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:44 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-27T03:08:44.872 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:44 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-27T03:08:44.872 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:44 smithi149 ceph-mon[37173]: from='osd.4 ' entity='osd.4' 2024-07-27T03:08:44.872 INFO:journalctl@ceph.osd.4.smithi149.stdout:Jul 27 03:08:44 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-4[48622]: 2024-07-27T03:08:44.610+0000 7fce8c149640 -1 osd.4 0 waiting for initial osdmap 2024-07-27T03:08:44.872 INFO:journalctl@ceph.osd.4.smithi149.stdout:Jul 27 03:08:44 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-4[48622]: 2024-07-27T03:08:44.616+0000 7fce83f3f640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-27T03:08:45.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:44 smithi175 ceph-mon[37125]: from='osd.4 ' entity='osd.4' 2024-07-27T03:08:45.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:44 smithi067 ceph-mon[32814]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-07-27T03:08:45.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:44 smithi067 ceph-mon[32814]: osdmap e31: 5 total, 4 up, 5 in 2024-07-27T03:08:45.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:44 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-27T03:08:45.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:44 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-27T03:08:45.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:44 smithi067 ceph-mon[32814]: from='osd.4 ' entity='osd.4' 2024-07-27T03:08:45.258 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_3 2024-07-27T03:08:45.515 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.c/config 2024-07-27T03:08:46.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:45 smithi175 ceph-mon[37125]: purged_snaps scrub starts 2024-07-27T03:08:46.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:45 smithi175 ceph-mon[37125]: purged_snaps scrub ok 2024-07-27T03:08:46.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:45 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-27T03:08:46.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:45 smithi175 ceph-mon[37125]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-07-27T03:08:46.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:45 smithi175 ceph-mon[37125]: Detected new or changed devices on smithi149 2024-07-27T03:08:46.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:45 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:46.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:45 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:46.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:45 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:08:46.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:45 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:08:46.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:45 smithi175 ceph-mon[37125]: Adjusting osd_memory_target on smithi149 to 2908M 2024-07-27T03:08:46.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:45 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:46.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:45 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:46.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:45 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:08:46.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:45 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:46.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:45 smithi175 ceph-mon[37125]: osd.4 [v2:172.21.15.149:6808/3393503532,v1:172.21.15.149:6809/3393503532] boot 2024-07-27T03:08:46.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:45 smithi175 ceph-mon[37125]: osdmap e32: 5 total, 5 up, 5 in 2024-07-27T03:08:46.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:45 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-27T03:08:46.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:45 smithi067 ceph-mon[32814]: purged_snaps scrub starts 2024-07-27T03:08:46.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:45 smithi067 ceph-mon[32814]: purged_snaps scrub ok 2024-07-27T03:08:46.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:45 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-27T03:08:46.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:45 smithi067 ceph-mon[32814]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-07-27T03:08:46.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:45 smithi067 ceph-mon[32814]: Detected new or changed devices on smithi149 2024-07-27T03:08:46.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:45 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:46.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:45 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:46.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:45 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:08:46.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:45 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:08:46.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:45 smithi067 ceph-mon[32814]: Adjusting osd_memory_target on smithi149 to 2908M 2024-07-27T03:08:46.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:45 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:46.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:45 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:46.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:45 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:08:46.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:45 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:46.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:45 smithi067 ceph-mon[32814]: osd.4 [v2:172.21.15.149:6808/3393503532,v1:172.21.15.149:6809/3393503532] boot 2024-07-27T03:08:46.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:45 smithi067 ceph-mon[32814]: osdmap e32: 5 total, 5 up, 5 in 2024-07-27T03:08:46.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:45 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-27T03:08:46.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:45 smithi149 ceph-mon[37173]: purged_snaps scrub starts 2024-07-27T03:08:46.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:45 smithi149 ceph-mon[37173]: purged_snaps scrub ok 2024-07-27T03:08:46.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:45 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-27T03:08:46.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:45 smithi149 ceph-mon[37173]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 129 MiB used, 357 GiB / 358 GiB avail 2024-07-27T03:08:46.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:45 smithi149 ceph-mon[37173]: Detected new or changed devices on smithi149 2024-07-27T03:08:46.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:45 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:46.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:45 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:46.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:45 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:08:46.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:45 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:08:46.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:45 smithi149 ceph-mon[37173]: Adjusting osd_memory_target on smithi149 to 2908M 2024-07-27T03:08:46.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:45 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:46.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:45 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:46.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:45 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:08:46.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:45 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:08:46.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:45 smithi149 ceph-mon[37173]: osd.4 [v2:172.21.15.149:6808/3393503532,v1:172.21.15.149:6809/3393503532] boot 2024-07-27T03:08:46.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:45 smithi149 ceph-mon[37173]: osdmap e32: 5 total, 5 up, 5 in 2024-07-27T03:08:46.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:45 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-07-27T03:08:47.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.070+0000 7f0ae57ab640 1 -- 172.21.15.175:0/2329239820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae01051c0 msgr2=0x7f0ae01076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:08:47.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.070+0000 7f0ae57ab640 1 --2- 172.21.15.175:0/2329239820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae01051c0 0x7f0ae01076a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8009900 tx=0x7f0ac802a410 comp rx=0 tx=0).stop 2024-07-27T03:08:47.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.070+0000 7f0ae57ab640 1 -- 172.21.15.175:0/2329239820 shutdown_connections 2024-07-27T03:08:47.079 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.070+0000 7f0ae57ab640 1 --2- 172.21.15.175:0/2329239820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ae0107de0 0x7f0ae010a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:47.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.070+0000 7f0ae57ab640 1 --2- 172.21.15.175:0/2329239820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae01051c0 0x7f0ae01076a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:47.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.070+0000 7f0ae57ab640 1 --2- 172.21.15.175:0/2329239820 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae0102760 0x7f0ae0104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:47.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.070+0000 7f0ae57ab640 1 -- 172.21.15.175:0/2329239820 >> 172.21.15.175:0/2329239820 conn(0x7f0ae00fd1e0 msgr2=0x7f0ae00ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:08:47.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.070+0000 7f0ae57ab640 1 -- 172.21.15.175:0/2329239820 shutdown_connections 2024-07-27T03:08:47.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.070+0000 7f0ae57ab640 1 -- 172.21.15.175:0/2329239820 wait complete. 2024-07-27T03:08:47.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.070+0000 7f0ae57ab640 1 Processor -- start 2024-07-27T03:08:47.080 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.071+0000 7f0ae57ab640 1 -- start start 2024-07-27T03:08:47.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.071+0000 7f0ae57ab640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ae0102760 0x7f0ae0199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:08:47.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.071+0000 7f0ae57ab640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae01051c0 0x7f0ae019a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:08:47.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.071+0000 7f0ae57ab640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae0107de0 0x7f0ae019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:08:47.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.071+0000 7f0ae57ab640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae010ee10 con 0x7f0ae01051c0 2024-07-27T03:08:47.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.071+0000 7f0ae57ab640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae010ec90 con 0x7f0ae0102760 2024-07-27T03:08:47.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.071+0000 7f0ae57ab640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae010ef90 con 0x7f0ae0107de0 2024-07-27T03:08:47.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.071+0000 7f0adf7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae0107de0 0x7f0ae019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:08:47.081 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.071+0000 7f0adf7fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae0107de0 0x7f0ae019ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:45376/0 (socket says 172.21.15.175:45376) 2024-07-27T03:08:47.082 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.071+0000 7f0adf7fe640 1 -- 172.21.15.175:0/2993131001 learned_addr learned my addr 172.21.15.175:0/2993131001 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-27T03:08:47.082 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.071+0000 7f0adeffd640 1 --2- 172.21.15.175:0/2993131001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ae0102760 0x7f0ae0199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:08:47.082 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.071+0000 7f0ade7fc640 1 --2- 172.21.15.175:0/2993131001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae01051c0 0x7f0ae019a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:08:47.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.071+0000 7f0adf7fe640 1 -- 172.21.15.175:0/2993131001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ae0102760 msgr2=0x7f0ae0199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:08:47.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.071+0000 7f0adf7fe640 1 --2- 172.21.15.175:0/2993131001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ae0102760 0x7f0ae0199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:47.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.071+0000 7f0adf7fe640 1 -- 172.21.15.175:0/2993131001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae01051c0 msgr2=0x7f0ae019a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:08:47.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.071+0000 7f0adf7fe640 1 --2- 172.21.15.175:0/2993131001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae01051c0 0x7f0ae019a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:08:47.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.071+0000 7f0adf7fe640 1 -- 172.21.15.175:0/2993131001 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae019b430 con 0x7f0ae0107de0 2024-07-27T03:08:47.083 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.071+0000 7f0ade7fc640 1 --2- 172.21.15.175:0/2993131001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae01051c0 0x7f0ae019a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:08:47.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.072+0000 7f0adf7fe640 1 --2- 172.21.15.175:0/2993131001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae0107de0 0x7f0ae019ac10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800d750 tx=0x7f0ad800dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:08:47.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.072+0000 7f0ad7fff640 1 -- 172.21.15.175:0/2993131001 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8017070 con 0x7f0ae0107de0 2024-07-27T03:08:47.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.072+0000 7f0ad7fff640 1 -- 172.21.15.175:0/2993131001 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad80054d0 con 0x7f0ae0107de0 2024-07-27T03:08:47.084 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.072+0000 7f0ad7fff640 1 -- 172.21.15.175:0/2993131001 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8004c60 con 0x7f0ae0107de0 2024-07-27T03:08:47.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.072+0000 7f0ae57ab640 1 -- 172.21.15.175:0/2993131001 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae019b720 con 0x7f0ae0107de0 2024-07-27T03:08:47.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.072+0000 7f0ae57ab640 1 -- 172.21.15.175:0/2993131001 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae01a1e20 con 0x7f0ae0107de0 2024-07-27T03:08:47.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.073+0000 7f0ae57ab640 1 -- 172.21.15.175:0/2993131001 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ae0105690 con 0x7f0ae0107de0 2024-07-27T03:08:47.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.074+0000 7f0ad7fff640 1 -- 172.21.15.175:0/2993131001 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f0ad8009bd0 con 0x7f0ae0107de0 2024-07-27T03:08:47.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.075+0000 7f0ad7fff640 1 --2- 172.21.15.175:0/2993131001 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0acc0b3bd0 0x7f0acc0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:08:47.085 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.075+0000 7f0ad7fff640 1 -- 172.21.15.175:0/2993131001 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4101+0+0 (secure 0 0 0) 0x7f0ad80e29b0 con 0x7f0ae0107de0 2024-07-27T03:08:47.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.075+0000 7f0adeffd640 1 --2- 172.21.15.175:0/2993131001 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0acc0b3bd0 0x7f0acc0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:08:47.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.076+0000 7f0adeffd640 1 --2- 172.21.15.175:0/2993131001 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0acc0b3bd0 0x7f0acc0b60e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0005fd0 tx=0x7f0ad0009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:08:47.086 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.079+0000 7f0ad7fff640 1 -- 172.21.15.175:0/2993131001 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f0ad80ab070 con 0x7f0ae0107de0 2024-07-27T03:08:47.260 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:08:47.258+0000 7f0ae57ab640 1 -- 172.21.15.175:0/2993131001 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0ae0063270 con 0x7f0acc0b3bd0 2024-07-27T03:08:47.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:47 smithi175 ceph-mon[37125]: osdmap e33: 5 total, 5 up, 5 in 2024-07-27T03:08:47.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:47 smithi175 ceph-mon[37125]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:08:47.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:47 smithi175 ceph-mon[37125]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:08:47.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:47 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:08:47.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:47 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:08:47.951 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:47 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:48.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:47 smithi067 ceph-mon[32814]: osdmap e33: 5 total, 5 up, 5 in 2024-07-27T03:08:48.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:47 smithi067 ceph-mon[32814]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:08:48.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:47 smithi067 ceph-mon[32814]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:08:48.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:47 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:08:48.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:47 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:08:48.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:47 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:48.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:47 smithi149 ceph-mon[37173]: osdmap e33: 5 total, 5 up, 5 in 2024-07-27T03:08:48.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:47 smithi149 ceph-mon[37173]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:08:48.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:47 smithi149 ceph-mon[37173]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:08:48.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:08:48.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:08:48.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:47 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:50.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:50 smithi175 ceph-mon[37125]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:08:50.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:50 smithi175 ceph-mon[37125]: from='client.? 172.21.15.175:0/4200156610' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c85abd6-2b42-4957-a9b7-59a9ee36b841"}]: dispatch 2024-07-27T03:08:50.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:50 smithi175 ceph-mon[37125]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c85abd6-2b42-4957-a9b7-59a9ee36b841"}]: dispatch 2024-07-27T03:08:50.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:50 smithi175 ceph-mon[37125]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c85abd6-2b42-4957-a9b7-59a9ee36b841"}]': finished 2024-07-27T03:08:50.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:50 smithi175 ceph-mon[37125]: osdmap e34: 6 total, 5 up, 6 in 2024-07-27T03:08:50.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:50 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-27T03:08:50.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:50 smithi067 ceph-mon[32814]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:08:50.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:50 smithi067 ceph-mon[32814]: from='client.? 172.21.15.175:0/4200156610' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c85abd6-2b42-4957-a9b7-59a9ee36b841"}]: dispatch 2024-07-27T03:08:50.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:50 smithi067 ceph-mon[32814]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c85abd6-2b42-4957-a9b7-59a9ee36b841"}]: dispatch 2024-07-27T03:08:50.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:50 smithi067 ceph-mon[32814]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c85abd6-2b42-4957-a9b7-59a9ee36b841"}]': finished 2024-07-27T03:08:50.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:50 smithi067 ceph-mon[32814]: osdmap e34: 6 total, 5 up, 6 in 2024-07-27T03:08:50.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:50 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-27T03:08:50.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:50 smithi149 ceph-mon[37173]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:08:50.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:50 smithi149 ceph-mon[37173]: from='client.? 172.21.15.175:0/4200156610' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c85abd6-2b42-4957-a9b7-59a9ee36b841"}]: dispatch 2024-07-27T03:08:50.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:50 smithi149 ceph-mon[37173]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c85abd6-2b42-4957-a9b7-59a9ee36b841"}]: dispatch 2024-07-27T03:08:50.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:50 smithi149 ceph-mon[37173]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c85abd6-2b42-4957-a9b7-59a9ee36b841"}]': finished 2024-07-27T03:08:50.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:50 smithi149 ceph-mon[37173]: osdmap e34: 6 total, 5 up, 6 in 2024-07-27T03:08:50.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:50 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-27T03:08:51.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:51 smithi175 ceph-mon[37125]: from='client.? 172.21.15.175:0/30815315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:08:51.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:51 smithi067 ceph-mon[32814]: from='client.? 172.21.15.175:0/30815315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:08:51.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:51 smithi149 ceph-mon[37173]: from='client.? 172.21.15.175:0/30815315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:08:52.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:52 smithi175 ceph-mon[37125]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:08:52.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:52 smithi067 ceph-mon[32814]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:08:52.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:52 smithi149 ceph-mon[37173]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:08:54.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:54 smithi175 ceph-mon[37125]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:08:54.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:54 smithi067 ceph-mon[32814]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:08:54.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:54 smithi149 ceph-mon[37173]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:08:56.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:56 smithi175 ceph-mon[37125]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:08:56.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:56 smithi067 ceph-mon[32814]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:08:56.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:56 smithi149 ceph-mon[37173]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:08:57.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-27T03:08:57.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:57.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:57 smithi175 ceph-mon[37125]: Deploying daemon osd.5 on smithi175 2024-07-27T03:08:57.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-27T03:08:57.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:57.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:57 smithi067 ceph-mon[32814]: Deploying daemon osd.5 on smithi175 2024-07-27T03:08:57.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-07-27T03:08:57.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:08:57.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:57 smithi149 ceph-mon[37173]: Deploying daemon osd.5 on smithi175 2024-07-27T03:08:58.365 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:08:58 smithi175 ceph-mon[37125]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:08:58.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:08:58 smithi067 ceph-mon[32814]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:08:58.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:08:58 smithi149 ceph-mon[37173]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:09:00.206 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:00 smithi175 ceph-mon[37125]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:09:00.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:00 smithi067 ceph-mon[32814]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:09:00.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:00 smithi149 ceph-mon[37173]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:09:02.328 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:02 smithi175 ceph-mon[37125]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:09:02.328 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:02 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:09:02.328 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:02 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:02.328 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:02 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:02.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:02 smithi067 ceph-mon[32814]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:09:02.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:02 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:09:02.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:02 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:02.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:02 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:02.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:02 smithi149 ceph-mon[37173]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:09:02.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:02 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:09:02.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:02 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:02.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:02 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:04.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:04 smithi175 ceph-mon[37125]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:09:04.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:04 smithi067 ceph-mon[32814]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:09:04.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:04 smithi149 ceph-mon[37173]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:09:05.217 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:05.213+0000 7f0ad7fff640 1 -- 172.21.15.175:0/2993131001 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0ae0063270 con 0x7f0acc0b3bd0 2024-07-27T03:09:05.217 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 5 on host 'smithi175' 2024-07-27T03:09:05.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:05.215+0000 7f0ae57ab640 1 -- 172.21.15.175:0/2993131001 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0acc0b3bd0 msgr2=0x7f0acc0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:09:05.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:05.215+0000 7f0ae57ab640 1 --2- 172.21.15.175:0/2993131001 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0acc0b3bd0 0x7f0acc0b60e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0005fd0 tx=0x7f0ad0009680 comp rx=0 tx=0).stop 2024-07-27T03:09:05.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:05.215+0000 7f0ae57ab640 1 -- 172.21.15.175:0/2993131001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae0107de0 msgr2=0x7f0ae019ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:09:05.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:05.215+0000 7f0ae57ab640 1 --2- 172.21.15.175:0/2993131001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae0107de0 0x7f0ae019ac10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800d750 tx=0x7f0ad800dc10 comp rx=0 tx=0).stop 2024-07-27T03:09:05.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:05.215+0000 7f0ae57ab640 1 -- 172.21.15.175:0/2993131001 shutdown_connections 2024-07-27T03:09:05.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:05.215+0000 7f0ae57ab640 1 --2- 172.21.15.175:0/2993131001 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0acc0b3bd0 0x7f0acc0b60e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:05.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:05.215+0000 7f0ae57ab640 1 --2- 172.21.15.175:0/2993131001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae0107de0 0x7f0ae019ac10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:05.218 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:05.215+0000 7f0ae57ab640 1 --2- 172.21.15.175:0/2993131001 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0ae01051c0 0x7f0ae019a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:05.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:05.215+0000 7f0ae57ab640 1 --2- 172.21.15.175:0/2993131001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ae0102760 0x7f0ae0199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:05.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:05.215+0000 7f0ae57ab640 1 -- 172.21.15.175:0/2993131001 >> 172.21.15.175:0/2993131001 conn(0x7f0ae00fd1e0 msgr2=0x7f0ae01035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:09:05.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:05.215+0000 7f0ae57ab640 1 -- 172.21.15.175:0/2993131001 shutdown_connections 2024-07-27T03:09:05.219 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:05.216+0000 7f0ae57ab640 1 -- 172.21.15.175:0/2993131001 wait complete. 2024-07-27T03:09:05.542 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:05 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:05.542 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:05 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:05.542 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:05 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:05.542 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:05 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:09:05.542 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:05 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:05.542 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:05 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:09:05.542 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:05 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:05.542 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:05 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:05.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:05 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:05.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:05 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:05.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:05 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:05.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:05 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:09:05.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:05 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:05.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:05 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:09:05.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:05 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:05.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:05 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:05.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:05 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:05.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:05 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:05.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:05 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:05.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:05 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:09:05.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:05 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:05.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:05 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:09:05.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:05 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:05.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:05 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:05.899 DEBUG:teuthology.orchestra.run.smithi175:osd.5> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.5.service 2024-07-27T03:09:05.901 INFO:tasks.cephadm:Deploying osd.6 on smithi067 with /dev/vg_nvme/lv_2... 2024-07-27T03:09:05.902 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-07-27T03:09:06.182 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:09:06.598 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:06 smithi067 ceph-mon[32814]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:09:06.598 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:06 smithi067 ceph-mon[32814]: from='osd.5 [v2:172.21.15.175:6808/2291204784,v1:172.21.15.175:6809/2291204784]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-27T03:09:06.598 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:06 smithi067 ceph-mon[32814]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-27T03:09:06.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:06 smithi175 ceph-mon[37125]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:09:06.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:06 smithi175 ceph-mon[37125]: from='osd.5 [v2:172.21.15.175:6808/2291204784,v1:172.21.15.175:6809/2291204784]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-27T03:09:06.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:06 smithi175 ceph-mon[37125]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-27T03:09:06.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:06 smithi149 ceph-mon[37173]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:09:06.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:06 smithi149 ceph-mon[37173]: from='osd.5 [v2:172.21.15.175:6808/2291204784,v1:172.21.15.175:6809/2291204784]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-27T03:09:06.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:06 smithi149 ceph-mon[37173]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-07-27T03:09:07.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:07 smithi175 ceph-mon[37125]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-27T03:09:07.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:07 smithi175 ceph-mon[37125]: from='osd.5 [v2:172.21.15.175:6808/2291204784,v1:172.21.15.175:6809/2291204784]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:09:07.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:07 smithi175 ceph-mon[37125]: osdmap e35: 6 total, 5 up, 6 in 2024-07-27T03:09:07.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:07 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-27T03:09:07.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:07 smithi175 ceph-mon[37125]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:09:07.625 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:09:07 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:09:07.385+0000 7fcb801cf640 -1 osd.5 0 waiting for initial osdmap 2024-07-27T03:09:07.625 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:09:07 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:09:07.399+0000 7fcb7bfcd640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-27T03:09:07.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:07 smithi067 ceph-mon[32814]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-27T03:09:07.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:07 smithi067 ceph-mon[32814]: from='osd.5 [v2:172.21.15.175:6808/2291204784,v1:172.21.15.175:6809/2291204784]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:09:07.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:07 smithi067 ceph-mon[32814]: osdmap e35: 6 total, 5 up, 6 in 2024-07-27T03:09:07.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:07 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-27T03:09:07.670 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:07 smithi067 ceph-mon[32814]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:09:07.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:07 smithi149 ceph-mon[37173]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-07-27T03:09:07.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:07 smithi149 ceph-mon[37173]: from='osd.5 [v2:172.21.15.175:6808/2291204784,v1:172.21.15.175:6809/2291204784]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:09:07.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:07 smithi149 ceph-mon[37173]: osdmap e35: 6 total, 5 up, 6 in 2024-07-27T03:09:07.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:07 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-27T03:09:07.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:07 smithi149 ceph-mon[37173]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:09:08.087 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-07-27T03:09:08.174 INFO:teuthology.orchestra.run.smithi067.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-07-27T03:09:08.187 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10+0 records in 2024-07-27T03:09:08.187 INFO:teuthology.orchestra.run.smithi067.stderr:10+0 records out 2024-07-27T03:09:08.188 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115796 s, 906 MB/s 2024-07-27T03:09:08.188 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping successful for: 2024-07-27T03:09:08.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:08 smithi067 ceph-mon[32814]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:09:08.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:08 smithi067 ceph-mon[32814]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-27T03:09:08.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:08 smithi067 ceph-mon[32814]: osdmap e36: 6 total, 5 up, 6 in 2024-07-27T03:09:08.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:08 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-27T03:09:08.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:08 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-27T03:09:08.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:08 smithi067 ceph-mon[32814]: osd.5 [v2:172.21.15.175:6808/2291204784,v1:172.21.15.175:6809/2291204784] boot 2024-07-27T03:09:08.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:08 smithi067 ceph-mon[32814]: osdmap e37: 6 total, 6 up, 6 in 2024-07-27T03:09:08.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:08 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-27T03:09:08.474 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:08 smithi175 ceph-mon[37125]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:09:08.474 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:08 smithi175 ceph-mon[37125]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-27T03:09:08.474 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:08 smithi175 ceph-mon[37125]: osdmap e36: 6 total, 5 up, 6 in 2024-07-27T03:09:08.474 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:08 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-27T03:09:08.474 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:08 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-27T03:09:08.474 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:08 smithi175 ceph-mon[37125]: osd.5 [v2:172.21.15.175:6808/2291204784,v1:172.21.15.175:6809/2291204784] boot 2024-07-27T03:09:08.474 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:08 smithi175 ceph-mon[37125]: osdmap e37: 6 total, 6 up, 6 in 2024-07-27T03:09:08.474 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:08 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-27T03:09:08.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:08 smithi149 ceph-mon[37173]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-07-27T03:09:08.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:08 smithi149 ceph-mon[37173]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-27T03:09:08.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:08 smithi149 ceph-mon[37173]: osdmap e36: 6 total, 5 up, 6 in 2024-07-27T03:09:08.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:08 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-27T03:09:08.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:08 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-27T03:09:08.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:08 smithi149 ceph-mon[37173]: osd.5 [v2:172.21.15.175:6808/2291204784,v1:172.21.15.175:6809/2291204784] boot 2024-07-27T03:09:08.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:08 smithi149 ceph-mon[37173]: osdmap e37: 6 total, 6 up, 6 in 2024-07-27T03:09:08.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:08 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-07-27T03:09:08.914 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph orch daemon add osd smithi067:vg_nvme/lv_2 2024-07-27T03:09:09.178 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:09:09.605 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:09 smithi067 ceph-mon[32814]: purged_snaps scrub starts 2024-07-27T03:09:09.606 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:09 smithi067 ceph-mon[32814]: purged_snaps scrub ok 2024-07-27T03:09:09.606 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:09 smithi067 ceph-mon[32814]: Detected new or changed devices on smithi175 2024-07-27T03:09:09.606 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:09.606 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:09.606 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:09.606 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:09.606 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:09 smithi067 ceph-mon[32814]: Adjusting osd_memory_target on smithi175 to 8544M 2024-07-27T03:09:09.607 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:09.607 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:09.607 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:09:09.607 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:09.607 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:09 smithi067 ceph-mon[32814]: osdmap e38: 6 total, 6 up, 6 in 2024-07-27T03:09:09.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:09 smithi175 ceph-mon[37125]: purged_snaps scrub starts 2024-07-27T03:09:09.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:09 smithi175 ceph-mon[37125]: purged_snaps scrub ok 2024-07-27T03:09:09.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:09 smithi175 ceph-mon[37125]: Detected new or changed devices on smithi175 2024-07-27T03:09:09.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:09.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:09.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:09.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:09.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:09 smithi175 ceph-mon[37125]: Adjusting osd_memory_target on smithi175 to 8544M 2024-07-27T03:09:09.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:09.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:09.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:09:09.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:09.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:09 smithi175 ceph-mon[37125]: osdmap e38: 6 total, 6 up, 6 in 2024-07-27T03:09:10.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:09 smithi149 ceph-mon[37173]: purged_snaps scrub starts 2024-07-27T03:09:10.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:09 smithi149 ceph-mon[37173]: purged_snaps scrub ok 2024-07-27T03:09:10.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:09 smithi149 ceph-mon[37173]: Detected new or changed devices on smithi175 2024-07-27T03:09:10.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:10.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:10.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:10.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:10.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:09 smithi149 ceph-mon[37173]: Adjusting osd_memory_target on smithi175 to 8544M 2024-07-27T03:09:10.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:10.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:10.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:09:10.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:10.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:09 smithi149 ceph-mon[37173]: osdmap e38: 6 total, 6 up, 6 in 2024-07-27T03:09:10.687 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:10 smithi067 ceph-mon[32814]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:10.687 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:10 smithi067 ceph-mon[32814]: osdmap e39: 6 total, 6 up, 6 in 2024-07-27T03:09:10.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:10 smithi175 ceph-mon[37125]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:10.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:10 smithi175 ceph-mon[37125]: osdmap e39: 6 total, 6 up, 6 in 2024-07-27T03:09:10.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.916+0000 7ff0f4ea2640 1 -- 172.21.15.67:0/3662933775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0f0104bf0 msgr2=0x7ff0f010b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:09:10.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.916+0000 7ff0f4ea2640 1 --2- 172.21.15.67:0/3662933775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0f0104bf0 0x7ff0f010b590 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009900 tx=0x7ff0e802a480 comp rx=0 tx=0).stop 2024-07-27T03:09:10.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.916+0000 7ff0f4ea2640 1 -- 172.21.15.67:0/3662933775 shutdown_connections 2024-07-27T03:09:10.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.916+0000 7ff0f4ea2640 1 --2- 172.21.15.67:0/3662933775 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0f0104bf0 0x7ff0f010b590 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:10.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.916+0000 7ff0f4ea2640 1 --2- 172.21.15.67:0/3662933775 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0f0104250 0x7ff0f0104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:10.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.916+0000 7ff0f4ea2640 1 --2- 172.21.15.67:0/3662933775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0f01031c0 0x7ff0f0103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:10.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.916+0000 7ff0f4ea2640 1 -- 172.21.15.67:0/3662933775 >> 172.21.15.67:0/3662933775 conn(0x7ff0f00feb30 msgr2=0x7ff0f0100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:09:10.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.916+0000 7ff0f4ea2640 1 -- 172.21.15.67:0/3662933775 shutdown_connections 2024-07-27T03:09:10.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.916+0000 7ff0f4ea2640 1 -- 172.21.15.67:0/3662933775 wait complete. 2024-07-27T03:09:10.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.917+0000 7ff0f4ea2640 1 Processor -- start 2024-07-27T03:09:10.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.917+0000 7ff0f4ea2640 1 -- start start 2024-07-27T03:09:10.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.917+0000 7ff0f4ea2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0f01031c0 0x7ff0f0199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:09:10.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.917+0000 7ff0f4ea2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0f0104250 0x7ff0f019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:09:10.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.917+0000 7ff0f4ea2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0f0104bf0 0x7ff0f019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:09:10.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.917+0000 7ff0f4ea2640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f010eee0 con 0x7ff0f0104bf0 2024-07-27T03:09:10.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.917+0000 7ff0f4ea2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f010ed60 con 0x7ff0f0104250 2024-07-27T03:09:10.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.917+0000 7ff0f4ea2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f010f060 con 0x7ff0f01031c0 2024-07-27T03:09:10.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.917+0000 7ff0eed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0f0104bf0 0x7ff0f019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:09:10.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.917+0000 7ff0eed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0f0104bf0 0x7ff0f019ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42208/0 (socket says 172.21.15.67:42208) 2024-07-27T03:09:10.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.917+0000 7ff0eed9d640 1 -- 172.21.15.67:0/1941034270 learned_addr learned my addr 172.21.15.67:0/1941034270 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:09:10.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.918+0000 7ff0ee59c640 1 --2- 172.21.15.67:0/1941034270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0f01031c0 0x7ff0f0199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:09:10.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.918+0000 7ff0edd9b640 1 --2- 172.21.15.67:0/1941034270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0f0104250 0x7ff0f019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:09:10.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.918+0000 7ff0eed9d640 1 -- 172.21.15.67:0/1941034270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0f01031c0 msgr2=0x7ff0f0199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:09:10.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.918+0000 7ff0eed9d640 1 --2- 172.21.15.67:0/1941034270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0f01031c0 0x7ff0f0199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:10.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.918+0000 7ff0eed9d640 1 -- 172.21.15.67:0/1941034270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0f0104250 msgr2=0x7ff0f019a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:09:10.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.918+0000 7ff0eed9d640 1 --2- 172.21.15.67:0/1941034270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0f0104250 0x7ff0f019a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:10.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.918+0000 7ff0eed9d640 1 -- 172.21.15.67:0/1941034270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f019b5c0 con 0x7ff0f0104bf0 2024-07-27T03:09:10.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.918+0000 7ff0ee59c640 1 --2- 172.21.15.67:0/1941034270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0f01031c0 0x7ff0f0199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:09:10.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.918+0000 7ff0eed9d640 1 --2- 172.21.15.67:0/1941034270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0f0104bf0 0x7ff0f019ada0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8005ab0 tx=0x7ff0e8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:09:10.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.918+0000 7ff0e77fe640 1 -- 172.21.15.67:0/1941034270 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e803f030 con 0x7ff0f0104bf0 2024-07-27T03:09:10.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.918+0000 7ff0f4ea2640 1 -- 172.21.15.67:0/1941034270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f019b850 con 0x7ff0f0104bf0 2024-07-27T03:09:10.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.918+0000 7ff0e77fe640 1 -- 172.21.15.67:0/1941034270 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0e8003cb0 con 0x7ff0f0104bf0 2024-07-27T03:09:10.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.918+0000 7ff0e77fe640 1 -- 172.21.15.67:0/1941034270 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8035070 con 0x7ff0f0104bf0 2024-07-27T03:09:10.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.918+0000 7ff0f4ea2640 1 -- 172.21.15.67:0/1941034270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f01a1fb0 con 0x7ff0f0104bf0 2024-07-27T03:09:10.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.919+0000 7ff0f4ea2640 1 -- 172.21.15.67:0/1941034270 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f01036b0 con 0x7ff0f0104bf0 2024-07-27T03:09:10.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.921+0000 7ff0e77fe640 1 -- 172.21.15.67:0/1941034270 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7ff0e8042030 con 0x7ff0f0104bf0 2024-07-27T03:09:10.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.922+0000 7ff0e77fe640 1 --2- 172.21.15.67:0/1941034270 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff0dc0b3c50 0x7ff0dc0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:09:10.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.922+0000 7ff0ee59c640 1 --2- 172.21.15.67:0/1941034270 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff0dc0b3c50 0x7ff0dc0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:09:10.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.922+0000 7ff0e77fe640 1 -- 172.21.15.67:0/1941034270 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4533+0+0 (secure 0 0 0) 0x7ff0e8103a60 con 0x7ff0f0104bf0 2024-07-27T03:09:10.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.922+0000 7ff0ee59c640 1 --2- 172.21.15.67:0/1941034270 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff0dc0b3c50 0x7ff0dc0b6160 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009fa0 tx=0x7ff0e0009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:09:10.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:10.925+0000 7ff0e77fe640 1 -- 172.21.15.67:0/1941034270 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7ff0e80cbf50 con 0x7ff0f0104bf0 2024-07-27T03:09:11.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:10 smithi149 ceph-mon[37173]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:11.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:10 smithi149 ceph-mon[37173]: osdmap e39: 6 total, 6 up, 6 in 2024-07-27T03:09:11.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:11.121+0000 7ff0f4ea2640 1 -- 172.21.15.67:0/1941034270 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff0f002e4e0 con 0x7ff0dc0b3c50 2024-07-27T03:09:11.866 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:11 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:09:11.866 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:11 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:09:11.866 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:11 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:11.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:11 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:09:11.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:11 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:09:11.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:11 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:12.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:11 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:09:12.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:11 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:09:12.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:11 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:12.854 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:12 smithi067 ceph-mon[32814]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:12.855 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:12 smithi067 ceph-mon[32814]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:09:12.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:12 smithi175 ceph-mon[37125]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:12.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:12 smithi175 ceph-mon[37125]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:09:13.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:12 smithi149 ceph-mon[37173]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:13.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:12 smithi149 ceph-mon[37173]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:09:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:14 smithi175 ceph-mon[37125]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-07-27T03:09:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:14 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/4057498657' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf6c277d-ec24-4470-98de-cf6f28876ccb"}]: dispatch 2024-07-27T03:09:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:14 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/4057498657' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf6c277d-ec24-4470-98de-cf6f28876ccb"}]': finished 2024-07-27T03:09:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:14 smithi175 ceph-mon[37125]: osdmap e40: 7 total, 6 up, 7 in 2024-07-27T03:09:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-27T03:09:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:14 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2047428459' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:09:14.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:14 smithi067 ceph-mon[32814]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-07-27T03:09:14.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:14 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/4057498657' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf6c277d-ec24-4470-98de-cf6f28876ccb"}]: dispatch 2024-07-27T03:09:14.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:14 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/4057498657' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf6c277d-ec24-4470-98de-cf6f28876ccb"}]': finished 2024-07-27T03:09:14.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:14 smithi067 ceph-mon[32814]: osdmap e40: 7 total, 6 up, 7 in 2024-07-27T03:09:14.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-27T03:09:14.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:14 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2047428459' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:09:15.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:14 smithi149 ceph-mon[37173]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-07-27T03:09:15.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:14 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/4057498657' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf6c277d-ec24-4470-98de-cf6f28876ccb"}]: dispatch 2024-07-27T03:09:15.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:14 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/4057498657' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf6c277d-ec24-4470-98de-cf6f28876ccb"}]': finished 2024-07-27T03:09:15.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:14 smithi149 ceph-mon[37173]: osdmap e40: 7 total, 6 up, 7 in 2024-07-27T03:09:15.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-27T03:09:15.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:14 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2047428459' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:09:16.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:16 smithi175 ceph-mon[37125]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-27T03:09:16.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:16 smithi067 ceph-mon[32814]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-27T03:09:17.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:16 smithi149 ceph-mon[37173]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-27T03:09:18.824 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:18 smithi067 ceph-mon[32814]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-07-27T03:09:18.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:18 smithi175 ceph-mon[37125]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-07-27T03:09:19.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:18 smithi149 ceph-mon[37173]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-07-27T03:09:20.830 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:20 smithi067 ceph-mon[32814]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-07-27T03:09:20.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:20 smithi175 ceph-mon[37125]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-07-27T03:09:21.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:20 smithi149 ceph-mon[37173]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-07-27T03:09:21.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-07-27T03:09:21.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:21.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-07-27T03:09:21.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:22.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-07-27T03:09:22.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:22.697 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:22 smithi067 ceph-mon[32814]: Deploying daemon osd.6 on smithi067 2024-07-27T03:09:22.698 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:22 smithi067 ceph-mon[32814]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-27T03:09:22.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:22 smithi175 ceph-mon[37125]: Deploying daemon osd.6 on smithi067 2024-07-27T03:09:22.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:22 smithi175 ceph-mon[37125]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-27T03:09:23.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:22 smithi149 ceph-mon[37173]: Deploying daemon osd.6 on smithi067 2024-07-27T03:09:23.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:22 smithi149 ceph-mon[37173]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-27T03:09:24.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:24 smithi067 ceph-mon[32814]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:24.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:24 smithi175 ceph-mon[37125]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:25.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:24 smithi149 ceph-mon[37173]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:26.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:26 smithi067 ceph-mon[32814]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:26.863 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:26 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:09:26.863 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:26 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:26.863 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:26 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:26.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:26 smithi175 ceph-mon[37125]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:26.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:26 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:09:26.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:26 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:26.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:26 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:27.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:26 smithi149 ceph-mon[37173]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:27.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:26 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:09:27.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:26 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:27.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:26 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:28.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:28 smithi067 ceph-mon[32814]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:28.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:28 smithi175 ceph-mon[37125]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:29.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:28 smithi149 ceph-mon[37173]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:30.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:30.116+0000 7ff0e77fe640 1 -- 172.21.15.67:0/1941034270 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff0f002e4e0 con 0x7ff0dc0b3c50 2024-07-27T03:09:30.120 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 6 on host 'smithi067' 2024-07-27T03:09:30.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:30.119+0000 7ff0f4ea2640 1 -- 172.21.15.67:0/1941034270 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff0dc0b3c50 msgr2=0x7ff0dc0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:09:30.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:30.119+0000 7ff0f4ea2640 1 --2- 172.21.15.67:0/1941034270 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff0dc0b3c50 0x7ff0dc0b6160 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009fa0 tx=0x7ff0e0009570 comp rx=0 tx=0).stop 2024-07-27T03:09:30.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:30.119+0000 7ff0f4ea2640 1 -- 172.21.15.67:0/1941034270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0f0104bf0 msgr2=0x7ff0f019ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:09:30.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:30.119+0000 7ff0f4ea2640 1 --2- 172.21.15.67:0/1941034270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0f0104bf0 0x7ff0f019ada0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8005ab0 tx=0x7ff0e8004360 comp rx=0 tx=0).stop 2024-07-27T03:09:30.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:30.119+0000 7ff0f4ea2640 1 -- 172.21.15.67:0/1941034270 shutdown_connections 2024-07-27T03:09:30.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:30.119+0000 7ff0f4ea2640 1 --2- 172.21.15.67:0/1941034270 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff0dc0b3c50 0x7ff0dc0b6160 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:30.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:30.119+0000 7ff0f4ea2640 1 --2- 172.21.15.67:0/1941034270 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0f0104bf0 0x7ff0f019ada0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:30.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:30.119+0000 7ff0f4ea2640 1 --2- 172.21.15.67:0/1941034270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0f0104250 0x7ff0f019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:30.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:30.119+0000 7ff0f4ea2640 1 --2- 172.21.15.67:0/1941034270 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0f01031c0 0x7ff0f0199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:30.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:30.119+0000 7ff0f4ea2640 1 -- 172.21.15.67:0/1941034270 >> 172.21.15.67:0/1941034270 conn(0x7ff0f00feb30 msgr2=0x7ff0f0107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:09:30.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:30.119+0000 7ff0f4ea2640 1 -- 172.21.15.67:0/1941034270 shutdown_connections 2024-07-27T03:09:30.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:09:30.119+0000 7ff0f4ea2640 1 -- 172.21.15.67:0/1941034270 wait complete. 2024-07-27T03:09:30.342 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:30 smithi067 ceph-mon[32814]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:30.342 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:30.343 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:30.343 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:30.343 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:09:30.343 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:30.343 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:09:30.343 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:30.343 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:30.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:30 smithi149 ceph-mon[37173]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:30.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:30.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:30.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:30.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:09:30.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:30.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:09:30.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:30.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:30.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:30 smithi175 ceph-mon[37125]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:30.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:30.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:30.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:30.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:09:30.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:30.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:09:30.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:30.627 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:30.743 DEBUG:teuthology.orchestra.run.smithi067:osd.6> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.6.service 2024-07-27T03:09:30.746 INFO:tasks.cephadm:Deploying osd.7 on smithi149 with /dev/vg_nvme/lv_2... 2024-07-27T03:09:30.746 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-07-27T03:09:31.009 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.b/config 2024-07-27T03:09:31.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:31 smithi067 ceph-mon[32814]: from='osd.6 [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-27T03:09:31.472 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:31 smithi149 ceph-mon[37173]: from='osd.6 [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-27T03:09:31.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:31 smithi175 ceph-mon[37125]: from='osd.6 [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-07-27T03:09:32.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:32 smithi067 ceph-mon[32814]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:32.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:32 smithi067 ceph-mon[32814]: from='osd.6 [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-07-27T03:09:32.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:32 smithi067 ceph-mon[32814]: osdmap e41: 7 total, 6 up, 7 in 2024-07-27T03:09:32.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:32 smithi067 ceph-mon[32814]: from='osd.6 [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-27T03:09:32.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:32 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-27T03:09:32.375 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:09:32 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:09:32.210+0000 7fb1a76dd640 -1 osd.6 0 waiting for initial osdmap 2024-07-27T03:09:32.375 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:09:32 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:09:32.221+0000 7fb1a34db640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-27T03:09:32.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:32 smithi149 ceph-mon[37173]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:32.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:32 smithi149 ceph-mon[37173]: from='osd.6 [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-07-27T03:09:32.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:32 smithi149 ceph-mon[37173]: osdmap e41: 7 total, 6 up, 7 in 2024-07-27T03:09:32.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:32 smithi149 ceph-mon[37173]: from='osd.6 [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-27T03:09:32.388 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:32 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-27T03:09:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:32 smithi175 ceph-mon[37125]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:32 smithi175 ceph-mon[37125]: from='osd.6 [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-07-27T03:09:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:32 smithi175 ceph-mon[37125]: osdmap e41: 7 total, 6 up, 7 in 2024-07-27T03:09:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:32 smithi175 ceph-mon[37125]: from='osd.6 [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-27T03:09:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:32 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-27T03:09:32.766 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-07-27T03:09:32.847 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-07-27T03:09:32.860 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-07-27T03:09:32.860 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-07-27T03:09:32.860 INFO:teuthology.orchestra.run.smithi149.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115654 s, 907 MB/s 2024-07-27T03:09:32.861 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 2024-07-27T03:09:32.861 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-07-27T03:09:33.462 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:33 smithi067 ceph-mon[32814]: from='osd.6 [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-27T03:09:33.462 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:33 smithi067 ceph-mon[32814]: osdmap e42: 7 total, 6 up, 7 in 2024-07-27T03:09:33.462 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:33 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-27T03:09:33.462 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:33 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-27T03:09:33.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:33 smithi149 ceph-mon[37173]: from='osd.6 [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-27T03:09:33.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:33 smithi149 ceph-mon[37173]: osdmap e42: 7 total, 6 up, 7 in 2024-07-27T03:09:33.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:33 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-27T03:09:33.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:33 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-27T03:09:33.553 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_2 2024-07-27T03:09:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:33 smithi175 ceph-mon[37125]: from='osd.6 [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-27T03:09:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:33 smithi175 ceph-mon[37125]: osdmap e42: 7 total, 6 up, 7 in 2024-07-27T03:09:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:33 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-27T03:09:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:33 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-27T03:09:33.818 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.b/config 2024-07-27T03:09:34.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:34 smithi067 ceph-mon[32814]: purged_snaps scrub starts 2024-07-27T03:09:34.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:34 smithi067 ceph-mon[32814]: purged_snaps scrub ok 2024-07-27T03:09:34.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:34 smithi067 ceph-mon[32814]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:34.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-27T03:09:34.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:34 smithi067 ceph-mon[32814]: osd.6 [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879] boot 2024-07-27T03:09:34.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:34 smithi067 ceph-mon[32814]: osdmap e43: 7 total, 7 up, 7 in 2024-07-27T03:09:34.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-27T03:09:34.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:34.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:34.471 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:34.471 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:34.471 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:34.471 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:34.471 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:34.471 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:09:34.471 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:34.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:34 smithi149 ceph-mon[37173]: purged_snaps scrub starts 2024-07-27T03:09:34.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:34 smithi149 ceph-mon[37173]: purged_snaps scrub ok 2024-07-27T03:09:34.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:34 smithi149 ceph-mon[37173]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:34.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-27T03:09:34.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:34 smithi149 ceph-mon[37173]: osd.6 [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879] boot 2024-07-27T03:09:34.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:34 smithi149 ceph-mon[37173]: osdmap e43: 7 total, 7 up, 7 in 2024-07-27T03:09:34.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-27T03:09:34.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:34.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:34.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:34.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:34.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:34.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:34.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:34.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:09:34.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:34.526 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:34 smithi175 ceph-mon[37125]: purged_snaps scrub starts 2024-07-27T03:09:34.526 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:34 smithi175 ceph-mon[37125]: purged_snaps scrub ok 2024-07-27T03:09:34.526 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:34 smithi175 ceph-mon[37125]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-07-27T03:09:34.526 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-27T03:09:34.526 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:34 smithi175 ceph-mon[37125]: osd.6 [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879] boot 2024-07-27T03:09:34.527 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:34 smithi175 ceph-mon[37125]: osdmap e43: 7 total, 7 up, 7 in 2024-07-27T03:09:34.527 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-07-27T03:09:34.527 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:34.527 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:34.527 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:34.527 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:34.527 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:34.527 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:34.527 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:34.527 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:09:34.527 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:35.419 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.407+0000 7f8696eb1640 1 -- 172.21.15.149:0/3021784388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8690108f10 msgr2=0x7f869010b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:09:35.449 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.407+0000 7f8696eb1640 1 --2- 172.21.15.149:0/3021784388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8690108f10 0x7f869010b420 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8688009ca0 tx=0x7f868802a8c0 comp rx=0 tx=0).stop 2024-07-27T03:09:35.449 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.407+0000 7f8696eb1640 1 -- 172.21.15.149:0/3021784388 shutdown_connections 2024-07-27T03:09:35.449 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.407+0000 7f8696eb1640 1 --2- 172.21.15.149:0/3021784388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8690108f10 0x7f869010b420 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:35.450 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.407+0000 7f8696eb1640 1 --2- 172.21.15.149:0/3021784388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8690106370 0x7f86901087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:35.450 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.407+0000 7f8696eb1640 1 --2- 172.21.15.149:0/3021784388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f869006c0d0 0x7f8690105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:35.450 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.407+0000 7f8696eb1640 1 -- 172.21.15.149:0/3021784388 >> 172.21.15.149:0/3021784388 conn(0x7f86900fda50 msgr2=0x7f86900fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:09:35.450 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.407+0000 7f8696eb1640 1 -- 172.21.15.149:0/3021784388 shutdown_connections 2024-07-27T03:09:35.450 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.407+0000 7f8696eb1640 1 -- 172.21.15.149:0/3021784388 wait complete. 2024-07-27T03:09:35.450 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.408+0000 7f8696eb1640 1 Processor -- start 2024-07-27T03:09:35.450 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.408+0000 7f8696eb1640 1 -- start start 2024-07-27T03:09:35.450 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.408+0000 7f8696eb1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f869006c0d0 0x7f86901999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:09:35.450 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.408+0000 7f8696eb1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8690106370 0x7f869019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:09:35.450 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.408+0000 7f8696eb1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8690108f10 0x7f869019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:09:35.450 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.408+0000 7f8696eb1640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f869010ee30 con 0x7f8690106370 2024-07-27T03:09:35.451 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.408+0000 7f8696eb1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f869010ecb0 con 0x7f8690108f10 2024-07-27T03:09:35.451 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.408+0000 7f8696eb1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f869010efb0 con 0x7f869006c0d0 2024-07-27T03:09:35.451 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.408+0000 7f869544e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8690108f10 0x7f869019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:09:35.451 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.408+0000 7f869544e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8690108f10 0x7f869019abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:44968/0 (socket says 172.21.15.149:44968) 2024-07-27T03:09:35.451 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.408+0000 7f869544e640 1 -- 172.21.15.149:0/1690075544 learned_addr learned my addr 172.21.15.149:0/1690075544 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-07-27T03:09:35.451 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.408+0000 7f8687fff640 1 --2- 172.21.15.149:0/1690075544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8690106370 0x7f869019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:09:35.451 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.408+0000 7f8694c4d640 1 --2- 172.21.15.149:0/1690075544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f869006c0d0 0x7f86901999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:09:35.451 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.408+0000 7f869544e640 1 -- 172.21.15.149:0/1690075544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f869006c0d0 msgr2=0x7f86901999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:09:35.451 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.408+0000 7f869544e640 1 --2- 172.21.15.149:0/1690075544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f869006c0d0 0x7f86901999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:35.452 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.408+0000 7f869544e640 1 -- 172.21.15.149:0/1690075544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8690106370 msgr2=0x7f869019a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:09:35.452 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.408+0000 7f869544e640 1 --2- 172.21.15.149:0/1690075544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8690106370 0x7f869019a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:35.452 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.408+0000 7f869544e640 1 -- 172.21.15.149:0/1690075544 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f869019b410 con 0x7f8690108f10 2024-07-27T03:09:35.452 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.408+0000 7f8694c4d640 1 --2- 172.21.15.149:0/1690075544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f869006c0d0 0x7f86901999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:09:35.452 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.408+0000 7f8687fff640 1 --2- 172.21.15.149:0/1690075544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8690106370 0x7f869019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:09:35.452 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.408+0000 7f869544e640 1 --2- 172.21.15.149:0/1690075544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8690108f10 0x7f869019abf0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f86880097b0 tx=0x7f8688003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:09:35.452 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.409+0000 7f8685ffb640 1 -- 172.21.15.149:0/1690075544 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8688004240 con 0x7f8690108f10 2024-07-27T03:09:35.452 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.409+0000 7f8696eb1640 1 -- 172.21.15.149:0/1690075544 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f869019b6a0 con 0x7f8690108f10 2024-07-27T03:09:35.452 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.409+0000 7f8685ffb640 1 -- 172.21.15.149:0/1690075544 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f868802f380 con 0x7f8690108f10 2024-07-27T03:09:35.452 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.409+0000 7f8685ffb640 1 -- 172.21.15.149:0/1690075544 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868802fc30 con 0x7f8690108f10 2024-07-27T03:09:35.452 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.409+0000 7f8696eb1640 1 -- 172.21.15.149:0/1690075544 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86901a1e00 con 0x7f8690108f10 2024-07-27T03:09:35.452 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.410+0000 7f8696eb1640 1 -- 172.21.15.149:0/1690075544 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86901067c0 con 0x7f8690108f10 2024-07-27T03:09:35.453 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.411+0000 7f8685ffb640 1 -- 172.21.15.149:0/1690075544 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f86880043e0 con 0x7f8690108f10 2024-07-27T03:09:35.453 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.412+0000 7f8685ffb640 1 --2- 172.21.15.149:0/1690075544 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f86780b3bd0 0x7f86780b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:09:35.453 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.412+0000 7f8685ffb640 1 -- 172.21.15.149:0/1690075544 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7f8688104940 con 0x7f8690108f10 2024-07-27T03:09:35.453 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.413+0000 7f8694c4d640 1 --2- 172.21.15.149:0/1690075544 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f86780b3bd0 0x7f86780b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:09:35.453 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.413+0000 7f8694c4d640 1 --2- 172.21.15.149:0/1690075544 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f86780b3bd0 0x7f86780b60e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f868c0058c0 tx=0x7f868c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:09:35.453 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.416+0000 7f8685ffb640 1 -- 172.21.15.149:0/1690075544 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f86880ccc00 con 0x7f8690108f10 2024-07-27T03:09:35.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:35 smithi149 ceph-mon[37173]: Detected new or changed devices on smithi067 2024-07-27T03:09:35.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:35 smithi149 ceph-mon[37173]: Adjusting osd_memory_target on smithi067 to 5696M 2024-07-27T03:09:35.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:35 smithi149 ceph-mon[37173]: osdmap e44: 7 total, 7 up, 7 in 2024-07-27T03:09:35.542 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:35 smithi175 ceph-mon[37125]: Detected new or changed devices on smithi067 2024-07-27T03:09:35.542 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:35 smithi175 ceph-mon[37125]: Adjusting osd_memory_target on smithi067 to 5696M 2024-07-27T03:09:35.542 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:35 smithi175 ceph-mon[37125]: osdmap e44: 7 total, 7 up, 7 in 2024-07-27T03:09:35.601 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:35.598+0000 7f8696eb1640 1 -- 172.21.15.149:0/1690075544 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8690063270 con 0x7f86780b3bd0 2024-07-27T03:09:35.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:35 smithi067 ceph-mon[32814]: Detected new or changed devices on smithi067 2024-07-27T03:09:35.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:35 smithi067 ceph-mon[32814]: Adjusting osd_memory_target on smithi067 to 5696M 2024-07-27T03:09:35.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:35 smithi067 ceph-mon[32814]: osdmap e44: 7 total, 7 up, 7 in 2024-07-27T03:09:36.335 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:36 smithi149 ceph-mon[37173]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:36.335 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:36 smithi149 ceph-mon[37173]: from='client.24401 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:09:36.335 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:36 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:09:36.335 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:36 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:09:36.335 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:36 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:36.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:36 smithi175 ceph-mon[37125]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:36.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:36 smithi175 ceph-mon[37125]: from='client.24401 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:09:36.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:36 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:09:36.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:36 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:09:36.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:36 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:36.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:36 smithi067 ceph-mon[32814]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:36.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:36 smithi067 ceph-mon[32814]: from='client.24401 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:09:36.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:36 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:09:36.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:36 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:09:36.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:36 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:38.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:38 smithi149 ceph-mon[37173]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:38.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:38 smithi149 ceph-mon[37173]: from='client.? 172.21.15.149:0/2841767224' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fea2c7cf-d5cb-45ae-9b29-f6e56c903891"}]: dispatch 2024-07-27T03:09:38.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:38 smithi149 ceph-mon[37173]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fea2c7cf-d5cb-45ae-9b29-f6e56c903891"}]: dispatch 2024-07-27T03:09:38.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:38 smithi149 ceph-mon[37173]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fea2c7cf-d5cb-45ae-9b29-f6e56c903891"}]': finished 2024-07-27T03:09:38.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:38 smithi149 ceph-mon[37173]: osdmap e45: 8 total, 7 up, 8 in 2024-07-27T03:09:38.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:38 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-27T03:09:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:38 smithi175 ceph-mon[37125]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:38 smithi175 ceph-mon[37125]: from='client.? 172.21.15.149:0/2841767224' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fea2c7cf-d5cb-45ae-9b29-f6e56c903891"}]: dispatch 2024-07-27T03:09:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:38 smithi175 ceph-mon[37125]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fea2c7cf-d5cb-45ae-9b29-f6e56c903891"}]: dispatch 2024-07-27T03:09:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:38 smithi175 ceph-mon[37125]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fea2c7cf-d5cb-45ae-9b29-f6e56c903891"}]': finished 2024-07-27T03:09:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:38 smithi175 ceph-mon[37125]: osdmap e45: 8 total, 7 up, 8 in 2024-07-27T03:09:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-27T03:09:38.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:38 smithi067 ceph-mon[32814]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 213 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:38.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:38 smithi067 ceph-mon[32814]: from='client.? 172.21.15.149:0/2841767224' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fea2c7cf-d5cb-45ae-9b29-f6e56c903891"}]: dispatch 2024-07-27T03:09:38.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:38 smithi067 ceph-mon[32814]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fea2c7cf-d5cb-45ae-9b29-f6e56c903891"}]: dispatch 2024-07-27T03:09:38.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:38 smithi067 ceph-mon[32814]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fea2c7cf-d5cb-45ae-9b29-f6e56c903891"}]': finished 2024-07-27T03:09:38.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:38 smithi067 ceph-mon[32814]: osdmap e45: 8 total, 7 up, 8 in 2024-07-27T03:09:38.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:38 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-27T03:09:39.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:39 smithi149 ceph-mon[37173]: from='client.? 172.21.15.149:0/731147169' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:09:39.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:39 smithi175 ceph-mon[37125]: from='client.? 172.21.15.149:0/731147169' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:09:39.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:39 smithi067 ceph-mon[32814]: from='client.? 172.21.15.149:0/731147169' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:09:40.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:40 smithi149 ceph-mon[37173]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:40.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:40 smithi175 ceph-mon[37125]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:40.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:40 smithi067 ceph-mon[32814]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:42.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:42 smithi149 ceph-mon[37173]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:42.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:42 smithi175 ceph-mon[37125]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:42.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:42 smithi067 ceph-mon[32814]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:44.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:44 smithi149 ceph-mon[37173]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:44.522 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:44 smithi067 ceph-mon[32814]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:44.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:44 smithi175 ceph-mon[37125]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:45.582 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:45 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-07-27T03:09:45.582 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:45 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:45.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:45 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-07-27T03:09:45.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:45 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:45.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:45 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-07-27T03:09:45.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:45 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:46.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:46 smithi149 ceph-mon[37173]: Deploying daemon osd.7 on smithi149 2024-07-27T03:09:46.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:46 smithi149 ceph-mon[37173]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:46.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:46 smithi175 ceph-mon[37125]: Deploying daemon osd.7 on smithi149 2024-07-27T03:09:46.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:46 smithi175 ceph-mon[37125]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:46.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:46 smithi067 ceph-mon[32814]: Deploying daemon osd.7 on smithi149 2024-07-27T03:09:46.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:46 smithi067 ceph-mon[32814]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:48.581 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:48 smithi149 ceph-mon[37173]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:48.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:48 smithi175 ceph-mon[37125]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:48.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:48 smithi067 ceph-mon[32814]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:50.578 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:50 smithi149 ceph-mon[37173]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:50.578 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:50 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:09:50.578 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:50 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:50.579 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:50 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:50.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:50 smithi175 ceph-mon[37125]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:50.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:50 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:09:50.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:50 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:50.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:50 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:50.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:50 smithi067 ceph-mon[32814]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:50.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:50 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:09:50.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:50 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:50.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:50 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:52.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:52 smithi149 ceph-mon[37173]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:52.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:52 smithi175 ceph-mon[37125]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:52.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:52 smithi067 ceph-mon[32814]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:53.706 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:53 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:53.706 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:53 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:53.707 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:53 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:53.707 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:53 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:09:53.707 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:53 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:53.749 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:53.747+0000 7f8685ffb640 1 -- 172.21.15.149:0/1690075544 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8690063270 con 0x7f86780b3bd0 2024-07-27T03:09:53.749 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 7 on host 'smithi149' 2024-07-27T03:09:53.752 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:53.749+0000 7f8696eb1640 1 -- 172.21.15.149:0/1690075544 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f86780b3bd0 msgr2=0x7f86780b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:09:53.752 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:53.749+0000 7f8696eb1640 1 --2- 172.21.15.149:0/1690075544 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f86780b3bd0 0x7f86780b60e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f868c0058c0 tx=0x7f868c005850 comp rx=0 tx=0).stop 2024-07-27T03:09:53.752 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:53.749+0000 7f8696eb1640 1 -- 172.21.15.149:0/1690075544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8690108f10 msgr2=0x7f869019abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:09:53.753 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:53.749+0000 7f8696eb1640 1 --2- 172.21.15.149:0/1690075544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8690108f10 0x7f869019abf0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f86880097b0 tx=0x7f8688003c20 comp rx=0 tx=0).stop 2024-07-27T03:09:53.753 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:53.750+0000 7f8696eb1640 1 -- 172.21.15.149:0/1690075544 shutdown_connections 2024-07-27T03:09:53.753 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:53.750+0000 7f8696eb1640 1 --2- 172.21.15.149:0/1690075544 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f86780b3bd0 0x7f86780b60e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:53.753 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:53.750+0000 7f8696eb1640 1 --2- 172.21.15.149:0/1690075544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8690108f10 0x7f869019abf0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:53.753 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:53.750+0000 7f8696eb1640 1 --2- 172.21.15.149:0/1690075544 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8690106370 0x7f869019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:53.753 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:53.750+0000 7f8696eb1640 1 --2- 172.21.15.149:0/1690075544 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f869006c0d0 0x7f86901999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:53.753 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:53.750+0000 7f8696eb1640 1 -- 172.21.15.149:0/1690075544 >> 172.21.15.149:0/1690075544 conn(0x7f86900fda50 msgr2=0x7f8690104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:09:53.753 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:53.750+0000 7f8696eb1640 1 -- 172.21.15.149:0/1690075544 shutdown_connections 2024-07-27T03:09:53.753 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:09:53.750+0000 7f8696eb1640 1 -- 172.21.15.149:0/1690075544 wait complete. 2024-07-27T03:09:53.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:53 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:53.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:53 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:53.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:53 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:53.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:53 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:09:53.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:53 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:53.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:53 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:53.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:53 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:53.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:53 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:53.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:53 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:09:53.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:53 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:54.474 DEBUG:teuthology.orchestra.run.smithi149:osd.7> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.7.service 2024-07-27T03:09:54.476 INFO:tasks.cephadm:Deploying osd.8 on smithi175 with /dev/vg_nvme/lv_2... 2024-07-27T03:09:54.477 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-07-27T03:09:54.711 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:54 smithi149 ceph-mon[37173]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:54.711 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:09:54.711 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:54.711 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:54 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:54.711 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:54 smithi149 ceph-mon[37173]: from='osd.7 [v2:172.21.15.149:6816/2289624404,v1:172.21.15.149:6817/2289624404]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-27T03:09:54.711 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:54 smithi149 ceph-mon[37173]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-27T03:09:54.736 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.c/config 2024-07-27T03:09:54.858 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:54 smithi175 ceph-mon[37125]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:54.858 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:09:54.858 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:54.858 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:54 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:54.858 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:54 smithi175 ceph-mon[37125]: from='osd.7 [v2:172.21.15.149:6816/2289624404,v1:172.21.15.149:6817/2289624404]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-27T03:09:54.858 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:54 smithi175 ceph-mon[37125]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-27T03:09:54.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:54 smithi067 ceph-mon[32814]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:54.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:09:54.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:54.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:54 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:54.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:54 smithi067 ceph-mon[32814]: from='osd.7 [v2:172.21.15.149:6816/2289624404,v1:172.21.15.149:6817/2289624404]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-27T03:09:54.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:54 smithi067 ceph-mon[32814]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-07-27T03:09:56.069 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:55 smithi175 ceph-mon[37125]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-07-27T03:09:56.069 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:55 smithi175 ceph-mon[37125]: osdmap e46: 8 total, 7 up, 8 in 2024-07-27T03:09:56.069 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:55 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-27T03:09:56.069 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:55 smithi175 ceph-mon[37125]: from='osd.7 [v2:172.21.15.149:6816/2289624404,v1:172.21.15.149:6817/2289624404]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:09:56.069 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:55 smithi175 ceph-mon[37125]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:09:56.069 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:55 smithi175 ceph-mon[37125]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:56.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:55 smithi067 ceph-mon[32814]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-07-27T03:09:56.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:55 smithi067 ceph-mon[32814]: osdmap e46: 8 total, 7 up, 8 in 2024-07-27T03:09:56.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-27T03:09:56.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:55 smithi067 ceph-mon[32814]: from='osd.7 [v2:172.21.15.149:6816/2289624404,v1:172.21.15.149:6817/2289624404]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:09:56.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:55 smithi067 ceph-mon[32814]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:09:56.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:55 smithi067 ceph-mon[32814]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:56.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:55 smithi149 ceph-mon[37173]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-07-27T03:09:56.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:55 smithi149 ceph-mon[37173]: osdmap e46: 8 total, 7 up, 8 in 2024-07-27T03:09:56.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:55 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-27T03:09:56.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:55 smithi149 ceph-mon[37173]: from='osd.7 [v2:172.21.15.149:6816/2289624404,v1:172.21.15.149:6817/2289624404]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:09:56.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:55 smithi149 ceph-mon[37173]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:09:56.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:55 smithi149 ceph-mon[37173]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-07-27T03:09:56.252 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:09:55 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:09:55.959+0000 7fe7cf092640 -1 osd.7 0 waiting for initial osdmap 2024-07-27T03:09:56.252 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:09:55 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:09:55.968+0000 7fe7cae90640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-27T03:09:56.440 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-07-27T03:09:56.542 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-07-27T03:09:56.555 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-07-27T03:09:56.555 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-07-27T03:09:56.555 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113236 s, 926 MB/s 2024-07-27T03:09:56.555 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-07-27T03:09:57.134 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:56 smithi149 ceph-mon[37173]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-07-27T03:09:57.135 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:56 smithi149 ceph-mon[37173]: osdmap e47: 8 total, 7 up, 8 in 2024-07-27T03:09:57.135 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:56 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-27T03:09:57.135 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:56 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-27T03:09:57.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:56 smithi175 ceph-mon[37125]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-07-27T03:09:57.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:56 smithi175 ceph-mon[37125]: osdmap e47: 8 total, 7 up, 8 in 2024-07-27T03:09:57.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:56 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-27T03:09:57.208 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:56 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-27T03:09:57.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:56 smithi067 ceph-mon[32814]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-07-27T03:09:57.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:56 smithi067 ceph-mon[32814]: osdmap e47: 8 total, 7 up, 8 in 2024-07-27T03:09:57.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:56 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-27T03:09:57.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:56 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-27T03:09:57.236 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_2 2024-07-27T03:09:57.497 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.c/config 2024-07-27T03:09:58.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:57 smithi175 ceph-mon[37125]: purged_snaps scrub starts 2024-07-27T03:09:58.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:57 smithi175 ceph-mon[37125]: purged_snaps scrub ok 2024-07-27T03:09:58.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:57 smithi175 ceph-mon[37125]: osd.7 [v2:172.21.15.149:6816/2289624404,v1:172.21.15.149:6817/2289624404] boot 2024-07-27T03:09:58.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:57 smithi175 ceph-mon[37125]: osdmap e48: 8 total, 8 up, 8 in 2024-07-27T03:09:58.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-27T03:09:58.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:57 smithi175 ceph-mon[37125]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-07-27T03:09:58.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:57 smithi175 ceph-mon[37125]: Detected new or changed devices on smithi149 2024-07-27T03:09:58.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:58.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:58.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:58.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:58.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:58.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:57 smithi175 ceph-mon[37125]: Adjusting osd_memory_target on smithi149 to 1938M 2024-07-27T03:09:58.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:58.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:58.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:09:58.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:58.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:57 smithi175 ceph-mon[37125]: osdmap e49: 8 total, 8 up, 8 in 2024-07-27T03:09:58.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:57 smithi067 ceph-mon[32814]: purged_snaps scrub starts 2024-07-27T03:09:58.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:57 smithi067 ceph-mon[32814]: purged_snaps scrub ok 2024-07-27T03:09:58.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:57 smithi067 ceph-mon[32814]: osd.7 [v2:172.21.15.149:6816/2289624404,v1:172.21.15.149:6817/2289624404] boot 2024-07-27T03:09:58.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:57 smithi067 ceph-mon[32814]: osdmap e48: 8 total, 8 up, 8 in 2024-07-27T03:09:58.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-27T03:09:58.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:57 smithi067 ceph-mon[32814]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-07-27T03:09:58.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:57 smithi067 ceph-mon[32814]: Detected new or changed devices on smithi149 2024-07-27T03:09:58.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:58.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:58.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:58.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:58.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:58.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:57 smithi067 ceph-mon[32814]: Adjusting osd_memory_target on smithi149 to 1938M 2024-07-27T03:09:58.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:58.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:58.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:09:58.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:58.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:57 smithi067 ceph-mon[32814]: osdmap e49: 8 total, 8 up, 8 in 2024-07-27T03:09:58.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:57 smithi149 ceph-mon[37173]: purged_snaps scrub starts 2024-07-27T03:09:58.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:57 smithi149 ceph-mon[37173]: purged_snaps scrub ok 2024-07-27T03:09:58.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:57 smithi149 ceph-mon[37173]: osd.7 [v2:172.21.15.149:6816/2289624404,v1:172.21.15.149:6817/2289624404] boot 2024-07-27T03:09:58.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:57 smithi149 ceph-mon[37173]: osdmap e48: 8 total, 8 up, 8 in 2024-07-27T03:09:58.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-07-27T03:09:58.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:57 smithi149 ceph-mon[37173]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-07-27T03:09:58.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:57 smithi149 ceph-mon[37173]: Detected new or changed devices on smithi149 2024-07-27T03:09:58.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:58.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:58.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:58.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:58.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:09:58.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:57 smithi149 ceph-mon[37173]: Adjusting osd_memory_target on smithi149 to 1938M 2024-07-27T03:09:58.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:58.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:09:58.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:09:58.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:09:58.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:57 smithi149 ceph-mon[37173]: osdmap e49: 8 total, 8 up, 8 in 2024-07-27T03:09:59.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.090+0000 7fea50d06640 1 -- 172.21.15.175:0/3936030805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea4c106350 msgr2=0x7fea4c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:09:59.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.090+0000 7fea50d06640 1 --2- 172.21.15.175:0/3936030805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea4c106350 0x7fea4c1087b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fea40009900 tx=0x7fea4002a410 comp rx=0 tx=0).stop 2024-07-27T03:09:59.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.090+0000 7fea50d06640 1 -- 172.21.15.175:0/3936030805 shutdown_connections 2024-07-27T03:09:59.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.090+0000 7fea50d06640 1 --2- 172.21.15.175:0/3936030805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea4c108ef0 0x7fea4c10b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:59.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.090+0000 7fea50d06640 1 --2- 172.21.15.175:0/3936030805 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea4c106350 0x7fea4c1087b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:59.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.090+0000 7fea50d06640 1 --2- 172.21.15.175:0/3936030805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea4c06c0d0 0x7fea4c105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:59.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.090+0000 7fea50d06640 1 -- 172.21.15.175:0/3936030805 >> 172.21.15.175:0/3936030805 conn(0x7fea4c0fda50 msgr2=0x7fea4c0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:09:59.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.090+0000 7fea50d06640 1 -- 172.21.15.175:0/3936030805 shutdown_connections 2024-07-27T03:09:59.093 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.090+0000 7fea50d06640 1 -- 172.21.15.175:0/3936030805 wait complete. 2024-07-27T03:09:59.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.091+0000 7fea50d06640 1 Processor -- start 2024-07-27T03:09:59.095 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.091+0000 7fea50d06640 1 -- start start 2024-07-27T03:09:59.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.091+0000 7fea50d06640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea4c06c0d0 0x7fea4c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:09:59.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.091+0000 7fea50d06640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea4c106350 0x7fea4c19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:09:59.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.091+0000 7fea50d06640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea4c108ef0 0x7fea4c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:09:59.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.091+0000 7fea50d06640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c10ee10 con 0x7fea4c108ef0 2024-07-27T03:09:59.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.091+0000 7fea50d06640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c10ec90 con 0x7fea4c106350 2024-07-27T03:09:59.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.091+0000 7fea50d06640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c10ef90 con 0x7fea4c06c0d0 2024-07-27T03:09:59.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.091+0000 7fea4a59c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea4c06c0d0 0x7fea4c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:09:59.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.091+0000 7fea4a59c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea4c06c0d0 0x7fea4c199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:55854/0 (socket says 172.21.15.175:55854) 2024-07-27T03:09:59.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.091+0000 7fea4a59c640 1 -- 172.21.15.175:0/2679031308 learned_addr learned my addr 172.21.15.175:0/2679031308 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-27T03:09:59.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.091+0000 7fea4ad9d640 1 --2- 172.21.15.175:0/2679031308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea4c108ef0 0x7fea4c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:09:59.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.091+0000 7fea49d9b640 1 --2- 172.21.15.175:0/2679031308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea4c106350 0x7fea4c19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:09:59.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.091+0000 7fea4a59c640 1 -- 172.21.15.175:0/2679031308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea4c106350 msgr2=0x7fea4c19a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:09:59.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.091+0000 7fea4a59c640 1 --2- 172.21.15.175:0/2679031308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea4c106350 0x7fea4c19a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:59.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.091+0000 7fea4a59c640 1 -- 172.21.15.175:0/2679031308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea4c108ef0 msgr2=0x7fea4c19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:09:59.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.091+0000 7fea4a59c640 1 --2- 172.21.15.175:0/2679031308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea4c108ef0 0x7fea4c19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:09:59.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.091+0000 7fea4a59c640 1 -- 172.21.15.175:0/2679031308 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4c19b3c0 con 0x7fea4c06c0d0 2024-07-27T03:09:59.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.092+0000 7fea4a59c640 1 --2- 172.21.15.175:0/2679031308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea4c06c0d0 0x7fea4c199a20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00c920 tx=0x7fea3c00cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:09:59.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.092+0000 7fea3b7fe640 1 -- 172.21.15.175:0/2679031308 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c007c70 con 0x7fea4c06c0d0 2024-07-27T03:09:59.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.092+0000 7fea50d06640 1 -- 172.21.15.175:0/2679031308 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4c19b6b0 con 0x7fea4c06c0d0 2024-07-27T03:09:59.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.092+0000 7fea3b7fe640 1 -- 172.21.15.175:0/2679031308 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea3c007e10 con 0x7fea4c06c0d0 2024-07-27T03:09:59.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.092+0000 7fea3b7fe640 1 -- 172.21.15.175:0/2679031308 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c018820 con 0x7fea4c06c0d0 2024-07-27T03:09:59.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.092+0000 7fea50d06640 1 -- 172.21.15.175:0/2679031308 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea4c1a1e40 con 0x7fea4c06c0d0 2024-07-27T03:09:59.097 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.093+0000 7fea50d06640 1 -- 172.21.15.175:0/2679031308 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea4c1067a0 con 0x7fea4c06c0d0 2024-07-27T03:09:59.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.094+0000 7fea3b7fe640 1 -- 172.21.15.175:0/2679031308 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7fea3c00f450 con 0x7fea4c06c0d0 2024-07-27T03:09:59.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.095+0000 7fea3b7fe640 1 --2- 172.21.15.175:0/2679031308 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fea340b3bd0 0x7fea340b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:09:59.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.095+0000 7fea3b7fe640 1 -- 172.21.15.175:0/2679031308 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5397+0+0 (secure 0 0 0) 0x7fea3c0e33c0 con 0x7fea4c06c0d0 2024-07-27T03:09:59.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.095+0000 7fea49d9b640 1 --2- 172.21.15.175:0/2679031308 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fea340b3bd0 0x7fea340b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:09:59.098 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.095+0000 7fea49d9b640 1 --2- 172.21.15.175:0/2679031308 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fea340b3bd0 0x7fea340b60e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fea40000c00 tx=0x7fea40031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:09:59.100 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.098+0000 7fea3b7fe640 1 -- 172.21.15.175:0/2679031308 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fea3c014030 con 0x7fea4c06c0d0 2024-07-27T03:09:59.281 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:09:59.279+0000 7fea50d06640 1 -- 172.21.15.175:0/2679031308 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fea4c063270 con 0x7fea340b3bd0 2024-07-27T03:10:00.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:59 smithi067 ceph-mon[32814]: osdmap e50: 8 total, 8 up, 8 in 2024-07-27T03:10:00.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:59 smithi067 ceph-mon[32814]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-07-27T03:10:00.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:59 smithi067 ceph-mon[32814]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:10:00.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:10:00.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:10:00.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:09:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:00.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:59 smithi149 ceph-mon[37173]: osdmap e50: 8 total, 8 up, 8 in 2024-07-27T03:10:00.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:59 smithi149 ceph-mon[37173]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-07-27T03:10:00.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:59 smithi149 ceph-mon[37173]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:10:00.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:10:00.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:10:00.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:09:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:00.300 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:59 smithi175 ceph-mon[37125]: osdmap e50: 8 total, 8 up, 8 in 2024-07-27T03:10:00.300 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:59 smithi175 ceph-mon[37125]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-07-27T03:10:00.300 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:59 smithi175 ceph-mon[37125]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:10:00.300 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:10:00.300 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:10:00.300 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:09:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:00.987 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:00 smithi175 ceph-mon[37125]: overall HEALTH_OK 2024-07-27T03:10:01.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:00 smithi067 ceph-mon[32814]: overall HEALTH_OK 2024-07-27T03:10:01.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:00 smithi149 ceph-mon[37173]: overall HEALTH_OK 2024-07-27T03:10:02.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:01 smithi067 ceph-mon[32814]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-07-27T03:10:02.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:01 smithi067 ceph-mon[32814]: from='client.? 172.21.15.175:0/1665224791' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d04d96-579a-4dc9-a4d4-50e4850b48e9"}]: dispatch 2024-07-27T03:10:02.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:01 smithi067 ceph-mon[32814]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d04d96-579a-4dc9-a4d4-50e4850b48e9"}]: dispatch 2024-07-27T03:10:02.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:01 smithi067 ceph-mon[32814]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14d04d96-579a-4dc9-a4d4-50e4850b48e9"}]': finished 2024-07-27T03:10:02.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:01 smithi067 ceph-mon[32814]: osdmap e51: 9 total, 8 up, 9 in 2024-07-27T03:10:02.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:01 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-27T03:10:02.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:01 smithi149 ceph-mon[37173]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-07-27T03:10:02.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:01 smithi149 ceph-mon[37173]: from='client.? 172.21.15.175:0/1665224791' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d04d96-579a-4dc9-a4d4-50e4850b48e9"}]: dispatch 2024-07-27T03:10:02.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:01 smithi149 ceph-mon[37173]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d04d96-579a-4dc9-a4d4-50e4850b48e9"}]: dispatch 2024-07-27T03:10:02.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:01 smithi149 ceph-mon[37173]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14d04d96-579a-4dc9-a4d4-50e4850b48e9"}]': finished 2024-07-27T03:10:02.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:01 smithi149 ceph-mon[37173]: osdmap e51: 9 total, 8 up, 9 in 2024-07-27T03:10:02.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:01 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-27T03:10:02.274 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:01 smithi175 ceph-mon[37125]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-07-27T03:10:02.274 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:01 smithi175 ceph-mon[37125]: from='client.? 172.21.15.175:0/1665224791' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d04d96-579a-4dc9-a4d4-50e4850b48e9"}]: dispatch 2024-07-27T03:10:02.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:01 smithi175 ceph-mon[37125]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14d04d96-579a-4dc9-a4d4-50e4850b48e9"}]: dispatch 2024-07-27T03:10:02.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:01 smithi175 ceph-mon[37125]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14d04d96-579a-4dc9-a4d4-50e4850b48e9"}]': finished 2024-07-27T03:10:02.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:01 smithi175 ceph-mon[37125]: osdmap e51: 9 total, 8 up, 9 in 2024-07-27T03:10:02.275 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:01 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-27T03:10:03.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:02 smithi067 ceph-mon[32814]: from='client.? 172.21.15.175:0/262014284' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:10:03.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:02 smithi149 ceph-mon[37173]: from='client.? 172.21.15.175:0/262014284' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:10:03.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:02 smithi175 ceph-mon[37125]: from='client.? 172.21.15.175:0/262014284' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:10:04.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:03 smithi149 ceph-mon[37173]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-27T03:10:04.276 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:03 smithi175 ceph-mon[37125]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-27T03:10:04.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:03 smithi067 ceph-mon[32814]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-27T03:10:06.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:06 smithi175 ceph-mon[37125]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-07-27T03:10:06.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:06 smithi067 ceph-mon[32814]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-07-27T03:10:06.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:06 smithi149 ceph-mon[37173]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-07-27T03:10:08.297 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:08 smithi175 ceph-mon[37125]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-27T03:10:08.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:08 smithi067 ceph-mon[32814]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-27T03:10:08.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:08 smithi149 ceph-mon[37173]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-27T03:10:09.350 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-07-27T03:10:09.350 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:09.350 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:09 smithi175 ceph-mon[37125]: Deploying daemon osd.8 on smithi175 2024-07-27T03:10:09.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-07-27T03:10:09.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:09.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:09 smithi067 ceph-mon[32814]: Deploying daemon osd.8 on smithi175 2024-07-27T03:10:09.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-07-27T03:10:09.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:09.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:09 smithi149 ceph-mon[37173]: Deploying daemon osd.8 on smithi175 2024-07-27T03:10:10.307 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:10 smithi175 ceph-mon[37125]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-27T03:10:10.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:10 smithi067 ceph-mon[32814]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-27T03:10:10.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:10 smithi149 ceph-mon[37173]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-27T03:10:12.106 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:12 smithi175 ceph-mon[37125]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-27T03:10:12.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:12 smithi067 ceph-mon[32814]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-27T03:10:12.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:12 smithi149 ceph-mon[37173]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-27T03:10:14.349 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:14 smithi175 ceph-mon[37125]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-07-27T03:10:14.350 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:10:14.350 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:14.350 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:14.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:14 smithi067 ceph-mon[32814]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-07-27T03:10:14.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:10:14.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:14.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:14.502 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:14 smithi149 ceph-mon[37173]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-07-27T03:10:14.502 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:10:14.502 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:14.502 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:16.326 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:16 smithi175 ceph-mon[37125]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-07-27T03:10:16.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:16 smithi067 ceph-mon[32814]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-07-27T03:10:16.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:16 smithi149 ceph-mon[37173]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-07-27T03:10:17.243 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 8 on host 'smithi175' 2024-07-27T03:10:17.244 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:10:17.239+0000 7fea3b7fe640 1 -- 172.21.15.175:0/2679031308 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fea4c063270 con 0x7fea340b3bd0 2024-07-27T03:10:17.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:10:17.242+0000 7fea50d06640 1 -- 172.21.15.175:0/2679031308 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fea340b3bd0 msgr2=0x7fea340b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:10:17.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:10:17.242+0000 7fea50d06640 1 --2- 172.21.15.175:0/2679031308 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fea340b3bd0 0x7fea340b60e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fea40000c00 tx=0x7fea40031400 comp rx=0 tx=0).stop 2024-07-27T03:10:17.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:10:17.242+0000 7fea50d06640 1 -- 172.21.15.175:0/2679031308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea4c06c0d0 msgr2=0x7fea4c199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:10:17.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:10:17.242+0000 7fea50d06640 1 --2- 172.21.15.175:0/2679031308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea4c06c0d0 0x7fea4c199a20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00c920 tx=0x7fea3c00cde0 comp rx=0 tx=0).stop 2024-07-27T03:10:17.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:10:17.242+0000 7fea50d06640 1 -- 172.21.15.175:0/2679031308 shutdown_connections 2024-07-27T03:10:17.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:10:17.242+0000 7fea50d06640 1 --2- 172.21.15.175:0/2679031308 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fea340b3bd0 0x7fea340b60e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:10:17.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:10:17.242+0000 7fea50d06640 1 --2- 172.21.15.175:0/2679031308 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea4c108ef0 0x7fea4c19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:10:17.245 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:10:17.242+0000 7fea50d06640 1 --2- 172.21.15.175:0/2679031308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea4c106350 0x7fea4c19a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:10:17.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:10:17.242+0000 7fea50d06640 1 --2- 172.21.15.175:0/2679031308 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea4c06c0d0 0x7fea4c199a20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:10:17.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:10:17.242+0000 7fea50d06640 1 -- 172.21.15.175:0/2679031308 >> 172.21.15.175:0/2679031308 conn(0x7fea4c0fda50 msgr2=0x7fea4c1048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:10:17.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:10:17.242+0000 7fea50d06640 1 -- 172.21.15.175:0/2679031308 shutdown_connections 2024-07-27T03:10:17.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:10:17.242+0000 7fea50d06640 1 -- 172.21.15.175:0/2679031308 wait complete. 2024-07-27T03:10:17.930 DEBUG:teuthology.orchestra.run.smithi175:osd.8> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.8.service 2024-07-27T03:10:17.933 INFO:tasks.cephadm:Deploying osd.9 on smithi067 with /dev/vg_nvme/lv_1... 2024-07-27T03:10:17.933 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-07-27T03:10:18.215 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:10:18.498 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:18 smithi067 ceph-mon[32814]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-07-27T03:10:18.498 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:18 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:18.498 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:18 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:18.498 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:18 smithi067 ceph-mon[32814]: from='osd.8 [v2:172.21.15.175:6816/2286032466,v1:172.21.15.175:6817/2286032466]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-27T03:10:18.498 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:18 smithi067 ceph-mon[32814]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-27T03:10:18.498 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:18 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:18.498 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:18 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:18.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:18 smithi149 ceph-mon[37173]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-07-27T03:10:18.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:18 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:18.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:18 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:18.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:18 smithi149 ceph-mon[37173]: from='osd.8 [v2:172.21.15.175:6816/2286032466,v1:172.21.15.175:6817/2286032466]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-27T03:10:18.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:18 smithi149 ceph-mon[37173]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-27T03:10:18.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:18 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:18.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:18 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:18.533 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:18 smithi175 ceph-mon[37125]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-07-27T03:10:18.533 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:18 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:18.533 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:18 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:18.533 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:18 smithi175 ceph-mon[37125]: from='osd.8 [v2:172.21.15.175:6816/2286032466,v1:172.21.15.175:6817/2286032466]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-27T03:10:18.533 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:18 smithi175 ceph-mon[37125]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-07-27T03:10:18.533 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:18 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:18.533 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:18 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:19.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:19 smithi175 ceph-mon[37125]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-07-27T03:10:19.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:19 smithi175 ceph-mon[37125]: from='osd.8 [v2:172.21.15.175:6816/2286032466,v1:172.21.15.175:6817/2286032466]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:10:19.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:19 smithi175 ceph-mon[37125]: osdmap e52: 9 total, 8 up, 9 in 2024-07-27T03:10:19.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:19 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-27T03:10:19.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:19 smithi175 ceph-mon[37125]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:10:19.375 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:10:19 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:10:19.253+0000 7f8c23921640 -1 osd.8 0 waiting for initial osdmap 2024-07-27T03:10:19.375 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:10:19 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:10:19.267+0000 7f8c1f71f640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-27T03:10:19.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:19 smithi149 ceph-mon[37173]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-07-27T03:10:19.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:19 smithi149 ceph-mon[37173]: from='osd.8 [v2:172.21.15.175:6816/2286032466,v1:172.21.15.175:6817/2286032466]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:10:19.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:19 smithi149 ceph-mon[37173]: osdmap e52: 9 total, 8 up, 9 in 2024-07-27T03:10:19.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:19 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-27T03:10:19.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:19 smithi149 ceph-mon[37173]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:10:19.586 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:19 smithi067 ceph-mon[32814]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-07-27T03:10:19.587 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:19 smithi067 ceph-mon[32814]: from='osd.8 [v2:172.21.15.175:6816/2286032466,v1:172.21.15.175:6817/2286032466]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:10:19.587 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:19 smithi067 ceph-mon[32814]: osdmap e52: 9 total, 8 up, 9 in 2024-07-27T03:10:19.587 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:19 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-27T03:10:19.587 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:19 smithi067 ceph-mon[32814]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:10:20.282 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-07-27T03:10:20.369 INFO:teuthology.orchestra.run.smithi067.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-07-27T03:10:20.383 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10+0 records in 2024-07-27T03:10:20.383 INFO:teuthology.orchestra.run.smithi067.stderr:10+0 records out 2024-07-27T03:10:20.383 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114343 s, 917 MB/s 2024-07-27T03:10:20.384 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping successful for: 2024-07-27T03:10:20.510 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:20 smithi067 ceph-mon[32814]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-07-27T03:10:20.511 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:20 smithi067 ceph-mon[32814]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-27T03:10:20.511 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:20 smithi067 ceph-mon[32814]: osdmap e53: 9 total, 8 up, 9 in 2024-07-27T03:10:20.511 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:20 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-27T03:10:20.511 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:20 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-27T03:10:20.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:20 smithi175 ceph-mon[37125]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-07-27T03:10:20.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:20 smithi175 ceph-mon[37125]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-27T03:10:20.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:20 smithi175 ceph-mon[37125]: osdmap e53: 9 total, 8 up, 9 in 2024-07-27T03:10:20.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:20 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-27T03:10:20.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:20 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-27T03:10:20.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:20 smithi149 ceph-mon[37173]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-07-27T03:10:20.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:20 smithi149 ceph-mon[37173]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-27T03:10:20.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:20 smithi149 ceph-mon[37173]: osdmap e53: 9 total, 8 up, 9 in 2024-07-27T03:10:20.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:20 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-27T03:10:20.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:20 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-27T03:10:21.026 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph orch daemon add osd smithi067:vg_nvme/lv_1 2024-07-27T03:10:21.266 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:21 smithi067 ceph-mon[32814]: purged_snaps scrub starts 2024-07-27T03:10:21.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:21 smithi067 ceph-mon[32814]: purged_snaps scrub ok 2024-07-27T03:10:21.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:21 smithi067 ceph-mon[32814]: osd.8 [v2:172.21.15.175:6816/2286032466,v1:172.21.15.175:6817/2286032466] boot 2024-07-27T03:10:21.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:21 smithi067 ceph-mon[32814]: osdmap e54: 9 total, 9 up, 9 in 2024-07-27T03:10:21.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-27T03:10:21.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:21.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:21.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:10:21.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:10:21.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:10:21.267 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:21.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:21.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:10:21.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:21.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:10:21.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:21.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:10:21.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:21.312 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:10:21.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:21 smithi175 ceph-mon[37125]: purged_snaps scrub starts 2024-07-27T03:10:21.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:21 smithi175 ceph-mon[37125]: purged_snaps scrub ok 2024-07-27T03:10:21.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:21 smithi175 ceph-mon[37125]: osd.8 [v2:172.21.15.175:6816/2286032466,v1:172.21.15.175:6817/2286032466] boot 2024-07-27T03:10:21.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:21 smithi175 ceph-mon[37125]: osdmap e54: 9 total, 9 up, 9 in 2024-07-27T03:10:21.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-27T03:10:21.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:21.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:21.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:10:21.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:10:21.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:10:21.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:21.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:21.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:10:21.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:21.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:10:21.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:21.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:10:21.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:21.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:21 smithi149 ceph-mon[37173]: purged_snaps scrub starts 2024-07-27T03:10:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:21 smithi149 ceph-mon[37173]: purged_snaps scrub ok 2024-07-27T03:10:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:21 smithi149 ceph-mon[37173]: osd.8 [v2:172.21.15.175:6816/2286032466,v1:172.21.15.175:6817/2286032466] boot 2024-07-27T03:10:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:21 smithi149 ceph-mon[37173]: osdmap e54: 9 total, 9 up, 9 in 2024-07-27T03:10:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-07-27T03:10:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:10:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:10:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:10:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:10:21.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:21.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:10:21.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:21.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:10:21.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:22.327 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:22 smithi067 ceph-mon[32814]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-07-27T03:10:22.327 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:22 smithi067 ceph-mon[32814]: Detected new or changed devices on smithi175 2024-07-27T03:10:22.327 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:22 smithi067 ceph-mon[32814]: Adjusting osd_memory_target on smithi175 to 5696M 2024-07-27T03:10:22.327 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:22 smithi067 ceph-mon[32814]: osdmap e55: 9 total, 9 up, 9 in 2024-07-27T03:10:22.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:22 smithi175 ceph-mon[37125]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-07-27T03:10:22.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:22 smithi175 ceph-mon[37125]: Detected new or changed devices on smithi175 2024-07-27T03:10:22.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:22 smithi175 ceph-mon[37125]: Adjusting osd_memory_target on smithi175 to 5696M 2024-07-27T03:10:22.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:22 smithi175 ceph-mon[37125]: osdmap e55: 9 total, 9 up, 9 in 2024-07-27T03:10:22.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:22 smithi149 ceph-mon[37173]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-07-27T03:10:22.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:22 smithi149 ceph-mon[37173]: Detected new or changed devices on smithi175 2024-07-27T03:10:22.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:22 smithi149 ceph-mon[37173]: Adjusting osd_memory_target on smithi175 to 5696M 2024-07-27T03:10:22.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:22 smithi149 ceph-mon[37173]: osdmap e55: 9 total, 9 up, 9 in 2024-07-27T03:10:22.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.824+0000 7f0b6644d640 1 -- 172.21.15.67:0/3872168272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b601020b0 msgr2=0x7f0b60102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:10:22.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.824+0000 7f0b6644d640 1 --2- 172.21.15.67:0/3872168272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b601020b0 0x7f0b60102520 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009900 tx=0x7f0b5002a480 comp rx=0 tx=0).stop 2024-07-27T03:10:22.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.824+0000 7f0b6644d640 1 -- 172.21.15.67:0/3872168272 shutdown_connections 2024-07-27T03:10:22.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.824+0000 7f0b6644d640 1 --2- 172.21.15.67:0/3872168272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b60103ae0 0x7f0b6010a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:10:22.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.824+0000 7f0b6644d640 1 --2- 172.21.15.67:0/3872168272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b60103140 0x7f0b60103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:10:22.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.824+0000 7f0b6644d640 1 --2- 172.21.15.67:0/3872168272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b601020b0 0x7f0b60102520 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:10:22.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.824+0000 7f0b6644d640 1 -- 172.21.15.67:0/3872168272 >> 172.21.15.67:0/3872168272 conn(0x7f0b600fda20 msgr2=0x7f0b600ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:10:22.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.824+0000 7f0b6644d640 1 -- 172.21.15.67:0/3872168272 shutdown_connections 2024-07-27T03:10:22.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.824+0000 7f0b6644d640 1 -- 172.21.15.67:0/3872168272 wait complete. 2024-07-27T03:10:22.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.825+0000 7f0b6644d640 1 Processor -- start 2024-07-27T03:10:22.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.825+0000 7f0b6644d640 1 -- start start 2024-07-27T03:10:22.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.825+0000 7f0b6644d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b601020b0 0x7f0b6019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:10:22.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.825+0000 7f0b6644d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b60103140 0x7f0b6019c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:10:22.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.825+0000 7f0b6644d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b60103ae0 0x7f0b6019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:10:22.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.825+0000 7f0b6644d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6010ee20 con 0x7f0b60103140 2024-07-27T03:10:22.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.825+0000 7f0b6644d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6010eca0 con 0x7f0b60103ae0 2024-07-27T03:10:22.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.825+0000 7f0b6644d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6010efa0 con 0x7f0b601020b0 2024-07-27T03:10:22.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.825+0000 7f0b5f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b60103140 0x7f0b6019c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:10:22.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.825+0000 7f0b5f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b60103140 0x7f0b6019c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46540/0 (socket says 172.21.15.67:46540) 2024-07-27T03:10:22.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.825+0000 7f0b5f7fe640 1 -- 172.21.15.67:0/3057727301 learned_addr learned my addr 172.21.15.67:0/3057727301 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:10:22.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.826+0000 7f0b5ffff640 1 --2- 172.21.15.67:0/3057727301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b601020b0 0x7f0b6019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:10:22.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.826+0000 7f0b649ea640 1 --2- 172.21.15.67:0/3057727301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b60103ae0 0x7f0b6019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:10:22.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.826+0000 7f0b5f7fe640 1 -- 172.21.15.67:0/3057727301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b601020b0 msgr2=0x7f0b6019bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:10:22.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.826+0000 7f0b5f7fe640 1 --2- 172.21.15.67:0/3057727301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b601020b0 0x7f0b6019bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:10:22.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.826+0000 7f0b5f7fe640 1 -- 172.21.15.67:0/3057727301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b60103ae0 msgr2=0x7f0b6019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:10:22.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.826+0000 7f0b5f7fe640 1 --2- 172.21.15.67:0/3057727301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b60103ae0 0x7f0b6019ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:10:22.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.826+0000 7f0b5f7fe640 1 -- 172.21.15.67:0/3057727301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6019d5f0 con 0x7f0b60103140 2024-07-27T03:10:22.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.826+0000 7f0b5f7fe640 1 --2- 172.21.15.67:0/3057727301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b60103140 0x7f0b6019c2c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400c920 tx=0x7f0b5400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:10:22.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.826+0000 7f0b5d7fa640 1 -- 172.21.15.67:0/3057727301 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b54007c70 con 0x7f0b60103140 2024-07-27T03:10:22.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.826+0000 7f0b6644d640 1 -- 172.21.15.67:0/3057727301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6019d8e0 con 0x7f0b60103140 2024-07-27T03:10:22.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.826+0000 7f0b5d7fa640 1 -- 172.21.15.67:0/3057727301 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b54007e10 con 0x7f0b60103140 2024-07-27T03:10:22.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.826+0000 7f0b5d7fa640 1 -- 172.21.15.67:0/3057727301 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b54018820 con 0x7f0b60103140 2024-07-27T03:10:22.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.826+0000 7f0b6644d640 1 -- 172.21.15.67:0/3057727301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b601a4070 con 0x7f0b60103140 2024-07-27T03:10:22.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.827+0000 7f0b6644d640 1 -- 172.21.15.67:0/3057727301 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b60102520 con 0x7f0b60103140 2024-07-27T03:10:22.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.829+0000 7f0b5d7fa640 1 -- 172.21.15.67:0/3057727301 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f0b5400f450 con 0x7f0b60103140 2024-07-27T03:10:22.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.829+0000 7f0b5d7fa640 1 --2- 172.21.15.67:0/3057727301 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0b480b3bd0 0x7f0b480b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:10:22.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.830+0000 7f0b5ffff640 1 --2- 172.21.15.67:0/3057727301 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0b480b3bd0 0x7f0b480b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:10:22.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.830+0000 7f0b5d7fa640 1 -- 172.21.15.67:0/3057727301 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5829+0+0 (secure 0 0 0) 0x7f0b540e3450 con 0x7f0b60103140 2024-07-27T03:10:22.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.830+0000 7f0b5ffff640 1 --2- 172.21.15.67:0/3057727301 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0b480b3bd0 0x7f0b480b60e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0b50031440 tx=0x7f0b500314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:10:22.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:22.834+0000 7f0b5d7fa640 1 -- 172.21.15.67:0/3057727301 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f0b54014030 con 0x7f0b60103140 2024-07-27T03:10:23.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:23.025+0000 7f0b6644d640 1 -- 172.21.15.67:0/3057727301 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0b601a4240 con 0x7f0b480b3bd0 2024-07-27T03:10:23.458 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:23 smithi067 ceph-mon[32814]: osdmap e56: 9 total, 9 up, 9 in 2024-07-27T03:10:23.458 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:23 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:10:23.458 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:23 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:10:23.458 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:23 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:23.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:23 smithi175 ceph-mon[37125]: osdmap e56: 9 total, 9 up, 9 in 2024-07-27T03:10:23.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:23 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:10:23.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:23 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:10:23.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:23 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:23.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:23 smithi149 ceph-mon[37173]: osdmap e56: 9 total, 9 up, 9 in 2024-07-27T03:10:23.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:23 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:10:23.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:23 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:10:23.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:23 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:24.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:24 smithi067 ceph-mon[32814]: pgmap v210: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-07-27T03:10:24.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:24 smithi067 ceph-mon[32814]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:10:24.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:24 smithi175 ceph-mon[37125]: pgmap v210: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-07-27T03:10:24.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:24 smithi175 ceph-mon[37125]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:10:24.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:24 smithi149 ceph-mon[37173]: pgmap v210: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-07-27T03:10:24.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:24 smithi149 ceph-mon[37173]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:10:26.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:26 smithi175 ceph-mon[37125]: pgmap v211: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-07-27T03:10:26.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:26 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2805291586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b65dfb6b-1efe-4314-a8be-ce6a295e99fd"}]: dispatch 2024-07-27T03:10:26.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:26 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2805291586' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b65dfb6b-1efe-4314-a8be-ce6a295e99fd"}]': finished 2024-07-27T03:10:26.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:26 smithi175 ceph-mon[37125]: osdmap e57: 10 total, 9 up, 10 in 2024-07-27T03:10:26.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:26 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-27T03:10:26.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:26 smithi067 ceph-mon[32814]: pgmap v211: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-07-27T03:10:26.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:26 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2805291586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b65dfb6b-1efe-4314-a8be-ce6a295e99fd"}]: dispatch 2024-07-27T03:10:26.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:26 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2805291586' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b65dfb6b-1efe-4314-a8be-ce6a295e99fd"}]': finished 2024-07-27T03:10:26.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:26 smithi067 ceph-mon[32814]: osdmap e57: 10 total, 9 up, 10 in 2024-07-27T03:10:26.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:26 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-27T03:10:26.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:26 smithi149 ceph-mon[37173]: pgmap v211: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-07-27T03:10:26.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:26 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2805291586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b65dfb6b-1efe-4314-a8be-ce6a295e99fd"}]: dispatch 2024-07-27T03:10:26.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:26 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2805291586' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b65dfb6b-1efe-4314-a8be-ce6a295e99fd"}]': finished 2024-07-27T03:10:26.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:26 smithi149 ceph-mon[37173]: osdmap e57: 10 total, 9 up, 10 in 2024-07-27T03:10:26.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:26 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-27T03:10:27.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:27 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3074964356' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:10:27.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:27 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3074964356' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:10:27.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:27 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3074964356' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:10:28.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:28 smithi175 ceph-mon[37125]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-27T03:10:28.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:28 smithi067 ceph-mon[32814]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-27T03:10:28.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:28 smithi149 ceph-mon[37173]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-07-27T03:10:30.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:30 smithi067 ceph-mon[32814]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-07-27T03:10:30.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:30 smithi175 ceph-mon[37125]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-07-27T03:10:30.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:30 smithi149 ceph-mon[37173]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-07-27T03:10:32.615 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:32 smithi067 ceph-mon[32814]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-07-27T03:10:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:32 smithi175 ceph-mon[37125]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-07-27T03:10:32.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:32 smithi149 ceph-mon[37173]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-07-27T03:10:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:33 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-07-27T03:10:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:33 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:33.627 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:33 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-07-27T03:10:33.627 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:33 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:33.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:33 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-07-27T03:10:33.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:33 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:34.552 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:34 smithi067 ceph-mon[32814]: Deploying daemon osd.9 on smithi067 2024-07-27T03:10:34.553 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:34 smithi067 ceph-mon[32814]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-27T03:10:34.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:34 smithi175 ceph-mon[37125]: Deploying daemon osd.9 on smithi067 2024-07-27T03:10:34.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:34 smithi175 ceph-mon[37125]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-27T03:10:34.651 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:34 smithi149 ceph-mon[37173]: Deploying daemon osd.9 on smithi067 2024-07-27T03:10:34.651 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:34 smithi149 ceph-mon[37173]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-27T03:10:36.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:36 smithi067 ceph-mon[32814]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-27T03:10:36.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:36 smithi175 ceph-mon[37125]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-27T03:10:36.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:36 smithi149 ceph-mon[37173]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-27T03:10:38.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:38 smithi067 ceph-mon[32814]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-07-27T03:10:38.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:38 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:10:38.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:38 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:38.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:38 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:38 smithi175 ceph-mon[37125]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-07-27T03:10:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:10:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:38 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:38.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:38 smithi149 ceph-mon[37173]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-07-27T03:10:38.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:38 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:10:38.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:38 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:38.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:38 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:40.601 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:40 smithi067 ceph-mon[32814]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-07-27T03:10:40.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:40 smithi175 ceph-mon[37125]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-07-27T03:10:40.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:40 smithi149 ceph-mon[37173]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-07-27T03:10:41.837 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 9 on host 'smithi067' 2024-07-27T03:10:41.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:41.834+0000 7f0b5d7fa640 1 -- 172.21.15.67:0/3057727301 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0b601a4240 con 0x7f0b480b3bd0 2024-07-27T03:10:41.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:41.836+0000 7f0b6644d640 1 -- 172.21.15.67:0/3057727301 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0b480b3bd0 msgr2=0x7f0b480b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:10:41.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:41.836+0000 7f0b6644d640 1 --2- 172.21.15.67:0/3057727301 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0b480b3bd0 0x7f0b480b60e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0b50031440 tx=0x7f0b500314a0 comp rx=0 tx=0).stop 2024-07-27T03:10:41.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:41.836+0000 7f0b6644d640 1 -- 172.21.15.67:0/3057727301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b60103140 msgr2=0x7f0b6019c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:10:41.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:41.836+0000 7f0b6644d640 1 --2- 172.21.15.67:0/3057727301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b60103140 0x7f0b6019c2c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400c920 tx=0x7f0b5400cde0 comp rx=0 tx=0).stop 2024-07-27T03:10:41.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:41.837+0000 7f0b6644d640 1 -- 172.21.15.67:0/3057727301 shutdown_connections 2024-07-27T03:10:41.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:41.837+0000 7f0b6644d640 1 --2- 172.21.15.67:0/3057727301 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0b480b3bd0 0x7f0b480b60e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:10:41.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:41.837+0000 7f0b6644d640 1 --2- 172.21.15.67:0/3057727301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b60103ae0 0x7f0b6019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:10:41.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:41.837+0000 7f0b6644d640 1 --2- 172.21.15.67:0/3057727301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b60103140 0x7f0b6019c2c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:10:41.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:41.837+0000 7f0b6644d640 1 --2- 172.21.15.67:0/3057727301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b601020b0 0x7f0b6019bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:10:41.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:41.837+0000 7f0b6644d640 1 -- 172.21.15.67:0/3057727301 >> 172.21.15.67:0/3057727301 conn(0x7f0b600fda20 msgr2=0x7f0b6010afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:10:41.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:41.837+0000 7f0b6644d640 1 -- 172.21.15.67:0/3057727301 shutdown_connections 2024-07-27T03:10:41.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:10:41.837+0000 7f0b6644d640 1 -- 172.21.15.67:0/3057727301 wait complete. 2024-07-27T03:10:42.133 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:42 smithi067 ceph-mon[32814]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-07-27T03:10:42.133 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:42 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:42.133 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:42 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:42.133 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:42 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:42.133 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:42 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:10:42.133 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:42 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:42.134 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:42 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:10:42.134 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:42 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:42.134 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:42 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:42.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:42 smithi175 ceph-mon[37125]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-07-27T03:10:42.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:42 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:42.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:42 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:42.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:42 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:42.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:42 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:10:42.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:42 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:42.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:42 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:10:42.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:42 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:42.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:42 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:42.427 DEBUG:teuthology.orchestra.run.smithi067:osd.9> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.9.service 2024-07-27T03:10:42.430 INFO:tasks.cephadm:Deploying osd.10 on smithi149 with /dev/vg_nvme/lv_1... 2024-07-27T03:10:42.430 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-07-27T03:10:42.456 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:42 smithi149 ceph-mon[37173]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-07-27T03:10:42.457 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:42 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:42.457 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:42 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:42.457 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:42 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:42.457 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:42 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:10:42.457 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:42 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:42.457 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:42 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:10:42.457 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:42 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:42.457 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:42 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:42.693 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.b/config 2024-07-27T03:10:43.135 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:43 smithi149 ceph-mon[37173]: from='osd.9 [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-07-27T03:10:43.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:43 smithi175 ceph-mon[37125]: from='osd.9 [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-07-27T03:10:43.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:43 smithi067 ceph-mon[32814]: from='osd.9 [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-07-27T03:10:44.090 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:44 smithi149 ceph-mon[37173]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-07-27T03:10:44.091 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:44 smithi149 ceph-mon[37173]: from='osd.9 [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-07-27T03:10:44.091 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:44 smithi149 ceph-mon[37173]: osdmap e58: 10 total, 9 up, 10 in 2024-07-27T03:10:44.091 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:44 smithi149 ceph-mon[37173]: from='osd.9 [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-27T03:10:44.091 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:44 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-27T03:10:44.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:44 smithi067 ceph-mon[32814]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-07-27T03:10:44.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:44 smithi067 ceph-mon[32814]: from='osd.9 [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-07-27T03:10:44.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:44 smithi067 ceph-mon[32814]: osdmap e58: 10 total, 9 up, 10 in 2024-07-27T03:10:44.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:44 smithi067 ceph-mon[32814]: from='osd.9 [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-27T03:10:44.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:44 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-27T03:10:44.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:44 smithi175 ceph-mon[37125]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-07-27T03:10:44.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:44 smithi175 ceph-mon[37125]: from='osd.9 [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-07-27T03:10:44.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:44 smithi175 ceph-mon[37125]: osdmap e58: 10 total, 9 up, 10 in 2024-07-27T03:10:44.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:44 smithi175 ceph-mon[37125]: from='osd.9 [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-07-27T03:10:44.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:44 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-27T03:10:44.570 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-07-27T03:10:44.659 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-07-27T03:10:44.673 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-07-27T03:10:44.673 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-07-27T03:10:44.673 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117145 s, 895 MB/s 2024-07-27T03:10:44.673 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-07-27T03:10:45.150 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:45 smithi067 ceph-mon[32814]: from='osd.9 [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-27T03:10:45.150 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:45 smithi067 ceph-mon[32814]: osdmap e59: 10 total, 9 up, 10 in 2024-07-27T03:10:45.150 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:45 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-27T03:10:45.150 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:45 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-27T03:10:45.150 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:45 smithi067 ceph-mon[32814]: from='osd.9 [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513]' entity='osd.9' 2024-07-27T03:10:45.150 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:45 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-27T03:10:45.150 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:10:44 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:10:44.832+0000 7fc090762640 -1 osd.9 0 waiting for initial osdmap 2024-07-27T03:10:45.151 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:10:44 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:10:44.845+0000 7fc088558640 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-27T03:10:45.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:45 smithi175 ceph-mon[37125]: from='osd.9 [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-27T03:10:45.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:45 smithi175 ceph-mon[37125]: osdmap e59: 10 total, 9 up, 10 in 2024-07-27T03:10:45.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:45 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-27T03:10:45.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:45 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-27T03:10:45.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:45 smithi175 ceph-mon[37125]: from='osd.9 [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513]' entity='osd.9' 2024-07-27T03:10:45.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:45 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-27T03:10:45.407 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:45 smithi149 ceph-mon[37173]: from='osd.9 [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-07-27T03:10:45.407 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:45 smithi149 ceph-mon[37173]: osdmap e59: 10 total, 9 up, 10 in 2024-07-27T03:10:45.407 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:45 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-27T03:10:45.407 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:45 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-27T03:10:45.407 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:45 smithi149 ceph-mon[37173]: from='osd.9 [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513]' entity='osd.9' 2024-07-27T03:10:45.407 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:45 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-27T03:10:45.438 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph orch daemon add osd smithi149:vg_nvme/lv_1 2024-07-27T03:10:45.737 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.b/config 2024-07-27T03:10:46.210 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:46 smithi149 ceph-mon[37173]: purged_snaps scrub starts 2024-07-27T03:10:46.210 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:46 smithi149 ceph-mon[37173]: purged_snaps scrub ok 2024-07-27T03:10:46.210 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:46 smithi149 ceph-mon[37173]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-07-27T03:10:46.210 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:46 smithi149 ceph-mon[37173]: Detected new or changed devices on smithi067 2024-07-27T03:10:46.210 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:46 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:46.211 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:46 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:46.211 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:46 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:10:46.211 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:46 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:10:46.211 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:46 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:10:46.211 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:46 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:10:46.211 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:46 smithi149 ceph-mon[37173]: Adjusting osd_memory_target on smithi067 to 4272M 2024-07-27T03:10:46.211 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:46 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:46.211 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:46 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:46.211 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:46 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:10:46.211 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:46 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:46.211 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:46 smithi149 ceph-mon[37173]: osd.9 [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513] boot 2024-07-27T03:10:46.211 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:46 smithi149 ceph-mon[37173]: osdmap e60: 10 total, 10 up, 10 in 2024-07-27T03:10:46.211 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:46 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-27T03:10:46.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:46 smithi175 ceph-mon[37125]: purged_snaps scrub starts 2024-07-27T03:10:46.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:46 smithi175 ceph-mon[37125]: purged_snaps scrub ok 2024-07-27T03:10:46.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:46 smithi175 ceph-mon[37125]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-07-27T03:10:46.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:46 smithi175 ceph-mon[37125]: Detected new or changed devices on smithi067 2024-07-27T03:10:46.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:46 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:46.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:46 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:46.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:46 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:10:46.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:46 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:10:46.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:46 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:10:46.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:46 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:10:46.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:46 smithi175 ceph-mon[37125]: Adjusting osd_memory_target on smithi067 to 4272M 2024-07-27T03:10:46.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:46 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:46.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:46 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:46.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:46 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:10:46.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:46 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:46.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:46 smithi175 ceph-mon[37125]: osd.9 [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513] boot 2024-07-27T03:10:46.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:46 smithi175 ceph-mon[37125]: osdmap e60: 10 total, 10 up, 10 in 2024-07-27T03:10:46.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:46 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-27T03:10:46.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:46 smithi067 ceph-mon[32814]: purged_snaps scrub starts 2024-07-27T03:10:46.471 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:46 smithi067 ceph-mon[32814]: purged_snaps scrub ok 2024-07-27T03:10:46.471 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:46 smithi067 ceph-mon[32814]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-07-27T03:10:46.471 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:46 smithi067 ceph-mon[32814]: Detected new or changed devices on smithi067 2024-07-27T03:10:46.471 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:46 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:46.472 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:46 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:46.472 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:46 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:10:46.472 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:46 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:10:46.472 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:46 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:10:46.472 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:46 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:10:46.472 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:46 smithi067 ceph-mon[32814]: Adjusting osd_memory_target on smithi067 to 4272M 2024-07-27T03:10:46.473 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:46 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:46.473 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:46 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:46.473 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:46 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:10:46.473 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:46 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:10:46.473 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:46 smithi067 ceph-mon[32814]: osd.9 [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513] boot 2024-07-27T03:10:46.474 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:46 smithi067 ceph-mon[32814]: osdmap e60: 10 total, 10 up, 10 in 2024-07-27T03:10:46.474 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:46 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-07-27T03:10:47.346 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.341+0000 7f9fa262e640 1 -- 172.21.15.149:0/4172403677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f9c103ae0 msgr2=0x7f9f9c10a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:10:47.346 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.341+0000 7f9fa262e640 1 --2- 172.21.15.149:0/4172403677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f9c103ae0 0x7f9f9c10a480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009ca0 tx=0x7f9f9402a950 comp rx=0 tx=0).stop 2024-07-27T03:10:47.346 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.341+0000 7f9fa262e640 1 -- 172.21.15.149:0/4172403677 shutdown_connections 2024-07-27T03:10:47.346 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.341+0000 7f9fa262e640 1 --2- 172.21.15.149:0/4172403677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f9c103ae0 0x7f9f9c10a480 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:10:47.346 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.341+0000 7f9fa262e640 1 --2- 172.21.15.149:0/4172403677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f9c103140 0x7f9f9c103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:10:47.346 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.341+0000 7f9fa262e640 1 --2- 172.21.15.149:0/4172403677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f9c1020b0 0x7f9f9c102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:10:47.346 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.341+0000 7f9fa262e640 1 -- 172.21.15.149:0/4172403677 >> 172.21.15.149:0/4172403677 conn(0x7f9f9c0fda20 msgr2=0x7f9f9c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:10:47.346 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.341+0000 7f9fa262e640 1 -- 172.21.15.149:0/4172403677 shutdown_connections 2024-07-27T03:10:47.347 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.341+0000 7f9fa262e640 1 -- 172.21.15.149:0/4172403677 wait complete. 2024-07-27T03:10:47.347 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.342+0000 7f9fa262e640 1 Processor -- start 2024-07-27T03:10:47.347 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.342+0000 7f9fa262e640 1 -- start start 2024-07-27T03:10:47.347 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.342+0000 7f9fa262e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f9c1020b0 0x7f9f9c06b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:10:47.347 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.342+0000 7f9fa262e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f9c103140 0x7f9f9c19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:10:47.347 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.342+0000 7f9fa262e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f9c103ae0 0x7f9f9c1a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:10:47.347 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.342+0000 7f9fa262e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c10dd10 con 0x7f9f9c103140 2024-07-27T03:10:47.347 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.342+0000 7f9fa262e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c10db90 con 0x7f9f9c103ae0 2024-07-27T03:10:47.347 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.342+0000 7f9fa262e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c10de90 con 0x7f9f9c1020b0 2024-07-27T03:10:47.347 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.342+0000 7f9fa0bcb640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f9c103ae0 0x7f9f9c1a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:10:47.347 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.342+0000 7f9fa0bcb640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f9c103ae0 0x7f9f9c1a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:51684/0 (socket says 172.21.15.149:51684) 2024-07-27T03:10:47.347 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.342+0000 7f9fa0bcb640 1 -- 172.21.15.149:0/3002373466 learned_addr learned my addr 172.21.15.149:0/3002373466 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-07-27T03:10:47.348 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.342+0000 7f9f9b7fe640 1 --2- 172.21.15.149:0/3002373466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f9c103140 0x7f9f9c19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:10:47.348 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.343+0000 7f9f9bfff640 1 --2- 172.21.15.149:0/3002373466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f9c1020b0 0x7f9f9c06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:10:47.348 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.343+0000 7f9fa0bcb640 1 -- 172.21.15.149:0/3002373466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f9c1020b0 msgr2=0x7f9f9c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:10:47.348 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.343+0000 7f9fa0bcb640 1 --2- 172.21.15.149:0/3002373466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f9c1020b0 0x7f9f9c06b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:10:47.348 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.343+0000 7f9fa0bcb640 1 -- 172.21.15.149:0/3002373466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f9c103140 msgr2=0x7f9f9c19e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:10:47.348 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.343+0000 7f9fa0bcb640 1 --2- 172.21.15.149:0/3002373466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f9c103140 0x7f9f9c19e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:10:47.348 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.343+0000 7f9fa0bcb640 1 -- 172.21.15.149:0/3002373466 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9c1a1470 con 0x7f9f9c103ae0 2024-07-27T03:10:47.348 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.343+0000 7f9fa0bcb640 1 --2- 172.21.15.149:0/3002373466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f9c103ae0 0x7f9f9c1a0c70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9f94006010 tx=0x7f9f9402fc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:10:47.348 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.343+0000 7f9f997fa640 1 -- 172.21.15.149:0/3002373466 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f94034040 con 0x7f9f9c103ae0 2024-07-27T03:10:47.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.343+0000 7f9fa262e640 1 -- 172.21.15.149:0/3002373466 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f9c1a1700 con 0x7f9f9c103ae0 2024-07-27T03:10:47.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.343+0000 7f9f997fa640 1 -- 172.21.15.149:0/3002373466 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f940308f0 con 0x7f9f9c103ae0 2024-07-27T03:10:47.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.343+0000 7f9f997fa640 1 -- 172.21.15.149:0/3002373466 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f94039450 con 0x7f9f9c103ae0 2024-07-27T03:10:47.349 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.343+0000 7f9fa262e640 1 -- 172.21.15.149:0/3002373466 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f9c1a1c30 con 0x7f9f9c103ae0 2024-07-27T03:10:47.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.344+0000 7f9fa262e640 1 -- 172.21.15.149:0/3002373466 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f9c109290 con 0x7f9f9c103ae0 2024-07-27T03:10:47.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.346+0000 7f9f997fa640 1 -- 172.21.15.149:0/3002373466 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f9f94037020 con 0x7f9f9c103ae0 2024-07-27T03:10:47.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.346+0000 7f9f997fa640 1 --2- 172.21.15.149:0/3002373466 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f9f840b3bd0 0x7f9f840b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:10:47.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.347+0000 7f9f997fa640 1 -- 172.21.15.149:0/3002373466 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6261+0+0 (secure 0 0 0) 0x7f9f941046d0 con 0x7f9f9c103ae0 2024-07-27T03:10:47.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.347+0000 7f9f9bfff640 1 --2- 172.21.15.149:0/3002373466 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f9f840b3bd0 0x7f9f840b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:10:47.350 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.347+0000 7f9f9bfff640 1 --2- 172.21.15.149:0/3002373466 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f9f840b3bd0 0x7f9f840b60e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c005d00 tx=0x7f9f8c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:10:47.352 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.350+0000 7f9f997fa640 1 -- 172.21.15.149:0/3002373466 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f9f940cc520 con 0x7f9f9c103ae0 2024-07-27T03:10:47.536 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:10:47.534+0000 7f9fa262e640 1 -- 172.21.15.149:0/3002373466 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9f9c063270 con 0x7f9f840b3bd0 2024-07-27T03:10:48.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:48 smithi149 ceph-mon[37173]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:10:48.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:48 smithi149 ceph-mon[37173]: osdmap e61: 10 total, 10 up, 10 in 2024-07-27T03:10:48.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:48 smithi149 ceph-mon[37173]: from='client.24461 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:10:48.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:48 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:10:48.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:48 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:10:48.252 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:48 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:48.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:48 smithi175 ceph-mon[37125]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:10:48.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:48 smithi175 ceph-mon[37125]: osdmap e61: 10 total, 10 up, 10 in 2024-07-27T03:10:48.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:48 smithi175 ceph-mon[37125]: from='client.24461 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:10:48.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:48 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:10:48.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:48 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:10:48.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:48 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:48.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:48 smithi067 ceph-mon[32814]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:10:48.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:48 smithi067 ceph-mon[32814]: osdmap e61: 10 total, 10 up, 10 in 2024-07-27T03:10:48.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:48 smithi067 ceph-mon[32814]: from='client.24461 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:10:48.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:48 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:10:48.471 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:48 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:10:48.471 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:48 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:50.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:50 smithi175 ceph-mon[37125]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:10:50.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:50 smithi175 ceph-mon[37125]: from='client.? 172.21.15.149:0/2458318770' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3dabae0-0d64-4222-984c-fc1cce883456"}]: dispatch 2024-07-27T03:10:50.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:50 smithi175 ceph-mon[37125]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3dabae0-0d64-4222-984c-fc1cce883456"}]: dispatch 2024-07-27T03:10:50.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:50 smithi175 ceph-mon[37125]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3dabae0-0d64-4222-984c-fc1cce883456"}]': finished 2024-07-27T03:10:50.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:50 smithi175 ceph-mon[37125]: osdmap e62: 11 total, 10 up, 11 in 2024-07-27T03:10:50.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:50 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-27T03:10:50.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:50 smithi067 ceph-mon[32814]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:10:50.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:50 smithi067 ceph-mon[32814]: from='client.? 172.21.15.149:0/2458318770' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3dabae0-0d64-4222-984c-fc1cce883456"}]: dispatch 2024-07-27T03:10:50.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:50 smithi067 ceph-mon[32814]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3dabae0-0d64-4222-984c-fc1cce883456"}]: dispatch 2024-07-27T03:10:50.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:50 smithi067 ceph-mon[32814]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3dabae0-0d64-4222-984c-fc1cce883456"}]': finished 2024-07-27T03:10:50.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:50 smithi067 ceph-mon[32814]: osdmap e62: 11 total, 10 up, 11 in 2024-07-27T03:10:50.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:50 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-27T03:10:50.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:50 smithi149 ceph-mon[37173]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:10:50.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:50 smithi149 ceph-mon[37173]: from='client.? 172.21.15.149:0/2458318770' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3dabae0-0d64-4222-984c-fc1cce883456"}]: dispatch 2024-07-27T03:10:50.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:50 smithi149 ceph-mon[37173]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3dabae0-0d64-4222-984c-fc1cce883456"}]: dispatch 2024-07-27T03:10:50.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:50 smithi149 ceph-mon[37173]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3dabae0-0d64-4222-984c-fc1cce883456"}]': finished 2024-07-27T03:10:50.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:50 smithi149 ceph-mon[37173]: osdmap e62: 11 total, 10 up, 11 in 2024-07-27T03:10:50.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:50 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-27T03:10:51.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:51 smithi175 ceph-mon[37125]: from='client.? 172.21.15.149:0/3054684545' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:10:51.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:51 smithi067 ceph-mon[32814]: from='client.? 172.21.15.149:0/3054684545' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:10:51.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:51 smithi149 ceph-mon[37173]: from='client.? 172.21.15.149:0/3054684545' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:10:52.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:52 smithi175 ceph-mon[37125]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:10:52.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:52 smithi067 ceph-mon[32814]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:10:52.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:52 smithi149 ceph-mon[37173]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:10:54.122 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:54 smithi149 ceph-mon[37173]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:10:54.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:54 smithi175 ceph-mon[37125]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:10:54.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:54 smithi067 ceph-mon[32814]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:10:56.230 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:56 smithi149 ceph-mon[37173]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:10:56.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:56 smithi175 ceph-mon[37125]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:10:56.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:56 smithi067 ceph-mon[32814]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:10:57.446 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-07-27T03:10:57.447 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:57.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-07-27T03:10:57.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:57.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-07-27T03:10:57.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:10:58.464 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:58 smithi149 ceph-mon[37173]: Deploying daemon osd.10 on smithi149 2024-07-27T03:10:58.464 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:10:58 smithi149 ceph-mon[37173]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:10:58.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:58 smithi067 ceph-mon[32814]: Deploying daemon osd.10 on smithi149 2024-07-27T03:10:58.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:10:58 smithi067 ceph-mon[32814]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:10:58.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:58 smithi175 ceph-mon[37125]: Deploying daemon osd.10 on smithi149 2024-07-27T03:10:58.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:10:58 smithi175 ceph-mon[37125]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:11:00.438 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:00 smithi149 ceph-mon[37173]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:11:00.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:00 smithi067 ceph-mon[32814]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:11:00.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:00 smithi175 ceph-mon[37125]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:11:02.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:02 smithi067 ceph-mon[32814]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:11:02.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:02 smithi149 ceph-mon[37173]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:11:02.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:02 smithi175 ceph-mon[37125]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:11:03.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:03 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:11:03.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:03 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:03.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:03 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:03.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:03 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:11:03.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:03 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:03.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:03 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:03.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:03 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:11:03.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:03 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:03.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:03 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:04.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:04 smithi149 ceph-mon[37173]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:11:04.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:04 smithi067 ceph-mon[32814]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:11:04.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:04 smithi175 ceph-mon[37125]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:11:05.862 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:11:05.858+0000 7f9f997fa640 1 -- 172.21.15.149:0/3002373466 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f9f9c063270 con 0x7f9f840b3bd0 2024-07-27T03:11:05.863 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 10 on host 'smithi149' 2024-07-27T03:11:05.864 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:11:05.861+0000 7f9fa262e640 1 -- 172.21.15.149:0/3002373466 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f9f840b3bd0 msgr2=0x7f9f840b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:05.864 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:11:05.861+0000 7f9fa262e640 1 --2- 172.21.15.149:0/3002373466 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f9f840b3bd0 0x7f9f840b60e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c005d00 tx=0x7f9f8c00b410 comp rx=0 tx=0).stop 2024-07-27T03:11:05.865 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:11:05.861+0000 7f9fa262e640 1 -- 172.21.15.149:0/3002373466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f9c103ae0 msgr2=0x7f9f9c1a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:05.865 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:11:05.861+0000 7f9fa262e640 1 --2- 172.21.15.149:0/3002373466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f9c103ae0 0x7f9f9c1a0c70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9f94006010 tx=0x7f9f9402fc00 comp rx=0 tx=0).stop 2024-07-27T03:11:05.865 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:11:05.861+0000 7f9fa262e640 1 -- 172.21.15.149:0/3002373466 shutdown_connections 2024-07-27T03:11:05.865 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:11:05.861+0000 7f9fa262e640 1 --2- 172.21.15.149:0/3002373466 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f9f840b3bd0 0x7f9f840b60e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:05.865 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:11:05.861+0000 7f9fa262e640 1 --2- 172.21.15.149:0/3002373466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f9c103ae0 0x7f9f9c1a0c70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:05.865 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:11:05.861+0000 7f9fa262e640 1 --2- 172.21.15.149:0/3002373466 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9f9c103140 0x7f9f9c19e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:05.865 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:11:05.861+0000 7f9fa262e640 1 --2- 172.21.15.149:0/3002373466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f9c1020b0 0x7f9f9c06b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:05.865 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:11:05.861+0000 7f9fa262e640 1 -- 172.21.15.149:0/3002373466 >> 172.21.15.149:0/3002373466 conn(0x7f9f9c0fda20 msgr2=0x7f9f9c106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:11:05.865 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:11:05.861+0000 7f9fa262e640 1 -- 172.21.15.149:0/3002373466 shutdown_connections 2024-07-27T03:11:05.865 INFO:teuthology.orchestra.run.smithi149.stderr:2024-07-27T03:11:05.861+0000 7f9fa262e640 1 -- 172.21.15.149:0/3002373466 wait complete. 2024-07-27T03:11:06.497 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:06 smithi149 ceph-mon[37173]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:11:06.497 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:06 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:06.497 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:06 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:06.497 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:06 smithi149 ceph-mon[37173]: from='osd.10 [v2:172.21.15.149:6824/2011495212,v1:172.21.15.149:6825/2011495212]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-27T03:11:06.497 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:06 smithi149 ceph-mon[37173]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-27T03:11:06.531 DEBUG:teuthology.orchestra.run.smithi149:osd.10> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.10.service 2024-07-27T03:11:06.533 INFO:tasks.cephadm:Deploying osd.11 on smithi175 with /dev/vg_nvme/lv_1... 2024-07-27T03:11:06.534 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-07-27T03:11:06.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:06 smithi067 ceph-mon[32814]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:11:06.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:06.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:06.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:06 smithi067 ceph-mon[32814]: from='osd.10 [v2:172.21.15.149:6824/2011495212,v1:172.21.15.149:6825/2011495212]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-27T03:11:06.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:06 smithi067 ceph-mon[32814]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-27T03:11:06.795 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.c/config 2024-07-27T03:11:06.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:06 smithi175 ceph-mon[37125]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:11:06.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:06 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:06.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:06 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:06.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:06 smithi175 ceph-mon[37125]: from='osd.10 [v2:172.21.15.149:6824/2011495212,v1:172.21.15.149:6825/2011495212]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-27T03:11:06.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:06 smithi175 ceph-mon[37125]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-07-27T03:11:07.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:07 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:07.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:07 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:07.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:07 smithi149 ceph-mon[37173]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-07-27T03:11:07.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:07 smithi149 ceph-mon[37173]: osdmap e63: 11 total, 10 up, 11 in 2024-07-27T03:11:07.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:07 smithi149 ceph-mon[37173]: from='osd.10 [v2:172.21.15.149:6824/2011495212,v1:172.21.15.149:6825/2011495212]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:11:07.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:07 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-27T03:11:07.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:07 smithi149 ceph-mon[37173]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:11:07.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:07 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:07.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:07 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:07.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:07 smithi067 ceph-mon[32814]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-07-27T03:11:07.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:07 smithi067 ceph-mon[32814]: osdmap e63: 11 total, 10 up, 11 in 2024-07-27T03:11:07.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:07 smithi067 ceph-mon[32814]: from='osd.10 [v2:172.21.15.149:6824/2011495212,v1:172.21.15.149:6825/2011495212]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:11:07.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:07 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-27T03:11:07.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:07 smithi067 ceph-mon[32814]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:11:07.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:07 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:07.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:07 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:07.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:07 smithi175 ceph-mon[37125]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-07-27T03:11:07.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:07 smithi175 ceph-mon[37125]: osdmap e63: 11 total, 10 up, 11 in 2024-07-27T03:11:07.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:07 smithi175 ceph-mon[37125]: from='osd.10 [v2:172.21.15.149:6824/2011495212,v1:172.21.15.149:6825/2011495212]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:11:07.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:07 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-27T03:11:07.998 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:07 smithi175 ceph-mon[37125]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-07-27T03:11:08.666 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-07-27T03:11:08.765 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-07-27T03:11:08.782 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-07-27T03:11:08.782 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-07-27T03:11:08.782 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0150527 s, 697 MB/s 2024-07-27T03:11:08.782 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-07-27T03:11:08.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:08 smithi149 ceph-mon[37173]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:11:08.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:08 smithi149 ceph-mon[37173]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-07-27T03:11:08.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:08 smithi149 ceph-mon[37173]: osdmap e64: 11 total, 10 up, 11 in 2024-07-27T03:11:08.858 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:08 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-27T03:11:08.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:08 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-27T03:11:08.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:08 smithi149 ceph-mon[37173]: osdmap e65: 11 total, 10 up, 11 in 2024-07-27T03:11:08.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:08 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-27T03:11:08.859 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:08 smithi149 ceph-mon[37173]: from='osd.10 ' entity='osd.10' 2024-07-27T03:11:08.859 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:11:08 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:11:08.623+0000 7f841a4b8640 -1 osd.10 0 waiting for initial osdmap 2024-07-27T03:11:08.859 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:11:08 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:11:08.637+0000 7f84122ae640 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-27T03:11:08.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:08 smithi067 ceph-mon[32814]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:11:08.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:08 smithi067 ceph-mon[32814]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-07-27T03:11:08.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:08 smithi067 ceph-mon[32814]: osdmap e64: 11 total, 10 up, 11 in 2024-07-27T03:11:08.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:08 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-27T03:11:08.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:08 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-27T03:11:08.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:08 smithi067 ceph-mon[32814]: osdmap e65: 11 total, 10 up, 11 in 2024-07-27T03:11:08.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:08 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-27T03:11:08.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:08 smithi067 ceph-mon[32814]: from='osd.10 ' entity='osd.10' 2024-07-27T03:11:09.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:08 smithi175 ceph-mon[37125]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:11:09.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:08 smithi175 ceph-mon[37125]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-07-27T03:11:09.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:08 smithi175 ceph-mon[37125]: osdmap e64: 11 total, 10 up, 11 in 2024-07-27T03:11:09.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:08 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-27T03:11:09.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:08 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-27T03:11:09.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:08 smithi175 ceph-mon[37125]: osdmap e65: 11 total, 10 up, 11 in 2024-07-27T03:11:09.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:08 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-27T03:11:09.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:08 smithi175 ceph-mon[37125]: from='osd.10 ' entity='osd.10' 2024-07-27T03:11:09.479 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_1 2024-07-27T03:11:09.740 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.c/config 2024-07-27T03:11:09.838 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:09 smithi149 ceph-mon[37173]: purged_snaps scrub starts 2024-07-27T03:11:09.839 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:09 smithi149 ceph-mon[37173]: purged_snaps scrub ok 2024-07-27T03:11:09.839 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-27T03:11:09.839 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:09 smithi149 ceph-mon[37173]: osd.10 [v2:172.21.15.149:6824/2011495212,v1:172.21.15.149:6825/2011495212] boot 2024-07-27T03:11:09.839 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:09 smithi149 ceph-mon[37173]: osdmap e66: 11 total, 11 up, 11 in 2024-07-27T03:11:09.839 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-27T03:11:09.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:09 smithi067 ceph-mon[32814]: purged_snaps scrub starts 2024-07-27T03:11:09.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:09 smithi067 ceph-mon[32814]: purged_snaps scrub ok 2024-07-27T03:11:09.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-27T03:11:09.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:09 smithi067 ceph-mon[32814]: osd.10 [v2:172.21.15.149:6824/2011495212,v1:172.21.15.149:6825/2011495212] boot 2024-07-27T03:11:09.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:09 smithi067 ceph-mon[32814]: osdmap e66: 11 total, 11 up, 11 in 2024-07-27T03:11:09.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-27T03:11:09.987 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:09 smithi175 ceph-mon[37125]: purged_snaps scrub starts 2024-07-27T03:11:09.987 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:09 smithi175 ceph-mon[37125]: purged_snaps scrub ok 2024-07-27T03:11:09.987 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-27T03:11:09.988 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:09 smithi175 ceph-mon[37125]: osd.10 [v2:172.21.15.149:6824/2011495212,v1:172.21.15.149:6825/2011495212] boot 2024-07-27T03:11:09.988 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:09 smithi175 ceph-mon[37125]: osdmap e66: 11 total, 11 up, 11 in 2024-07-27T03:11:09.988 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-07-27T03:11:10.854 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:10 smithi175 ceph-mon[37125]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:11:10.854 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:10 smithi175 ceph-mon[37125]: osdmap e67: 11 total, 11 up, 11 in 2024-07-27T03:11:10.855 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:10 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:10.855 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:10 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:10.855 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:10 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:10.855 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:10 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:10.855 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:10 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:10.855 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:10 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:10.855 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:10 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:10.855 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:10 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:10.855 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:10 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:11:10.855 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:10 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:10.855 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:10 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:11:10.855 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:10 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:10.855 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:10 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:11:10.856 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:10 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:10.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:10 smithi067 ceph-mon[32814]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:11:10.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:10 smithi067 ceph-mon[32814]: osdmap e67: 11 total, 11 up, 11 in 2024-07-27T03:11:10.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:10 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:10.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:10 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:10.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:10 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:10.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:10 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:10.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:10 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:10.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:10 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:10.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:10 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:10.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:10 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:10.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:10 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:11:10.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:10 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:10.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:10 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:11:10.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:10 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:10.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:10 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:11:10.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:10 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:11.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:10 smithi149 ceph-mon[37173]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-07-27T03:11:11.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:10 smithi149 ceph-mon[37173]: osdmap e67: 11 total, 11 up, 11 in 2024-07-27T03:11:11.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:10 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:11.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:10 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:11.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:10 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:11.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:10 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:11.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:10 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:11.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:10 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:11.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:10 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:11.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:10 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:11.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:10 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:11:11.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:10 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:11.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:10 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:11:11.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:10 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:11.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:10 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:11:11.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:10 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:11.399 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.397+0000 7f536d5e0640 1 -- 172.21.15.175:0/4030733912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5368103ae0 msgr2=0x7f536810a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:11.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.397+0000 7f536d5e0640 1 --2- 172.21.15.175:0/4030733912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5368103ae0 0x7f536810a480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5360009ca0 tx=0x7f536002a950 comp rx=0 tx=0).stop 2024-07-27T03:11:11.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.397+0000 7f536d5e0640 1 -- 172.21.15.175:0/4030733912 shutdown_connections 2024-07-27T03:11:11.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.397+0000 7f536d5e0640 1 --2- 172.21.15.175:0/4030733912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5368103ae0 0x7f536810a480 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:11.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.397+0000 7f536d5e0640 1 --2- 172.21.15.175:0/4030733912 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5368103140 0x7f5368103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:11.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.397+0000 7f536d5e0640 1 --2- 172.21.15.175:0/4030733912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53681020b0 0x7f5368102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:11.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.397+0000 7f536d5e0640 1 -- 172.21.15.175:0/4030733912 >> 172.21.15.175:0/4030733912 conn(0x7f53680fda20 msgr2=0x7f53680ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:11:11.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.397+0000 7f536d5e0640 1 -- 172.21.15.175:0/4030733912 shutdown_connections 2024-07-27T03:11:11.400 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.397+0000 7f536d5e0640 1 -- 172.21.15.175:0/4030733912 wait complete. 2024-07-27T03:11:11.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.398+0000 7f536d5e0640 1 Processor -- start 2024-07-27T03:11:11.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.398+0000 7f536d5e0640 1 -- start start 2024-07-27T03:11:11.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.398+0000 7f536d5e0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53681020b0 0x7f53681999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:11.402 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.398+0000 7f536d5e0640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5368103140 0x7f536819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:11.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.398+0000 7f536d5e0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5368103ae0 0x7f536819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:11.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.398+0000 7f536d5e0640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f536810dd10 con 0x7f53681020b0 2024-07-27T03:11:11.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.398+0000 7f536d5e0640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f536810db90 con 0x7f5368103140 2024-07-27T03:11:11.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.398+0000 7f536d5e0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f536810de90 con 0x7f5368103ae0 2024-07-27T03:11:11.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.398+0000 7f53677fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5368103ae0 0x7f536819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:11.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.398+0000 7f53677fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5368103ae0 0x7f536819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:50972/0 (socket says 172.21.15.175:50972) 2024-07-27T03:11:11.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.398+0000 7f53677fe640 1 -- 172.21.15.175:0/4230477198 learned_addr learned my addr 172.21.15.175:0/4230477198 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-27T03:11:11.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.398+0000 7f5366ffd640 1 --2- 172.21.15.175:0/4230477198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53681020b0 0x7f53681999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:11.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.398+0000 7f535ffff640 1 --2- 172.21.15.175:0/4230477198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5368103140 0x7f536819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:11.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.398+0000 7f53677fe640 1 -- 172.21.15.175:0/4230477198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5368103140 msgr2=0x7f536819a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:11.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.398+0000 7f53677fe640 1 --2- 172.21.15.175:0/4230477198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5368103140 0x7f536819a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:11.403 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.398+0000 7f53677fe640 1 -- 172.21.15.175:0/4230477198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53681020b0 msgr2=0x7f53681999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:11.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.398+0000 7f53677fe640 1 --2- 172.21.15.175:0/4230477198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53681020b0 0x7f53681999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:11.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.398+0000 7f53677fe640 1 -- 172.21.15.175:0/4230477198 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536819b3d0 con 0x7f5368103ae0 2024-07-27T03:11:11.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.399+0000 7f53677fe640 1 --2- 172.21.15.175:0/4230477198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5368103ae0 0x7f536819abb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5360006010 tx=0x7f536002fc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:11:11.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.399+0000 7f5364ff9640 1 -- 172.21.15.175:0/4230477198 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360034040 con 0x7f5368103ae0 2024-07-27T03:11:11.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.399+0000 7f5364ff9640 1 -- 172.21.15.175:0/4230477198 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53600308f0 con 0x7f5368103ae0 2024-07-27T03:11:11.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.399+0000 7f5364ff9640 1 -- 172.21.15.175:0/4230477198 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360039450 con 0x7f5368103ae0 2024-07-27T03:11:11.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.399+0000 7f536d5e0640 1 -- 172.21.15.175:0/4230477198 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536819b660 con 0x7f5368103ae0 2024-07-27T03:11:11.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.399+0000 7f536d5e0640 1 -- 172.21.15.175:0/4230477198 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53681a1dc0 con 0x7f5368103ae0 2024-07-27T03:11:11.404 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.400+0000 7f536d5e0640 1 -- 172.21.15.175:0/4230477198 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53681a1f90 con 0x7f5368103ae0 2024-07-27T03:11:11.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.402+0000 7f5364ff9640 1 -- 172.21.15.175:0/4230477198 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f5360037020 con 0x7f5368103ae0 2024-07-27T03:11:11.405 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.403+0000 7f5364ff9640 1 --2- 172.21.15.175:0/4230477198 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f53540b3c50 0x7f53540b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:11.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.403+0000 7f5364ff9640 1 -- 172.21.15.175:0/4230477198 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6694+0+0 (secure 0 0 0) 0x7f53601049a0 con 0x7f5368103ae0 2024-07-27T03:11:11.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.403+0000 7f5366ffd640 1 --2- 172.21.15.175:0/4230477198 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f53540b3c50 0x7f53540b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:11.406 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.404+0000 7f5366ffd640 1 --2- 172.21.15.175:0/4230477198 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f53540b3c50 0x7f53540b6160 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f53580058c0 tx=0x7f535800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:11:11.409 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.406+0000 7f5364ff9640 1 -- 172.21.15.175:0/4230477198 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f53600cc5a0 con 0x7f5368103ae0 2024-07-27T03:11:11.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:11.587+0000 7f536d5e0640 1 -- 172.21.15.175:0/4230477198 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f53680ff610 con 0x7f53540b3c50 2024-07-27T03:11:11.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:11 smithi175 ceph-mon[37125]: Detected new or changed devices on smithi149 2024-07-27T03:11:11.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:11 smithi175 ceph-mon[37125]: Adjusting osd_memory_target on smithi149 to 1454M 2024-07-27T03:11:11.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:11 smithi175 ceph-mon[37125]: osdmap e68: 11 total, 11 up, 11 in 2024-07-27T03:11:11.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:11 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:11:11.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:11 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:11:11.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:11 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:11.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:11 smithi067 ceph-mon[32814]: Detected new or changed devices on smithi149 2024-07-27T03:11:11.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:11 smithi067 ceph-mon[32814]: Adjusting osd_memory_target on smithi149 to 1454M 2024-07-27T03:11:11.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:11 smithi067 ceph-mon[32814]: osdmap e68: 11 total, 11 up, 11 in 2024-07-27T03:11:11.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:11 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:11:11.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:11 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:11:11.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:11 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:12.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:11 smithi149 ceph-mon[37173]: Detected new or changed devices on smithi149 2024-07-27T03:11:12.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:11 smithi149 ceph-mon[37173]: Adjusting osd_memory_target on smithi149 to 1454M 2024-07-27T03:11:12.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:11 smithi149 ceph-mon[37173]: osdmap e68: 11 total, 11 up, 11 in 2024-07-27T03:11:12.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:11 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-07-27T03:11:12.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:11 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-07-27T03:11:12.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:11 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:12.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:12 smithi067 ceph-mon[32814]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-07-27T03:11:12.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:12 smithi067 ceph-mon[32814]: from='client.24475 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:11:12.999 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:12 smithi175 ceph-mon[37125]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-07-27T03:11:12.999 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:12 smithi175 ceph-mon[37125]: from='client.24475 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:11:13.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:12 smithi149 ceph-mon[37173]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-07-27T03:11:13.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:12 smithi149 ceph-mon[37173]: from='client.24475 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:11:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:14 smithi175 ceph-mon[37125]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-07-27T03:11:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:14 smithi175 ceph-mon[37125]: from='client.? 172.21.15.175:0/906184149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "406c082c-9b5f-4781-8cbf-3c04fa49e410"}]: dispatch 2024-07-27T03:11:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:14 smithi175 ceph-mon[37125]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "406c082c-9b5f-4781-8cbf-3c04fa49e410"}]: dispatch 2024-07-27T03:11:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:14 smithi175 ceph-mon[37125]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "406c082c-9b5f-4781-8cbf-3c04fa49e410"}]': finished 2024-07-27T03:11:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:14 smithi175 ceph-mon[37125]: osdmap e69: 12 total, 11 up, 12 in 2024-07-27T03:11:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-27T03:11:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:14 smithi175 ceph-mon[37125]: from='client.? 172.21.15.175:0/2117156267' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:11:14.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:14 smithi067 ceph-mon[32814]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-07-27T03:11:14.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:14 smithi067 ceph-mon[32814]: from='client.? 172.21.15.175:0/906184149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "406c082c-9b5f-4781-8cbf-3c04fa49e410"}]: dispatch 2024-07-27T03:11:14.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:14 smithi067 ceph-mon[32814]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "406c082c-9b5f-4781-8cbf-3c04fa49e410"}]: dispatch 2024-07-27T03:11:14.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:14 smithi067 ceph-mon[32814]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "406c082c-9b5f-4781-8cbf-3c04fa49e410"}]': finished 2024-07-27T03:11:14.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:14 smithi067 ceph-mon[32814]: osdmap e69: 12 total, 11 up, 12 in 2024-07-27T03:11:14.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-27T03:11:14.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:14 smithi067 ceph-mon[32814]: from='client.? 172.21.15.175:0/2117156267' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:11:15.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:14 smithi149 ceph-mon[37173]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-07-27T03:11:15.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:14 smithi149 ceph-mon[37173]: from='client.? 172.21.15.175:0/906184149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "406c082c-9b5f-4781-8cbf-3c04fa49e410"}]: dispatch 2024-07-27T03:11:15.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:14 smithi149 ceph-mon[37173]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "406c082c-9b5f-4781-8cbf-3c04fa49e410"}]: dispatch 2024-07-27T03:11:15.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:14 smithi149 ceph-mon[37173]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "406c082c-9b5f-4781-8cbf-3c04fa49e410"}]': finished 2024-07-27T03:11:15.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:14 smithi149 ceph-mon[37173]: osdmap e69: 12 total, 11 up, 12 in 2024-07-27T03:11:15.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-27T03:11:15.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:14 smithi149 ceph-mon[37173]: from='client.? 172.21.15.175:0/2117156267' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-07-27T03:11:17.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:16 smithi149 ceph-mon[37173]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-07-27T03:11:17.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:16 smithi175 ceph-mon[37125]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-07-27T03:11:17.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:16 smithi067 ceph-mon[32814]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-07-27T03:11:18.976 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:18 smithi175 ceph-mon[37125]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-07-27T03:11:19.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:18 smithi149 ceph-mon[37173]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-07-27T03:11:19.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:18 smithi067 ceph-mon[32814]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-07-27T03:11:21.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:20 smithi149 ceph-mon[37173]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-27T03:11:21.092 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:20 smithi175 ceph-mon[37125]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-27T03:11:21.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:20 smithi067 ceph-mon[32814]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-07-27T03:11:21.790 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-07-27T03:11:21.791 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:22.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-07-27T03:11:22.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:22.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-07-27T03:11:22.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:22.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:22 smithi175 ceph-mon[37125]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-27T03:11:22.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:22 smithi175 ceph-mon[37125]: Deploying daemon osd.11 on smithi175 2024-07-27T03:11:23.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:22 smithi149 ceph-mon[37173]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-27T03:11:23.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:22 smithi149 ceph-mon[37173]: Deploying daemon osd.11 on smithi175 2024-07-27T03:11:23.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:22 smithi067 ceph-mon[32814]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-07-27T03:11:23.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:22 smithi067 ceph-mon[32814]: Deploying daemon osd.11 on smithi175 2024-07-27T03:11:24.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:24 smithi175 ceph-mon[37125]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-07-27T03:11:25.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:24 smithi149 ceph-mon[37173]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-07-27T03:11:25.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:24 smithi067 ceph-mon[32814]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-07-27T03:11:27.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:26 smithi149 ceph-mon[37173]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-07-27T03:11:27.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:26 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:11:27.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:26 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:27.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:26 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:27.073 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:26 smithi175 ceph-mon[37125]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-07-27T03:11:27.073 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:26 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:11:27.073 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:26 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:27.073 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:26 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:27.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:26 smithi067 ceph-mon[32814]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-07-27T03:11:27.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:26 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:11:27.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:26 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:27.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:26 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:28.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:28 smithi175 ceph-mon[37125]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-07-27T03:11:29.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:28 smithi149 ceph-mon[37173]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-07-27T03:11:29.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:28 smithi067 ceph-mon[32814]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-07-27T03:11:30.061 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 11 on host 'smithi175' 2024-07-27T03:11:30.062 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:30.058+0000 7f5364ff9640 1 -- 172.21.15.175:0/4230477198 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f53680ff610 con 0x7f53540b3c50 2024-07-27T03:11:30.063 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:30.061+0000 7f536d5e0640 1 -- 172.21.15.175:0/4230477198 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f53540b3c50 msgr2=0x7f53540b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:30.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:30.061+0000 7f536d5e0640 1 --2- 172.21.15.175:0/4230477198 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f53540b3c50 0x7f53540b6160 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f53580058c0 tx=0x7f535800a4c0 comp rx=0 tx=0).stop 2024-07-27T03:11:30.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:30.061+0000 7f536d5e0640 1 -- 172.21.15.175:0/4230477198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5368103ae0 msgr2=0x7f536819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:30.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:30.061+0000 7f536d5e0640 1 --2- 172.21.15.175:0/4230477198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5368103ae0 0x7f536819abb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5360006010 tx=0x7f536002fc00 comp rx=0 tx=0).stop 2024-07-27T03:11:30.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:30.061+0000 7f536d5e0640 1 -- 172.21.15.175:0/4230477198 shutdown_connections 2024-07-27T03:11:30.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:30.061+0000 7f536d5e0640 1 --2- 172.21.15.175:0/4230477198 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f53540b3c50 0x7f53540b6160 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:30.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:30.061+0000 7f536d5e0640 1 --2- 172.21.15.175:0/4230477198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5368103ae0 0x7f536819abb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:30.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:30.061+0000 7f536d5e0640 1 --2- 172.21.15.175:0/4230477198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5368103140 0x7f536819a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:30.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:30.061+0000 7f536d5e0640 1 --2- 172.21.15.175:0/4230477198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53681020b0 0x7f53681999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:30.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:30.061+0000 7f536d5e0640 1 -- 172.21.15.175:0/4230477198 >> 172.21.15.175:0/4230477198 conn(0x7f53680fda20 msgr2=0x7f53680fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:11:30.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:30.061+0000 7f536d5e0640 1 -- 172.21.15.175:0/4230477198 shutdown_connections 2024-07-27T03:11:30.064 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:30.061+0000 7f536d5e0640 1 -- 172.21.15.175:0/4230477198 wait complete. 2024-07-27T03:11:30.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:30 smithi175 ceph-mon[37125]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-07-27T03:11:30.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:30.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:30.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:30.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:11:30.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:30.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:11:30.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:30.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:30.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:30 smithi067 ceph-mon[32814]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-07-27T03:11:30.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:30.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:30.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:30.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:11:30.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:30.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:11:30.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:30.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:30.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:30 smithi149 ceph-mon[37173]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-07-27T03:11:30.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:30.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:30.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:30.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:11:30.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:30.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:11:30.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:30.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:30.758 DEBUG:teuthology.orchestra.run.smithi175:osd.11> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.11.service 2024-07-27T03:11:30.761 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-07-27T03:11:30.761 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd stat -f json 2024-07-27T03:11:31.043 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:11:31.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:31 smithi067 ceph-mon[32814]: from='osd.11 [v2:172.21.15.175:6824/2565514213,v1:172.21.15.175:6825/2565514213]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-27T03:11:31.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:31 smithi067 ceph-mon[32814]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-27T03:11:31.622 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:31 smithi175 ceph-mon[37125]: from='osd.11 [v2:172.21.15.175:6824/2565514213,v1:172.21.15.175:6825/2565514213]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-27T03:11:31.622 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:31 smithi175 ceph-mon[37125]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-27T03:11:31.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:31 smithi149 ceph-mon[37173]: from='osd.11 [v2:172.21.15.175:6824/2565514213,v1:172.21.15.175:6825/2565514213]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-27T03:11:31.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:31 smithi149 ceph-mon[37173]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-07-27T03:11:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:32 smithi175 ceph-mon[37125]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-07-27T03:11:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:32 smithi175 ceph-mon[37125]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-07-27T03:11:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:32 smithi175 ceph-mon[37125]: osdmap e70: 12 total, 11 up, 12 in 2024-07-27T03:11:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:32 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-27T03:11:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:32 smithi175 ceph-mon[37125]: from='osd.11 [v2:172.21.15.175:6824/2565514213,v1:172.21.15.175:6825/2565514213]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:11:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:32 smithi175 ceph-mon[37125]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:11:32.626 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:11:32 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:11:32.318+0000 7f06efbfd640 -1 osd.11 0 waiting for initial osdmap 2024-07-27T03:11:32.626 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:11:32 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:11:32.331+0000 7f06eb9fb640 -1 osd.11 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-07-27T03:11:32.664 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:32 smithi067 ceph-mon[32814]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-07-27T03:11:32.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:32 smithi067 ceph-mon[32814]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-07-27T03:11:32.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:32 smithi067 ceph-mon[32814]: osdmap e70: 12 total, 11 up, 12 in 2024-07-27T03:11:32.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:32 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-27T03:11:32.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:32 smithi067 ceph-mon[32814]: from='osd.11 [v2:172.21.15.175:6824/2565514213,v1:172.21.15.175:6825/2565514213]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:11:32.665 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:32 smithi067 ceph-mon[32814]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:11:32.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:32 smithi149 ceph-mon[37173]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-07-27T03:11:32.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:32 smithi149 ceph-mon[37173]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-07-27T03:11:32.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:32 smithi149 ceph-mon[37173]: osdmap e70: 12 total, 11 up, 12 in 2024-07-27T03:11:32.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:32 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-27T03:11:32.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:32 smithi149 ceph-mon[37173]: from='osd.11 [v2:172.21.15.175:6824/2565514213,v1:172.21.15.175:6825/2565514213]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:11:32.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:32 smithi149 ceph-mon[37173]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-07-27T03:11:32.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.889+0000 7f8eee733640 1 -- 172.21.15.67:0/4103263515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ee8103170 msgr2=0x7f8ee8103580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:32.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.889+0000 7f8eee733640 1 --2- 172.21.15.67:0/4103263515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ee8103170 0x7f8ee8103580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8edc009900 tx=0x7f8edc02a480 comp rx=0 tx=0).stop 2024-07-27T03:11:32.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.889+0000 7f8eee733640 1 -- 172.21.15.67:0/4103263515 shutdown_connections 2024-07-27T03:11:32.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.889+0000 7f8eee733640 1 --2- 172.21.15.67:0/4103263515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ee8103b10 0x7f8ee810a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:32.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.889+0000 7f8eee733640 1 --2- 172.21.15.67:0/4103263515 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ee8103170 0x7f8ee8103580 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:32.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.889+0000 7f8eee733640 1 --2- 172.21.15.67:0/4103263515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ee81020e0 0x7f8ee8102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:32.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.889+0000 7f8eee733640 1 -- 172.21.15.67:0/4103263515 >> 172.21.15.67:0/4103263515 conn(0x7f8ee80fda50 msgr2=0x7f8ee80fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:11:32.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.889+0000 7f8eee733640 1 -- 172.21.15.67:0/4103263515 shutdown_connections 2024-07-27T03:11:32.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.889+0000 7f8eee733640 1 -- 172.21.15.67:0/4103263515 wait complete. 2024-07-27T03:11:32.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.890+0000 7f8eee733640 1 Processor -- start 2024-07-27T03:11:32.893 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.890+0000 7f8eee733640 1 -- start start 2024-07-27T03:11:32.895 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.890+0000 7f8eee733640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ee81020e0 0x7f8ee8199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:32.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.890+0000 7f8eee733640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ee8103170 0x7f8ee819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:32.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.890+0000 7f8eee733640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ee8103b10 0x7f8ee819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:32.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.890+0000 7f8eee733640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee810ee50 con 0x7f8ee8103b10 2024-07-27T03:11:32.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.890+0000 7f8eee733640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee810ecd0 con 0x7f8ee8103170 2024-07-27T03:11:32.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.890+0000 7f8eee733640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee810efd0 con 0x7f8ee81020e0 2024-07-27T03:11:32.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.890+0000 7f8eeccd0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ee8103b10 0x7f8ee819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:32.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.891+0000 7f8ee7fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ee81020e0 0x7f8ee8199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:32.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.891+0000 7f8ee77fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ee8103170 0x7f8ee819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:32.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.891+0000 7f8ee7fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ee81020e0 0x7f8ee8199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.67:53498/0 (socket says 172.21.15.67:53498) 2024-07-27T03:11:32.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.891+0000 7f8eeccd0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ee8103b10 0x7f8ee819ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38498/0 (socket says 172.21.15.67:38498) 2024-07-27T03:11:32.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.891+0000 7f8ee7fff640 1 -- 172.21.15.67:0/1611810393 learned_addr learned my addr 172.21.15.67:0/1611810393 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:11:32.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.891+0000 7f8ee7fff640 1 -- 172.21.15.67:0/1611810393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ee8103170 msgr2=0x7f8ee819a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:32.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.891+0000 7f8ee7fff640 1 --2- 172.21.15.67:0/1611810393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ee8103170 0x7f8ee819a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:32.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.891+0000 7f8ee7fff640 1 -- 172.21.15.67:0/1611810393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ee8103b10 msgr2=0x7f8ee819ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:32.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.891+0000 7f8ee7fff640 1 --2- 172.21.15.67:0/1611810393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ee8103b10 0x7f8ee819ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:32.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.891+0000 7f8ee7fff640 1 -- 172.21.15.67:0/1611810393 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee819b4a0 con 0x7f8ee81020e0 2024-07-27T03:11:32.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.891+0000 7f8eeccd0640 1 --2- 172.21.15.67:0/1611810393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ee8103b10 0x7f8ee819ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:11:32.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.891+0000 7f8ee7fff640 1 --2- 172.21.15.67:0/1611810393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ee81020e0 0x7f8ee8199a70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800d950 tx=0x7f8ed800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:11:32.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.891+0000 7f8ee57fa640 1 -- 172.21.15.67:0/1611810393 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8017070 con 0x7f8ee81020e0 2024-07-27T03:11:32.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.891+0000 7f8eee733640 1 -- 172.21.15.67:0/1611810393 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee819b790 con 0x7f8ee81020e0 2024-07-27T03:11:32.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.891+0000 7f8ee57fa640 1 -- 172.21.15.67:0/1611810393 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ed8009c30 con 0x7f8ee81020e0 2024-07-27T03:11:32.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.891+0000 7f8ee57fa640 1 -- 172.21.15.67:0/1611810393 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed800f620 con 0x7f8ee81020e0 2024-07-27T03:11:32.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.891+0000 7f8eee733640 1 -- 172.21.15.67:0/1611810393 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee81a1e90 con 0x7f8ee81020e0 2024-07-27T03:11:32.898 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.893+0000 7f8eee733640 1 -- 172.21.15.67:0/1611810393 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ee8102550 con 0x7f8ee81020e0 2024-07-27T03:11:32.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.894+0000 7f8ee57fa640 1 -- 172.21.15.67:0/1611810393 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f8ed8005030 con 0x7f8ee81020e0 2024-07-27T03:11:32.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.895+0000 7f8ee57fa640 1 --2- 172.21.15.67:0/1611810393 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f8ed00b3bd0 0x7f8ed00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:32.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.895+0000 7f8ee77fe640 1 --2- 172.21.15.67:0/1611810393 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f8ed00b3bd0 0x7f8ed00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:32.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.895+0000 7f8ee57fa640 1 -- 172.21.15.67:0/1611810393 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6847+0+0 (secure 0 0 0) 0x7f8ed80e3200 con 0x7f8ee81020e0 2024-07-27T03:11:32.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.895+0000 7f8ee77fe640 1 --2- 172.21.15.67:0/1611810393 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f8ed00b3bd0 0x7f8ed00b60e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8edc000c00 tx=0x7f8edc031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:11:32.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:32.901+0000 7f8ee57fa640 1 -- 172.21.15.67:0/1611810393 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f8ed8009de0 con 0x7f8ee81020e0 2024-07-27T03:11:33.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:33.083+0000 7f8eee733640 1 -- 172.21.15.67:0/1611810393 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f8ee81092a0 con 0x7f8ee81020e0 2024-07-27T03:11:33.087 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:33.084+0000 7f8ee57fa640 1 -- 172.21.15.67:0/1611810393 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f8ee81092a0 con 0x7f8ee81020e0 2024-07-27T03:11:33.087 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:11:33.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:33.086+0000 7f8eee733640 1 -- 172.21.15.67:0/1611810393 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f8ed00b3bd0 msgr2=0x7f8ed00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:33.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:33.086+0000 7f8eee733640 1 --2- 172.21.15.67:0/1611810393 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f8ed00b3bd0 0x7f8ed00b60e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8edc000c00 tx=0x7f8edc031400 comp rx=0 tx=0).stop 2024-07-27T03:11:33.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:33.086+0000 7f8eee733640 1 -- 172.21.15.67:0/1611810393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ee81020e0 msgr2=0x7f8ee8199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:33.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:33.086+0000 7f8eee733640 1 --2- 172.21.15.67:0/1611810393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ee81020e0 0x7f8ee8199a70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800d950 tx=0x7f8ed800de10 comp rx=0 tx=0).stop 2024-07-27T03:11:33.089 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:33.086+0000 7f8eee733640 1 -- 172.21.15.67:0/1611810393 shutdown_connections 2024-07-27T03:11:33.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:33.086+0000 7f8eee733640 1 --2- 172.21.15.67:0/1611810393 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f8ed00b3bd0 0x7f8ed00b60e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:33.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:33.086+0000 7f8eee733640 1 --2- 172.21.15.67:0/1611810393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ee8103b10 0x7f8ee819ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:33.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:33.086+0000 7f8eee733640 1 --2- 172.21.15.67:0/1611810393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ee8103170 0x7f8ee819a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:33.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:33.086+0000 7f8eee733640 1 --2- 172.21.15.67:0/1611810393 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ee81020e0 0x7f8ee8199a70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:33.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:33.087+0000 7f8eee733640 1 -- 172.21.15.67:0/1611810393 >> 172.21.15.67:0/1611810393 conn(0x7f8ee80fda50 msgr2=0x7f8ee8106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:11:33.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:33.087+0000 7f8eee733640 1 -- 172.21.15.67:0/1611810393 shutdown_connections 2024-07-27T03:11:33.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:33.087+0000 7f8eee733640 1 -- 172.21.15.67:0/1611810393 wait complete. 2024-07-27T03:11:33.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:33 smithi067 ceph-mon[32814]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-27T03:11:33.395 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:33 smithi067 ceph-mon[32814]: osdmap e71: 12 total, 11 up, 12 in 2024-07-27T03:11:33.395 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:33 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-27T03:11:33.395 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:33 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-27T03:11:33.395 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:33 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1611810393' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-27T03:11:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:33 smithi175 ceph-mon[37125]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-27T03:11:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:33 smithi175 ceph-mon[37125]: osdmap e71: 12 total, 11 up, 12 in 2024-07-27T03:11:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:33 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-27T03:11:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:33 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-27T03:11:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:33 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1611810393' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-27T03:11:33.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:33 smithi149 ceph-mon[37173]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-07-27T03:11:33.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:33 smithi149 ceph-mon[37173]: osdmap e71: 12 total, 11 up, 12 in 2024-07-27T03:11:33.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:33 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-27T03:11:33.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:33 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-27T03:11:33.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:33 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1611810393' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-27T03:11:33.773 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1722049869,"num_in_osds":12,"osd_in_since":1722049874,"num_remapped_pgs":0} 2024-07-27T03:11:34.388 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:34 smithi175 ceph-mon[37125]: purged_snaps scrub starts 2024-07-27T03:11:34.388 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:34 smithi175 ceph-mon[37125]: purged_snaps scrub ok 2024-07-27T03:11:34.388 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:34 smithi175 ceph-mon[37125]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-07-27T03:11:34.388 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-27T03:11:34.388 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:34 smithi175 ceph-mon[37125]: osd.11 [v2:172.21.15.175:6824/2565514213,v1:172.21.15.175:6825/2565514213] boot 2024-07-27T03:11:34.388 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:34 smithi175 ceph-mon[37125]: osdmap e72: 12 total, 12 up, 12 in 2024-07-27T03:11:34.389 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-27T03:11:34.389 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:34.389 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:34.389 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:34.389 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:34.389 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:34.389 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:34.389 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:34.389 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:34.389 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:11:34.389 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:34.652 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:34 smithi149 ceph-mon[37173]: purged_snaps scrub starts 2024-07-27T03:11:34.652 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:34 smithi149 ceph-mon[37173]: purged_snaps scrub ok 2024-07-27T03:11:34.652 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:34 smithi149 ceph-mon[37173]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-07-27T03:11:34.652 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-27T03:11:34.652 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:34 smithi149 ceph-mon[37173]: osd.11 [v2:172.21.15.175:6824/2565514213,v1:172.21.15.175:6825/2565514213] boot 2024-07-27T03:11:34.652 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:34 smithi149 ceph-mon[37173]: osdmap e72: 12 total, 12 up, 12 in 2024-07-27T03:11:34.652 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-27T03:11:34.652 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:34.653 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:34.653 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:34.653 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:34.653 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:34.653 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:34.653 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:34.653 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:34.653 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:11:34.653 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:34.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:34 smithi067 ceph-mon[32814]: purged_snaps scrub starts 2024-07-27T03:11:34.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:34 smithi067 ceph-mon[32814]: purged_snaps scrub ok 2024-07-27T03:11:34.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:34 smithi067 ceph-mon[32814]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-07-27T03:11:34.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-27T03:11:34.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:34 smithi067 ceph-mon[32814]: osd.11 [v2:172.21.15.175:6824/2565514213,v1:172.21.15.175:6825/2565514213] boot 2024-07-27T03:11:34.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:34 smithi067 ceph-mon[32814]: osdmap e72: 12 total, 12 up, 12 in 2024-07-27T03:11:34.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-07-27T03:11:34.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:34.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:34.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:34.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:34.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:34.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-07-27T03:11:34.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:34.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:34.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:11:34.722 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:34.775 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd stat -f json 2024-07-27T03:11:35.039 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:11:35.416 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:35 smithi067 ceph-mon[32814]: Detected new or changed devices on smithi175 2024-07-27T03:11:35.416 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:35 smithi067 ceph-mon[32814]: Adjusting osd_memory_target on smithi175 to 4272M 2024-07-27T03:11:35.416 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:35 smithi067 ceph-mon[32814]: osdmap e73: 12 total, 12 up, 12 in 2024-07-27T03:11:35.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:35 smithi175 ceph-mon[37125]: Detected new or changed devices on smithi175 2024-07-27T03:11:35.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:35 smithi175 ceph-mon[37125]: Adjusting osd_memory_target on smithi175 to 4272M 2024-07-27T03:11:35.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:35 smithi175 ceph-mon[37125]: osdmap e73: 12 total, 12 up, 12 in 2024-07-27T03:11:35.669 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:35 smithi149 ceph-mon[37173]: Detected new or changed devices on smithi175 2024-07-27T03:11:35.669 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:35 smithi149 ceph-mon[37173]: Adjusting osd_memory_target on smithi175 to 4272M 2024-07-27T03:11:35.669 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:35 smithi149 ceph-mon[37173]: osdmap e73: 12 total, 12 up, 12 in 2024-07-27T03:11:36.604 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:36 smithi067 ceph-mon[32814]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:36.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:36 smithi175 ceph-mon[37125]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:36.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:36 smithi149 ceph-mon[37173]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:36.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.819+0000 7fc518c6e640 1 -- 172.21.15.67:0/3382021433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5141031e0 msgr2=0x7fc514103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:36.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.819+0000 7fc518c6e640 1 --2- 172.21.15.67:0/3382021433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5141031e0 0x7fc514103650 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc504009ca0 tx=0x7fc50402a950 comp rx=0 tx=0).stop 2024-07-27T03:11:36.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.820+0000 7fc518c6e640 1 -- 172.21.15.67:0/3382021433 shutdown_connections 2024-07-27T03:11:36.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.820+0000 7fc518c6e640 1 --2- 172.21.15.67:0/3382021433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc514104c10 0x7fc51410b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:36.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.820+0000 7fc518c6e640 1 --2- 172.21.15.67:0/3382021433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc514104270 0x7fc514104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:36.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.820+0000 7fc518c6e640 1 --2- 172.21.15.67:0/3382021433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc5141031e0 0x7fc514103650 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:36.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.820+0000 7fc518c6e640 1 -- 172.21.15.67:0/3382021433 >> 172.21.15.67:0/3382021433 conn(0x7fc5140feb30 msgr2=0x7fc514101000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:11:36.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.820+0000 7fc518c6e640 1 -- 172.21.15.67:0/3382021433 shutdown_connections 2024-07-27T03:11:36.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.820+0000 7fc518c6e640 1 -- 172.21.15.67:0/3382021433 wait complete. 2024-07-27T03:11:36.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.820+0000 7fc518c6e640 1 Processor -- start 2024-07-27T03:11:36.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.821+0000 7fc518c6e640 1 -- start start 2024-07-27T03:11:36.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.821+0000 7fc518c6e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5141031e0 0x7fc514199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:36.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.821+0000 7fc518c6e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc514104270 0x7fc51419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:36.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.821+0000 7fc518c6e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc514104c10 0x7fc51419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:36.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.821+0000 7fc518c6e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc51410ee40 con 0x7fc514104c10 2024-07-27T03:11:36.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.821+0000 7fc518c6e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc51410ecc0 con 0x7fc514104270 2024-07-27T03:11:36.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.821+0000 7fc518c6e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc51410efc0 con 0x7fc5141031e0 2024-07-27T03:11:36.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.821+0000 7fc512d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc514104c10 0x7fc51419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:36.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.821+0000 7fc512d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc514104c10 0x7fc51419ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54860/0 (socket says 172.21.15.67:54860) 2024-07-27T03:11:36.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.821+0000 7fc512d9d640 1 -- 172.21.15.67:0/1138277350 learned_addr learned my addr 172.21.15.67:0/1138277350 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:11:36.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.821+0000 7fc511d9b640 1 --2- 172.21.15.67:0/1138277350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc514104270 0x7fc51419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:36.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.821+0000 7fc51259c640 1 --2- 172.21.15.67:0/1138277350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5141031e0 0x7fc514199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:36.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.821+0000 7fc512d9d640 1 -- 172.21.15.67:0/1138277350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5141031e0 msgr2=0x7fc514199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:36.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.821+0000 7fc512d9d640 1 --2- 172.21.15.67:0/1138277350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5141031e0 0x7fc514199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:36.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.821+0000 7fc512d9d640 1 -- 172.21.15.67:0/1138277350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc514104270 msgr2=0x7fc51419a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:36.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.821+0000 7fc512d9d640 1 --2- 172.21.15.67:0/1138277350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc514104270 0x7fc51419a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:36.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.821+0000 7fc512d9d640 1 -- 172.21.15.67:0/1138277350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51419b440 con 0x7fc514104c10 2024-07-27T03:11:36.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.821+0000 7fc512d9d640 1 --2- 172.21.15.67:0/1138277350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc514104c10 0x7fc51419ac20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00c970 tx=0x7fc50c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:11:36.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.822+0000 7fc50b7fe640 1 -- 172.21.15.67:0/1138277350 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c007bb0 con 0x7fc514104c10 2024-07-27T03:11:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.822+0000 7fc50b7fe640 1 -- 172.21.15.67:0/1138277350 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc50c007d50 con 0x7fc514104c10 2024-07-27T03:11:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.822+0000 7fc518c6e640 1 -- 172.21.15.67:0/1138277350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc51419b730 con 0x7fc514104c10 2024-07-27T03:11:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.822+0000 7fc50b7fe640 1 -- 172.21.15.67:0/1138277350 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c004e60 con 0x7fc514104c10 2024-07-27T03:11:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.822+0000 7fc518c6e640 1 -- 172.21.15.67:0/1138277350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5141a1e30 con 0x7fc514104c10 2024-07-27T03:11:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.823+0000 7fc518c6e640 1 -- 172.21.15.67:0/1138277350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc51410a3c0 con 0x7fc514104c10 2024-07-27T03:11:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.824+0000 7fc50b7fe640 1 -- 172.21.15.67:0/1138277350 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7fc50c005000 con 0x7fc514104c10 2024-07-27T03:11:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.825+0000 7fc50b7fe640 1 --2- 172.21.15.67:0/1138277350 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc5000b3c50 0x7fc5000b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.826+0000 7fc51259c640 1 --2- 172.21.15.67:0/1138277350 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc5000b3c50 0x7fc5000b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.826+0000 7fc50b7fe640 1 -- 172.21.15.67:0/1138277350 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fc50c0e36f0 con 0x7fc514104c10 2024-07-27T03:11:36.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.826+0000 7fc51259c640 1 --2- 172.21.15.67:0/1138277350 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc5000b3c50 0x7fc5000b6160 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc50402f8d0 tx=0x7fc504009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:11:36.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:36.829+0000 7fc50b7fe640 1 -- 172.21.15.67:0/1138277350 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fc50c014030 con 0x7fc514104c10 2024-07-27T03:11:37.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:37.012+0000 7fc518c6e640 1 -- 172.21.15.67:0/1138277350 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc514063a30 con 0x7fc514104c10 2024-07-27T03:11:37.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:37.013+0000 7fc50b7fe640 1 -- 172.21.15.67:0/1138277350 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7fc50c0aab70 con 0x7fc514104c10 2024-07-27T03:11:37.016 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:11:37.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:37.015+0000 7fc518c6e640 1 -- 172.21.15.67:0/1138277350 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc5000b3c50 msgr2=0x7fc5000b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:37.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:37.015+0000 7fc518c6e640 1 --2- 172.21.15.67:0/1138277350 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc5000b3c50 0x7fc5000b6160 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc50402f8d0 tx=0x7fc504009360 comp rx=0 tx=0).stop 2024-07-27T03:11:37.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:37.015+0000 7fc518c6e640 1 -- 172.21.15.67:0/1138277350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc514104c10 msgr2=0x7fc51419ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:37.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:37.015+0000 7fc518c6e640 1 --2- 172.21.15.67:0/1138277350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc514104c10 0x7fc51419ac20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00c970 tx=0x7fc50c00ce30 comp rx=0 tx=0).stop 2024-07-27T03:11:37.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:37.015+0000 7fc518c6e640 1 -- 172.21.15.67:0/1138277350 shutdown_connections 2024-07-27T03:11:37.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:37.015+0000 7fc518c6e640 1 --2- 172.21.15.67:0/1138277350 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc5000b3c50 0x7fc5000b6160 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:37.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:37.015+0000 7fc518c6e640 1 --2- 172.21.15.67:0/1138277350 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc514104c10 0x7fc51419ac20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:37.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:37.015+0000 7fc518c6e640 1 --2- 172.21.15.67:0/1138277350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc514104270 0x7fc51419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:37.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:37.015+0000 7fc518c6e640 1 --2- 172.21.15.67:0/1138277350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5141031e0 0x7fc514199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:37.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:37.015+0000 7fc518c6e640 1 -- 172.21.15.67:0/1138277350 >> 172.21.15.67:0/1138277350 conn(0x7fc5140feb30 msgr2=0x7fc514107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:11:37.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:37.015+0000 7fc518c6e640 1 -- 172.21.15.67:0/1138277350 shutdown_connections 2024-07-27T03:11:37.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:37.015+0000 7fc518c6e640 1 -- 172.21.15.67:0/1138277350 wait complete. 2024-07-27T03:11:37.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:37 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1138277350' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-27T03:11:37.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:37 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1138277350' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-27T03:11:37.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:37 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1138277350' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-07-27T03:11:37.790 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":73,"num_osds":12,"num_up_osds":12,"osd_up_since":1722049893,"num_in_osds":12,"osd_in_since":1722049874,"num_remapped_pgs":0} 2024-07-27T03:11:37.791 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd dump --format=json 2024-07-27T03:11:38.057 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:11:38.438 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:38 smithi067 ceph-mon[32814]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:38.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:38 smithi175 ceph-mon[37125]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:38.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:38 smithi149 ceph-mon[37173]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:39.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.705+0000 7f189eb3d640 1 -- 172.21.15.67:0/1232868019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1898102550 msgr2=0x7f1898102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:39.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.705+0000 7f189eb3d640 1 --2- 172.21.15.67:0/1232868019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1898102550 0x7f1898102960 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f188c009900 tx=0x7f188c02a410 comp rx=0 tx=0).stop 2024-07-27T03:11:39.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.705+0000 7f189eb3d640 1 -- 172.21.15.67:0/1232868019 shutdown_connections 2024-07-27T03:11:39.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.705+0000 7f189eb3d640 1 --2- 172.21.15.67:0/1232868019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1898108790 0x7f189810efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:39.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.705+0000 7f189eb3d640 1 --2- 172.21.15.67:0/1232868019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1898107e40 0x7f1898108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:39.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.705+0000 7f189eb3d640 1 --2- 172.21.15.67:0/1232868019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1898102550 0x7f1898102960 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:39.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.705+0000 7f189eb3d640 1 -- 172.21.15.67:0/1232868019 >> 172.21.15.67:0/1232868019 conn(0x7f18980791d0 msgr2=0x7f18980795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:11:39.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.705+0000 7f189eb3d640 1 -- 172.21.15.67:0/1232868019 shutdown_connections 2024-07-27T03:11:39.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.705+0000 7f189eb3d640 1 -- 172.21.15.67:0/1232868019 wait complete. 2024-07-27T03:11:39.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.706+0000 7f189eb3d640 1 Processor -- start 2024-07-27T03:11:39.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.706+0000 7f189eb3d640 1 -- start start 2024-07-27T03:11:39.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.706+0000 7f189eb3d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1898102550 0x7f1898199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:39.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.706+0000 7f189eb3d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1898107e40 0x7f189819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:39.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.706+0000 7f189eb3d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1898108790 0x7f189819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:39.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.706+0000 7f189eb3d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1898112460 con 0x7f1898108790 2024-07-27T03:11:39.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.706+0000 7f189eb3d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f18981122e0 con 0x7f1898107e40 2024-07-27T03:11:39.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.706+0000 7f189eb3d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18981125e0 con 0x7f1898102550 2024-07-27T03:11:39.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.706+0000 7f189d0da640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1898108790 0x7f189819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:39.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.706+0000 7f189d0da640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1898108790 0x7f189819f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54878/0 (socket says 172.21.15.67:54878) 2024-07-27T03:11:39.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.706+0000 7f189d0da640 1 -- 172.21.15.67:0/401183050 learned_addr learned my addr 172.21.15.67:0/401183050 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:11:39.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.706+0000 7f189c8d9640 1 --2- 172.21.15.67:0/401183050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1898102550 0x7f1898199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.706+0000 7f1897fff640 1 --2- 172.21.15.67:0/401183050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1898107e40 0x7f189819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.706+0000 7f189d0da640 1 -- 172.21.15.67:0/401183050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1898102550 msgr2=0x7f1898199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.706+0000 7f189d0da640 1 --2- 172.21.15.67:0/401183050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1898102550 0x7f1898199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.706+0000 7f189d0da640 1 -- 172.21.15.67:0/401183050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1898107e40 msgr2=0x7f189819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.706+0000 7f189d0da640 1 --2- 172.21.15.67:0/401183050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1898107e40 0x7f189819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.706+0000 7f189d0da640 1 -- 172.21.15.67:0/401183050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189819fe30 con 0x7f1898108790 2024-07-27T03:11:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.707+0000 7f189d0da640 1 --2- 172.21.15.67:0/401183050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1898108790 0x7f189819f700 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f189000eaa0 tx=0x7f189000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:11:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.707+0000 7f1895ffb640 1 -- 172.21.15.67:0/401183050 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189000cdb0 con 0x7f1898108790 2024-07-27T03:11:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.707+0000 7f1895ffb640 1 -- 172.21.15.67:0/401183050 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1890005020 con 0x7f1898108790 2024-07-27T03:11:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.707+0000 7f1895ffb640 1 -- 172.21.15.67:0/401183050 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18900116d0 con 0x7f1898108790 2024-07-27T03:11:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.707+0000 7f189eb3d640 1 -- 172.21.15.67:0/401183050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18981a0120 con 0x7f1898108790 2024-07-27T03:11:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.707+0000 7f189eb3d640 1 -- 172.21.15.67:0/401183050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18981a6880 con 0x7f1898108790 2024-07-27T03:11:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.708+0000 7f189eb3d640 1 -- 172.21.15.67:0/401183050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1898103780 con 0x7f1898108790 2024-07-27T03:11:39.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.709+0000 7f1895ffb640 1 -- 172.21.15.67:0/401183050 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f18900051d0 con 0x7f1898108790 2024-07-27T03:11:39.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.710+0000 7f1895ffb640 1 --2- 172.21.15.67:0/401183050 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f18880b3bd0 0x7f18880b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:39.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.710+0000 7f189c8d9640 1 --2- 172.21.15.67:0/401183050 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f18880b3bd0 0x7f18880b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:39.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.710+0000 7f1895ffb640 1 -- 172.21.15.67:0/401183050 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f18900e38e0 con 0x7f1898108790 2024-07-27T03:11:39.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.711+0000 7f189c8d9640 1 --2- 172.21.15.67:0/401183050 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f18880b3bd0 0x7f18880b60e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f188c005ab0 tx=0x7f188c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:11:39.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.715+0000 7f1895ffb640 1 -- 172.21.15.67:0/401183050 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f1890016030 con 0x7f1898108790 2024-07-27T03:11:39.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.899+0000 7f189eb3d640 1 -- 172.21.15.67:0/401183050 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f189819aff0 con 0x7f1898108790 2024-07-27T03:11:39.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.900+0000 7f1895ffb640 1 -- 172.21.15.67:0/401183050 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+18492 (secure 0 0 0) 0x7f18900ab3d0 con 0x7f1898108790 2024-07-27T03:11:39.904 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:11:39.904 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":73,"fsid":"c0782a00-4bc4-11ef-bcc7-c7b262605968","created":"2024-07-27T03:04:08.096824+0000","modified":"2024-07-27T03:11:34.321550+0000","last_up_change":"2024-07-27T03:11:33.314800+0000","last_in_change":"2024-07-27T03:11:14.028095+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-27T03:08:01.042479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9f614272-18e8-4562-812c-4f43e40efd2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6803","nonce":2648129674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6805","nonce":2648129674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6809","nonce":2648129674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6807","nonce":2648129674}]},"public_addr":"172.21.15.67:6803/2648129674","cluster_addr":"172.21.15.67:6805/2648129674","heartbeat_back_addr":"172.21.15.67:6809/2648129674","heartbeat_front_addr":"172.21.15.67:6807/2648129674","state":["exists","up"]},{"osd":1,"uuid":"f59f5879-163a-4269-bbe7-1200eea65341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6801","nonce":2561753152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6803","nonce":2561753152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6807","nonce":2561753152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6805","nonce":2561753152}]},"public_addr":"172.21.15.149:6801/2561753152","cluster_addr":"172.21.15.149:6803/2561753152","heartbeat_back_addr":"172.21.15.149:6807/2561753152","heartbeat_front_addr":"172.21.15.149:6805/2561753152","state":["exists","up"]},{"osd":2,"uuid":"444e0ebe-7ca3-4be9-8bd5-db859f824445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6801","nonce":3770940078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6803","nonce":3770940078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6807","nonce":3770940078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6805","nonce":3770940078}]},"public_addr":"172.21.15.175:6801/3770940078","cluster_addr":"172.21.15.175:6803/3770940078","heartbeat_back_addr":"172.21.15.175:6807/3770940078","heartbeat_front_addr":"172.21.15.175:6805/3770940078","state":["exists","up"]},{"osd":3,"uuid":"d554db3d-9604-427c-8736-b1bb379f8d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6811","nonce":3159144693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6813","nonce":3159144693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6817","nonce":3159144693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6815","nonce":3159144693}]},"public_addr":"172.21.15.67:6811/3159144693","cluster_addr":"172.21.15.67:6813/3159144693","heartbeat_back_addr":"172.21.15.67:6817/3159144693","heartbeat_front_addr":"172.21.15.67:6815/3159144693","state":["exists","up"]},{"osd":4,"uuid":"c317dbe5-3b49-47a6-8408-7042a3dac345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3393503532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3393503532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3393503532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3393503532}]},"public_addr":"172.21.15.149:6809/3393503532","cluster_addr":"172.21.15.149:6811/3393503532","heartbeat_back_addr":"172.21.15.149:6815/3393503532","heartbeat_front_addr":"172.21.15.149:6813/3393503532","state":["exists","up"]},{"osd":5,"uuid":"5c85abd6-2b42-4957-a9b7-59a9ee36b841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6809","nonce":2291204784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6811","nonce":2291204784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6815","nonce":2291204784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6813","nonce":2291204784}]},"public_addr":"172.21.15.175:6809/2291204784","cluster_addr":"172.21.15.175:6811/2291204784","heartbeat_back_addr":"172.21.15.175:6815/2291204784","heartbeat_front_addr":"172.21.15.175:6813/2291204784","state":["exists","up"]},{"osd":6,"uuid":"bf6c277d-ec24-4470-98de-cf6f28876ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6819","nonce":2660182879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6821","nonce":2660182879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6825","nonce":2660182879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6823","nonce":2660182879}]},"public_addr":"172.21.15.67:6819/2660182879","cluster_addr":"172.21.15.67:6821/2660182879","heartbeat_back_addr":"172.21.15.67:6825/2660182879","heartbeat_front_addr":"172.21.15.67:6823/2660182879","state":["exists","up"]},{"osd":7,"uuid":"fea2c7cf-d5cb-45ae-9b29-f6e56c903891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6817","nonce":2289624404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6819","nonce":2289624404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6823","nonce":2289624404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6821","nonce":2289624404}]},"public_addr":"172.21.15.149:6817/2289624404","cluster_addr":"172.21.15.149:6819/2289624404","heartbeat_back_addr":"172.21.15.149:6823/2289624404","heartbeat_front_addr":"172.21.15.149:6821/2289624404","state":["exists","up"]},{"osd":8,"uuid":"14d04d96-579a-4dc9-a4d4-50e4850b48e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6817","nonce":2286032466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6819","nonce":2286032466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6823","nonce":2286032466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6821","nonce":2286032466}]},"public_addr":"172.21.15.175:6817/2286032466","cluster_addr":"172.21.15.175:6819/2286032466","heartbeat_back_addr":"172.21.15.175:6823/2286032466","heartbeat_front_addr":"172.21.15.175:6821/2286032466","state":["exists","up"]},{"osd":9,"uuid":"b65dfb6b-1efe-4314-a8be-ce6a295e99fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6827","nonce":3411081513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6829","nonce":3411081513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6833","nonce":3411081513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6831","nonce":3411081513}]},"public_addr":"172.21.15.67:6827/3411081513","cluster_addr":"172.21.15.67:6829/3411081513","heartbeat_back_addr":"172.21.15.67:6833/3411081513","heartbeat_front_addr":"172.21.15.67:6831/3411081513","state":["exists","up"]},{"osd":10,"uuid":"e3dabae0-0d64-4222-984c-fc1cce883456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6825","nonce":2011495212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6827","nonce":2011495212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6831","nonce":2011495212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6829","nonce":2011495212}]},"public_addr":"172.21.15.149:6825/2011495212","cluster_addr":"172.21.15.149:6827/2011495212","heartbeat_back_addr":"172.21.15.149:6831/2011495212","heartbeat_front_addr":"172.21.15.149:6829/2011495212","state":["exists","up"]},{"osd":11,"uuid":"406c082c-9b5f-4781-8cbf-3c04fa49e410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6825","nonce":2565514213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6827","nonce":2565514213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6831","nonce":2565514213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6829","nonce":2565514213}]},"public_addr":"172.21.15.175:6825/2565514213","cluster_addr":"172.21.15.175:6827/2565514213","heartbeat_back_addr":"172.21.15.175:6831/2565514213","heartbeat_front_addr":"172.21.15.175:6829/2565514213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:11.399380+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:33.728518+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:56.411181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:19.871731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:42.994576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:06.712815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:31.321967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:55.070898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:18.686108+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:43.187795+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:07.318538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:31.374843+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/3477601235":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/4130679770":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6800/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/1029332005":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6801/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6801/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/284078926":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6800/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/2559121542":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/925406021":"2024-07-28T03:04:30.600273+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-27T03:11:39.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.903+0000 7f189eb3d640 1 -- 172.21.15.67:0/401183050 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f18880b3bd0 msgr2=0x7f18880b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:39.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.903+0000 7f189eb3d640 1 --2- 172.21.15.67:0/401183050 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f18880b3bd0 0x7f18880b60e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f188c005ab0 tx=0x7f188c005a20 comp rx=0 tx=0).stop 2024-07-27T03:11:39.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.903+0000 7f189eb3d640 1 -- 172.21.15.67:0/401183050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1898108790 msgr2=0x7f189819f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:39.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.903+0000 7f189eb3d640 1 --2- 172.21.15.67:0/401183050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1898108790 0x7f189819f700 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f189000eaa0 tx=0x7f189000ef60 comp rx=0 tx=0).stop 2024-07-27T03:11:39.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.903+0000 7f189eb3d640 1 -- 172.21.15.67:0/401183050 shutdown_connections 2024-07-27T03:11:39.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.903+0000 7f189eb3d640 1 --2- 172.21.15.67:0/401183050 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f18880b3bd0 0x7f18880b60e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:39.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.903+0000 7f189eb3d640 1 --2- 172.21.15.67:0/401183050 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1898108790 0x7f189819f700 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:39.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.903+0000 7f189eb3d640 1 --2- 172.21.15.67:0/401183050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1898107e40 0x7f189819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:39.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.903+0000 7f189eb3d640 1 --2- 172.21.15.67:0/401183050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1898102550 0x7f1898199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:39.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.903+0000 7f189eb3d640 1 -- 172.21.15.67:0/401183050 >> 172.21.15.67:0/401183050 conn(0x7f18980791d0 msgr2=0x7f1898106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:11:39.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.903+0000 7f189eb3d640 1 -- 172.21.15.67:0/401183050 shutdown_connections 2024-07-27T03:11:39.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:39.903+0000 7f189eb3d640 1 -- 172.21.15.67:0/401183050 wait complete. 2024-07-27T03:11:40.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:40 smithi067 ceph-mon[32814]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:40.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:40 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/401183050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:11:40.573 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-27T03:08:01.042479+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-07-27T03:11:40.573 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-07-27T03:11:40.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:40 smithi175 ceph-mon[37125]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:40.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:40 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/401183050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:11:40.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:40 smithi149 ceph-mon[37173]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:40.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:40 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/401183050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:11:40.840 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:11:42.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.582+0000 7f4ba8315640 1 -- 172.21.15.67:0/238591272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ba0107e40 msgr2=0x7f4ba0108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:42.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.582+0000 7f4ba8315640 1 --2- 172.21.15.67:0/238591272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ba0107e40 0x7f4ba0108250 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009900 tx=0x7f4b9002a410 comp rx=0 tx=0).stop 2024-07-27T03:11:42.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.582+0000 7f4ba8315640 1 -- 172.21.15.67:0/238591272 shutdown_connections 2024-07-27T03:11:42.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.582+0000 7f4ba8315640 1 --2- 172.21.15.67:0/238591272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ba0108790 0x7f4ba010efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:42.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.582+0000 7f4ba8315640 1 --2- 172.21.15.67:0/238591272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ba0107e40 0x7f4ba0108250 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:42.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.582+0000 7f4ba8315640 1 --2- 172.21.15.67:0/238591272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ba0102550 0x7f4ba0102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:42.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.582+0000 7f4ba8315640 1 -- 172.21.15.67:0/238591272 >> 172.21.15.67:0/238591272 conn(0x7f4ba00791d0 msgr2=0x7f4ba00795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:11:42.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.582+0000 7f4ba8315640 1 -- 172.21.15.67:0/238591272 shutdown_connections 2024-07-27T03:11:42.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.582+0000 7f4ba8315640 1 -- 172.21.15.67:0/238591272 wait complete. 2024-07-27T03:11:42.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.583+0000 7f4ba8315640 1 Processor -- start 2024-07-27T03:11:42.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.583+0000 7f4ba8315640 1 -- start start 2024-07-27T03:11:42.686 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:42 smithi175 ceph-mon[37125]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:42.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.583+0000 7f4ba8315640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ba0102550 0x7f4ba0199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:42.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.583+0000 7f4ba8315640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ba0107e40 0x7f4ba019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:42.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.583+0000 7f4ba8315640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ba0108790 0x7f4ba019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:42.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.583+0000 7f4ba8315640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba0112440 con 0x7f4ba0108790 2024-07-27T03:11:42.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.583+0000 7f4ba8315640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba01122c0 con 0x7f4ba0107e40 2024-07-27T03:11:42.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.583+0000 7f4ba8315640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba01125c0 con 0x7f4ba0102550 2024-07-27T03:11:42.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.583+0000 7f4ba68b2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ba0108790 0x7f4ba019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:42.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.584+0000 7f4ba58b0640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ba0107e40 0x7f4ba019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:42.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.584+0000 7f4ba68b2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ba0108790 0x7f4ba019f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54898/0 (socket says 172.21.15.67:54898) 2024-07-27T03:11:42.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.584+0000 7f4ba68b2640 1 -- 172.21.15.67:0/3046959058 learned_addr learned my addr 172.21.15.67:0/3046959058 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:11:42.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.584+0000 7f4ba60b1640 1 --2- 172.21.15.67:0/3046959058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ba0102550 0x7f4ba0199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:42.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.584+0000 7f4ba68b2640 1 -- 172.21.15.67:0/3046959058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ba0102550 msgr2=0x7f4ba0199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:42.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.584+0000 7f4ba68b2640 1 --2- 172.21.15.67:0/3046959058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ba0102550 0x7f4ba0199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:42.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.584+0000 7f4ba68b2640 1 -- 172.21.15.67:0/3046959058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ba0107e40 msgr2=0x7f4ba019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:42.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.584+0000 7f4ba68b2640 1 --2- 172.21.15.67:0/3046959058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ba0107e40 0x7f4ba019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:42.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.584+0000 7f4ba68b2640 1 -- 172.21.15.67:0/3046959058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba019ff50 con 0x7f4ba0108790 2024-07-27T03:11:42.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.584+0000 7f4ba60b1640 1 --2- 172.21.15.67:0/3046959058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ba0102550 0x7f4ba0199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:11:42.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.584+0000 7f4ba68b2640 1 --2- 172.21.15.67:0/3046959058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ba0108790 0x7f4ba019f820 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4b9800b970 tx=0x7f4b9800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:11:42.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.584+0000 7f4b977fe640 1 -- 172.21.15.67:0/3046959058 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b98019070 con 0x7f4ba0108790 2024-07-27T03:11:42.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.584+0000 7f4b977fe640 1 -- 172.21.15.67:0/3046959058 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b98014440 con 0x7f4ba0108790 2024-07-27T03:11:42.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.584+0000 7f4ba8315640 1 -- 172.21.15.67:0/3046959058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ba01a0240 con 0x7f4ba0108790 2024-07-27T03:11:42.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.584+0000 7f4b977fe640 1 -- 172.21.15.67:0/3046959058 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b980075d0 con 0x7f4ba0108790 2024-07-27T03:11:42.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.584+0000 7f4ba8315640 1 -- 172.21.15.67:0/3046959058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ba01a69a0 con 0x7f4ba0108790 2024-07-27T03:11:42.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.585+0000 7f4ba8315640 1 -- 172.21.15.67:0/3046959058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ba0103780 con 0x7f4ba0108790 2024-07-27T03:11:42.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.587+0000 7f4b977fe640 1 -- 172.21.15.67:0/3046959058 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f4b9801c070 con 0x7f4ba0108790 2024-07-27T03:11:42.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.588+0000 7f4b977fe640 1 --2- 172.21.15.67:0/3046959058 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4b8c0b3bd0 0x7f4b8c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:42.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.588+0000 7f4ba60b1640 1 --2- 172.21.15.67:0/3046959058 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4b8c0b3bd0 0x7f4b8c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:42.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.588+0000 7f4b977fe640 1 -- 172.21.15.67:0/3046959058 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f4b980e31a0 con 0x7f4ba0108790 2024-07-27T03:11:42.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.588+0000 7f4ba60b1640 1 --2- 172.21.15.67:0/3046959058 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4b8c0b3bd0 0x7f4b8c0b60e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c0058c0 tx=0x7f4b9c005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:11:42.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.591+0000 7f4b977fe640 1 -- 172.21.15.67:0/3046959058 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f4b980aa640 con 0x7f4ba0108790 2024-07-27T03:11:42.692 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:42 smithi067 ceph-mon[32814]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:42.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:42 smithi149 ceph-mon[37173]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:42.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.774+0000 7f4ba8315640 1 -- 172.21.15.67:0/3046959058 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4ba019b110 con 0x7f4ba0108790 2024-07-27T03:11:42.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.774+0000 7f4b977fe640 1 -- 172.21.15.67:0/3046959058 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7f4b980aeb90 con 0x7f4ba0108790 2024-07-27T03:11:42.777 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-07-27T03:11:42.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.776+0000 7f4ba8315640 1 -- 172.21.15.67:0/3046959058 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4b8c0b3bd0 msgr2=0x7f4b8c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:42.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.776+0000 7f4ba8315640 1 --2- 172.21.15.67:0/3046959058 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4b8c0b3bd0 0x7f4b8c0b60e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c0058c0 tx=0x7f4b9c005ec0 comp rx=0 tx=0).stop 2024-07-27T03:11:42.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.776+0000 7f4ba8315640 1 -- 172.21.15.67:0/3046959058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ba0108790 msgr2=0x7f4ba019f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:42.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.776+0000 7f4ba8315640 1 --2- 172.21.15.67:0/3046959058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ba0108790 0x7f4ba019f820 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4b9800b970 tx=0x7f4b9800be30 comp rx=0 tx=0).stop 2024-07-27T03:11:42.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.776+0000 7f4ba8315640 1 -- 172.21.15.67:0/3046959058 shutdown_connections 2024-07-27T03:11:42.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.776+0000 7f4ba8315640 1 --2- 172.21.15.67:0/3046959058 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4b8c0b3bd0 0x7f4b8c0b60e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:42.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.776+0000 7f4ba8315640 1 --2- 172.21.15.67:0/3046959058 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4ba0108790 0x7f4ba019f820 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:42.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.776+0000 7f4ba8315640 1 --2- 172.21.15.67:0/3046959058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ba0107e40 0x7f4ba019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:42.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.776+0000 7f4ba8315640 1 --2- 172.21.15.67:0/3046959058 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ba0102550 0x7f4ba0199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:42.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.776+0000 7f4ba8315640 1 -- 172.21.15.67:0/3046959058 >> 172.21.15.67:0/3046959058 conn(0x7f4ba00791d0 msgr2=0x7f4ba006b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:11:42.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.776+0000 7f4ba8315640 1 -- 172.21.15.67:0/3046959058 shutdown_connections 2024-07-27T03:11:42.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:42.776+0000 7f4ba8315640 1 -- 172.21.15.67:0/3046959058 wait complete. 2024-07-27T03:11:43.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:43 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3046959058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-27T03:11:43.556 INFO:tasks.cephadm:Adding mds.a on smithi067 2024-07-27T03:11:43.557 INFO:tasks.cephadm:Adding mds.d on smithi067 2024-07-27T03:11:43.557 INFO:tasks.cephadm:Adding mds.g on smithi067 2024-07-27T03:11:43.557 INFO:tasks.cephadm:Adding mds.j on smithi067 2024-07-27T03:11:43.557 INFO:tasks.cephadm:Adding mds.b on smithi149 2024-07-27T03:11:43.557 INFO:tasks.cephadm:Adding mds.e on smithi149 2024-07-27T03:11:43.558 INFO:tasks.cephadm:Adding mds.h on smithi149 2024-07-27T03:11:43.558 INFO:tasks.cephadm:Adding mds.k on smithi149 2024-07-27T03:11:43.558 INFO:tasks.cephadm:Adding mds.c on smithi175 2024-07-27T03:11:43.558 INFO:tasks.cephadm:Adding mds.f on smithi175 2024-07-27T03:11:43.558 INFO:tasks.cephadm:Adding mds.i on smithi175 2024-07-27T03:11:43.558 INFO:tasks.cephadm:Adding mds.l on smithi175 2024-07-27T03:11:43.558 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph orch apply mds all '12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l' 2024-07-27T03:11:43.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:43 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3046959058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-27T03:11:43.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:43 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3046959058' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-27T03:11:43.826 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.c/config 2024-07-27T03:11:44.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:44 smithi175 ceph-mon[37125]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:44.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:44 smithi067 ceph-mon[32814]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:44.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:44 smithi149 ceph-mon[37173]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:45.546 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.533+0000 7f3f188e7640 1 -- 172.21.15.175:0/3455743940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f1406b370 msgr2=0x7f3f14105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:45.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.533+0000 7f3f188e7640 1 --2- 172.21.15.175:0/3455743940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f1406b370 0x7f3f14105e80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009900 tx=0x7f3f0402a410 comp rx=0 tx=0).stop 2024-07-27T03:11:45.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.533+0000 7f3f188e7640 1 -- 172.21.15.175:0/3455743940 shutdown_connections 2024-07-27T03:11:45.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.533+0000 7f3f188e7640 1 --2- 172.21.15.175:0/3455743940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f14108f00 0x7f3f1410b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:45.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.533+0000 7f3f188e7640 1 --2- 172.21.15.175:0/3455743940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f141063c0 0x7f3f141087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:45.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.533+0000 7f3f188e7640 1 --2- 172.21.15.175:0/3455743940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f1406b370 0x7f3f14105e80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:45.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.533+0000 7f3f188e7640 1 -- 172.21.15.175:0/3455743940 >> 172.21.15.175:0/3455743940 conn(0x7f3f140fda20 msgr2=0x7f3f140ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:11:45.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.533+0000 7f3f188e7640 1 -- 172.21.15.175:0/3455743940 shutdown_connections 2024-07-27T03:11:45.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.533+0000 7f3f188e7640 1 -- 172.21.15.175:0/3455743940 wait complete. 2024-07-27T03:11:45.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.534+0000 7f3f188e7640 1 Processor -- start 2024-07-27T03:11:45.547 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.534+0000 7f3f188e7640 1 -- start start 2024-07-27T03:11:45.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.534+0000 7f3f188e7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f1406b370 0x7f3f14199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:45.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.534+0000 7f3f188e7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f141063c0 0x7f3f1419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:45.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.534+0000 7f3f188e7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f14108f00 0x7f3f1419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:45.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.534+0000 7f3f188e7640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1410ee00 con 0x7f3f14108f00 2024-07-27T03:11:45.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.534+0000 7f3f188e7640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1410ec80 con 0x7f3f1406b370 2024-07-27T03:11:45.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.534+0000 7f3f188e7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1410ef80 con 0x7f3f141063c0 2024-07-27T03:11:45.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.534+0000 7f3f11d9b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f141063c0 0x7f3f1419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:45.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.534+0000 7f3f11d9b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f141063c0 0x7f3f1419a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:56986/0 (socket says 172.21.15.175:56986) 2024-07-27T03:11:45.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.534+0000 7f3f11d9b640 1 -- 172.21.15.175:0/1446548516 learned_addr learned my addr 172.21.15.175:0/1446548516 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-07-27T03:11:45.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.534+0000 7f3f1259c640 1 --2- 172.21.15.175:0/1446548516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f1406b370 0x7f3f14199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:45.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.534+0000 7f3f12d9d640 1 --2- 172.21.15.175:0/1446548516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f14108f00 0x7f3f1419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:45.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.535+0000 7f3f11d9b640 1 -- 172.21.15.175:0/1446548516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f1406b370 msgr2=0x7f3f14199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:45.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.535+0000 7f3f11d9b640 1 --2- 172.21.15.175:0/1446548516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f1406b370 0x7f3f14199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:45.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.535+0000 7f3f11d9b640 1 -- 172.21.15.175:0/1446548516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f14108f00 msgr2=0x7f3f1419ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:45.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.535+0000 7f3f11d9b640 1 --2- 172.21.15.175:0/1446548516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f14108f00 0x7f3f1419ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:45.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.535+0000 7f3f11d9b640 1 -- 172.21.15.175:0/1446548516 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1419b430 con 0x7f3f141063c0 2024-07-27T03:11:45.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.535+0000 7f3f11d9b640 1 --2- 172.21.15.175:0/1446548516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f141063c0 0x7f3f1419a070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800e9f0 tx=0x7f3f0800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:11:45.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.535+0000 7f3f037fe640 1 -- 172.21.15.175:0/1446548516 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0800cd80 con 0x7f3f141063c0 2024-07-27T03:11:45.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.535+0000 7f3f188e7640 1 -- 172.21.15.175:0/1446548516 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1419b720 con 0x7f3f141063c0 2024-07-27T03:11:45.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.535+0000 7f3f037fe640 1 -- 172.21.15.175:0/1446548516 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f080108e0 con 0x7f3f141063c0 2024-07-27T03:11:45.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.535+0000 7f3f037fe640 1 -- 172.21.15.175:0/1446548516 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08018980 con 0x7f3f141063c0 2024-07-27T03:11:45.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.535+0000 7f3f188e7640 1 -- 172.21.15.175:0/1446548516 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f141a1e20 con 0x7f3f141063c0 2024-07-27T03:11:45.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.536+0000 7f3f188e7640 1 -- 172.21.15.175:0/1446548516 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f140ff540 con 0x7f3f141063c0 2024-07-27T03:11:45.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.537+0000 7f3f037fe640 1 -- 172.21.15.175:0/1446548516 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f3f08010430 con 0x7f3f141063c0 2024-07-27T03:11:45.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.538+0000 7f3f037fe640 1 --2- 172.21.15.175:0/1446548516 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3efc0b3bd0 0x7f3efc0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:45.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.538+0000 7f3f037fe640 1 -- 172.21.15.175:0/1446548516 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f3f080e3f30 con 0x7f3f141063c0 2024-07-27T03:11:45.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.538+0000 7f3f1259c640 1 --2- 172.21.15.175:0/1446548516 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3efc0b3bd0 0x7f3efc0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:45.549 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.539+0000 7f3f1259c640 1 --2- 172.21.15.175:0/1446548516 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3efc0b3bd0 0x7f3efc0b60e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3f04000c00 tx=0x7f3f04031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:11:45.550 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.541+0000 7f3f037fe640 1 -- 172.21.15.175:0/1446548516 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f3f080ab3d0 con 0x7f3f141063c0 2024-07-27T03:11:45.728 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.724+0000 7f3f188e7640 1 -- 172.21.15.175:0/1446548516 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l", "target": ["mon-mgr", ""]}) -- 0x7f3f14063370 con 0x7f3efc0b3bd0 2024-07-27T03:11:45.735 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.732+0000 7f3f037fe640 1 -- 172.21.15.175:0/1446548516 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f3f14063370 con 0x7f3efc0b3bd0 2024-07-27T03:11:45.736 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled mds.all update... 2024-07-27T03:11:45.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.734+0000 7f3f188e7640 1 -- 172.21.15.175:0/1446548516 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3efc0b3bd0 msgr2=0x7f3efc0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:45.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.734+0000 7f3f188e7640 1 --2- 172.21.15.175:0/1446548516 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3efc0b3bd0 0x7f3efc0b60e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3f04000c00 tx=0x7f3f04031400 comp rx=0 tx=0).stop 2024-07-27T03:11:45.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.734+0000 7f3f188e7640 1 -- 172.21.15.175:0/1446548516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f141063c0 msgr2=0x7f3f1419a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:45.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.734+0000 7f3f188e7640 1 --2- 172.21.15.175:0/1446548516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f141063c0 0x7f3f1419a070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800e9f0 tx=0x7f3f0800eeb0 comp rx=0 tx=0).stop 2024-07-27T03:11:45.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.734+0000 7f3f188e7640 1 -- 172.21.15.175:0/1446548516 shutdown_connections 2024-07-27T03:11:45.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.734+0000 7f3f188e7640 1 --2- 172.21.15.175:0/1446548516 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3efc0b3bd0 0x7f3efc0b60e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:45.738 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.734+0000 7f3f188e7640 1 --2- 172.21.15.175:0/1446548516 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3f14108f00 0x7f3f1419ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:45.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.734+0000 7f3f188e7640 1 --2- 172.21.15.175:0/1446548516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f141063c0 0x7f3f1419a070 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:45.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.734+0000 7f3f188e7640 1 --2- 172.21.15.175:0/1446548516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f1406b370 0x7f3f14199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:45.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.734+0000 7f3f188e7640 1 -- 172.21.15.175:0/1446548516 >> 172.21.15.175:0/1446548516 conn(0x7f3f140fda20 msgr2=0x7f3f14104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:11:45.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.734+0000 7f3f188e7640 1 -- 172.21.15.175:0/1446548516 shutdown_connections 2024-07-27T03:11:45.739 INFO:teuthology.orchestra.run.smithi175.stderr:2024-07-27T03:11:45.734+0000 7f3f188e7640 1 -- 172.21.15.175:0/1446548516 wait complete. 2024-07-27T03:11:46.427 DEBUG:teuthology.orchestra.run.smithi067:mds.a> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.a.service 2024-07-27T03:11:46.430 DEBUG:teuthology.orchestra.run.smithi067:mds.d> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.d.service 2024-07-27T03:11:46.472 DEBUG:teuthology.orchestra.run.smithi067:mds.g> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.g.service 2024-07-27T03:11:46.515 DEBUG:teuthology.orchestra.run.smithi067:mds.j> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.j.service 2024-07-27T03:11:46.559 DEBUG:teuthology.orchestra.run.smithi149:mds.b> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.b.service 2024-07-27T03:11:46.562 DEBUG:teuthology.orchestra.run.smithi149:mds.e> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.e.service 2024-07-27T03:11:46.606 DEBUG:teuthology.orchestra.run.smithi149:mds.h> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.h.service 2024-07-27T03:11:46.648 DEBUG:teuthology.orchestra.run.smithi149:mds.k> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.k.service 2024-07-27T03:11:46.691 DEBUG:teuthology.orchestra.run.smithi175:mds.c> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.c.service 2024-07-27T03:11:46.694 DEBUG:teuthology.orchestra.run.smithi175:mds.f> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.f.service 2024-07-27T03:11:46.738 DEBUG:teuthology.orchestra.run.smithi175:mds.i> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.i.service 2024-07-27T03:11:46.781 DEBUG:teuthology.orchestra.run.smithi175:mds.l> sudo journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.l.service 2024-07-27T03:11:46.824 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-07-27T03:11:46.825 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:11:46.908 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:46 smithi149 ceph-mon[37173]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:46.908 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:46 smithi149 ceph-mon[37173]: from='client.24520 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:11:46.908 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:46 smithi149 ceph-mon[37173]: Saving service mds.all spec with placement smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l;count:12 2024-07-27T03:11:46.908 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:46 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:46.908 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:46 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:11:46.909 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:46 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:46.909 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:46 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:11:46.909 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:46 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:46.909 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:46 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:11:46.909 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:46 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:11:46.909 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:46 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:47.085 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:11:47.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:46 smithi175 ceph-mon[37125]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:47.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:46 smithi175 ceph-mon[37125]: from='client.24520 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:11:47.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:46 smithi175 ceph-mon[37125]: Saving service mds.all spec with placement smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l;count:12 2024-07-27T03:11:47.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:46 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:47.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:46 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:11:47.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:46 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:47.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:46 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:11:47.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:46 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:47.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:46 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:11:47.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:46 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:11:47.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:46 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:47.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:46 smithi067 ceph-mon[32814]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:47.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:46 smithi067 ceph-mon[32814]: from='client.24520 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:11:47.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:46 smithi067 ceph-mon[32814]: Saving service mds.all spec with placement smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l;count:12 2024-07-27T03:11:47.208 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:46 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:47.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:46 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:11:47.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:46 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:47.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:46 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:11:47.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:46 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:47.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:46 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:11:47.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:46 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:11:47.209 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:46 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:47.844 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:47 smithi175 ceph-mon[37125]: Deploying daemon mds.l on smithi175 2024-07-27T03:11:47.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:47 smithi067 ceph-mon[32814]: Deploying daemon mds.l on smithi175 2024-07-27T03:11:48.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:47 smithi149 ceph-mon[37173]: Deploying daemon mds.l on smithi175 2024-07-27T03:11:48.125 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jul 27 03:11:48 smithi175 systemd[1]: Starting Ceph mds.l for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:11:48.519 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jul 27 03:11:48 smithi175 podman[66894]: 2024-07-27 03:11:48.244923365 +0000 UTC m=+0.112138143 container create c5738d3a87e5e5bd67334cbf61a45738faf18aaa0bd29c12417546a4f0019fac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf) 2024-07-27T03:11:48.519 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jul 27 03:11:48 smithi175 podman[66894]: 2024-07-27 03:11:48.150108445 +0000 UTC m=+0.017323235 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c 2024-07-27T03:11:48.519 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jul 27 03:11:48 smithi175 podman[66894]: 2024-07-27 03:11:48.481323426 +0000 UTC m=+0.348538211 container init c5738d3a87e5e5bd67334cbf61a45738faf18aaa0bd29c12417546a4f0019fac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-l, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240724, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-07-27T03:11:48.519 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jul 27 03:11:48 smithi175 podman[66894]: 2024-07-27 03:11:48.484505719 +0000 UTC m=+0.351720497 container start c5738d3a87e5e5bd67334cbf61a45738faf18aaa0bd29c12417546a4f0019fac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-l, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf) 2024-07-27T03:11:48.778 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:48 smithi067 ceph-mon[32814]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:48.778 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:48 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.175:6832/3804638701,v1:172.21.15.175:6833/3804638701] up:boot 2024-07-27T03:11:48.778 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:48 smithi067 ceph-mon[32814]: fsmap 1 up:standby 2024-07-27T03:11:48.778 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:48 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-27T03:11:48.778 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:48 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:48.778 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:48 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:48.778 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:48 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:48.778 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:48 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:11:48.778 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:48 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:11:48.779 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:48 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:48.875 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jul 27 03:11:48 smithi175 ceph-mds[66913]: -- 172.21.15.175:0/744755811 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5590ec779860 con 0x5590f091f600 2024-07-27T03:11:48.875 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jul 27 03:11:48 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-l[66909]: starting mds.l at 2024-07-27T03:11:48.875 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jul 27 03:11:48 smithi175 bash[66894]: c5738d3a87e5e5bd67334cbf61a45738faf18aaa0bd29c12417546a4f0019fac 2024-07-27T03:11:48.875 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jul 27 03:11:48 smithi175 systemd[1]: Started Ceph mds.l for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:11:48.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:48 smithi175 ceph-mon[37125]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:48.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:48 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.175:6832/3804638701,v1:172.21.15.175:6833/3804638701] up:boot 2024-07-27T03:11:48.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:48 smithi175 ceph-mon[37125]: fsmap 1 up:standby 2024-07-27T03:11:48.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:48 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-27T03:11:48.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:48 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:48.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:48 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:48.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:48 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:48.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:48 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:11:48.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:48 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:11:48.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:48 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.863+0000 7fb7e99ed640 1 -- 172.21.15.67:0/331480883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e4109880 msgr2=0x7fb7e41100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.863+0000 7fb7e99ed640 1 --2- 172.21.15.67:0/331480883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e4109880 0x7fb7e41100b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc009ca0 tx=0x7fb7dc02a8c0 comp rx=0 tx=0).stop 2024-07-27T03:11:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.863+0000 7fb7e99ed640 1 -- 172.21.15.67:0/331480883 shutdown_connections 2024-07-27T03:11:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.863+0000 7fb7e99ed640 1 --2- 172.21.15.67:0/331480883 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e4109880 0x7fb7e41100b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.863+0000 7fb7e99ed640 1 --2- 172.21.15.67:0/331480883 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7e4108f30 0x7fb7e4109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.863+0000 7fb7e99ed640 1 --2- 172.21.15.67:0/331480883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7e41035f0 0x7fb7e4103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:48.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.863+0000 7fb7e99ed640 1 -- 172.21.15.67:0/331480883 >> 172.21.15.67:0/331480883 conn(0x7fb7e40ff040 msgr2=0x7fb7e4101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:11:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.863+0000 7fb7e99ed640 1 -- 172.21.15.67:0/331480883 shutdown_connections 2024-07-27T03:11:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.863+0000 7fb7e99ed640 1 -- 172.21.15.67:0/331480883 wait complete. 2024-07-27T03:11:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.864+0000 7fb7e99ed640 1 Processor -- start 2024-07-27T03:11:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.864+0000 7fb7e99ed640 1 -- start start 2024-07-27T03:11:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.864+0000 7fb7e99ed640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e41035f0 0x7fb7e4199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.864+0000 7fb7e99ed640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7e4108f30 0x7fb7e419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.864+0000 7fb7e99ed640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7e4109880 0x7fb7e419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.864+0000 7fb7e99ed640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e4113530 con 0x7fb7e41035f0 2024-07-27T03:11:48.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.864+0000 7fb7e99ed640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e41133b0 con 0x7fb7e4109880 2024-07-27T03:11:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.864+0000 7fb7e99ed640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e41136b0 con 0x7fb7e4108f30 2024-07-27T03:11:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.865+0000 7fb7e2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e41035f0 0x7fb7e4199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.865+0000 7fb7e2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e41035f0 0x7fb7e4199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39582/0 (socket says 172.21.15.67:39582) 2024-07-27T03:11:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.865+0000 7fb7e2ffd640 1 -- 172.21.15.67:0/3775467931 learned_addr learned my addr 172.21.15.67:0/3775467931 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:11:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.865+0000 7fb7e37fe640 1 --2- 172.21.15.67:0/3775467931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7e4109880 0x7fb7e419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.865+0000 7fb7e27fc640 1 --2- 172.21.15.67:0/3775467931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7e4108f30 0x7fb7e419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.865+0000 7fb7e2ffd640 1 -- 172.21.15.67:0/3775467931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7e4108f30 msgr2=0x7fb7e419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.865+0000 7fb7e2ffd640 1 --2- 172.21.15.67:0/3775467931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7e4108f30 0x7fb7e419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.865+0000 7fb7e2ffd640 1 -- 172.21.15.67:0/3775467931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7e4109880 msgr2=0x7fb7e419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.865+0000 7fb7e2ffd640 1 --2- 172.21.15.67:0/3775467931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7e4109880 0x7fb7e419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.865+0000 7fb7e2ffd640 1 -- 172.21.15.67:0/3775467931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e419fe50 con 0x7fb7e41035f0 2024-07-27T03:11:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.865+0000 7fb7e2ffd640 1 --2- 172.21.15.67:0/3775467931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e41035f0 0x7fb7e4199ee0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb7d400d950 tx=0x7fb7d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:11:48.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.865+0000 7fb7e89eb640 1 -- 172.21.15.67:0/3775467931 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4017030 con 0x7fb7e41035f0 2024-07-27T03:11:48.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.865+0000 7fb7e99ed640 1 -- 172.21.15.67:0/3775467931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e41a0140 con 0x7fb7e41035f0 2024-07-27T03:11:48.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.865+0000 7fb7e89eb640 1 -- 172.21.15.67:0/3775467931 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7d4004020 con 0x7fb7e41035f0 2024-07-27T03:11:48.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.865+0000 7fb7e89eb640 1 -- 172.21.15.67:0/3775467931 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d40047a0 con 0x7fb7e41035f0 2024-07-27T03:11:48.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.865+0000 7fb7e99ed640 1 -- 172.21.15.67:0/3775467931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e41a68a0 con 0x7fb7e41035f0 2024-07-27T03:11:48.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.866+0000 7fb7e99ed640 1 -- 172.21.15.67:0/3775467931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7e4103a00 con 0x7fb7e41035f0 2024-07-27T03:11:48.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.868+0000 7fb7e89eb640 1 -- 172.21.15.67:0/3775467931 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7fb7d4009a30 con 0x7fb7e41035f0 2024-07-27T03:11:48.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.869+0000 7fb7e89eb640 1 --2- 172.21.15.67:0/3775467931 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb7cc0b3bd0 0x7fb7cc0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:48.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.869+0000 7fb7e27fc640 1 --2- 172.21.15.67:0/3775467931 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb7cc0b3bd0 0x7fb7cc0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:48.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.869+0000 7fb7e89eb640 1 -- 172.21.15.67:0/3775467931 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fb7d40e37d0 con 0x7fb7e41035f0 2024-07-27T03:11:48.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.869+0000 7fb7e27fc640 1 --2- 172.21.15.67:0/3775467931 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb7cc0b3bd0 0x7fb7cc0b60e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4107950 tx=0x7fb7d800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:11:48.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:48.872+0000 7fb7e89eb640 1 -- 172.21.15.67:0/3775467931 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fb7d40ab2c0 con 0x7fb7e41035f0 2024-07-27T03:11:49.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:48 smithi149 ceph-mon[37173]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:49.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:48 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.175:6832/3804638701,v1:172.21.15.175:6833/3804638701] up:boot 2024-07-27T03:11:49.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:48 smithi149 ceph-mon[37173]: fsmap 1 up:standby 2024-07-27T03:11:49.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:48 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-27T03:11:49.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:48 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:49.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:48 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:49.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:48 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:49.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:48 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:11:49.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:48 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:11:49.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:48 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:49.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:49.091+0000 7fb7e99ed640 1 -- 172.21.15.67:0/3775467931 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7e419b010 con 0x7fb7e41035f0 2024-07-27T03:11:49.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:49.091+0000 7fb7e89eb640 1 -- 172.21.15.67:0/3775467931 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7fb7d40aabc0 con 0x7fb7e41035f0 2024-07-27T03:11:49.095 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:11:49.095 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":2,"btime":"2024-07-27T03:11:48:522108+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24526,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-07-27T03:11:49.095 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 2 2024-07-27T03:11:49.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:49.093+0000 7fb7e99ed640 1 -- 172.21.15.67:0/3775467931 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb7cc0b3bd0 msgr2=0x7fb7cc0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:49.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:49.093+0000 7fb7e99ed640 1 --2- 172.21.15.67:0/3775467931 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb7cc0b3bd0 0x7fb7cc0b60e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4107950 tx=0x7fb7d800b410 comp rx=0 tx=0).stop 2024-07-27T03:11:49.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:49.094+0000 7fb7e99ed640 1 -- 172.21.15.67:0/3775467931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e41035f0 msgr2=0x7fb7e4199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:49.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:49.094+0000 7fb7e99ed640 1 --2- 172.21.15.67:0/3775467931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e41035f0 0x7fb7e4199ee0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb7d400d950 tx=0x7fb7d400de10 comp rx=0 tx=0).stop 2024-07-27T03:11:49.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:49.094+0000 7fb7e99ed640 1 -- 172.21.15.67:0/3775467931 shutdown_connections 2024-07-27T03:11:49.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:49.094+0000 7fb7e99ed640 1 --2- 172.21.15.67:0/3775467931 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb7cc0b3bd0 0x7fb7cc0b60e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:49.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:49.094+0000 7fb7e99ed640 1 --2- 172.21.15.67:0/3775467931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7e4109880 0x7fb7e419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:49.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:49.094+0000 7fb7e99ed640 1 --2- 172.21.15.67:0/3775467931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7e4108f30 0x7fb7e419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:49.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:49.094+0000 7fb7e99ed640 1 --2- 172.21.15.67:0/3775467931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb7e41035f0 0x7fb7e4199ee0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:49.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:49.094+0000 7fb7e99ed640 1 -- 172.21.15.67:0/3775467931 >> 172.21.15.67:0/3775467931 conn(0x7fb7e40ff040 msgr2=0x7fb7e4101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:11:49.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:49.094+0000 7fb7e99ed640 1 -- 172.21.15.67:0/3775467931 shutdown_connections 2024-07-27T03:11:49.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:49.094+0000 7fb7e99ed640 1 -- 172.21.15.67:0/3775467931 wait complete. 2024-07-27T03:11:50.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:49 smithi149 ceph-mon[37173]: Deploying daemon mds.k on smithi149 2024-07-27T03:11:50.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:49 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3775467931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:11:50.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:49 smithi175 ceph-mon[37125]: Deploying daemon mds.k on smithi149 2024-07-27T03:11:50.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:49 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3775467931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:11:50.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:49 smithi067 ceph-mon[32814]: Deploying daemon mds.k on smithi149 2024-07-27T03:11:50.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:49 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3775467931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:11:50.750 INFO:journalctl@ceph.mds.k.smithi149.stdout:Jul 27 03:11:50 smithi149 systemd[1]: Starting Ceph mds.k for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:11:51.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:50 smithi149 ceph-mon[37173]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:51.001 INFO:journalctl@ceph.mds.k.smithi149.stdout:Jul 27 03:11:50 smithi149 podman[66359]: 2024-07-27 03:11:50.655840156 +0000 UTC m=+0.017362675 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c 2024-07-27T03:11:51.001 INFO:journalctl@ceph.mds.k.smithi149.stdout:Jul 27 03:11:50 smithi149 podman[66359]: 2024-07-27 03:11:50.762497529 +0000 UTC m=+0.124020075 container create 15e7bc64f44a0d62fac47d8e090c6b46c0e8343719abe26cd2f9038791a6dcbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-07-27T03:11:51.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:50 smithi175 ceph-mon[37125]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:51.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:50 smithi067 ceph-mon[32814]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:51.500 INFO:journalctl@ceph.mds.k.smithi149.stdout:Jul 27 03:11:51 smithi149 podman[66359]: 2024-07-27 03:11:51.023314943 +0000 UTC m=+0.384837451 container init 15e7bc64f44a0d62fac47d8e090c6b46c0e8343719abe26cd2f9038791a6dcbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-07-27T03:11:51.501 INFO:journalctl@ceph.mds.k.smithi149.stdout:Jul 27 03:11:51 smithi149 podman[66359]: 2024-07-27 03:11:51.026004119 +0000 UTC m=+0.387526632 container start 15e7bc64f44a0d62fac47d8e090c6b46c0e8343719abe26cd2f9038791a6dcbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, ceph=True, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-07-27T03:11:51.501 INFO:journalctl@ceph.mds.k.smithi149.stdout:Jul 27 03:11:51 smithi149 ceph-mds[66378]: -- 172.21.15.149:0/1484338416 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56337153f860 con 0x5633756e5a80 2024-07-27T03:11:51.501 INFO:journalctl@ceph.mds.k.smithi149.stdout:Jul 27 03:11:51 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-k[66374]: starting mds.k at 2024-07-27T03:11:51.501 INFO:journalctl@ceph.mds.k.smithi149.stdout:Jul 27 03:11:51 smithi149 bash[66359]: 15e7bc64f44a0d62fac47d8e090c6b46c0e8343719abe26cd2f9038791a6dcbd 2024-07-27T03:11:51.501 INFO:journalctl@ceph.mds.k.smithi149.stdout:Jul 27 03:11:51 smithi149 systemd[1]: Started Ceph mds.k for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:11:51.828 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:11:52.223 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:11:52.410 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:52 smithi149 ceph-mon[37173]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:52.410 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:52.410 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:52.411 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:52.411 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:11:52.411 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:11:52.411 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:52.411 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:52 smithi149 ceph-mon[37173]: Deploying daemon mds.j on smithi067 2024-07-27T03:11:52.411 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:52 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.149:6832/3868325215,v1:172.21.15.149:6833/3868325215] up:boot 2024-07-27T03:11:52.411 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:52 smithi149 ceph-mon[37173]: fsmap 2 up:standby 2024-07-27T03:11:52.411 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:52 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-27T03:11:52.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:52 smithi067 ceph-mon[32814]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:52.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:52.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:52.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:52.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:11:52.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:11:52.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:52.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:52 smithi067 ceph-mon[32814]: Deploying daemon mds.j on smithi067 2024-07-27T03:11:52.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:52 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.149:6832/3868325215,v1:172.21.15.149:6833/3868325215] up:boot 2024-07-27T03:11:52.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:52 smithi067 ceph-mon[32814]: fsmap 2 up:standby 2024-07-27T03:11:52.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:52 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-27T03:11:52.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:52 smithi175 ceph-mon[37125]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:52.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:52.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:52.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:52.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:11:52.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:11:52.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:52.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:52 smithi175 ceph-mon[37125]: Deploying daemon mds.j on smithi067 2024-07-27T03:11:52.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:52 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.149:6832/3868325215,v1:172.21.15.149:6833/3868325215] up:boot 2024-07-27T03:11:52.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:52 smithi175 ceph-mon[37125]: fsmap 2 up:standby 2024-07-27T03:11:52.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:52 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-27T03:11:54.356 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:54 smithi067 ceph-mon[32814]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:54.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:54 smithi149 ceph-mon[37173]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:54.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:54 smithi175 ceph-mon[37125]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:54.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.819+0000 7fac7d4bc640 1 -- 172.21.15.67:0/3437236018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac78108760 msgr2=0x7fac7810ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:54.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.819+0000 7fac7d4bc640 1 --2- 172.21.15.67:0/3437236018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac78108760 0x7fac7810ef90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fac70009ca0 tx=0x7fac7002a8c0 comp rx=0 tx=0).stop 2024-07-27T03:11:54.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.819+0000 7fac7d4bc640 1 -- 172.21.15.67:0/3437236018 shutdown_connections 2024-07-27T03:11:54.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.819+0000 7fac7d4bc640 1 --2- 172.21.15.67:0/3437236018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac78108760 0x7fac7810ef90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:54.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.819+0000 7fac7d4bc640 1 --2- 172.21.15.67:0/3437236018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac78107e10 0x7fac78108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:54.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.819+0000 7fac7d4bc640 1 --2- 172.21.15.67:0/3437236018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac781024d0 0x7fac781028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:54.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.819+0000 7fac7d4bc640 1 -- 172.21.15.67:0/3437236018 >> 172.21.15.67:0/3437236018 conn(0x7fac780791a0 msgr2=0x7fac780795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:11:54.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.819+0000 7fac7d4bc640 1 -- 172.21.15.67:0/3437236018 shutdown_connections 2024-07-27T03:11:54.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.819+0000 7fac7d4bc640 1 -- 172.21.15.67:0/3437236018 wait complete. 2024-07-27T03:11:54.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.820+0000 7fac7d4bc640 1 Processor -- start 2024-07-27T03:11:54.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.820+0000 7fac7d4bc640 1 -- start start 2024-07-27T03:11:54.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.820+0000 7fac7d4bc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac781024d0 0x7fac7819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:54.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.820+0000 7fac7d4bc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac78107e10 0x7fac7819b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:54.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.820+0000 7fac7d4bc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac78108760 0x7fac7819b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:54.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.820+0000 7fac7d4bc640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fac781123e0 con 0x7fac781024d0 2024-07-27T03:11:54.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.820+0000 7fac7d4bc640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fac78112260 con 0x7fac78108760 2024-07-27T03:11:54.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.820+0000 7fac7d4bc640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fac78112560 con 0x7fac78107e10 2024-07-27T03:11:54.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.821+0000 7fac76ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac781024d0 0x7fac7819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:54.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.821+0000 7fac76ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac781024d0 0x7fac7819ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39604/0 (socket says 172.21.15.67:39604) 2024-07-27T03:11:54.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.821+0000 7fac76ffd640 1 -- 172.21.15.67:0/1662150551 learned_addr learned my addr 172.21.15.67:0/1662150551 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:11:54.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.821+0000 7fac777fe640 1 --2- 172.21.15.67:0/1662150551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac78108760 0x7fac7819b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:54.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.821+0000 7fac767fc640 1 --2- 172.21.15.67:0/1662150551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac78107e10 0x7fac7819b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:54.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.821+0000 7fac76ffd640 1 -- 172.21.15.67:0/1662150551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac78107e10 msgr2=0x7fac7819b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:54.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.821+0000 7fac76ffd640 1 --2- 172.21.15.67:0/1662150551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac78107e10 0x7fac7819b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:54.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.821+0000 7fac76ffd640 1 -- 172.21.15.67:0/1662150551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac78108760 msgr2=0x7fac7819b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:54.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.821+0000 7fac76ffd640 1 --2- 172.21.15.67:0/1662150551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac78108760 0x7fac7819b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:54.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.821+0000 7fac76ffd640 1 -- 172.21.15.67:0/1662150551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac781a5fa0 con 0x7fac781024d0 2024-07-27T03:11:54.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.821+0000 7fac76ffd640 1 --2- 172.21.15.67:0/1662150551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac781024d0 0x7fac7819ad10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fac6000d770 tx=0x7fac6000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:11:54.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.821+0000 7fac6ffff640 1 -- 172.21.15.67:0/1662150551 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60017070 con 0x7fac781024d0 2024-07-27T03:11:54.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.821+0000 7fac6ffff640 1 -- 172.21.15.67:0/1662150551 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac600054d0 con 0x7fac781024d0 2024-07-27T03:11:54.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.821+0000 7fac7d4bc640 1 -- 172.21.15.67:0/1662150551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac781a6290 con 0x7fac781024d0 2024-07-27T03:11:54.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.821+0000 7fac6ffff640 1 -- 172.21.15.67:0/1662150551 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60004c30 con 0x7fac781024d0 2024-07-27T03:11:54.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.821+0000 7fac7d4bc640 1 -- 172.21.15.67:0/1662150551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac781a67d0 con 0x7fac781024d0 2024-07-27T03:11:54.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.823+0000 7fac7d4bc640 1 -- 172.21.15.67:0/1662150551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac781028e0 con 0x7fac781024d0 2024-07-27T03:11:54.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.824+0000 7fac6ffff640 1 -- 172.21.15.67:0/1662150551 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7fac60004dd0 con 0x7fac781024d0 2024-07-27T03:11:54.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.825+0000 7fac6ffff640 1 --2- 172.21.15.67:0/1662150551 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fac640b3c80 0x7fac640b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:54.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.825+0000 7fac767fc640 1 --2- 172.21.15.67:0/1662150551 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fac640b3c80 0x7fac640b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:54.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.825+0000 7fac6ffff640 1 -- 172.21.15.67:0/1662150551 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fac600e3630 con 0x7fac781024d0 2024-07-27T03:11:54.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.826+0000 7fac767fc640 1 --2- 172.21.15.67:0/1662150551 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fac640b3c80 0x7fac640b6190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fac68005fd0 tx=0x7fac68005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:11:54.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:54.828+0000 7fac6ffff640 1 -- 172.21.15.67:0/1662150551 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fac600ab1a0 con 0x7fac781024d0 2024-07-27T03:11:54.992 INFO:journalctl@ceph.mds.j.smithi067.stdout:Jul 27 03:11:54 smithi067 podman[76859]: 2024-07-27 03:11:54.95613922 +0000 UTC m=+0.853210394 container start c8f67424c9aad325e97cb70ffc560d9ba02730b62f62523718c6996122837429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-j, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf) 2024-07-27T03:11:54.993 INFO:journalctl@ceph.mds.j.smithi067.stdout:Jul 27 03:11:54 smithi067 ceph-mds[76911]: -- 172.21.15.67:0/1827743941 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c93ed5f860 con 0x55c942f05a80 2024-07-27T03:11:55.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:55.053+0000 7fac7d4bc640 1 -- 172.21.15.67:0/1662150551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac7806ace0 con 0x7fac781024d0 2024-07-27T03:11:55.057 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:55.054+0000 7fac6ffff640 1 -- 172.21.15.67:0/1662150551 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2266 (secure 0 0 0) 0x7fac600aaaa0 con 0x7fac781024d0 2024-07-27T03:11:55.057 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:11:55.057 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":3,"btime":"2024-07-27T03:11:51:747248+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24526,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24533,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3868325215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3868325215},{"type":"v1","addr":"172.21.15.149:6833","nonce":3868325215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-07-27T03:11:55.058 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 3 2024-07-27T03:11:55.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:55.056+0000 7fac7d4bc640 1 -- 172.21.15.67:0/1662150551 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fac640b3c80 msgr2=0x7fac640b6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:55.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:55.056+0000 7fac7d4bc640 1 --2- 172.21.15.67:0/1662150551 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fac640b3c80 0x7fac640b6190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fac68005fd0 tx=0x7fac68005850 comp rx=0 tx=0).stop 2024-07-27T03:11:55.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:55.056+0000 7fac7d4bc640 1 -- 172.21.15.67:0/1662150551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac781024d0 msgr2=0x7fac7819ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:55.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:55.056+0000 7fac7d4bc640 1 --2- 172.21.15.67:0/1662150551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac781024d0 0x7fac7819ad10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fac6000d770 tx=0x7fac6000dc30 comp rx=0 tx=0).stop 2024-07-27T03:11:55.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:55.056+0000 7fac7d4bc640 1 -- 172.21.15.67:0/1662150551 shutdown_connections 2024-07-27T03:11:55.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:55.056+0000 7fac7d4bc640 1 --2- 172.21.15.67:0/1662150551 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fac640b3c80 0x7fac640b6190 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:55.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:55.056+0000 7fac7d4bc640 1 --2- 172.21.15.67:0/1662150551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac78108760 0x7fac7819b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:55.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:55.056+0000 7fac7d4bc640 1 --2- 172.21.15.67:0/1662150551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fac78107e10 0x7fac7819b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:55.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:55.056+0000 7fac7d4bc640 1 --2- 172.21.15.67:0/1662150551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fac781024d0 0x7fac7819ad10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:55.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:55.056+0000 7fac7d4bc640 1 -- 172.21.15.67:0/1662150551 >> 172.21.15.67:0/1662150551 conn(0x7fac780791a0 msgr2=0x7fac78104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:11:55.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:55.056+0000 7fac7d4bc640 1 -- 172.21.15.67:0/1662150551 shutdown_connections 2024-07-27T03:11:55.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:55.056+0000 7fac7d4bc640 1 -- 172.21.15.67:0/1662150551 wait complete. 2024-07-27T03:11:55.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:55 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1662150551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:11:55.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:55.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:55.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:55.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:11:55.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:11:55.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:55.470 INFO:journalctl@ceph.mds.j.smithi067.stdout:Jul 27 03:11:54 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-j[76886]: starting mds.j at 2024-07-27T03:11:55.470 INFO:journalctl@ceph.mds.j.smithi067.stdout:Jul 27 03:11:55 smithi067 bash[76859]: c8f67424c9aad325e97cb70ffc560d9ba02730b62f62523718c6996122837429 2024-07-27T03:11:55.470 INFO:journalctl@ceph.mds.j.smithi067.stdout:Jul 27 03:11:55 smithi067 systemd[1]: Started Ceph mds.j for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:11:55.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:55 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1662150551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:11:55.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:55 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:55.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:55 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:55.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:55 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:55.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:55 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:11:55.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:55 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:11:55.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:55 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:55.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:55 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1662150551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:11:55.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:55 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:55.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:55 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:55.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:55 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:55.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:55 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:11:55.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:55 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:11:55.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:55 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:56.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:56 smithi067 ceph-mon[32814]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:56.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:56 smithi067 ceph-mon[32814]: Deploying daemon mds.d on smithi067 2024-07-27T03:11:56.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:56 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.67:6834/252686369,v1:172.21.15.67:6835/252686369] up:boot 2024-07-27T03:11:56.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:56 smithi067 ceph-mon[32814]: fsmap 3 up:standby 2024-07-27T03:11:56.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:56 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-27T03:11:56.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:56 smithi149 ceph-mon[37173]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:56.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:56 smithi149 ceph-mon[37173]: Deploying daemon mds.d on smithi067 2024-07-27T03:11:56.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:56 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.67:6834/252686369,v1:172.21.15.67:6835/252686369] up:boot 2024-07-27T03:11:56.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:56 smithi149 ceph-mon[37173]: fsmap 3 up:standby 2024-07-27T03:11:56.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:56 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-27T03:11:56.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:56 smithi175 ceph-mon[37125]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:56.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:56 smithi175 ceph-mon[37125]: Deploying daemon mds.d on smithi067 2024-07-27T03:11:56.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:56 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.67:6834/252686369,v1:172.21.15.67:6835/252686369] up:boot 2024-07-27T03:11:56.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:56 smithi175 ceph-mon[37125]: fsmap 3 up:standby 2024-07-27T03:11:56.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:56 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-07-27T03:11:57.697 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:11:57.720 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 27 03:11:57 smithi067 podman[77215]: 2024-07-27 03:11:57.626962918 +0000 UTC m=+0.131768914 container create a30b577e5ac20082a16ef5a179b674a6ca75e594b6d768b5f703c1da9501f7db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-07-27T03:11:58.012 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:11:58.138 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 27 03:11:57 smithi067 podman[77215]: 2024-07-27 03:11:57.84646352 +0000 UTC m=+0.351269529 container init a30b577e5ac20082a16ef5a179b674a6ca75e594b6d768b5f703c1da9501f7db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-d, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-07-27T03:11:58.138 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 27 03:11:57 smithi067 podman[77215]: 2024-07-27 03:11:57.849937052 +0000 UTC m=+0.354743048 container start a30b577e5ac20082a16ef5a179b674a6ca75e594b6d768b5f703c1da9501f7db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_CLEAN=True) 2024-07-27T03:11:58.138 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 27 03:11:57 smithi067 ceph-mds[77252]: -- 172.21.15.67:0/3089319505 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557ba87bb860 con 0x557bac961600 2024-07-27T03:11:58.138 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 27 03:11:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-d[77248]: starting mds.d at 2024-07-27T03:11:58.138 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 27 03:11:57 smithi067 bash[77215]: a30b577e5ac20082a16ef5a179b674a6ca75e594b6d768b5f703c1da9501f7db 2024-07-27T03:11:58.138 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 27 03:11:57 smithi067 systemd[1]: Started Ceph mds.d for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:11:58.394 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:58 smithi067 ceph-mon[32814]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:58.395 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:58 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:58.395 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:58 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:58.395 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:58 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:58.395 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:58 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:11:58.395 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:58 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:11:58.395 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:58 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:58.444 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:58 smithi149 ceph-mon[37173]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:58.444 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:58 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:58.444 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:58 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:58.444 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:58 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:58.444 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:58 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:11:58.444 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:58 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:11:58.445 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:58 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:58.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:58 smithi175 ceph-mon[37125]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:11:58.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:58 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:58.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:58 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:58.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:58 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:11:58.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:58 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:11:58.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:58 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:11:58.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:58 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:11:59.434 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:59 smithi149 ceph-mon[37173]: Deploying daemon mds.e on smithi149 2024-07-27T03:11:59.434 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:59 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.67:6836/3358832762,v1:172.21.15.67:6837/3358832762] up:boot 2024-07-27T03:11:59.434 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:59 smithi149 ceph-mon[37173]: fsmap 4 up:standby 2024-07-27T03:11:59.434 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:11:59 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-27T03:11:59.445 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:59 smithi067 ceph-mon[32814]: Deploying daemon mds.e on smithi149 2024-07-27T03:11:59.445 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:59 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.67:6836/3358832762,v1:172.21.15.67:6837/3358832762] up:boot 2024-07-27T03:11:59.445 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:59 smithi067 ceph-mon[32814]: fsmap 4 up:standby 2024-07-27T03:11:59.445 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:11:59 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-27T03:11:59.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:59 smithi175 ceph-mon[37125]: Deploying daemon mds.e on smithi149 2024-07-27T03:11:59.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:59 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.67:6836/3358832762,v1:172.21.15.67:6837/3358832762] up:boot 2024-07-27T03:11:59.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:59 smithi175 ceph-mon[37125]: fsmap 4 up:standby 2024-07-27T03:11:59.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:11:59 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-07-27T03:11:59.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.820+0000 7f22e9a22640 1 -- 172.21.15.67:0/1982619434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22e4109860 msgr2=0x7f22e4110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:59.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.820+0000 7f22e9a22640 1 --2- 172.21.15.67:0/1982619434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22e4109860 0x7f22e4110090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009ca0 tx=0x7f22dc02a8c0 comp rx=0 tx=0).stop 2024-07-27T03:11:59.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.820+0000 7f22e9a22640 1 -- 172.21.15.67:0/1982619434 shutdown_connections 2024-07-27T03:11:59.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.820+0000 7f22e9a22640 1 --2- 172.21.15.67:0/1982619434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22e4109860 0x7f22e4110090 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:59.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.820+0000 7f22e9a22640 1 --2- 172.21.15.67:0/1982619434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22e4108f10 0x7f22e4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:59.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.820+0000 7f22e9a22640 1 --2- 172.21.15.67:0/1982619434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22e41035d0 0x7f22e41039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:59.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.820+0000 7f22e9a22640 1 -- 172.21.15.67:0/1982619434 >> 172.21.15.67:0/1982619434 conn(0x7f22e407b2c0 msgr2=0x7f22e407b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:11:59.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.820+0000 7f22e9a22640 1 -- 172.21.15.67:0/1982619434 shutdown_connections 2024-07-27T03:11:59.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.820+0000 7f22e9a22640 1 -- 172.21.15.67:0/1982619434 wait complete. 2024-07-27T03:11:59.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.821+0000 7f22e9a22640 1 Processor -- start 2024-07-27T03:11:59.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.821+0000 7f22e9a22640 1 -- start start 2024-07-27T03:11:59.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.821+0000 7f22e9a22640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22e41035d0 0x7f22e4075180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:59.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.821+0000 7f22e9a22640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22e4108f10 0x7f22e40756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:59.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.821+0000 7f22e9a22640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22e4109860 0x7f22e4075c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:59.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.821+0000 7f22e9a22640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f22e4113510 con 0x7f22e4109860 2024-07-27T03:11:59.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.821+0000 7f22e9a22640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f22e4113390 con 0x7f22e4108f10 2024-07-27T03:11:59.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.821+0000 7f22e9a22640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22e4113690 con 0x7f22e41035d0 2024-07-27T03:11:59.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.821+0000 7f22e37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22e4109860 0x7f22e4075c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:59.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.821+0000 7f22e37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22e4109860 0x7f22e4075c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50958/0 (socket says 172.21.15.67:50958) 2024-07-27T03:11:59.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.821+0000 7f22e37fe640 1 -- 172.21.15.67:0/944200903 learned_addr learned my addr 172.21.15.67:0/944200903 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:11:59.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.821+0000 7f22e27fc640 1 --2- 172.21.15.67:0/944200903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22e4108f10 0x7f22e40756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:59.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.822+0000 7f22e2ffd640 1 --2- 172.21.15.67:0/944200903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22e41035d0 0x7f22e4075180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:59.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.822+0000 7f22e37fe640 1 -- 172.21.15.67:0/944200903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22e41035d0 msgr2=0x7f22e4075180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:59.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.822+0000 7f22e37fe640 1 --2- 172.21.15.67:0/944200903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22e41035d0 0x7f22e4075180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:59.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.822+0000 7f22e37fe640 1 -- 172.21.15.67:0/944200903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22e4108f10 msgr2=0x7f22e40756c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:11:59.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.822+0000 7f22e37fe640 1 --2- 172.21.15.67:0/944200903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22e4108f10 0x7f22e40756c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:11:59.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.822+0000 7f22e37fe640 1 -- 172.21.15.67:0/944200903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22e41a80d0 con 0x7f22e4109860 2024-07-27T03:11:59.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.822+0000 7f22e27fc640 1 --2- 172.21.15.67:0/944200903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22e4108f10 0x7f22e40756c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:11:59.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.822+0000 7f22e37fe640 1 --2- 172.21.15.67:0/944200903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22e4109860 0x7f22e4075c00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0098d0 tx=0x7f22dc0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:11:59.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.822+0000 7f22e8a20640 1 -- 172.21.15.67:0/944200903 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc003b70 con 0x7f22e4109860 2024-07-27T03:11:59.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.822+0000 7f22e8a20640 1 -- 172.21.15.67:0/944200903 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22dc003d10 con 0x7f22e4109860 2024-07-27T03:11:59.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.822+0000 7f22e9a22640 1 -- 172.21.15.67:0/944200903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22e41a8360 con 0x7f22e4109860 2024-07-27T03:11:59.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.822+0000 7f22e8a20640 1 -- 172.21.15.67:0/944200903 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22dc02fa20 con 0x7f22e4109860 2024-07-27T03:11:59.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.822+0000 7f22e9a22640 1 -- 172.21.15.67:0/944200903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22e41a88f0 con 0x7f22e4109860 2024-07-27T03:11:59.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.823+0000 7f22e9a22640 1 -- 172.21.15.67:0/944200903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22e41039e0 con 0x7f22e4109860 2024-07-27T03:11:59.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.825+0000 7f22e8a20640 1 -- 172.21.15.67:0/944200903 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f22dc02fbe0 con 0x7f22e4109860 2024-07-27T03:11:59.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.825+0000 7f22e8a20640 1 --2- 172.21.15.67:0/944200903 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f22cc0b3c50 0x7f22cc0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:11:59.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.825+0000 7f22e2ffd640 1 --2- 172.21.15.67:0/944200903 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f22cc0b3c50 0x7f22cc0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:11:59.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.826+0000 7f22e8a20640 1 -- 172.21.15.67:0/944200903 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f22dc104e10 con 0x7f22e4109860 2024-07-27T03:11:59.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.826+0000 7f22e2ffd640 1 --2- 172.21.15.67:0/944200903 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f22cc0b3c50 0x7f22cc0b6160 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f22d4005e10 tx=0x7f22d4005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:11:59.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:11:59.829+0000 7f22e8a20640 1 -- 172.21.15.67:0/944200903 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f22dc0cc990 con 0x7f22e4109860 2024-07-27T03:12:00.000 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:11:59 smithi149 systemd[1]: Starting Ceph mds.e for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:12:00.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:00.048+0000 7f22e9a22640 1 -- 172.21.15.67:0/944200903 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22e4076480 con 0x7f22e4109860 2024-07-27T03:12:00.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:00.049+0000 7f22e8a20640 1 -- 172.21.15.67:0/944200903 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3913 (secure 0 0 0) 0x7f22dc0cc290 con 0x7f22e4109860 2024-07-27T03:12:00.052 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:12:00.052 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":5,"btime":"2024-07-27T03:11:58:152940+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14679,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/252686369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":252686369},{"type":"v1","addr":"172.21.15.67:6835","nonce":252686369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/3358832762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3358832762},{"type":"v1","addr":"172.21.15.67:6837","nonce":3358832762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24526,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24533,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3868325215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3868325215},{"type":"v1","addr":"172.21.15.149:6833","nonce":3868325215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-07-27T03:12:00.052 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 5 2024-07-27T03:12:00.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:00.051+0000 7f22e9a22640 1 -- 172.21.15.67:0/944200903 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f22cc0b3c50 msgr2=0x7f22cc0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:00.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:00.051+0000 7f22e9a22640 1 --2- 172.21.15.67:0/944200903 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f22cc0b3c50 0x7f22cc0b6160 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f22d4005e10 tx=0x7f22d4005da0 comp rx=0 tx=0).stop 2024-07-27T03:12:00.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:00.051+0000 7f22e9a22640 1 -- 172.21.15.67:0/944200903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22e4109860 msgr2=0x7f22e4075c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:00.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:00.051+0000 7f22e9a22640 1 --2- 172.21.15.67:0/944200903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22e4109860 0x7f22e4075c00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0098d0 tx=0x7f22dc0317b0 comp rx=0 tx=0).stop 2024-07-27T03:12:00.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:00.051+0000 7f22e9a22640 1 -- 172.21.15.67:0/944200903 shutdown_connections 2024-07-27T03:12:00.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:00.051+0000 7f22e9a22640 1 --2- 172.21.15.67:0/944200903 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f22cc0b3c50 0x7f22cc0b6160 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:00.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:00.051+0000 7f22e9a22640 1 --2- 172.21.15.67:0/944200903 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22e4109860 0x7f22e4075c00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:00.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:00.051+0000 7f22e9a22640 1 --2- 172.21.15.67:0/944200903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22e4108f10 0x7f22e40756c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:00.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:00.051+0000 7f22e9a22640 1 --2- 172.21.15.67:0/944200903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22e41035d0 0x7f22e4075180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:00.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:00.051+0000 7f22e9a22640 1 -- 172.21.15.67:0/944200903 >> 172.21.15.67:0/944200903 conn(0x7f22e407b2c0 msgr2=0x7f22e4106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:00.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:00.051+0000 7f22e9a22640 1 -- 172.21.15.67:0/944200903 shutdown_connections 2024-07-27T03:12:00.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:00.051+0000 7f22e9a22640 1 -- 172.21.15.67:0/944200903 wait complete. 2024-07-27T03:12:00.365 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:00 smithi067 ceph-mon[32814]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:00.365 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:00 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/944200903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:12:00.371 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:00 smithi149 ceph-mon[37173]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:00.372 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:00 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/944200903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:12:00.372 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:12:00 smithi149 podman[66674]: 2024-07-27 03:12:00.092468839 +0000 UTC m=+0.103381971 container create 4ef3b65849c3912ae2210bc394e4dad42b6ea83c5325a3a4ea02bba9129e7857 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-e, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-07-27T03:12:00.372 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:12:00 smithi149 podman[66674]: 2024-07-27 03:12:00.006872172 +0000 UTC m=+0.017785274 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c 2024-07-27T03:12:00.372 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:12:00 smithi149 podman[66674]: 2024-07-27 03:12:00.37001189 +0000 UTC m=+0.380924981 container init 4ef3b65849c3912ae2210bc394e4dad42b6ea83c5325a3a4ea02bba9129e7857 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-e, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-07-27T03:12:00.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:00 smithi175 ceph-mon[37125]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:00.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:00 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/944200903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:12:00.750 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:12:00 smithi149 podman[66674]: 2024-07-27 03:12:00.372763496 +0000 UTC m=+0.383676589 container start 4ef3b65849c3912ae2210bc394e4dad42b6ea83c5325a3a4ea02bba9129e7857 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-07-27T03:12:00.750 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:12:00 smithi149 ceph-mds[66693]: -- 172.21.15.149:0/3636746885 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d5ad80d860 con 0x55d5b19b3600 2024-07-27T03:12:00.750 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:12:00 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-e[66689]: starting mds.e at 2024-07-27T03:12:00.751 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:12:00 smithi149 bash[66674]: 4ef3b65849c3912ae2210bc394e4dad42b6ea83c5325a3a4ea02bba9129e7857 2024-07-27T03:12:00.751 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:12:00 smithi149 systemd[1]: Started Ceph mds.e for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:12:01.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:01 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:01.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:01 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:01.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:01 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:01.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:01 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:12:01.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:01 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:12:01.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:01 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:01.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:01 smithi175 ceph-mon[37125]: Deploying daemon mds.f on smithi175 2024-07-27T03:12:01.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:01 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] up:boot 2024-07-27T03:12:01.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:01 smithi175 ceph-mon[37125]: fsmap 5 up:standby 2024-07-27T03:12:01.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:01 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-27T03:12:01.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:01 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:01.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:01 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:01.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:01 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:01.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:01 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:12:01.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:01 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:12:01.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:01 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:01.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:01 smithi067 ceph-mon[32814]: Deploying daemon mds.f on smithi175 2024-07-27T03:12:01.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:01 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] up:boot 2024-07-27T03:12:01.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:01 smithi067 ceph-mon[32814]: fsmap 5 up:standby 2024-07-27T03:12:01.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:01 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-27T03:12:02.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:01 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:02.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:01 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:02.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:01 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:02.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:01 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:12:02.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:01 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:12:02.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:01 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:02.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:01 smithi149 ceph-mon[37173]: Deploying daemon mds.f on smithi175 2024-07-27T03:12:02.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:01 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] up:boot 2024-07-27T03:12:02.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:01 smithi149 ceph-mon[37173]: fsmap 5 up:standby 2024-07-27T03:12:02.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:01 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-27T03:12:02.546 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jul 27 03:12:02 smithi175 systemd[1]: Starting Ceph mds.f for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:12:02.791 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:12:02.821 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:02 smithi067 ceph-mon[32814]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:02.839 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:02 smithi149 ceph-mon[37173]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:02.842 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:02 smithi175 ceph-mon[37125]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:02.843 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jul 27 03:12:02 smithi175 podman[67196]: 2024-07-27 03:12:02.574577927 +0000 UTC m=+0.017002460 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c 2024-07-27T03:12:02.843 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jul 27 03:12:02 smithi175 podman[67196]: 2024-07-27 03:12:02.701146104 +0000 UTC m=+0.143570665 container create 73a994aac40f55053ce097e9c8d068cbd87107b501cdbc8ed0b1a7a31acc0afc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-f, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-27T03:12:03.072 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:12:03.125 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jul 27 03:12:02 smithi175 podman[67196]: 2024-07-27 03:12:02.953635737 +0000 UTC m=+0.396060252 container init 73a994aac40f55053ce097e9c8d068cbd87107b501cdbc8ed0b1a7a31acc0afc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-07-27T03:12:03.125 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jul 27 03:12:02 smithi175 podman[67196]: 2024-07-27 03:12:02.956185113 +0000 UTC m=+0.398609628 container start 73a994aac40f55053ce097e9c8d068cbd87107b501cdbc8ed0b1a7a31acc0afc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-07-27T03:12:03.125 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jul 27 03:12:02 smithi175 ceph-mds[67215]: -- 172.21.15.175:0/2252290411 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557477aad860 con 0x55747bc53a80 2024-07-27T03:12:03.125 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jul 27 03:12:02 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-f[67211]: starting mds.f at 2024-07-27T03:12:03.125 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jul 27 03:12:03 smithi175 bash[67196]: 73a994aac40f55053ce097e9c8d068cbd87107b501cdbc8ed0b1a7a31acc0afc 2024-07-27T03:12:03.125 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jul 27 03:12:03 smithi175 systemd[1]: Started Ceph mds.f for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:12:04.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:04 smithi067 ceph-mon[32814]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:04.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:04 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:04.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:04 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:04.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:04 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:04.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:04 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:12:04.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:04 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:12:04.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:04 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:04.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:04 smithi067 ceph-mon[32814]: Deploying daemon mds.c on smithi175 2024-07-27T03:12:04.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:04 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.175:6834/2462749027,v1:172.21.15.175:6835/2462749027] up:boot 2024-07-27T03:12:04.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:04 smithi067 ceph-mon[32814]: fsmap 6 up:standby 2024-07-27T03:12:04.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:04 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-27T03:12:04.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:04 smithi175 ceph-mon[37125]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:04.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:04 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:04.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:04 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:04.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:04 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:04.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:04 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:12:04.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:04 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:12:04.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:04 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:04.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:04 smithi175 ceph-mon[37125]: Deploying daemon mds.c on smithi175 2024-07-27T03:12:04.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:04 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.175:6834/2462749027,v1:172.21.15.175:6835/2462749027] up:boot 2024-07-27T03:12:04.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:04 smithi175 ceph-mon[37125]: fsmap 6 up:standby 2024-07-27T03:12:04.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:04 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-27T03:12:04.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:04 smithi149 ceph-mon[37173]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:04.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:04 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:04.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:04 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:04.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:04 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:04.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:04 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:12:04.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:04 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:12:04.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:04 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:04.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:04 smithi149 ceph-mon[37173]: Deploying daemon mds.c on smithi175 2024-07-27T03:12:04.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:04 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.175:6834/2462749027,v1:172.21.15.175:6835/2462749027] up:boot 2024-07-27T03:12:04.502 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:04 smithi149 ceph-mon[37173]: fsmap 6 up:standby 2024-07-27T03:12:04.502 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:04 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-27T03:12:04.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.762+0000 7f549f2bf640 1 -- 172.21.15.67:0/1830168756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5498108760 msgr2=0x7f549810ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:04.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.762+0000 7f549f2bf640 1 --2- 172.21.15.67:0/1830168756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5498108760 0x7f549810ef90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5490009ca0 tx=0x7f549002a8c0 comp rx=0 tx=0).stop 2024-07-27T03:12:04.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.762+0000 7f549f2bf640 1 -- 172.21.15.67:0/1830168756 shutdown_connections 2024-07-27T03:12:04.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.762+0000 7f549f2bf640 1 --2- 172.21.15.67:0/1830168756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5498108760 0x7f549810ef90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:04.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.762+0000 7f549f2bf640 1 --2- 172.21.15.67:0/1830168756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5498107e10 0x7f5498108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:04.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.762+0000 7f549f2bf640 1 --2- 172.21.15.67:0/1830168756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54981024d0 0x7f54981028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:04.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.762+0000 7f549f2bf640 1 -- 172.21.15.67:0/1830168756 >> 172.21.15.67:0/1830168756 conn(0x7f54980791a0 msgr2=0x7f54980795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:04.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.762+0000 7f549f2bf640 1 -- 172.21.15.67:0/1830168756 shutdown_connections 2024-07-27T03:12:04.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.762+0000 7f549f2bf640 1 -- 172.21.15.67:0/1830168756 wait complete. 2024-07-27T03:12:04.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.763+0000 7f549f2bf640 1 Processor -- start 2024-07-27T03:12:04.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.763+0000 7f549f2bf640 1 -- start start 2024-07-27T03:12:04.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.763+0000 7f549f2bf640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54981024d0 0x7f549810fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:04.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.763+0000 7f549f2bf640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5498107e10 0x7f549810ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:04.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.764+0000 7f549f2bf640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5498108760 0x7f54981104a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:04.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.764+0000 7f549f2bf640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f54981134f0 con 0x7f54981024d0 2024-07-27T03:12:04.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.764+0000 7f549f2bf640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5498113370 con 0x7f5498107e10 2024-07-27T03:12:04.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.764+0000 7f549f2bf640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5498113670 con 0x7f5498108760 2024-07-27T03:12:04.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.764+0000 7f549d05b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54981024d0 0x7f549810fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:04.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.764+0000 7f549d05b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54981024d0 0x7f549810fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50978/0 (socket says 172.21.15.67:50978) 2024-07-27T03:12:04.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.764+0000 7f549d05b640 1 -- 172.21.15.67:0/2656447167 learned_addr learned my addr 172.21.15.67:0/2656447167 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:12:04.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.764+0000 7f549d85c640 1 --2- 172.21.15.67:0/2656447167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5498108760 0x7f54981104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:04.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.764+0000 7f549c85a640 1 --2- 172.21.15.67:0/2656447167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5498107e10 0x7f549810ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:04.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.764+0000 7f549d05b640 1 -- 172.21.15.67:0/2656447167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5498108760 msgr2=0x7f54981104a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:04.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.764+0000 7f549d05b640 1 --2- 172.21.15.67:0/2656447167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5498108760 0x7f54981104a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:04.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.764+0000 7f549d05b640 1 -- 172.21.15.67:0/2656447167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5498107e10 msgr2=0x7f549810ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:04.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.764+0000 7f549d05b640 1 --2- 172.21.15.67:0/2656447167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5498107e10 0x7f549810ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:04.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.764+0000 7f549d05b640 1 -- 172.21.15.67:0/2656447167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54981a8220 con 0x7f54981024d0 2024-07-27T03:12:04.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.764+0000 7f549d05b640 1 --2- 172.21.15.67:0/2656447167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54981024d0 0x7f549810fa20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f549400d950 tx=0x7f549400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:04.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.764+0000 7f548e7fc640 1 -- 172.21.15.67:0/2656447167 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494017030 con 0x7f54981024d0 2024-07-27T03:12:04.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.764+0000 7f548e7fc640 1 -- 172.21.15.67:0/2656447167 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5494004020 con 0x7f54981024d0 2024-07-27T03:12:04.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.764+0000 7f548e7fc640 1 -- 172.21.15.67:0/2656447167 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54940047a0 con 0x7f54981024d0 2024-07-27T03:12:04.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.765+0000 7f549f2bf640 1 -- 172.21.15.67:0/2656447167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54981a8510 con 0x7f54981024d0 2024-07-27T03:12:04.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.765+0000 7f549f2bf640 1 -- 172.21.15.67:0/2656447167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54981a8a00 con 0x7f54981024d0 2024-07-27T03:12:04.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.766+0000 7f549f2bf640 1 -- 172.21.15.67:0/2656447167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54981028e0 con 0x7f54981024d0 2024-07-27T03:12:04.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.767+0000 7f548e7fc640 1 -- 172.21.15.67:0/2656447167 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f5494009a30 con 0x7f54981024d0 2024-07-27T03:12:04.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.768+0000 7f548e7fc640 1 --2- 172.21.15.67:0/2656447167 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f54840b3bd0 0x7f54840b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:04.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.768+0000 7f549c85a640 1 --2- 172.21.15.67:0/2656447167 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f54840b3bd0 0x7f54840b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:04.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.768+0000 7f548e7fc640 1 -- 172.21.15.67:0/2656447167 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f54940e3770 con 0x7f54981024d0 2024-07-27T03:12:04.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.769+0000 7f549c85a640 1 --2- 172.21.15.67:0/2656447167 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f54840b3bd0 0x7f54840b60e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f54980797a0 tx=0x7f548800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:04.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.772+0000 7f548e7fc640 1 -- 172.21.15.67:0/2656447167 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f54940ab240 con 0x7f54981024d0 2024-07-27T03:12:04.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.993+0000 7f549f2bf640 1 -- 172.21.15.67:0/2656447167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5498110d20 con 0x7f54981024d0 2024-07-27T03:12:04.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.994+0000 7f548e7fc640 1 -- 172.21.15.67:0/2656447167 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5569 (secure 0 0 0) 0x7f5498110d20 con 0x7f54981024d0 2024-07-27T03:12:04.997 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:12:04.997 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":7,"btime":"2024-07-27T03:12:03:544064+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14679,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/252686369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":252686369},{"type":"v1","addr":"172.21.15.67:6835","nonce":252686369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/3358832762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3358832762},{"type":"v1","addr":"172.21.15.67:6837","nonce":3358832762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24526,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24533,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3868325215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3868325215},{"type":"v1","addr":"172.21.15.149:6833","nonce":3868325215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24562,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2462749027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2462749027},{"type":"v1","addr":"172.21.15.175:6835","nonce":2462749027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24563,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3730964806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3730964806},{"type":"v1","addr":"172.21.15.149:6835","nonce":3730964806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-07-27T03:12:04.997 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 7 2024-07-27T03:12:04.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.996+0000 7f549f2bf640 1 -- 172.21.15.67:0/2656447167 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f54840b3bd0 msgr2=0x7f54840b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:04.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.996+0000 7f549f2bf640 1 --2- 172.21.15.67:0/2656447167 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f54840b3bd0 0x7f54840b60e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f54980797a0 tx=0x7f548800b410 comp rx=0 tx=0).stop 2024-07-27T03:12:04.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.996+0000 7f549f2bf640 1 -- 172.21.15.67:0/2656447167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54981024d0 msgr2=0x7f549810fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:04.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.996+0000 7f549f2bf640 1 --2- 172.21.15.67:0/2656447167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54981024d0 0x7f549810fa20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f549400d950 tx=0x7f549400de10 comp rx=0 tx=0).stop 2024-07-27T03:12:04.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.996+0000 7f549f2bf640 1 -- 172.21.15.67:0/2656447167 shutdown_connections 2024-07-27T03:12:04.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.996+0000 7f549f2bf640 1 --2- 172.21.15.67:0/2656447167 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f54840b3bd0 0x7f54840b60e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:04.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.996+0000 7f549f2bf640 1 --2- 172.21.15.67:0/2656447167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5498108760 0x7f54981104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:04.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.996+0000 7f549f2bf640 1 --2- 172.21.15.67:0/2656447167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5498107e10 0x7f549810ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:05.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.996+0000 7f549f2bf640 1 --2- 172.21.15.67:0/2656447167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f54981024d0 0x7f549810fa20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:05.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.996+0000 7f549f2bf640 1 -- 172.21.15.67:0/2656447167 >> 172.21.15.67:0/2656447167 conn(0x7f54980791a0 msgr2=0x7f5498104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:05.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.996+0000 7f549f2bf640 1 -- 172.21.15.67:0/2656447167 shutdown_connections 2024-07-27T03:12:05.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:04.996+0000 7f549f2bf640 1 -- 172.21.15.67:0/2656447167 wait complete. 2024-07-27T03:12:05.111 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jul 27 03:12:05 smithi175 systemd[1]: Starting Ceph mds.c for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:12:05.299 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:05 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2656447167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:12:05.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:05 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2656447167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:12:05.375 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jul 27 03:12:05 smithi175 podman[67498]: 2024-07-27 03:12:05.148787604 +0000 UTC m=+0.017617166 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c 2024-07-27T03:12:05.375 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jul 27 03:12:05 smithi175 podman[67498]: 2024-07-27 03:12:05.282430539 +0000 UTC m=+0.151260090 container create 443e7d24d3cde7feb95c34f15d388f2a36e5d8378bd28296d038f8ceaa9b7976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-c, ceph=True, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-07-27T03:12:05.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:05 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2656447167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:12:05.875 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jul 27 03:12:05 smithi175 podman[67498]: 2024-07-27 03:12:05.510483106 +0000 UTC m=+0.379312676 container init 443e7d24d3cde7feb95c34f15d388f2a36e5d8378bd28296d038f8ceaa9b7976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-c, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-07-27T03:12:05.875 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jul 27 03:12:05 smithi175 podman[67498]: 2024-07-27 03:12:05.51362943 +0000 UTC m=+0.382458982 container start 443e7d24d3cde7feb95c34f15d388f2a36e5d8378bd28296d038f8ceaa9b7976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-07-27T03:12:05.875 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jul 27 03:12:05 smithi175 ceph-mds[67525]: -- 172.21.15.175:0/2818750973 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c352b21860 con 0x55c356cc7a80 2024-07-27T03:12:05.875 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jul 27 03:12:05 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-c[67516]: starting mds.c at 2024-07-27T03:12:05.876 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jul 27 03:12:05 smithi175 bash[67498]: 443e7d24d3cde7feb95c34f15d388f2a36e5d8378bd28296d038f8ceaa9b7976 2024-07-27T03:12:05.876 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jul 27 03:12:05 smithi175 systemd[1]: Started Ceph mds.c for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:12:06.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:06 smithi067 ceph-mon[32814]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:06.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:06.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:06.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:06.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:12:06.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:12:06.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:06.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:06 smithi067 ceph-mon[32814]: Deploying daemon mds.h on smithi149 2024-07-27T03:12:06.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:06 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] up:boot 2024-07-27T03:12:06.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:06 smithi067 ceph-mon[32814]: fsmap 7 up:standby 2024-07-27T03:12:06.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:06 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-27T03:12:07.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:06 smithi149 ceph-mon[37173]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:07.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:06 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:07.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:06 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:07.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:06 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:07.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:06 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:12:07.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:06 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:12:07.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:06 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:07.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:06 smithi149 ceph-mon[37173]: Deploying daemon mds.h on smithi149 2024-07-27T03:12:07.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:06 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] up:boot 2024-07-27T03:12:07.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:06 smithi149 ceph-mon[37173]: fsmap 7 up:standby 2024-07-27T03:12:07.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:06 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-27T03:12:07.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:06 smithi175 ceph-mon[37125]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:07.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:06 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:07.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:06 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:07.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:06 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:07.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:06 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:12:07.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:06 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:12:07.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:06 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:07.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:06 smithi175 ceph-mon[37125]: Deploying daemon mds.h on smithi149 2024-07-27T03:12:07.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:06 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] up:boot 2024-07-27T03:12:07.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:06 smithi175 ceph-mon[37125]: fsmap 7 up:standby 2024-07-27T03:12:07.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:06 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-27T03:12:07.750 INFO:journalctl@ceph.mds.h.smithi149.stdout:Jul 27 03:12:07 smithi149 systemd[1]: Starting Ceph mds.h for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:12:07.792 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:12:08.069 INFO:journalctl@ceph.mds.h.smithi149.stdout:Jul 27 03:12:07 smithi149 podman[66987]: 2024-07-27 03:12:07.665638006 +0000 UTC m=+0.018019558 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c 2024-07-27T03:12:08.069 INFO:journalctl@ceph.mds.h.smithi149.stdout:Jul 27 03:12:07 smithi149 podman[66987]: 2024-07-27 03:12:07.788446346 +0000 UTC m=+0.140827891 container create 7b2d25a41b9125fd0b4e619dfeb67515e479b736028fe144b562c84c9b4219fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.build-date=20240724, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-27T03:12:08.072 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:12:08.500 INFO:journalctl@ceph.mds.h.smithi149.stdout:Jul 27 03:12:08 smithi149 podman[66987]: 2024-07-27 03:12:08.066883717 +0000 UTC m=+0.419265282 container init 7b2d25a41b9125fd0b4e619dfeb67515e479b736028fe144b562c84c9b4219fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-h, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, GIT_BRANCH=HEAD) 2024-07-27T03:12:08.500 INFO:journalctl@ceph.mds.h.smithi149.stdout:Jul 27 03:12:08 smithi149 podman[66987]: 2024-07-27 03:12:08.070068666 +0000 UTC m=+0.422450212 container start 7b2d25a41b9125fd0b4e619dfeb67515e479b736028fe144b562c84c9b4219fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-h, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-07-27T03:12:08.501 INFO:journalctl@ceph.mds.h.smithi149.stdout:Jul 27 03:12:08 smithi149 ceph-mds[67006]: -- 172.21.15.149:0/3432066257 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e0852d9860 con 0x55e08947fa80 2024-07-27T03:12:08.501 INFO:journalctl@ceph.mds.h.smithi149.stdout:Jul 27 03:12:08 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-h[67002]: starting mds.h at 2024-07-27T03:12:08.501 INFO:journalctl@ceph.mds.h.smithi149.stdout:Jul 27 03:12:08 smithi149 bash[66987]: 7b2d25a41b9125fd0b4e619dfeb67515e479b736028fe144b562c84c9b4219fe 2024-07-27T03:12:08.501 INFO:journalctl@ceph.mds.h.smithi149.stdout:Jul 27 03:12:08 smithi149 systemd[1]: Started Ceph mds.h for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:12:08.895 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:08 smithi067 ceph-mon[32814]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:08.895 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:08 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:08.895 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:08 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:08.895 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:08 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:08.895 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:08 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:12:08.895 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:08 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:12:08.895 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:08 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:08.917 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:08 smithi175 ceph-mon[37125]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:08.917 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:08 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:08.917 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:08 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:08.917 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:08 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:08.917 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:08 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:12:08.917 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:08 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:12:08.917 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:08 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:09.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:08 smithi149 ceph-mon[37173]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:09.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:08 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:09.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:08 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:09.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:08 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:09.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:08 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:12:09.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:08 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:12:09.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:08 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:09.796 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:09 smithi175 ceph-mon[37125]: Deploying daemon mds.i on smithi175 2024-07-27T03:12:09.796 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:09 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] up:boot 2024-07-27T03:12:09.796 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:09 smithi175 ceph-mon[37125]: fsmap 8 up:standby 2024-07-27T03:12:09.796 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:09 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-27T03:12:09.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.790+0000 7f5546fbb640 1 -- 172.21.15.67:0/982222401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5540108760 msgr2=0x7f554010ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:09.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.790+0000 7f5546fbb640 1 --2- 172.21.15.67:0/982222401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5540108760 0x7f554010ef90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5538009ca0 tx=0x7f553802a8c0 comp rx=0 tx=0).stop 2024-07-27T03:12:09.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.790+0000 7f5546fbb640 1 -- 172.21.15.67:0/982222401 shutdown_connections 2024-07-27T03:12:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.790+0000 7f5546fbb640 1 --2- 172.21.15.67:0/982222401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5540108760 0x7f554010ef90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.790+0000 7f5546fbb640 1 --2- 172.21.15.67:0/982222401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5540107e10 0x7f5540108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.790+0000 7f5546fbb640 1 --2- 172.21.15.67:0/982222401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55401024d0 0x7f55401028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.790+0000 7f5546fbb640 1 -- 172.21.15.67:0/982222401 >> 172.21.15.67:0/982222401 conn(0x7f55400791a0 msgr2=0x7f55400795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.790+0000 7f5546fbb640 1 -- 172.21.15.67:0/982222401 shutdown_connections 2024-07-27T03:12:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.790+0000 7f5546fbb640 1 -- 172.21.15.67:0/982222401 wait complete. 2024-07-27T03:12:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.791+0000 7f5546fbb640 1 Processor -- start 2024-07-27T03:12:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.791+0000 7f5546fbb640 1 -- start start 2024-07-27T03:12:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.791+0000 7f5546fbb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55401024d0 0x7f5540198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:09.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.791+0000 7f5546fbb640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5540107e10 0x7f5540199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:09.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.791+0000 7f5546fbb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5540108760 0x7f554019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:09.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.791+0000 7f5546fbb640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f55401123c0 con 0x7f55401024d0 2024-07-27T03:12:09.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.791+0000 7f5546fbb640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5540112240 con 0x7f5540107e10 2024-07-27T03:12:09.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.791+0000 7f5546fbb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5540112540 con 0x7f5540108760 2024-07-27T03:12:09.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.791+0000 7f5544d57640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55401024d0 0x7f5540198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:09.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.791+0000 7f5544d57640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55401024d0 0x7f5540198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58930/0 (socket says 172.21.15.67:58930) 2024-07-27T03:12:09.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.791+0000 7f5544d57640 1 -- 172.21.15.67:0/1042703976 learned_addr learned my addr 172.21.15.67:0/1042703976 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:12:09.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.792+0000 7f553ffff640 1 --2- 172.21.15.67:0/1042703976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5540107e10 0x7f5540199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:09.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.792+0000 7f5545558640 1 --2- 172.21.15.67:0/1042703976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5540108760 0x7f554019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:09.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.792+0000 7f5544d57640 1 -- 172.21.15.67:0/1042703976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5540108760 msgr2=0x7f554019e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:09.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.792+0000 7f5544d57640 1 --2- 172.21.15.67:0/1042703976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5540108760 0x7f554019e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:09.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.792+0000 7f5544d57640 1 -- 172.21.15.67:0/1042703976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5540107e10 msgr2=0x7f5540199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:09.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.792+0000 7f5544d57640 1 --2- 172.21.15.67:0/1042703976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5540107e10 0x7f5540199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:09.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.792+0000 7f5544d57640 1 -- 172.21.15.67:0/1042703976 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554019ec20 con 0x7f55401024d0 2024-07-27T03:12:09.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.792+0000 7f5545558640 1 --2- 172.21.15.67:0/1042703976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5540108760 0x7f554019e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:12:09.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.792+0000 7f5544d57640 1 --2- 172.21.15.67:0/1042703976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55401024d0 0x7f5540198d70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f553400c920 tx=0x7f553400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:09.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.792+0000 7f553dffb640 1 -- 172.21.15.67:0/1042703976 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553401c070 con 0x7f55401024d0 2024-07-27T03:12:09.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.792+0000 7f5546fbb640 1 -- 172.21.15.67:0/1042703976 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f554019ef10 con 0x7f55401024d0 2024-07-27T03:12:09.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.792+0000 7f553dffb640 1 -- 172.21.15.67:0/1042703976 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5534007c70 con 0x7f55401024d0 2024-07-27T03:12:09.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.792+0000 7f553dffb640 1 -- 172.21.15.67:0/1042703976 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55340177b0 con 0x7f55401024d0 2024-07-27T03:12:09.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.792+0000 7f5546fbb640 1 -- 172.21.15.67:0/1042703976 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f554006a8d0 con 0x7f55401024d0 2024-07-27T03:12:09.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.793+0000 7f5546fbb640 1 -- 172.21.15.67:0/1042703976 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5540103750 con 0x7f55401024d0 2024-07-27T03:12:09.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.795+0000 7f553dffb640 1 -- 172.21.15.67:0/1042703976 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f553400f550 con 0x7f55401024d0 2024-07-27T03:12:09.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.796+0000 7f553dffb640 1 --2- 172.21.15.67:0/1042703976 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f55300b3c00 0x7f55300b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:09.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.796+0000 7f553ffff640 1 --2- 172.21.15.67:0/1042703976 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f55300b3c00 0x7f55300b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:09.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.796+0000 7f553dffb640 1 -- 172.21.15.67:0/1042703976 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f55340e5230 con 0x7f55401024d0 2024-07-27T03:12:09.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.796+0000 7f553ffff640 1 --2- 172.21.15.67:0/1042703976 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f55300b3c00 0x7f55300b6110 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f55400795f0 tx=0x7f552c00c4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:09.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:09.800+0000 7f553dffb640 1 -- 172.21.15.67:0/1042703976 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f5534054600 con 0x7f55401024d0 2024-07-27T03:12:09.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:09 smithi067 ceph-mon[32814]: Deploying daemon mds.i on smithi175 2024-07-27T03:12:09.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:09 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] up:boot 2024-07-27T03:12:09.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:09 smithi067 ceph-mon[32814]: fsmap 8 up:standby 2024-07-27T03:12:09.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:09 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-27T03:12:10.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:09 smithi149 ceph-mon[37173]: Deploying daemon mds.i on smithi175 2024-07-27T03:12:10.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:09 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] up:boot 2024-07-27T03:12:10.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:09 smithi149 ceph-mon[37173]: fsmap 8 up:standby 2024-07-27T03:12:10.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:09 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-27T03:12:10.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:10.021+0000 7f5546fbb640 1 -- 172.21.15.67:0/1042703976 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55401028e0 con 0x7f55401024d0 2024-07-27T03:12:10.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:10.022+0000 7f553dffb640 1 -- 172.21.15.67:0/1042703976 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7225 (secure 0 0 0) 0x7f55340acd20 con 0x7f55401024d0 2024-07-27T03:12:10.024 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:12:10.025 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":9,"btime":"2024-07-27T03:12:08:667757+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14679,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/252686369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":252686369},{"type":"v1","addr":"172.21.15.67:6835","nonce":252686369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/3358832762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3358832762},{"type":"v1","addr":"172.21.15.67:6837","nonce":3358832762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24526,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24533,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3868325215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3868325215},{"type":"v1","addr":"172.21.15.149:6833","nonce":3868325215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24562,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2462749027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2462749027},{"type":"v1","addr":"172.21.15.175:6835","nonce":2462749027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24563,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3730964806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3730964806},{"type":"v1","addr":"172.21.15.149:6835","nonce":3730964806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24581,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1304803502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1304803502},{"type":"v1","addr":"172.21.15.175:6837","nonce":1304803502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24587,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2932951845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2932951845},{"type":"v1","addr":"172.21.15.149:6837","nonce":2932951845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-07-27T03:12:10.025 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 9 2024-07-27T03:12:10.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:10.024+0000 7f5546fbb640 1 -- 172.21.15.67:0/1042703976 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f55300b3c00 msgr2=0x7f55300b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:10.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:10.024+0000 7f5546fbb640 1 --2- 172.21.15.67:0/1042703976 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f55300b3c00 0x7f55300b6110 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f55400795f0 tx=0x7f552c00c4a0 comp rx=0 tx=0).stop 2024-07-27T03:12:10.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:10.024+0000 7f5546fbb640 1 -- 172.21.15.67:0/1042703976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55401024d0 msgr2=0x7f5540198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:10.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:10.024+0000 7f5546fbb640 1 --2- 172.21.15.67:0/1042703976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55401024d0 0x7f5540198d70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f553400c920 tx=0x7f553400cde0 comp rx=0 tx=0).stop 2024-07-27T03:12:10.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:10.024+0000 7f5546fbb640 1 -- 172.21.15.67:0/1042703976 shutdown_connections 2024-07-27T03:12:10.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:10.024+0000 7f5546fbb640 1 --2- 172.21.15.67:0/1042703976 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f55300b3c00 0x7f55300b6110 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:10.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:10.024+0000 7f5546fbb640 1 --2- 172.21.15.67:0/1042703976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5540108760 0x7f554019e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:10.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:10.024+0000 7f5546fbb640 1 --2- 172.21.15.67:0/1042703976 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5540107e10 0x7f5540199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:10.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:10.024+0000 7f5546fbb640 1 --2- 172.21.15.67:0/1042703976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f55401024d0 0x7f5540198d70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:10.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:10.024+0000 7f5546fbb640 1 -- 172.21.15.67:0/1042703976 >> 172.21.15.67:0/1042703976 conn(0x7f55400791a0 msgr2=0x7f5540104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:10.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:10.024+0000 7f5546fbb640 1 -- 172.21.15.67:0/1042703976 shutdown_connections 2024-07-27T03:12:10.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:10.024+0000 7f5546fbb640 1 -- 172.21.15.67:0/1042703976 wait complete. 2024-07-27T03:12:10.375 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jul 27 03:12:10 smithi175 systemd[1]: Starting Ceph mds.i for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:12:10.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:10 smithi067 ceph-mon[32814]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:10.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:10 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1042703976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:12:10.737 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:10 smithi175 ceph-mon[37125]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:10.737 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:10 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1042703976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:12:10.738 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jul 27 03:12:10 smithi175 podman[67812]: 2024-07-27 03:12:10.386860183 +0000 UTC m=+0.017388032 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c 2024-07-27T03:12:10.738 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jul 27 03:12:10 smithi175 podman[67812]: 2024-07-27 03:12:10.496610523 +0000 UTC m=+0.127138359 container create 2cf721d5493231d6ae04a66d2498d4caecf8a0a87b0b037ace8f21899528e900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-i, org.label-schema.build-date=20240724, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-27T03:12:10.738 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jul 27 03:12:10 smithi175 podman[67812]: 2024-07-27 03:12:10.732864392 +0000 UTC m=+0.363392230 container init 2cf721d5493231d6ae04a66d2498d4caecf8a0a87b0b037ace8f21899528e900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-i, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-07-27T03:12:10.738 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jul 27 03:12:10 smithi175 podman[67812]: 2024-07-27 03:12:10.735463544 +0000 UTC m=+0.365991381 container start 2cf721d5493231d6ae04a66d2498d4caecf8a0a87b0b037ace8f21899528e900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-i, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, io.buildah.version=1.36.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-27T03:12:11.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:10 smithi149 ceph-mon[37173]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:11.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:10 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1042703976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:12:11.125 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jul 27 03:12:10 smithi175 ceph-mds[67831]: -- 172.21.15.175:0/3218394708 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555f731af860 con 0x555f77355600 2024-07-27T03:12:11.125 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jul 27 03:12:10 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-i[67827]: starting mds.i at 2024-07-27T03:12:11.125 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jul 27 03:12:10 smithi175 bash[67812]: 2cf721d5493231d6ae04a66d2498d4caecf8a0a87b0b037ace8f21899528e900 2024-07-27T03:12:11.125 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jul 27 03:12:10 smithi175 systemd[1]: Started Ceph mds.i for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:12:12.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:11 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:12.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:11 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:12.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:11 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:12.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:11 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:12:12.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:11 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:12:12.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:11 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:12.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:11 smithi067 ceph-mon[32814]: Deploying daemon mds.g on smithi067 2024-07-27T03:12:12.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:11 smithi067 ceph-mon[32814]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:12.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:11 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] up:boot 2024-07-27T03:12:12.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:11 smithi067 ceph-mon[32814]: fsmap 9 up:standby 2024-07-27T03:12:12.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:11 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-27T03:12:12.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:11 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:12.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:11 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:12.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:11 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:12.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:11 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:12:12.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:11 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:12:12.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:11 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:12.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:11 smithi149 ceph-mon[37173]: Deploying daemon mds.g on smithi067 2024-07-27T03:12:12.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:11 smithi149 ceph-mon[37173]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:12.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:11 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] up:boot 2024-07-27T03:12:12.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:11 smithi149 ceph-mon[37173]: fsmap 9 up:standby 2024-07-27T03:12:12.251 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:11 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-27T03:12:12.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:11 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:12.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:11 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:12.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:11 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:12.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:11 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:12:12.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:11 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:12:12.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:11 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:12.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:11 smithi175 ceph-mon[37125]: Deploying daemon mds.g on smithi067 2024-07-27T03:12:12.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:11 smithi175 ceph-mon[37125]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:12.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:11 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] up:boot 2024-07-27T03:12:12.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:11 smithi175 ceph-mon[37125]: fsmap 9 up:standby 2024-07-27T03:12:12.376 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:11 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-27T03:12:12.787 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:12:12.970 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 27 03:12:12 smithi067 systemd[1]: Starting Ceph mds.g for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:12:13.360 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 27 03:12:13 smithi067 podman[78014]: 2024-07-27 03:12:12.994443258 +0000 UTC m=+0.018256735 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c 2024-07-27T03:12:13.361 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 27 03:12:13 smithi067 podman[78014]: 2024-07-27 03:12:13.113248735 +0000 UTC m=+0.137062202 container create 2799be9e5935f99a2bc58273b326b5ba33eff03ddea6914ed5317920f76cb57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.36.0) 2024-07-27T03:12:13.361 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 27 03:12:13 smithi067 podman[78014]: 2024-07-27 03:12:13.358508757 +0000 UTC m=+0.382322228 container init 2799be9e5935f99a2bc58273b326b5ba33eff03ddea6914ed5317920f76cb57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-g, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724) 2024-07-27T03:12:13.489 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:12:13.614 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 27 03:12:13 smithi067 podman[78014]: 2024-07-27 03:12:13.361708875 +0000 UTC m=+0.385522350 container start 2799be9e5935f99a2bc58273b326b5ba33eff03ddea6914ed5317920f76cb57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-g, org.label-schema.build-date=20240724, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-27T03:12:13.614 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 27 03:12:13 smithi067 ceph-mds[78058]: -- 172.21.15.67:0/3759012246 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ffd15af860 con 0x55ffd5755180 2024-07-27T03:12:13.614 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 27 03:12:13 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-g[78048]: starting mds.g at 2024-07-27T03:12:13.614 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 27 03:12:13 smithi067 bash[78014]: 2799be9e5935f99a2bc58273b326b5ba33eff03ddea6914ed5317920f76cb57e 2024-07-27T03:12:13.614 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 27 03:12:13 smithi067 systemd[1]: Started Ceph mds.g for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:12:14.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:14 smithi067 ceph-mon[32814]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:14.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:14.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:14.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:14.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:12:14.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:12:14.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:14.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:14 smithi067 ceph-mon[32814]: Deploying daemon mds.a on smithi067 2024-07-27T03:12:14.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:14 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.67:6838/2162708317,v1:172.21.15.67:6839/2162708317] up:boot 2024-07-27T03:12:14.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:14 smithi067 ceph-mon[32814]: fsmap 10 up:standby 2024-07-27T03:12:14.810 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-27T03:12:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:14 smithi175 ceph-mon[37125]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:12:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:12:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:14.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:14 smithi175 ceph-mon[37125]: Deploying daemon mds.a on smithi067 2024-07-27T03:12:14.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:14 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.67:6838/2162708317,v1:172.21.15.67:6839/2162708317] up:boot 2024-07-27T03:12:14.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:14 smithi175 ceph-mon[37125]: fsmap 10 up:standby 2024-07-27T03:12:14.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-27T03:12:15.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:14 smithi149 ceph-mon[37173]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:15.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:15.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:15.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:15.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:12:15.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:12:15.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:15.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:14 smithi149 ceph-mon[37173]: Deploying daemon mds.a on smithi067 2024-07-27T03:12:15.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:14 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.67:6838/2162708317,v1:172.21.15.67:6839/2162708317] up:boot 2024-07-27T03:12:15.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:14 smithi149 ceph-mon[37173]: fsmap 10 up:standby 2024-07-27T03:12:15.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-27T03:12:15.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.349+0000 7f6638549640 1 -- 172.21.15.67:0/974304838 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f663010fc70 msgr2=0x7f6630112090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:15.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.349+0000 7f6638549640 1 --2- 172.21.15.67:0/974304838 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f663010fc70 0x7f6630112090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6628009ca0 tx=0x7f662802a950 comp rx=0 tx=0).stop 2024-07-27T03:12:15.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.349+0000 7f6638549640 1 -- 172.21.15.67:0/974304838 shutdown_connections 2024-07-27T03:12:15.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.349+0000 7f6638549640 1 --2- 172.21.15.67:0/974304838 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f663010fc70 0x7f6630112090 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:15.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.349+0000 7f6638549640 1 --2- 172.21.15.67:0/974304838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6630102a10 0x7f663010f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:15.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.349+0000 7f6638549640 1 --2- 172.21.15.67:0/974304838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66301020c0 0x7f66301024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:15.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.349+0000 7f6638549640 1 -- 172.21.15.67:0/974304838 >> 172.21.15.67:0/974304838 conn(0x7f66300791d0 msgr2=0x7f66300795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:15.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.349+0000 7f6638549640 1 -- 172.21.15.67:0/974304838 shutdown_connections 2024-07-27T03:12:15.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.349+0000 7f6638549640 1 -- 172.21.15.67:0/974304838 wait complete. 2024-07-27T03:12:15.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.350+0000 7f6638549640 1 Processor -- start 2024-07-27T03:12:15.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.350+0000 7f6638549640 1 -- start start 2024-07-27T03:12:15.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.350+0000 7f6638549640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66301020c0 0x7f663019c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:15.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.350+0000 7f6638549640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6630102a10 0x7f663019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:15.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.350+0000 7f6638549640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f663010fc70 0x7f66301a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:15.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.350+0000 7f6638549640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f66301156c0 con 0x7f66301020c0 2024-07-27T03:12:15.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.350+0000 7f6638549640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6630115540 con 0x7f6630102a10 2024-07-27T03:12:15.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.350+0000 7f6638549640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6630115840 con 0x7f663010fc70 2024-07-27T03:12:15.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.350+0000 7f66362e5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66301020c0 0x7f663019c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:15.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.350+0000 7f66362e5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66301020c0 0x7f663019c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49020/0 (socket says 172.21.15.67:49020) 2024-07-27T03:12:15.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.350+0000 7f66362e5640 1 -- 172.21.15.67:0/1072749099 learned_addr learned my addr 172.21.15.67:0/1072749099 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:12:15.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.350+0000 7f6635ae4640 1 --2- 172.21.15.67:0/1072749099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6630102a10 0x7f663019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:15.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.350+0000 7f6636ae6640 1 --2- 172.21.15.67:0/1072749099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f663010fc70 0x7f66301a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:15.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.350+0000 7f66362e5640 1 -- 172.21.15.67:0/1072749099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f663010fc70 msgr2=0x7f66301a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:15.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.350+0000 7f66362e5640 1 --2- 172.21.15.67:0/1072749099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f663010fc70 0x7f66301a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:15.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.350+0000 7f66362e5640 1 -- 172.21.15.67:0/1072749099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6630102a10 msgr2=0x7f663019c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:15.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.350+0000 7f66362e5640 1 --2- 172.21.15.67:0/1072749099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6630102a10 0x7f663019c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:15.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.350+0000 7f66362e5640 1 -- 172.21.15.67:0/1072749099 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66301a2000 con 0x7f66301020c0 2024-07-27T03:12:15.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.350+0000 7f6636ae6640 1 --2- 172.21.15.67:0/1072749099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f663010fc70 0x7f66301a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:12:15.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.351+0000 7f66362e5640 1 --2- 172.21.15.67:0/1072749099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66301020c0 0x7f663019c090 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f662c00d950 tx=0x7f662c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:15.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.351+0000 7f66277fe640 1 -- 172.21.15.67:0/1072749099 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c017030 con 0x7f66301020c0 2024-07-27T03:12:15.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.351+0000 7f6638549640 1 -- 172.21.15.67:0/1072749099 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66301a22f0 con 0x7f66301020c0 2024-07-27T03:12:15.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.351+0000 7f66277fe640 1 -- 172.21.15.67:0/1072749099 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f662c004020 con 0x7f66301020c0 2024-07-27T03:12:15.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.351+0000 7f66277fe640 1 -- 172.21.15.67:0/1072749099 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c0047a0 con 0x7f66301020c0 2024-07-27T03:12:15.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.351+0000 7f6638549640 1 -- 172.21.15.67:0/1072749099 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66301a8a50 con 0x7f66301020c0 2024-07-27T03:12:15.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.352+0000 7f6638549640 1 -- 172.21.15.67:0/1072749099 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f663006bc60 con 0x7f66301020c0 2024-07-27T03:12:15.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.353+0000 7f66277fe640 1 -- 172.21.15.67:0/1072749099 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f662c009a30 con 0x7f66301020c0 2024-07-27T03:12:15.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.354+0000 7f66277fe640 1 --2- 172.21.15.67:0/1072749099 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f661c0b3bd0 0x7f661c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:15.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.354+0000 7f6635ae4640 1 --2- 172.21.15.67:0/1072749099 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f661c0b3bd0 0x7f661c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:15.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.354+0000 7f66277fe640 1 -- 172.21.15.67:0/1072749099 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f662c0e3860 con 0x7f66301020c0 2024-07-27T03:12:15.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.355+0000 7f6635ae4640 1 --2- 172.21.15.67:0/1072749099 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f661c0b3bd0 0x7f661c0b60e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6630111b40 tx=0x7f662000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:15.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.357+0000 7f66277fe640 1 -- 172.21.15.67:0/1072749099 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f662c0ab350 con 0x7f66301020c0 2024-07-27T03:12:15.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.582+0000 7f6638549640 1 -- 172.21.15.67:0/1072749099 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66301024d0 con 0x7f66301020c0 2024-07-27T03:12:15.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.583+0000 7f66277fe640 1 -- 172.21.15.67:0/1072749099 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8881 (secure 0 0 0) 0x7f662c0aac50 con 0x7f66301020c0 2024-07-27T03:12:15.586 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:12:15.586 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":11,"btime":"2024-07-27T03:12:14:134410+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14679,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/252686369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":252686369},{"type":"v1","addr":"172.21.15.67:6835","nonce":252686369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/3358832762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3358832762},{"type":"v1","addr":"172.21.15.67:6837","nonce":3358832762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/2162708317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2162708317},{"type":"v1","addr":"172.21.15.67:6839","nonce":2162708317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24526,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24533,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3868325215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3868325215},{"type":"v1","addr":"172.21.15.149:6833","nonce":3868325215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24562,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2462749027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2462749027},{"type":"v1","addr":"172.21.15.175:6835","nonce":2462749027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24563,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3730964806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3730964806},{"type":"v1","addr":"172.21.15.149:6835","nonce":3730964806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24581,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1304803502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1304803502},{"type":"v1","addr":"172.21.15.175:6837","nonce":1304803502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24583,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/1670568853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1670568853},{"type":"v1","addr":"172.21.15.175:6839","nonce":1670568853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24587,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2932951845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2932951845},{"type":"v1","addr":"172.21.15.149:6837","nonce":2932951845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-07-27T03:12:15.586 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 11 2024-07-27T03:12:15.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.586+0000 7f6638549640 1 -- 172.21.15.67:0/1072749099 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f661c0b3bd0 msgr2=0x7f661c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:15.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.586+0000 7f6638549640 1 --2- 172.21.15.67:0/1072749099 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f661c0b3bd0 0x7f661c0b60e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6630111b40 tx=0x7f662000a2d0 comp rx=0 tx=0).stop 2024-07-27T03:12:15.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.586+0000 7f6638549640 1 -- 172.21.15.67:0/1072749099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66301020c0 msgr2=0x7f663019c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:15.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.586+0000 7f6638549640 1 --2- 172.21.15.67:0/1072749099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66301020c0 0x7f663019c090 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f662c00d950 tx=0x7f662c00de10 comp rx=0 tx=0).stop 2024-07-27T03:12:15.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.586+0000 7f6638549640 1 -- 172.21.15.67:0/1072749099 shutdown_connections 2024-07-27T03:12:15.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.586+0000 7f6638549640 1 --2- 172.21.15.67:0/1072749099 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f661c0b3bd0 0x7f661c0b60e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:15.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.586+0000 7f6638549640 1 --2- 172.21.15.67:0/1072749099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f663010fc70 0x7f66301a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:15.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.586+0000 7f6638549640 1 --2- 172.21.15.67:0/1072749099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6630102a10 0x7f663019c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:15.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.586+0000 7f6638549640 1 --2- 172.21.15.67:0/1072749099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f66301020c0 0x7f663019c090 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:15.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.586+0000 7f6638549640 1 -- 172.21.15.67:0/1072749099 >> 172.21.15.67:0/1072749099 conn(0x7f66300791d0 msgr2=0x7f6630109810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:15.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.586+0000 7f6638549640 1 -- 172.21.15.67:0/1072749099 shutdown_connections 2024-07-27T03:12:15.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:15.586+0000 7f6638549640 1 -- 172.21.15.67:0/1072749099 wait complete. 2024-07-27T03:12:16.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:16 smithi175 ceph-mon[37125]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:16.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:16 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1072749099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:12:16.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:16 smithi067 ceph-mon[32814]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:16.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:16 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1072749099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:12:17.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:16 smithi149 ceph-mon[37173]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:17.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:16 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1072749099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:12:17.470 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:12:17 smithi067 systemd[1]: Starting Ceph mds.a for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:12:17.816 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:12:17 smithi067 podman[78455]: 2024-07-27 03:12:17.451644726 +0000 UTC m=+0.017734236 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c 2024-07-27T03:12:17.816 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:12:17 smithi067 podman[78455]: 2024-07-27 03:12:17.578195352 +0000 UTC m=+0.144284851 container create ae35ee434933d6980741f938a36213a5ff09d50daf78c79ab1cfef7a41674c46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-a, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0) 2024-07-27T03:12:17.816 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:12:17 smithi067 podman[78455]: 2024-07-27 03:12:17.814419825 +0000 UTC m=+0.380509326 container init ae35ee434933d6980741f938a36213a5ff09d50daf78c79ab1cfef7a41674c46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, io.buildah.version=1.36.0, ceph=True, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-07-27T03:12:18.220 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:12:17 smithi067 podman[78455]: 2024-07-27 03:12:17.817525661 +0000 UTC m=+0.383615170 container start ae35ee434933d6980741f938a36213a5ff09d50daf78c79ab1cfef7a41674c46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-07-27T03:12:18.220 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:12:17 smithi067 ceph-mds[78474]: -- 172.21.15.67:0/490431073 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5588c0089860 con 0x5588c422f600 2024-07-27T03:12:18.220 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:12:17 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-a[78470]: starting mds.a at 2024-07-27T03:12:18.220 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:12:17 smithi067 bash[78455]: ae35ee434933d6980741f938a36213a5ff09d50daf78c79ab1cfef7a41674c46 2024-07-27T03:12:18.220 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:12:17 smithi067 systemd[1]: Started Ceph mds.a for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:12:18.447 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:12:18.672 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:18 smithi149 ceph-mon[37173]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:18.672 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:18 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:18.672 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:18 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:18.672 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:18 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:18.672 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:18 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:12:18.672 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:18 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:12:18.672 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:18 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:18.726 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:12:18.848 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:18 smithi067 ceph-mon[32814]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:18.849 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:18 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:18.849 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:18 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:18.849 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:18 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:18.849 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:18 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:12:18.849 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:18 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:12:18.849 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:18 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:18.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:18 smithi175 ceph-mon[37125]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:18.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:18 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:18.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:18 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:18.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:18 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:18.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:18 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-07-27T03:12:18.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:18 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-07-27T03:12:18.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:18 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:19.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:19 smithi149 ceph-mon[37173]: Deploying daemon mds.b on smithi149 2024-07-27T03:12:19.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:19 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.67:6840/795704244,v1:172.21.15.67:6841/795704244] up:boot 2024-07-27T03:12:19.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:19 smithi149 ceph-mon[37173]: fsmap 11 up:standby 2024-07-27T03:12:19.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:19 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-27T03:12:19.825 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:19 smithi067 ceph-mon[32814]: Deploying daemon mds.b on smithi149 2024-07-27T03:12:19.826 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:19 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.67:6840/795704244,v1:172.21.15.67:6841/795704244] up:boot 2024-07-27T03:12:19.826 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:19 smithi067 ceph-mon[32814]: fsmap 11 up:standby 2024-07-27T03:12:19.827 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:19 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-27T03:12:19.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:19 smithi175 ceph-mon[37125]: Deploying daemon mds.b on smithi149 2024-07-27T03:12:19.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:19 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.67:6840/795704244,v1:172.21.15.67:6841/795704244] up:boot 2024-07-27T03:12:19.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:19 smithi175 ceph-mon[37125]: fsmap 11 up:standby 2024-07-27T03:12:19.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:19 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-27T03:12:20.229 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:12:20 smithi149 systemd[1]: Starting Ceph mds.b for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:12:20.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.345+0000 7fd1fadde640 1 -- 172.21.15.67:0/1957271244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1f41066f0 msgr2=0x7fd1f4110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:20.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.345+0000 7fd1fadde640 1 --2- 172.21.15.67:0/1957271244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1f41066f0 0x7fd1f4110140 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009ca0 tx=0x7fd1ec02a8c0 comp rx=0 tx=0).stop 2024-07-27T03:12:20.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.345+0000 7fd1fadde640 1 -- 172.21.15.67:0/1957271244 shutdown_connections 2024-07-27T03:12:20.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.345+0000 7fd1fadde640 1 --2- 172.21.15.67:0/1957271244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1f41066f0 0x7fd1f4110140 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:20.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.345+0000 7fd1fadde640 1 --2- 172.21.15.67:0/1957271244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1f4105da0 0x7fd1f41061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:20.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.345+0000 7fd1fadde640 1 --2- 172.21.15.67:0/1957271244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1f4101bb0 0x7fd1f4101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:20.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.345+0000 7fd1fadde640 1 -- 172.21.15.67:0/1957271244 >> 172.21.15.67:0/1957271244 conn(0x7fd1f4079910 msgr2=0x7fd1f4079d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:20.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.346+0000 7fd1fadde640 1 -- 172.21.15.67:0/1957271244 shutdown_connections 2024-07-27T03:12:20.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.346+0000 7fd1fadde640 1 -- 172.21.15.67:0/1957271244 wait complete. 2024-07-27T03:12:20.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.346+0000 7fd1fadde640 1 Processor -- start 2024-07-27T03:12:20.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.346+0000 7fd1fadde640 1 -- start start 2024-07-27T03:12:20.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.346+0000 7fd1fadde640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1f4101bb0 0x7fd1f4199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:20.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.347+0000 7fd1fadde640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1f4105da0 0x7fd1f419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:20.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.347+0000 7fd1fadde640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1f41066f0 0x7fd1f419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:20.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.347+0000 7fd1fadde640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4113590 con 0x7fd1f4101bb0 2024-07-27T03:12:20.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.347+0000 7fd1fadde640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4113410 con 0x7fd1f4105da0 2024-07-27T03:12:20.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.347+0000 7fd1fadde640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4113710 con 0x7fd1f41066f0 2024-07-27T03:12:20.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.347+0000 7fd1f8b7a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1f4101bb0 0x7fd1f4199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:20.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.347+0000 7fd1f8b7a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1f4101bb0 0x7fd1f4199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49048/0 (socket says 172.21.15.67:49048) 2024-07-27T03:12:20.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.347+0000 7fd1f8b7a640 1 -- 172.21.15.67:0/3557212702 learned_addr learned my addr 172.21.15.67:0/3557212702 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:12:20.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.347+0000 7fd1f3fff640 1 --2- 172.21.15.67:0/3557212702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1f4105da0 0x7fd1f419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:20.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.347+0000 7fd1f937b640 1 --2- 172.21.15.67:0/3557212702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1f41066f0 0x7fd1f419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.347+0000 7fd1f8b7a640 1 -- 172.21.15.67:0/3557212702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1f41066f0 msgr2=0x7fd1f419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.347+0000 7fd1f8b7a640 1 --2- 172.21.15.67:0/3557212702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1f41066f0 0x7fd1f419f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.347+0000 7fd1f8b7a640 1 -- 172.21.15.67:0/3557212702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1f4105da0 msgr2=0x7fd1f419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.347+0000 7fd1f8b7a640 1 --2- 172.21.15.67:0/3557212702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1f4105da0 0x7fd1f419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.347+0000 7fd1f8b7a640 1 -- 172.21.15.67:0/3557212702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f419fe90 con 0x7fd1f4101bb0 2024-07-27T03:12:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.347+0000 7fd1f8b7a640 1 --2- 172.21.15.67:0/3557212702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1f4101bb0 0x7fd1f4199f20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800e9f0 tx=0x7fd1e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.347+0000 7fd1f1ffb640 1 -- 172.21.15.67:0/3557212702 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e800cd80 con 0x7fd1f4101bb0 2024-07-27T03:12:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.347+0000 7fd1f1ffb640 1 -- 172.21.15.67:0/3557212702 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1e80108e0 con 0x7fd1f4101bb0 2024-07-27T03:12:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.347+0000 7fd1f1ffb640 1 -- 172.21.15.67:0/3557212702 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8018980 con 0x7fd1f4101bb0 2024-07-27T03:12:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.347+0000 7fd1fadde640 1 -- 172.21.15.67:0/3557212702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f41a0180 con 0x7fd1f4101bb0 2024-07-27T03:12:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.348+0000 7fd1fadde640 1 -- 172.21.15.67:0/3557212702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f41a68e0 con 0x7fd1f4101bb0 2024-07-27T03:12:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.349+0000 7fd1fadde640 1 -- 172.21.15.67:0/3557212702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f4101fc0 con 0x7fd1f4101bb0 2024-07-27T03:12:20.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.350+0000 7fd1f1ffb640 1 -- 172.21.15.67:0/3557212702 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7fd1e8010430 con 0x7fd1f4101bb0 2024-07-27T03:12:20.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.351+0000 7fd1f1ffb640 1 --2- 172.21.15.67:0/3557212702 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd1e40b3bd0 0x7fd1e40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:20.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.351+0000 7fd1f3fff640 1 --2- 172.21.15.67:0/3557212702 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd1e40b3bd0 0x7fd1e40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:20.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.351+0000 7fd1f1ffb640 1 -- 172.21.15.67:0/3557212702 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fd1e80e3fe0 con 0x7fd1f4101bb0 2024-07-27T03:12:20.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.351+0000 7fd1f3fff640 1 --2- 172.21.15.67:0/3557212702 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd1e40b3bd0 0x7fd1e40b60e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4108ae0 tx=0x7fd1e000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:20.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.354+0000 7fd1f1ffb640 1 -- 172.21.15.67:0/3557212702 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fd1e80abad0 con 0x7fd1f4101bb0 2024-07-27T03:12:20.495 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:12:20 smithi149 podman[67291]: 2024-07-27 03:12:20.129881914 +0000 UTC m=+0.017543598 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c 2024-07-27T03:12:20.496 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:12:20 smithi149 podman[67291]: 2024-07-27 03:12:20.245321687 +0000 UTC m=+0.132983413 container create e6c2c871b8ee6267eb99eff183a84319999f3b8341281a7ffc0d4e615fdc9ef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, io.buildah.version=1.36.0, ceph=True) 2024-07-27T03:12:20.496 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:12:20 smithi149 podman[67291]: 2024-07-27 03:12:20.457173215 +0000 UTC m=+0.344834920 container init e6c2c871b8ee6267eb99eff183a84319999f3b8341281a7ffc0d4e615fdc9ef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-07-27T03:12:20.496 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:12:20 smithi149 podman[67291]: 2024-07-27 03:12:20.459955993 +0000 UTC m=+0.347617680 container start e6c2c871b8ee6267eb99eff183a84319999f3b8341281a7ffc0d4e615fdc9ef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-b, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, io.buildah.version=1.36.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724) 2024-07-27T03:12:20.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.574+0000 7fd1fadde640 1 -- 172.21.15.67:0/3557212702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1f402e100 con 0x7fd1f4101bb0 2024-07-27T03:12:20.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.575+0000 7fd1f1ffb640 1 -- 172.21.15.67:0/3557212702 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10530 (secure 0 0 0) 0x7fd1e80ab3d0 con 0x7fd1f4101bb0 2024-07-27T03:12:20.578 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:12:20.578 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":13,"btime":"2024-07-27T03:12:20:527634+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14679,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/252686369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":252686369},{"type":"v1","addr":"172.21.15.67:6835","nonce":252686369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/3358832762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3358832762},{"type":"v1","addr":"172.21.15.67:6837","nonce":3358832762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/2162708317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2162708317},{"type":"v1","addr":"172.21.15.67:6839","nonce":2162708317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/795704244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":795704244},{"type":"v1","addr":"172.21.15.67:6841","nonce":795704244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24526,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24533,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3868325215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3868325215},{"type":"v1","addr":"172.21.15.149:6833","nonce":3868325215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24562,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2462749027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2462749027},{"type":"v1","addr":"172.21.15.175:6835","nonce":2462749027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24563,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3730964806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3730964806},{"type":"v1","addr":"172.21.15.149:6835","nonce":3730964806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24581,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1304803502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1304803502},{"type":"v1","addr":"172.21.15.175:6837","nonce":1304803502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24583,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/1670568853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1670568853},{"type":"v1","addr":"172.21.15.175:6839","nonce":1670568853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24587,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2932951845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2932951845},{"type":"v1","addr":"172.21.15.149:6837","nonce":2932951845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24620,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/281746498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":281746498},{"type":"v1","addr":"172.21.15.149:6839","nonce":281746498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-07-27T03:12:20.578 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 13 2024-07-27T03:12:20.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.577+0000 7fd1fadde640 1 -- 172.21.15.67:0/3557212702 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd1e40b3bd0 msgr2=0x7fd1e40b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:20.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.577+0000 7fd1fadde640 1 --2- 172.21.15.67:0/3557212702 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd1e40b3bd0 0x7fd1e40b60e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4108ae0 tx=0x7fd1e000b390 comp rx=0 tx=0).stop 2024-07-27T03:12:20.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.577+0000 7fd1fadde640 1 -- 172.21.15.67:0/3557212702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1f4101bb0 msgr2=0x7fd1f4199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:20.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.577+0000 7fd1fadde640 1 --2- 172.21.15.67:0/3557212702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1f4101bb0 0x7fd1f4199f20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800e9f0 tx=0x7fd1e800eeb0 comp rx=0 tx=0).stop 2024-07-27T03:12:20.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.577+0000 7fd1fadde640 1 -- 172.21.15.67:0/3557212702 shutdown_connections 2024-07-27T03:12:20.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.577+0000 7fd1fadde640 1 --2- 172.21.15.67:0/3557212702 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd1e40b3bd0 0x7fd1e40b60e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:20.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.577+0000 7fd1fadde640 1 --2- 172.21.15.67:0/3557212702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1f41066f0 0x7fd1f419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:20.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.577+0000 7fd1fadde640 1 --2- 172.21.15.67:0/3557212702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd1f4105da0 0x7fd1f419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:20.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.577+0000 7fd1fadde640 1 --2- 172.21.15.67:0/3557212702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1f4101bb0 0x7fd1f4199f20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:20.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.577+0000 7fd1fadde640 1 -- 172.21.15.67:0/3557212702 >> 172.21.15.67:0/3557212702 conn(0x7fd1f4079910 msgr2=0x7fd1f4107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:20.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.577+0000 7fd1fadde640 1 -- 172.21.15.67:0/3557212702 shutdown_connections 2024-07-27T03:12:20.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:20.577+0000 7fd1fadde640 1 -- 172.21.15.67:0/3557212702 wait complete. 2024-07-27T03:12:20.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:20 smithi149 ceph-mon[37173]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:20.750 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:12:20 smithi149 ceph-mds[67310]: -- 172.21.15.149:0/1305215232 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557041589860 con 0x55704572f180 2024-07-27T03:12:20.750 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:12:20 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-b[67306]: starting mds.b at 2024-07-27T03:12:20.750 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:12:20 smithi149 bash[67291]: e6c2c871b8ee6267eb99eff183a84319999f3b8341281a7ffc0d4e615fdc9ef2 2024-07-27T03:12:20.751 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:12:20 smithi149 systemd[1]: Started Ceph mds.b for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:12:20.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:20 smithi175 ceph-mon[37125]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:20.900 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:20 smithi067 ceph-mon[32814]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:21.196 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-07-27T03:12:21.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:21 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] up:boot 2024-07-27T03:12:21.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:21 smithi175 ceph-mon[37125]: fsmap 12 up:standby 2024-07-27T03:12:21.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-27T03:12:21.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:21 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3557212702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:12:21.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:21.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:21.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:21.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:21.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:21.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:21 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:12:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:21 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] up:boot 2024-07-27T03:12:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:21 smithi149 ceph-mon[37173]: fsmap 12 up:standby 2024-07-27T03:12:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-27T03:12:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:21 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3557212702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:12:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:21.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:21.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:21.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:21 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:12:21.863 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:21 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] up:boot 2024-07-27T03:12:21.863 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:21 smithi067 ceph-mon[32814]: fsmap 12 up:standby 2024-07-27T03:12:21.863 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-27T03:12:21.863 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:21 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3557212702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:12:21.863 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:21.863 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:21.864 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:21.864 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:21.864 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:21.864 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:21 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:12:22.168 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:12:22.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:22 smithi067 ceph-mon[32814]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:22.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:22 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:22.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:22 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:22.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:22 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:22.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:22 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:23.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:22 smithi149 ceph-mon[37173]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:23.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:22 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:23.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:22 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:23.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:22 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:23.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:22 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:23.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:22 smithi175 ceph-mon[37125]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:23.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:22 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:23.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:22 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:23.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:22 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:23.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:22 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:23.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.815+0000 7f69ea445640 1 -- 172.21.15.67:0/1453010642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e4108f10 msgr2=0x7f69e4109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:23.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.815+0000 7f69ea445640 1 --2- 172.21.15.67:0/1453010642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e4108f10 0x7f69e4109320 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009900 tx=0x7f69d802a410 comp rx=0 tx=0).stop 2024-07-27T03:12:23.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.815+0000 7f69ea445640 1 -- 172.21.15.67:0/1453010642 shutdown_connections 2024-07-27T03:12:23.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.815+0000 7f69ea445640 1 --2- 172.21.15.67:0/1453010642 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69e4109860 0x7f69e4110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:23.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.815+0000 7f69ea445640 1 --2- 172.21.15.67:0/1453010642 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e4108f10 0x7f69e4109320 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:23.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.815+0000 7f69ea445640 1 --2- 172.21.15.67:0/1453010642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69e41035d0 0x7f69e41039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:23.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.815+0000 7f69ea445640 1 -- 172.21.15.67:0/1453010642 >> 172.21.15.67:0/1453010642 conn(0x7f69e407b2c0 msgr2=0x7f69e407b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:23.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.815+0000 7f69ea445640 1 -- 172.21.15.67:0/1453010642 shutdown_connections 2024-07-27T03:12:23.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.815+0000 7f69ea445640 1 -- 172.21.15.67:0/1453010642 wait complete. 2024-07-27T03:12:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.816+0000 7f69ea445640 1 Processor -- start 2024-07-27T03:12:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.816+0000 7f69ea445640 1 -- start start 2024-07-27T03:12:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.817+0000 7f69ea445640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e41035d0 0x7f69e4199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.817+0000 7f69ea445640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69e4108f10 0x7f69e419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.817+0000 7f69ea445640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69e4109860 0x7f69e419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.817+0000 7f69ea445640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f69e41134f0 con 0x7f69e41035d0 2024-07-27T03:12:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.817+0000 7f69ea445640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f69e4113370 con 0x7f69e4109860 2024-07-27T03:12:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.817+0000 7f69ea445640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69e4113670 con 0x7f69e4108f10 2024-07-27T03:12:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.817+0000 7f69e3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e41035d0 0x7f69e4199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.817+0000 7f69e3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e41035d0 0x7f69e4199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49064/0 (socket says 172.21.15.67:49064) 2024-07-27T03:12:23.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.817+0000 7f69e3fff640 1 -- 172.21.15.67:0/3222888009 learned_addr learned my addr 172.21.15.67:0/3222888009 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:12:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.817+0000 7f69e37fe640 1 --2- 172.21.15.67:0/3222888009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69e4108f10 0x7f69e419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.817+0000 7f69e89e2640 1 --2- 172.21.15.67:0/3222888009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69e4109860 0x7f69e419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.817+0000 7f69e3fff640 1 -- 172.21.15.67:0/3222888009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69e4108f10 msgr2=0x7f69e419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.817+0000 7f69e3fff640 1 --2- 172.21.15.67:0/3222888009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69e4108f10 0x7f69e419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.817+0000 7f69e3fff640 1 -- 172.21.15.67:0/3222888009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69e4109860 msgr2=0x7f69e419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.817+0000 7f69e3fff640 1 --2- 172.21.15.67:0/3222888009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69e4109860 0x7f69e419f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.817+0000 7f69e3fff640 1 -- 172.21.15.67:0/3222888009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e419fdd0 con 0x7f69e41035d0 2024-07-27T03:12:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.817+0000 7f69e89e2640 1 --2- 172.21.15.67:0/3222888009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69e4109860 0x7f69e419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:12:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.817+0000 7f69e3fff640 1 --2- 172.21.15.67:0/3222888009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e41035d0 0x7f69e4199f20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f69d400c920 tx=0x7f69d400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.817+0000 7f69e17fa640 1 -- 172.21.15.67:0/3222888009 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4007c70 con 0x7f69e41035d0 2024-07-27T03:12:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.818+0000 7f69e17fa640 1 -- 172.21.15.67:0/3222888009 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69d4007e10 con 0x7f69e41035d0 2024-07-27T03:12:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.818+0000 7f69ea445640 1 -- 172.21.15.67:0/3222888009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e41a00c0 con 0x7f69e41035d0 2024-07-27T03:12:23.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.818+0000 7f69e17fa640 1 -- 172.21.15.67:0/3222888009 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d4018820 con 0x7f69e41035d0 2024-07-27T03:12:23.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.818+0000 7f69ea445640 1 -- 172.21.15.67:0/3222888009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e41a68e0 con 0x7f69e41035d0 2024-07-27T03:12:23.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.819+0000 7f69ea445640 1 -- 172.21.15.67:0/3222888009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e41039e0 con 0x7f69e41035d0 2024-07-27T03:12:23.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.821+0000 7f69e17fa640 1 -- 172.21.15.67:0/3222888009 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f69d400f450 con 0x7f69e41035d0 2024-07-27T03:12:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.822+0000 7f69e17fa640 1 --2- 172.21.15.67:0/3222888009 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f69cc0b3c00 0x7f69cc0b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.822+0000 7f69e37fe640 1 --2- 172.21.15.67:0/3222888009 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f69cc0b3c00 0x7f69cc0b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.822+0000 7f69e17fa640 1 -- 172.21.15.67:0/3222888009 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f69d40e3ac0 con 0x7f69e41035d0 2024-07-27T03:12:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.823+0000 7f69e37fe640 1 --2- 172.21.15.67:0/3222888009 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f69cc0b3c00 0x7f69cc0b6110 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009230 tx=0x7f69d8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:23.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:23.825+0000 7f69e17fa640 1 -- 172.21.15.67:0/3222888009 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f69d4014030 con 0x7f69e41035d0 2024-07-27T03:12:24.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:24.019+0000 7f69ea445640 1 -- 172.21.15.67:0/3222888009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f69e410b080 con 0x7f69e41035d0 2024-07-27T03:12:24.025 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:24.022+0000 7f69e17fa640 1 -- 172.21.15.67:0/3222888009 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) ==== 84+0+65 (secure 0 0 0) 0x7f69d40ab640 con 0x7f69e41035d0 2024-07-27T03:12:24.025 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:12:24.025 INFO:teuthology.orchestra.run.smithi067.stdout:[ 2024-07-27T03:12:24.025 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:12:24.025 INFO:teuthology.orchestra.run.smithi067.stdout: "poolnum": 1, 2024-07-27T03:12:24.025 INFO:teuthology.orchestra.run.smithi067.stdout: "poolname": ".mgr" 2024-07-27T03:12:24.025 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:12:24.025 INFO:teuthology.orchestra.run.smithi067.stdout:] 2024-07-27T03:12:24.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:24.024+0000 7f69ea445640 1 -- 172.21.15.67:0/3222888009 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f69cc0b3c00 msgr2=0x7f69cc0b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:24.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:24.024+0000 7f69ea445640 1 --2- 172.21.15.67:0/3222888009 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f69cc0b3c00 0x7f69cc0b6110 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009230 tx=0x7f69d8031400 comp rx=0 tx=0).stop 2024-07-27T03:12:24.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:24.024+0000 7f69ea445640 1 -- 172.21.15.67:0/3222888009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e41035d0 msgr2=0x7f69e4199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:24.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:24.024+0000 7f69ea445640 1 --2- 172.21.15.67:0/3222888009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e41035d0 0x7f69e4199f20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f69d400c920 tx=0x7f69d400cde0 comp rx=0 tx=0).stop 2024-07-27T03:12:24.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:24.024+0000 7f69ea445640 1 -- 172.21.15.67:0/3222888009 shutdown_connections 2024-07-27T03:12:24.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:24.024+0000 7f69ea445640 1 --2- 172.21.15.67:0/3222888009 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f69cc0b3c00 0x7f69cc0b6110 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:24.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:24.024+0000 7f69ea445640 1 --2- 172.21.15.67:0/3222888009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f69e4109860 0x7f69e419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:24.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:24.024+0000 7f69ea445640 1 --2- 172.21.15.67:0/3222888009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69e4108f10 0x7f69e419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:24.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:24.024+0000 7f69ea445640 1 --2- 172.21.15.67:0/3222888009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f69e41035d0 0x7f69e4199f20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:24.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:24.024+0000 7f69ea445640 1 -- 172.21.15.67:0/3222888009 >> 172.21.15.67:0/3222888009 conn(0x7f69e407b2c0 msgr2=0x7f69e4107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:24.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:24.024+0000 7f69ea445640 1 -- 172.21.15.67:0/3222888009 shutdown_connections 2024-07-27T03:12:24.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:24.024+0000 7f69ea445640 1 -- 172.21.15.67:0/3222888009 wait complete. 2024-07-27T03:12:24.756 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-07-27T03:12:24.757 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-07-27T03:12:24.908 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:24 smithi149 ceph-mon[37173]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:24.908 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:24 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:24.908 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:24 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3222888009' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-07-27T03:12:24.908 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:24 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:24.908 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:24 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:24.909 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:24 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:12:24.909 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:24 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:24.909 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:24 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:12:24.911 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:24 smithi175 ceph-mon[37125]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:24.911 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:24 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:24.911 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:24 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3222888009' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-07-27T03:12:24.911 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:24 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:24.911 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:24 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:24.911 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:24 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:12:24.912 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:24 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:24.912 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:24 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:12:24.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:24 smithi067 ceph-mon[32814]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:24.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:24 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:24.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:24 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3222888009' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-07-27T03:12:24.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:24 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:24.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:24 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:24.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:24 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:12:24.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:24 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:24.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:24 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:12:25.685 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:12:26.073 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:26 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:26.073 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:26 smithi067 ceph-mon[32814]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:26.073 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:26 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:26.074 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:26 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:26.074 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:26 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:26.074 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:26 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:26.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:26 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:26.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:26 smithi175 ceph-mon[37125]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:26.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:26 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:26.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:26 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:26.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:26 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:26.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:26 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:26.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:26 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:26.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:26 smithi149 ceph-mon[37173]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:26.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:26 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:26.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:26 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:26.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:26 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:26.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:26 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:27.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.360+0000 7f185cb4d640 1 -- 172.21.15.67:0/1883773701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1858102550 msgr2=0x7f1858102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:27.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.360+0000 7f185cb4d640 1 --2- 172.21.15.67:0/1883773701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1858102550 0x7f1858102960 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1848009900 tx=0x7f184802a410 comp rx=0 tx=0).stop 2024-07-27T03:12:27.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.360+0000 7f185cb4d640 1 -- 172.21.15.67:0/1883773701 shutdown_connections 2024-07-27T03:12:27.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.360+0000 7f185cb4d640 1 --2- 172.21.15.67:0/1883773701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1858108790 0x7f185810efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:27.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.360+0000 7f185cb4d640 1 --2- 172.21.15.67:0/1883773701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1858107e40 0x7f1858108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:27.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.360+0000 7f185cb4d640 1 --2- 172.21.15.67:0/1883773701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1858102550 0x7f1858102960 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:27.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.360+0000 7f185cb4d640 1 -- 172.21.15.67:0/1883773701 >> 172.21.15.67:0/1883773701 conn(0x7f18580791d0 msgr2=0x7f18580795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:27.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.360+0000 7f185cb4d640 1 -- 172.21.15.67:0/1883773701 shutdown_connections 2024-07-27T03:12:27.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.360+0000 7f185cb4d640 1 -- 172.21.15.67:0/1883773701 wait complete. 2024-07-27T03:12:27.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.361+0000 7f185cb4d640 1 Processor -- start 2024-07-27T03:12:27.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.361+0000 7f185cb4d640 1 -- start start 2024-07-27T03:12:27.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.362+0000 7f185cb4d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1858102550 0x7f185819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:27.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.362+0000 7f185cb4d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1858107e40 0x7f185819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:27.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.362+0000 7f185cb4d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1858108790 0x7f185819f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:27.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.362+0000 7f185cb4d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f18581135e0 con 0x7f1858107e40 2024-07-27T03:12:27.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.362+0000 7f185cb4d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1858113460 con 0x7f1858102550 2024-07-27T03:12:27.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.362+0000 7f185cb4d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1858113760 con 0x7f1858108790 2024-07-27T03:12:27.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.362+0000 7f1855d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1858107e40 0x7f185819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:27.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.362+0000 7f1855d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1858107e40 0x7f185819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34788/0 (socket says 172.21.15.67:34788) 2024-07-27T03:12:27.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.362+0000 7f1855d9b640 1 -- 172.21.15.67:0/3668247120 learned_addr learned my addr 172.21.15.67:0/3668247120 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:12:27.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.362+0000 7f1856d9d640 1 --2- 172.21.15.67:0/3668247120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1858108790 0x7f185819f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:27.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.362+0000 7f185659c640 1 --2- 172.21.15.67:0/3668247120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1858102550 0x7f185819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:27.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.362+0000 7f1855d9b640 1 -- 172.21.15.67:0/3668247120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1858108790 msgr2=0x7f185819f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:27.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.362+0000 7f1855d9b640 1 --2- 172.21.15.67:0/3668247120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1858108790 0x7f185819f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:27.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.362+0000 7f1855d9b640 1 -- 172.21.15.67:0/3668247120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1858102550 msgr2=0x7f185819a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:27.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.362+0000 7f1855d9b640 1 --2- 172.21.15.67:0/3668247120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1858102550 0x7f185819a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:27.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.362+0000 7f1855d9b640 1 -- 172.21.15.67:0/3668247120 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185819ffa0 con 0x7f1858107e40 2024-07-27T03:12:27.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.362+0000 7f185659c640 1 --2- 172.21.15.67:0/3668247120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1858102550 0x7f185819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:12:27.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.362+0000 7f1855d9b640 1 --2- 172.21.15.67:0/3668247120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1858107e40 0x7f185819a5d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f184000b450 tx=0x7f184000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:27.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.362+0000 7f184f7fe640 1 -- 172.21.15.67:0/3668247120 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840015020 con 0x7f1858107e40 2024-07-27T03:12:27.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.363+0000 7f185cb4d640 1 -- 172.21.15.67:0/3668247120 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18581a0290 con 0x7f1858107e40 2024-07-27T03:12:27.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.363+0000 7f184f7fe640 1 -- 172.21.15.67:0/3668247120 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1840011d40 con 0x7f1858107e40 2024-07-27T03:12:27.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.363+0000 7f184f7fe640 1 -- 172.21.15.67:0/3668247120 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184000f340 con 0x7f1858107e40 2024-07-27T03:12:27.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.363+0000 7f185cb4d640 1 -- 172.21.15.67:0/3668247120 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18581a69f0 con 0x7f1858107e40 2024-07-27T03:12:27.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.364+0000 7f185cb4d640 1 -- 172.21.15.67:0/3668247120 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1858103780 con 0x7f1858107e40 2024-07-27T03:12:27.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.365+0000 7f184f7fe640 1 -- 172.21.15.67:0/3668247120 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f1840018020 con 0x7f1858107e40 2024-07-27T03:12:27.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.366+0000 7f184f7fe640 1 --2- 172.21.15.67:0/3668247120 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f18440b3bd0 0x7f18440b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:27.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.366+0000 7f185659c640 1 --2- 172.21.15.67:0/3668247120 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f18440b3bd0 0x7f18440b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:27.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.366+0000 7f184f7fe640 1 -- 172.21.15.67:0/3668247120 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f18400e37d0 con 0x7f1858107e40 2024-07-27T03:12:27.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.366+0000 7f185659c640 1 --2- 172.21.15.67:0/3668247120 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f18440b3bd0 0x7f18440b60e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1848031440 tx=0x7f1848005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:27.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.369+0000 7f184f7fe640 1 -- 172.21.15.67:0/3668247120 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f184001f020 con 0x7f1858107e40 2024-07-27T03:12:27.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:27.561+0000 7f185cb4d640 1 -- 172.21.15.67:0/3668247120 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f18581029e0 con 0x7f1858107e40 2024-07-27T03:12:28.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:28.515+0000 7f184f7fe640 1 -- 172.21.15.67:0/3668247120 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) ==== 138+0+0 (secure 0 0 0) 0x7f18581029e0 con 0x7f1858107e40 2024-07-27T03:12:28.518 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' created 2024-07-27T03:12:28.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:28.518+0000 7f185cb4d640 1 -- 172.21.15.67:0/3668247120 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f18440b3bd0 msgr2=0x7f18440b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:28.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:28.518+0000 7f185cb4d640 1 --2- 172.21.15.67:0/3668247120 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f18440b3bd0 0x7f18440b60e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1848031440 tx=0x7f1848005d50 comp rx=0 tx=0).stop 2024-07-27T03:12:28.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:28.519+0000 7f185cb4d640 1 -- 172.21.15.67:0/3668247120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1858107e40 msgr2=0x7f185819a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:28.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:28.519+0000 7f185cb4d640 1 --2- 172.21.15.67:0/3668247120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1858107e40 0x7f185819a5d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f184000b450 tx=0x7f184000b910 comp rx=0 tx=0).stop 2024-07-27T03:12:28.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:28.519+0000 7f185cb4d640 1 -- 172.21.15.67:0/3668247120 shutdown_connections 2024-07-27T03:12:28.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:28.519+0000 7f185cb4d640 1 --2- 172.21.15.67:0/3668247120 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f18440b3bd0 0x7f18440b60e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:28.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:28.519+0000 7f185cb4d640 1 --2- 172.21.15.67:0/3668247120 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1858108790 0x7f185819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:28.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:28.519+0000 7f185cb4d640 1 --2- 172.21.15.67:0/3668247120 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1858107e40 0x7f185819a5d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:28.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:28.519+0000 7f185cb4d640 1 --2- 172.21.15.67:0/3668247120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1858102550 0x7f185819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:28.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:28.519+0000 7f185cb4d640 1 -- 172.21.15.67:0/3668247120 >> 172.21.15.67:0/3668247120 conn(0x7f18580791d0 msgr2=0x7f1858101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:28.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:28.519+0000 7f185cb4d640 1 -- 172.21.15.67:0/3668247120 shutdown_connections 2024-07-27T03:12:28.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:28.519+0000 7f185cb4d640 1 -- 172.21.15.67:0/3668247120 wait complete. 2024-07-27T03:12:28.863 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:28 smithi067 ceph-mon[32814]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:28.863 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:28 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:28.863 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:28 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:28.863 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:28 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:28.863 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:28 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:12:28.863 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:28 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:28.863 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:28 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3668247120' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-27T03:12:28.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:28 smithi175 ceph-mon[37125]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:28.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:28 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:28.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:28 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:28.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:28 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:28.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:28 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:12:28.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:28 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:28.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:28 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3668247120' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-27T03:12:29.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:28 smithi149 ceph-mon[37173]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:29.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:28 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:29.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:28 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:29.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:28 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:12:29.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:28 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:12:29.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:28 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:12:29.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:28 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3668247120' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-07-27T03:12:29.208 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-07-27T03:12:29.483 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:12:29.858 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:29 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3668247120' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-07-27T03:12:29.859 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:29 smithi067 ceph-mon[32814]: osdmap e74: 12 total, 12 up, 12 in 2024-07-27T03:12:29.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:29 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3668247120' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-07-27T03:12:29.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:29 smithi175 ceph-mon[37125]: osdmap e74: 12 total, 12 up, 12 in 2024-07-27T03:12:30.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:29 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3668247120' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-07-27T03:12:30.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:29 smithi149 ceph-mon[37173]: osdmap e74: 12 total, 12 up, 12 in 2024-07-27T03:12:30.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:30 smithi067 ceph-mon[32814]: pgmap v291: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:30.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:30 smithi067 ceph-mon[32814]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-27T03:12:30.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:30 smithi067 ceph-mon[32814]: osdmap e75: 12 total, 12 up, 12 in 2024-07-27T03:12:30.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:30 smithi175 ceph-mon[37125]: pgmap v291: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:30.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:30 smithi175 ceph-mon[37125]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-27T03:12:30.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:30 smithi175 ceph-mon[37125]: osdmap e75: 12 total, 12 up, 12 in 2024-07-27T03:12:31.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:30 smithi149 ceph-mon[37173]: pgmap v291: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:31.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:30 smithi149 ceph-mon[37173]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-27T03:12:31.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:30 smithi149 ceph-mon[37173]: osdmap e75: 12 total, 12 up, 12 in 2024-07-27T03:12:31.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.155+0000 7fae93a3d640 1 -- 172.21.15.67:0/2724016174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae8c108760 msgr2=0x7fae8c10ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:31.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.155+0000 7fae93a3d640 1 --2- 172.21.15.67:0/2724016174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae8c108760 0x7fae8c10ef90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009900 tx=0x7fae7c02a410 comp rx=0 tx=0).stop 2024-07-27T03:12:31.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.155+0000 7fae93a3d640 1 -- 172.21.15.67:0/2724016174 shutdown_connections 2024-07-27T03:12:31.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.155+0000 7fae93a3d640 1 --2- 172.21.15.67:0/2724016174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae8c108760 0x7fae8c10ef90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:31.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.155+0000 7fae93a3d640 1 --2- 172.21.15.67:0/2724016174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae8c107e10 0x7fae8c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:31.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.155+0000 7fae93a3d640 1 --2- 172.21.15.67:0/2724016174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae8c1024d0 0x7fae8c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:31.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.155+0000 7fae93a3d640 1 -- 172.21.15.67:0/2724016174 >> 172.21.15.67:0/2724016174 conn(0x7fae8c0791a0 msgr2=0x7fae8c0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:31.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.156+0000 7fae93a3d640 1 -- 172.21.15.67:0/2724016174 shutdown_connections 2024-07-27T03:12:31.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.156+0000 7fae93a3d640 1 -- 172.21.15.67:0/2724016174 wait complete. 2024-07-27T03:12:31.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.156+0000 7fae93a3d640 1 Processor -- start 2024-07-27T03:12:31.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.156+0000 7fae93a3d640 1 -- start start 2024-07-27T03:12:31.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.157+0000 7fae93a3d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae8c1024d0 0x7fae8c19ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:31.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.157+0000 7fae93a3d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae8c107e10 0x7fae8c19b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:31.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.157+0000 7fae93a3d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae8c108760 0x7fae8c19b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:31.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.157+0000 7fae93a3d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c1123c0 con 0x7fae8c1024d0 2024-07-27T03:12:31.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.157+0000 7fae93a3d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c112240 con 0x7fae8c108760 2024-07-27T03:12:31.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.157+0000 7fae93a3d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c112540 con 0x7fae8c107e10 2024-07-27T03:12:31.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.157+0000 7fae917d9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae8c1024d0 0x7fae8c19ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:31.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.157+0000 7fae917d9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae8c1024d0 0x7fae8c19ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34804/0 (socket says 172.21.15.67:34804) 2024-07-27T03:12:31.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.157+0000 7fae917d9640 1 -- 172.21.15.67:0/1091121731 learned_addr learned my addr 172.21.15.67:0/1091121731 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:12:31.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.157+0000 7fae90fd8640 1 --2- 172.21.15.67:0/1091121731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae8c107e10 0x7fae8c19b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:31.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.157+0000 7fae91fda640 1 --2- 172.21.15.67:0/1091121731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae8c108760 0x7fae8c19b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:31.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.157+0000 7fae917d9640 1 -- 172.21.15.67:0/1091121731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae8c107e10 msgr2=0x7fae8c19b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:31.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.157+0000 7fae917d9640 1 --2- 172.21.15.67:0/1091121731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae8c107e10 0x7fae8c19b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:31.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.157+0000 7fae917d9640 1 -- 172.21.15.67:0/1091121731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae8c108760 msgr2=0x7fae8c19b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:31.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.157+0000 7fae917d9640 1 --2- 172.21.15.67:0/1091121731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae8c108760 0x7fae8c19b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:31.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.157+0000 7fae917d9640 1 -- 172.21.15.67:0/1091121731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8c1a5fa0 con 0x7fae8c1024d0 2024-07-27T03:12:31.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.157+0000 7fae91fda640 1 --2- 172.21.15.67:0/1091121731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae8c108760 0x7fae8c19b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:12:31.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.157+0000 7fae917d9640 1 --2- 172.21.15.67:0/1091121731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae8c1024d0 0x7fae8c19ad10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fae8400c890 tx=0x7fae8400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:31.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.158+0000 7fae8a7fc640 1 -- 172.21.15.67:0/1091121731 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84004220 con 0x7fae8c1024d0 2024-07-27T03:12:31.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.158+0000 7fae93a3d640 1 -- 172.21.15.67:0/1091121731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae8c1a6290 con 0x7fae8c1024d0 2024-07-27T03:12:31.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.158+0000 7fae8a7fc640 1 -- 172.21.15.67:0/1091121731 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae840043c0 con 0x7fae8c1024d0 2024-07-27T03:12:31.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.158+0000 7fae8a7fc640 1 -- 172.21.15.67:0/1091121731 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84004a00 con 0x7fae8c1024d0 2024-07-27T03:12:31.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.158+0000 7fae93a3d640 1 -- 172.21.15.67:0/1091121731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae8c1a6820 con 0x7fae8c1024d0 2024-07-27T03:12:31.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.159+0000 7fae93a3d640 1 -- 172.21.15.67:0/1091121731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae8c1028e0 con 0x7fae8c1024d0 2024-07-27T03:12:31.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.161+0000 7fae8a7fc640 1 -- 172.21.15.67:0/1091121731 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7fae840079b0 con 0x7fae8c1024d0 2024-07-27T03:12:31.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.161+0000 7fae8a7fc640 1 --2- 172.21.15.67:0/1091121731 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fae740b3c00 0x7fae740b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:31.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.162+0000 7fae90fd8640 1 --2- 172.21.15.67:0/1091121731 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fae740b3c00 0x7fae740b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:31.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.162+0000 7fae8a7fc640 1 -- 172.21.15.67:0/1091121731 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7497+0+0 (secure 0 0 0) 0x7fae840e39f0 con 0x7fae8c1024d0 2024-07-27T03:12:31.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.162+0000 7fae90fd8640 1 --2- 172.21.15.67:0/1091121731 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fae740b3c00 0x7fae740b6110 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fae8c105920 tx=0x7fae80009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:31.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.165+0000 7fae8a7fc640 1 -- 172.21.15.67:0/1091121731 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fae84014030 con 0x7fae8c1024d0 2024-07-27T03:12:31.351 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.349+0000 7fae93a3d640 1 -- 172.21.15.67:0/1091121731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fae8c06ace0 con 0x7fae8c1024d0 2024-07-27T03:12:31.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.539+0000 7fae8a7fc640 1 -- 172.21.15.67:0/1091121731 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v77) ==== 171+0+0 (secure 0 0 0) 0x7fae8c06ace0 con 0x7fae8c1024d0 2024-07-27T03:12:31.542 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' created 2024-07-27T03:12:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.542+0000 7fae93a3d640 1 -- 172.21.15.67:0/1091121731 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fae740b3c00 msgr2=0x7fae740b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.542+0000 7fae93a3d640 1 --2- 172.21.15.67:0/1091121731 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fae740b3c00 0x7fae740b6110 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fae8c105920 tx=0x7fae80009680 comp rx=0 tx=0).stop 2024-07-27T03:12:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.542+0000 7fae93a3d640 1 -- 172.21.15.67:0/1091121731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae8c1024d0 msgr2=0x7fae8c19ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.542+0000 7fae93a3d640 1 --2- 172.21.15.67:0/1091121731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae8c1024d0 0x7fae8c19ad10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fae8400c890 tx=0x7fae8400cd50 comp rx=0 tx=0).stop 2024-07-27T03:12:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.542+0000 7fae93a3d640 1 -- 172.21.15.67:0/1091121731 shutdown_connections 2024-07-27T03:12:31.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.542+0000 7fae93a3d640 1 --2- 172.21.15.67:0/1091121731 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fae740b3c00 0x7fae740b6110 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:31.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.542+0000 7fae93a3d640 1 --2- 172.21.15.67:0/1091121731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae8c108760 0x7fae8c19b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:31.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.542+0000 7fae93a3d640 1 --2- 172.21.15.67:0/1091121731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae8c107e10 0x7fae8c19b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:31.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.542+0000 7fae93a3d640 1 --2- 172.21.15.67:0/1091121731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fae8c1024d0 0x7fae8c19ad10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:31.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.542+0000 7fae93a3d640 1 -- 172.21.15.67:0/1091121731 >> 172.21.15.67:0/1091121731 conn(0x7fae8c0791a0 msgr2=0x7fae8c103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:31.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.543+0000 7fae93a3d640 1 -- 172.21.15.67:0/1091121731 shutdown_connections 2024-07-27T03:12:31.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:31.543+0000 7fae93a3d640 1 -- 172.21.15.67:0/1091121731 wait complete. 2024-07-27T03:12:31.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:31 smithi067 ceph-mon[32814]: osdmap e76: 12 total, 12 up, 12 in 2024-07-27T03:12:31.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:31 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1091121731' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-07-27T03:12:31.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:31 smithi175 ceph-mon[37125]: osdmap e76: 12 total, 12 up, 12 in 2024-07-27T03:12:31.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:31 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1091121731' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-07-27T03:12:32.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:31 smithi149 ceph-mon[37173]: osdmap e76: 12 total, 12 up, 12 in 2024-07-27T03:12:32.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:31 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1091121731' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-07-27T03:12:32.220 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-07-27T03:12:32.488 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:12:32.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:32 smithi067 ceph-mon[32814]: pgmap v294: 33 pgs: 6 creating+peering, 14 unknown, 13 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:32.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:32 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1091121731' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-07-27T03:12:32.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:32 smithi067 ceph-mon[32814]: osdmap e77: 12 total, 12 up, 12 in 2024-07-27T03:12:32.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:32 smithi175 ceph-mon[37125]: pgmap v294: 33 pgs: 6 creating+peering, 14 unknown, 13 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:32.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:32 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1091121731' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-07-27T03:12:32.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:32 smithi175 ceph-mon[37125]: osdmap e77: 12 total, 12 up, 12 in 2024-07-27T03:12:33.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:32 smithi149 ceph-mon[37173]: pgmap v294: 33 pgs: 6 creating+peering, 14 unknown, 13 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:33.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:32 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1091121731' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-07-27T03:12:33.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:32 smithi149 ceph-mon[37173]: osdmap e77: 12 total, 12 up, 12 in 2024-07-27T03:12:33.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:33 smithi067 ceph-mon[32814]: osdmap e78: 12 total, 12 up, 12 in 2024-07-27T03:12:33.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:33 smithi067 ceph-mon[32814]: osdmap e79: 12 total, 12 up, 12 in 2024-07-27T03:12:33.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:33 smithi175 ceph-mon[37125]: osdmap e78: 12 total, 12 up, 12 in 2024-07-27T03:12:33.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:33 smithi175 ceph-mon[37125]: osdmap e79: 12 total, 12 up, 12 in 2024-07-27T03:12:34.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:33 smithi149 ceph-mon[37173]: osdmap e78: 12 total, 12 up, 12 in 2024-07-27T03:12:34.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:33 smithi149 ceph-mon[37173]: osdmap e79: 12 total, 12 up, 12 in 2024-07-27T03:12:34.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.084+0000 7f88beca7640 1 -- 172.21.15.67:0/1250707567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88b81024d0 msgr2=0x7f88b81028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:34.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.084+0000 7f88beca7640 1 --2- 172.21.15.67:0/1250707567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88b81024d0 0x7f88b81028e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009900 tx=0x7f88ac02a410 comp rx=0 tx=0).stop 2024-07-27T03:12:34.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.084+0000 7f88beca7640 1 -- 172.21.15.67:0/1250707567 shutdown_connections 2024-07-27T03:12:34.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.084+0000 7f88beca7640 1 --2- 172.21.15.67:0/1250707567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88b8108760 0x7f88b810ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:34.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.084+0000 7f88beca7640 1 --2- 172.21.15.67:0/1250707567 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88b8107e10 0x7f88b8108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:34.090 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.084+0000 7f88beca7640 1 --2- 172.21.15.67:0/1250707567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88b81024d0 0x7f88b81028e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:34.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.084+0000 7f88beca7640 1 -- 172.21.15.67:0/1250707567 >> 172.21.15.67:0/1250707567 conn(0x7f88b80791a0 msgr2=0x7f88b80795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:34.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.084+0000 7f88beca7640 1 -- 172.21.15.67:0/1250707567 shutdown_connections 2024-07-27T03:12:34.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.084+0000 7f88beca7640 1 -- 172.21.15.67:0/1250707567 wait complete. 2024-07-27T03:12:34.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.085+0000 7f88beca7640 1 Processor -- start 2024-07-27T03:12:34.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.085+0000 7f88beca7640 1 -- start start 2024-07-27T03:12:34.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.085+0000 7f88beca7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88b81024d0 0x7f88b819acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:34.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.085+0000 7f88beca7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88b8107e10 0x7f88b819b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:34.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.085+0000 7f88beca7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88b8108760 0x7f88b819b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:34.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.085+0000 7f88beca7640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81123c0 con 0x7f88b81024d0 2024-07-27T03:12:34.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.085+0000 7f88beca7640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f88b8112240 con 0x7f88b8108760 2024-07-27T03:12:34.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.085+0000 7f88beca7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88b8112540 con 0x7f88b8107e10 2024-07-27T03:12:34.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.085+0000 7f88bca43640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88b81024d0 0x7f88b819acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:34.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.085+0000 7f88bca43640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88b81024d0 0x7f88b819acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34816/0 (socket says 172.21.15.67:34816) 2024-07-27T03:12:34.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.085+0000 7f88bca43640 1 -- 172.21.15.67:0/134576687 learned_addr learned my addr 172.21.15.67:0/134576687 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:12:34.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.086+0000 7f88bca43640 1 -- 172.21.15.67:0/134576687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88b8107e10 msgr2=0x7f88b819b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:34.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.086+0000 7f88b7fff640 1 --2- 172.21.15.67:0/134576687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88b8107e10 0x7f88b819b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:34.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.086+0000 7f88bd244640 1 --2- 172.21.15.67:0/134576687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88b8108760 0x7f88b819b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:34.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.086+0000 7f88bca43640 1 --2- 172.21.15.67:0/134576687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88b8107e10 0x7f88b819b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:34.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.086+0000 7f88bca43640 1 -- 172.21.15.67:0/134576687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88b8108760 msgr2=0x7f88b819b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:34.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.086+0000 7f88bca43640 1 --2- 172.21.15.67:0/134576687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88b8108760 0x7f88b819b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:34.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.086+0000 7f88bca43640 1 -- 172.21.15.67:0/134576687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b81a5ef0 con 0x7f88b81024d0 2024-07-27T03:12:34.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.086+0000 7f88bca43640 1 --2- 172.21.15.67:0/134576687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88b81024d0 0x7f88b819acf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f88ac005940 tx=0x7f88ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:34.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.086+0000 7f88b5ffb640 1 -- 172.21.15.67:0/134576687 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac03f030 con 0x7f88b81024d0 2024-07-27T03:12:34.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.086+0000 7f88b5ffb640 1 -- 172.21.15.67:0/134576687 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88ac003cb0 con 0x7f88b81024d0 2024-07-27T03:12:34.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.086+0000 7f88beca7640 1 -- 172.21.15.67:0/134576687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b81a6180 con 0x7f88b81024d0 2024-07-27T03:12:34.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.086+0000 7f88b5ffb640 1 -- 172.21.15.67:0/134576687 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac035070 con 0x7f88b81024d0 2024-07-27T03:12:34.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.086+0000 7f88beca7640 1 -- 172.21.15.67:0/134576687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b81a6660 con 0x7f88b81024d0 2024-07-27T03:12:34.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.087+0000 7f88beca7640 1 -- 172.21.15.67:0/134576687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88b81028e0 con 0x7f88b81024d0 2024-07-27T03:12:34.093 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.089+0000 7f88b5ffb640 1 -- 172.21.15.67:0/134576687 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f88ac042030 con 0x7f88b81024d0 2024-07-27T03:12:34.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.090+0000 7f88b5ffb640 1 --2- 172.21.15.67:0/134576687 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f88a80b3c00 0x7f88a80b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:34.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.090+0000 7f88b7fff640 1 --2- 172.21.15.67:0/134576687 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f88a80b3c00 0x7f88a80b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:34.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.090+0000 7f88b5ffb640 1 -- 172.21.15.67:0/134576687 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7f88ac104290 con 0x7f88b81024d0 2024-07-27T03:12:34.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.090+0000 7f88b7fff640 1 --2- 172.21.15.67:0/134576687 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f88a80b3c00 0x7f88a80b6110 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f88b8100730 tx=0x7f88a4005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:34.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.093+0000 7f88b5ffb640 1 -- 172.21.15.67:0/134576687 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f88ac0cba90 con 0x7f88b81024d0 2024-07-27T03:12:34.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.317+0000 7f88beca7640 1 -- 172.21.15.67:0/134576687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f88b802e100 con 0x7f88b81024d0 2024-07-27T03:12:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.559+0000 7f88b5ffb640 1 -- 172.21.15.67:0/134576687 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f88ac0cb390 con 0x7f88b81024d0 2024-07-27T03:12:34.563 INFO:teuthology.orchestra.run.smithi067.stderr:new fs with metadata pool 2 and data pool 3 2024-07-27T03:12:34.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.562+0000 7f88beca7640 1 -- 172.21.15.67:0/134576687 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f88a80b3c00 msgr2=0x7f88a80b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:34.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.562+0000 7f88beca7640 1 --2- 172.21.15.67:0/134576687 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f88a80b3c00 0x7f88a80b6110 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f88b8100730 tx=0x7f88a4005cc0 comp rx=0 tx=0).stop 2024-07-27T03:12:34.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.562+0000 7f88beca7640 1 -- 172.21.15.67:0/134576687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88b81024d0 msgr2=0x7f88b819acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:34.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.562+0000 7f88beca7640 1 --2- 172.21.15.67:0/134576687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88b81024d0 0x7f88b819acf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f88ac005940 tx=0x7f88ac0042f0 comp rx=0 tx=0).stop 2024-07-27T03:12:34.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.563+0000 7f88beca7640 1 -- 172.21.15.67:0/134576687 shutdown_connections 2024-07-27T03:12:34.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.563+0000 7f88beca7640 1 --2- 172.21.15.67:0/134576687 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f88a80b3c00 0x7f88a80b6110 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:34.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.563+0000 7f88beca7640 1 --2- 172.21.15.67:0/134576687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88b8108760 0x7f88b819b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:34.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.563+0000 7f88beca7640 1 --2- 172.21.15.67:0/134576687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88b8107e10 0x7f88b819b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:34.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.563+0000 7f88beca7640 1 --2- 172.21.15.67:0/134576687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f88b81024d0 0x7f88b819acf0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:34.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.563+0000 7f88beca7640 1 -- 172.21.15.67:0/134576687 >> 172.21.15.67:0/134576687 conn(0x7f88b80791a0 msgr2=0x7f88b8100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:34.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.563+0000 7f88beca7640 1 -- 172.21.15.67:0/134576687 shutdown_connections 2024-07-27T03:12:34.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:34.563+0000 7f88beca7640 1 -- 172.21.15.67:0/134576687 wait complete. 2024-07-27T03:12:34.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:34 smithi175 ceph-mon[37125]: pgmap v297: 97 pgs: 14 creating+peering, 53 unknown, 30 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:34.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:34 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/134576687' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-27T03:12:34.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:34 smithi175 ceph-mon[37125]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-07-27T03:12:34.902 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:34 smithi149 ceph-mon[37173]: pgmap v297: 97 pgs: 14 creating+peering, 53 unknown, 30 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:34.902 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:34 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/134576687' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-27T03:12:34.902 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:34 smithi149 ceph-mon[37173]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-07-27T03:12:34.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:34 smithi067 ceph-mon[32814]: pgmap v297: 97 pgs: 14 creating+peering, 53 unknown, 30 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:34.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:34 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/134576687' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-07-27T03:12:34.914 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:34 smithi067 ceph-mon[32814]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-07-27T03:12:35.268 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-07-27T03:12:35.268 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-07-27T03:12:35.544 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:12:35.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:35 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/134576687' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-07-27T03:12:35.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:35 smithi067 ceph-mon[32814]: osdmap e80: 12 total, 12 up, 12 in 2024-07-27T03:12:35.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:35 smithi067 ceph-mon[32814]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-07-27T03:12:35.804 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:35 smithi067 ceph-mon[32814]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-07-27T03:12:35.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:35 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/134576687' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-07-27T03:12:35.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:35 smithi175 ceph-mon[37125]: osdmap e80: 12 total, 12 up, 12 in 2024-07-27T03:12:35.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:35 smithi175 ceph-mon[37125]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-07-27T03:12:35.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:35 smithi175 ceph-mon[37125]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-07-27T03:12:36.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:35 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/134576687' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-07-27T03:12:36.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:35 smithi149 ceph-mon[37173]: osdmap e80: 12 total, 12 up, 12 in 2024-07-27T03:12:36.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:35 smithi149 ceph-mon[37173]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-07-27T03:12:36.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:35 smithi149 ceph-mon[37173]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-07-27T03:12:36.656 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:36 smithi067 ceph-mon[32814]: pgmap v300: 97 pgs: 8 creating+peering, 14 unknown, 75 active+clean; 578 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-07-27T03:12:36.656 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:36 smithi067 ceph-mon[32814]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-07-27T03:12:36.656 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:36 smithi067 ceph-mon[32814]: Cluster is now healthy 2024-07-27T03:12:36.656 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:36 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] up:active 2024-07-27T03:12:36.656 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:36 smithi067 ceph-mon[32814]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-27T03:12:36.657 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:36 smithi067 ceph-mon[32814]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-27T03:12:36.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:36 smithi175 ceph-mon[37125]: pgmap v300: 97 pgs: 8 creating+peering, 14 unknown, 75 active+clean; 578 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-07-27T03:12:36.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:36 smithi175 ceph-mon[37125]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-07-27T03:12:36.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:36 smithi175 ceph-mon[37125]: Cluster is now healthy 2024-07-27T03:12:36.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:36 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] up:active 2024-07-27T03:12:36.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:36 smithi175 ceph-mon[37125]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-27T03:12:36.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:36 smithi175 ceph-mon[37125]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-27T03:12:37.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:36 smithi149 ceph-mon[37173]: pgmap v300: 97 pgs: 8 creating+peering, 14 unknown, 75 active+clean; 578 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-07-27T03:12:37.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:36 smithi149 ceph-mon[37173]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-07-27T03:12:37.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:36 smithi149 ceph-mon[37173]: Cluster is now healthy 2024-07-27T03:12:37.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:36 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] up:active 2024-07-27T03:12:37.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:36 smithi149 ceph-mon[37173]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-27T03:12:37.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:36 smithi149 ceph-mon[37173]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-27T03:12:37.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.190+0000 7f3c8606e640 1 -- 172.21.15.67:0/2476967230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c80109860 msgr2=0x7f3c80110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:37.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.190+0000 7f3c8606e640 1 --2- 172.21.15.67:0/2476967230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c80109860 0x7f3c80110090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3c78009ca0 tx=0x7f3c7802a8c0 comp rx=0 tx=0).stop 2024-07-27T03:12:37.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.190+0000 7f3c8606e640 1 -- 172.21.15.67:0/2476967230 shutdown_connections 2024-07-27T03:12:37.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.190+0000 7f3c8606e640 1 --2- 172.21.15.67:0/2476967230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c80109860 0x7f3c80110090 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:37.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.190+0000 7f3c8606e640 1 --2- 172.21.15.67:0/2476967230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c80108f10 0x7f3c80109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:37.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.190+0000 7f3c8606e640 1 --2- 172.21.15.67:0/2476967230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c801035d0 0x7f3c801039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:37.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.190+0000 7f3c8606e640 1 -- 172.21.15.67:0/2476967230 >> 172.21.15.67:0/2476967230 conn(0x7f3c8007b2c0 msgr2=0x7f3c8007b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:37.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.190+0000 7f3c8606e640 1 -- 172.21.15.67:0/2476967230 shutdown_connections 2024-07-27T03:12:37.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.190+0000 7f3c8606e640 1 -- 172.21.15.67:0/2476967230 wait complete. 2024-07-27T03:12:37.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.191+0000 7f3c8606e640 1 Processor -- start 2024-07-27T03:12:37.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.191+0000 7f3c8606e640 1 -- start start 2024-07-27T03:12:37.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.191+0000 7f3c8606e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c801035d0 0x7f3c8019c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:37.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.191+0000 7f3c8606e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c80108f10 0x7f3c8019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:37.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.191+0000 7f3c8606e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c80109860 0x7f3c801a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:37.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.191+0000 7f3c8606e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3c80113510 con 0x7f3c80109860 2024-07-27T03:12:37.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.191+0000 7f3c8606e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3c80113390 con 0x7f3c801035d0 2024-07-27T03:12:37.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.191+0000 7f3c8606e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3c80113690 con 0x7f3c80108f10 2024-07-27T03:12:37.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.191+0000 7f3c7ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c80109860 0x7f3c801a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:37.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.191+0000 7f3c7ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c80109860 0x7f3c801a1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37248/0 (socket says 172.21.15.67:37248) 2024-07-27T03:12:37.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.191+0000 7f3c7ffff640 1 -- 172.21.15.67:0/2817959112 learned_addr learned my addr 172.21.15.67:0/2817959112 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:12:37.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.191+0000 7f3c7f7fe640 1 --2- 172.21.15.67:0/2817959112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c801035d0 0x7f3c8019c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:37.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.191+0000 7f3c76ffd640 1 --2- 172.21.15.67:0/2817959112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c80108f10 0x7f3c8019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:37.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.192+0000 7f3c7ffff640 1 -- 172.21.15.67:0/2817959112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c80108f10 msgr2=0x7f3c8019c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:37.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.192+0000 7f3c7ffff640 1 --2- 172.21.15.67:0/2817959112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c80108f10 0x7f3c8019c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:37.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.192+0000 7f3c7ffff640 1 -- 172.21.15.67:0/2817959112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c801035d0 msgr2=0x7f3c8019c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:37.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.192+0000 7f3c7ffff640 1 --2- 172.21.15.67:0/2817959112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c801035d0 0x7f3c8019c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:37.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.192+0000 7f3c7ffff640 1 -- 172.21.15.67:0/2817959112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c801a20b0 con 0x7f3c80109860 2024-07-27T03:12:37.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.192+0000 7f3c76ffd640 1 --2- 172.21.15.67:0/2817959112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c80108f10 0x7f3c8019c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:12:37.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.192+0000 7f3c7ffff640 1 --2- 172.21.15.67:0/2817959112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c80109860 0x7f3c801a1980 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3c78005cc0 tx=0x7f3c7802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:37.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.192+0000 7f3c7d7fa640 1 -- 172.21.15.67:0/2817959112 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78003bf0 con 0x7f3c80109860 2024-07-27T03:12:37.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.192+0000 7f3c7d7fa640 1 -- 172.21.15.67:0/2817959112 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c78003d90 con 0x7f3c80109860 2024-07-27T03:12:37.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.192+0000 7f3c8606e640 1 -- 172.21.15.67:0/2817959112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c801a2340 con 0x7f3c80109860 2024-07-27T03:12:37.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.192+0000 7f3c7d7fa640 1 -- 172.21.15.67:0/2817959112 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7802fbc0 con 0x7f3c80109860 2024-07-27T03:12:37.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.192+0000 7f3c8606e640 1 -- 172.21.15.67:0/2817959112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c801a8b00 con 0x7f3c80109860 2024-07-27T03:12:37.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.193+0000 7f3c8606e640 1 -- 172.21.15.67:0/2817959112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c801039e0 con 0x7f3c80109860 2024-07-27T03:12:37.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.195+0000 7f3c7d7fa640 1 -- 172.21.15.67:0/2817959112 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f3c7802fd60 con 0x7f3c80109860 2024-07-27T03:12:37.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.196+0000 7f3c7d7fa640 1 --2- 172.21.15.67:0/2817959112 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3c6c0b3bd0 0x7f3c6c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:37.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.196+0000 7f3c7f7fe640 1 --2- 172.21.15.67:0/2817959112 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3c6c0b3bd0 0x7f3c6c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:37.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.196+0000 7f3c7d7fa640 1 -- 172.21.15.67:0/2817959112 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f3c78105410 con 0x7f3c80109860 2024-07-27T03:12:37.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.196+0000 7f3c7f7fe640 1 --2- 172.21.15.67:0/2817959112 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3c6c0b3bd0 0x7f3c6c0b60e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3c70005fd0 tx=0x7f3c70005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:37.241 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.199+0000 7f3c7d7fa640 1 -- 172.21.15.67:0/2817959112 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f3c780ccbb0 con 0x7f3c80109860 2024-07-27T03:12:37.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.391+0000 7f3c8606e640 1 -- 172.21.15.67:0/2817959112 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f3c801a8d30 con 0x7f3c80109860 2024-07-27T03:12:37.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.584+0000 7f3c7d7fa640 1 -- 172.21.15.67:0/2817959112 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v81) ==== 157+0+0 (secure 0 0 0) 0x7f3c780cc4b0 con 0x7f3c80109860 2024-07-27T03:12:37.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.586+0000 7f3c8606e640 1 -- 172.21.15.67:0/2817959112 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3c6c0b3bd0 msgr2=0x7f3c6c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:37.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.586+0000 7f3c8606e640 1 --2- 172.21.15.67:0/2817959112 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3c6c0b3bd0 0x7f3c6c0b60e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3c70005fd0 tx=0x7f3c70005850 comp rx=0 tx=0).stop 2024-07-27T03:12:37.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.586+0000 7f3c8606e640 1 -- 172.21.15.67:0/2817959112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c80109860 msgr2=0x7f3c801a1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:37.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.586+0000 7f3c8606e640 1 --2- 172.21.15.67:0/2817959112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c80109860 0x7f3c801a1980 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3c78005cc0 tx=0x7f3c7802f380 comp rx=0 tx=0).stop 2024-07-27T03:12:37.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.586+0000 7f3c8606e640 1 -- 172.21.15.67:0/2817959112 shutdown_connections 2024-07-27T03:12:37.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.586+0000 7f3c8606e640 1 --2- 172.21.15.67:0/2817959112 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3c6c0b3bd0 0x7f3c6c0b60e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:37.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.586+0000 7f3c8606e640 1 --2- 172.21.15.67:0/2817959112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3c80109860 0x7f3c801a1980 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:37.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.586+0000 7f3c8606e640 1 --2- 172.21.15.67:0/2817959112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c80108f10 0x7f3c8019c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:37.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.586+0000 7f3c8606e640 1 --2- 172.21.15.67:0/2817959112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c801035d0 0x7f3c8019c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:37.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.586+0000 7f3c8606e640 1 -- 172.21.15.67:0/2817959112 >> 172.21.15.67:0/2817959112 conn(0x7f3c8007b2c0 msgr2=0x7f3c80106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:37.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.587+0000 7f3c8606e640 1 -- 172.21.15.67:0/2817959112 shutdown_connections 2024-07-27T03:12:37.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:37.587+0000 7f3c8606e640 1 -- 172.21.15.67:0/2817959112 wait complete. 2024-07-27T03:12:37.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:37 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2817959112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-07-27T03:12:37.906 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:37 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2817959112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-07-27T03:12:38.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:37 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2817959112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-07-27T03:12:38.232 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-07-27T03:12:38.502 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:12:38.843 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:38 smithi067 ceph-mon[32814]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 931 B/s wr, 6 op/s 2024-07-27T03:12:38.843 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:38 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2817959112' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-07-27T03:12:38.843 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:38 smithi067 ceph-mon[32814]: osdmap e81: 12 total, 12 up, 12 in 2024-07-27T03:12:38.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:38 smithi175 ceph-mon[37125]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 931 B/s wr, 6 op/s 2024-07-27T03:12:38.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:38 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2817959112' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-07-27T03:12:38.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:38 smithi175 ceph-mon[37125]: osdmap e81: 12 total, 12 up, 12 in 2024-07-27T03:12:39.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:38 smithi149 ceph-mon[37173]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 931 B/s wr, 6 op/s 2024-07-27T03:12:39.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:38 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2817959112' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-07-27T03:12:39.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:38 smithi149 ceph-mon[37173]: osdmap e81: 12 total, 12 up, 12 in 2024-07-27T03:12:40.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.125+0000 7f700f064640 1 -- 172.21.15.67:0/784867309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7008109870 msgr2=0x7f70081100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:40.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.125+0000 7f700f064640 1 --2- 172.21.15.67:0/784867309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7008109870 0x7f70081100a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7000009ca0 tx=0x7f700002a8c0 comp rx=0 tx=0).stop 2024-07-27T03:12:40.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.125+0000 7f700f064640 1 -- 172.21.15.67:0/784867309 shutdown_connections 2024-07-27T03:12:40.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.125+0000 7f700f064640 1 --2- 172.21.15.67:0/784867309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7008109870 0x7f70081100a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:40.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.125+0000 7f700f064640 1 --2- 172.21.15.67:0/784867309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7008108f20 0x7f7008109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:40.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.125+0000 7f700f064640 1 --2- 172.21.15.67:0/784867309 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f70081035e0 0x7f70081039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:40.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.125+0000 7f700f064640 1 -- 172.21.15.67:0/784867309 >> 172.21.15.67:0/784867309 conn(0x7f70080ff010 msgr2=0x7f7008101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:40.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.125+0000 7f700f064640 1 -- 172.21.15.67:0/784867309 shutdown_connections 2024-07-27T03:12:40.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.125+0000 7f700f064640 1 -- 172.21.15.67:0/784867309 wait complete. 2024-07-27T03:12:40.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.126+0000 7f700f064640 1 Processor -- start 2024-07-27T03:12:40.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.126+0000 7f700f064640 1 -- start start 2024-07-27T03:12:40.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.126+0000 7f700f064640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70081035e0 0x7f7008199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:40.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.126+0000 7f700f064640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7008108f20 0x7f700819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:40.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.126+0000 7f700f064640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7008109870 0x7f700819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:40.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.126+0000 7f700f064640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7008113540 con 0x7f7008109870 2024-07-27T03:12:40.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.126+0000 7f700f064640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f70081133c0 con 0x7f70081035e0 2024-07-27T03:12:40.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.126+0000 7f700f064640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f70081136c0 con 0x7f7008108f20 2024-07-27T03:12:40.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.126+0000 7f700d601640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7008109870 0x7f700819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:40.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.126+0000 7f700d601640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7008109870 0x7f700819f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37256/0 (socket says 172.21.15.67:37256) 2024-07-27T03:12:40.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.126+0000 7f700d601640 1 -- 172.21.15.67:0/335325804 learned_addr learned my addr 172.21.15.67:0/335325804 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:12:40.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.126+0000 7f700ce00640 1 --2- 172.21.15.67:0/335325804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70081035e0 0x7f7008199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:40.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.126+0000 7f6ffffff640 1 --2- 172.21.15.67:0/335325804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7008108f20 0x7f700819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:40.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.127+0000 7f700d601640 1 -- 172.21.15.67:0/335325804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7008108f20 msgr2=0x7f700819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:40.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.127+0000 7f700d601640 1 --2- 172.21.15.67:0/335325804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7008108f20 0x7f700819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:40.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.127+0000 7f700d601640 1 -- 172.21.15.67:0/335325804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70081035e0 msgr2=0x7f7008199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:40.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.127+0000 7f700d601640 1 --2- 172.21.15.67:0/335325804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70081035e0 0x7f7008199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:40.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.127+0000 7f700d601640 1 -- 172.21.15.67:0/335325804 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700819fe40 con 0x7f7008109870 2024-07-27T03:12:40.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.127+0000 7f700d601640 1 --2- 172.21.15.67:0/335325804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7008109870 0x7f700819f710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7000005cc0 tx=0x7f700002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:40.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.127+0000 7f6ffdffb640 1 -- 172.21.15.67:0/335325804 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000003bf0 con 0x7f7008109870 2024-07-27T03:12:40.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.127+0000 7f700f064640 1 -- 172.21.15.67:0/335325804 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70081a00d0 con 0x7f7008109870 2024-07-27T03:12:40.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.127+0000 7f6ffdffb640 1 -- 172.21.15.67:0/335325804 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7000003d90 con 0x7f7008109870 2024-07-27T03:12:40.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.127+0000 7f6ffdffb640 1 -- 172.21.15.67:0/335325804 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700002fbc0 con 0x7f7008109870 2024-07-27T03:12:40.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.127+0000 7f700f064640 1 -- 172.21.15.67:0/335325804 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70081a6890 con 0x7f7008109870 2024-07-27T03:12:40.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.128+0000 7f700f064640 1 -- 172.21.15.67:0/335325804 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70081039f0 con 0x7f7008109870 2024-07-27T03:12:40.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.130+0000 7f6ffdffb640 1 -- 172.21.15.67:0/335325804 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f700002fd60 con 0x7f7008109870 2024-07-27T03:12:40.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.131+0000 7f6ffdffb640 1 --2- 172.21.15.67:0/335325804 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f6ff00b3bd0 0x7f6ff00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:40.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.131+0000 7f700ce00640 1 --2- 172.21.15.67:0/335325804 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f6ff00b3bd0 0x7f6ff00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:40.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.131+0000 7f6ffdffb640 1 -- 172.21.15.67:0/335325804 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8277+0+0 (secure 0 0 0) 0x7f7000105440 con 0x7f7008109870 2024-07-27T03:12:40.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.131+0000 7f700ce00640 1 --2- 172.21.15.67:0/335325804 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f6ff00b3bd0 0x7f6ff00b60e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7004005fd0 tx=0x7f700400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:40.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.134+0000 7f6ffdffb640 1 -- 172.21.15.67:0/335325804 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f70000cca60 con 0x7f7008109870 2024-07-27T03:12:40.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:40.317+0000 7f700f064640 1 -- 172.21.15.67:0/335325804 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f700819b000 con 0x7f7008109870 2024-07-27T03:12:40.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:40 smithi175 ceph-mon[37125]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-07-27T03:12:40.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:40 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/335325804' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-27T03:12:40.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:40 smithi067 ceph-mon[32814]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-07-27T03:12:40.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:40 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/335325804' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-27T03:12:41.000 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:12:40 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-b[67306]: 2024-07-27T03:12:40.536+0000 7f3a9fa3c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-07-27T03:12:41.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:40 smithi149 ceph-mon[37173]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-07-27T03:12:41.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:40 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/335325804' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-07-27T03:12:41.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:41.591+0000 7f6ffdffb640 1 -- 172.21.15.67:0/335325804 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v83) ==== 228+0+0 (secure 0 0 0) 0x7f700819b000 con 0x7f7008109870 2024-07-27T03:12:41.595 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data_ec' created 2024-07-27T03:12:41.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:41.595+0000 7f700f064640 1 -- 172.21.15.67:0/335325804 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f6ff00b3bd0 msgr2=0x7f6ff00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:41.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:41.595+0000 7f700f064640 1 --2- 172.21.15.67:0/335325804 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f6ff00b3bd0 0x7f6ff00b60e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7004005fd0 tx=0x7f700400b390 comp rx=0 tx=0).stop 2024-07-27T03:12:41.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:41.595+0000 7f700f064640 1 -- 172.21.15.67:0/335325804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7008109870 msgr2=0x7f700819f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:41.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:41.595+0000 7f700f064640 1 --2- 172.21.15.67:0/335325804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7008109870 0x7f700819f710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7000005cc0 tx=0x7f700002f380 comp rx=0 tx=0).stop 2024-07-27T03:12:41.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:41.595+0000 7f700f064640 1 -- 172.21.15.67:0/335325804 shutdown_connections 2024-07-27T03:12:41.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:41.595+0000 7f700f064640 1 --2- 172.21.15.67:0/335325804 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f6ff00b3bd0 0x7f6ff00b60e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:41.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:41.595+0000 7f700f064640 1 --2- 172.21.15.67:0/335325804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7008109870 0x7f700819f710 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:41.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:41.595+0000 7f700f064640 1 --2- 172.21.15.67:0/335325804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7008108f20 0x7f700819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:41.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:41.595+0000 7f700f064640 1 --2- 172.21.15.67:0/335325804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70081035e0 0x7f7008199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:41.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:41.595+0000 7f700f064640 1 -- 172.21.15.67:0/335325804 >> 172.21.15.67:0/335325804 conn(0x7f70080ff010 msgr2=0x7f7008106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:41.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:41.595+0000 7f700f064640 1 -- 172.21.15.67:0/335325804 shutdown_connections 2024-07-27T03:12:41.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:41.595+0000 7f700f064640 1 -- 172.21.15.67:0/335325804 wait complete. 2024-07-27T03:12:41.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:41 smithi175 ceph-mon[37125]: osdmap e82: 12 total, 12 up, 12 in 2024-07-27T03:12:41.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:41 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/335325804' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-07-27T03:12:41.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:41 smithi175 ceph-mon[37125]: osdmap e83: 12 total, 12 up, 12 in 2024-07-27T03:12:41.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:41 smithi067 ceph-mon[32814]: osdmap e82: 12 total, 12 up, 12 in 2024-07-27T03:12:41.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:41 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/335325804' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-07-27T03:12:41.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:41 smithi067 ceph-mon[32814]: osdmap e83: 12 total, 12 up, 12 in 2024-07-27T03:12:42.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:41 smithi149 ceph-mon[37173]: osdmap e82: 12 total, 12 up, 12 in 2024-07-27T03:12:42.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:41 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/335325804' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-07-27T03:12:42.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:41 smithi149 ceph-mon[37173]: osdmap e83: 12 total, 12 up, 12 in 2024-07-27T03:12:42.302 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-07-27T03:12:42.578 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:12:42.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:42 smithi175 ceph-mon[37125]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-07-27T03:12:42.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:42 smithi175 ceph-mon[37125]: osdmap e84: 12 total, 12 up, 12 in 2024-07-27T03:12:42.951 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:42 smithi067 ceph-mon[32814]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-07-27T03:12:42.952 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:42 smithi067 ceph-mon[32814]: osdmap e84: 12 total, 12 up, 12 in 2024-07-27T03:12:43.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:42 smithi149 ceph-mon[37173]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-07-27T03:12:43.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:42 smithi149 ceph-mon[37173]: osdmap e84: 12 total, 12 up, 12 in 2024-07-27T03:12:44.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.311+0000 7f6e93c5f640 1 -- 172.21.15.67:0/167599996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e8c108f40 msgr2=0x7f6e8c109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:44.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.311+0000 7f6e93c5f640 1 --2- 172.21.15.67:0/167599996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e8c108f40 0x7f6e8c109350 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c009900 tx=0x7f6e7c02a410 comp rx=0 tx=0).stop 2024-07-27T03:12:44.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.311+0000 7f6e93c5f640 1 -- 172.21.15.67:0/167599996 shutdown_connections 2024-07-27T03:12:44.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.311+0000 7f6e93c5f640 1 --2- 172.21.15.67:0/167599996 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e8c109890 0x7f6e8c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:44.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.311+0000 7f6e93c5f640 1 --2- 172.21.15.67:0/167599996 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e8c108f40 0x7f6e8c109350 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:44.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.311+0000 7f6e93c5f640 1 --2- 172.21.15.67:0/167599996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e8c103600 0x7f6e8c103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:44.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.311+0000 7f6e93c5f640 1 -- 172.21.15.67:0/167599996 >> 172.21.15.67:0/167599996 conn(0x7f6e8c0ff070 msgr2=0x7f6e8c1014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:44.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.311+0000 7f6e93c5f640 1 -- 172.21.15.67:0/167599996 shutdown_connections 2024-07-27T03:12:44.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.311+0000 7f6e93c5f640 1 -- 172.21.15.67:0/167599996 wait complete. 2024-07-27T03:12:44.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.312+0000 7f6e93c5f640 1 Processor -- start 2024-07-27T03:12:44.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.312+0000 7f6e93c5f640 1 -- start start 2024-07-27T03:12:44.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.312+0000 7f6e93c5f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e8c103600 0x7f6e8c075180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:44.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.312+0000 7f6e93c5f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e8c108f40 0x7f6e8c0756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:44.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.312+0000 7f6e93c5f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e8c109890 0x7f6e8c075c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:44.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.312+0000 7f6e93c5f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c113520 con 0x7f6e8c103600 2024-07-27T03:12:44.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.312+0000 7f6e93c5f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c1133a0 con 0x7f6e8c108f40 2024-07-27T03:12:44.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.312+0000 7f6e93c5f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c1136a0 con 0x7f6e8c109890 2024-07-27T03:12:44.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.312+0000 7f6e919fb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e8c103600 0x7f6e8c075180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:44.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.312+0000 7f6e919fb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e8c103600 0x7f6e8c075180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:37272/0 (socket says 172.21.15.67:37272) 2024-07-27T03:12:44.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.312+0000 7f6e919fb640 1 -- 172.21.15.67:0/953127008 learned_addr learned my addr 172.21.15.67:0/953127008 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:12:44.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.312+0000 7f6e911fa640 1 --2- 172.21.15.67:0/953127008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e8c108f40 0x7f6e8c0756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:44.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.313+0000 7f6e921fc640 1 --2- 172.21.15.67:0/953127008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e8c109890 0x7f6e8c075c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:44.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.313+0000 7f6e919fb640 1 -- 172.21.15.67:0/953127008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e8c109890 msgr2=0x7f6e8c075c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:44.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.313+0000 7f6e919fb640 1 --2- 172.21.15.67:0/953127008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e8c109890 0x7f6e8c075c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:44.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.313+0000 7f6e919fb640 1 -- 172.21.15.67:0/953127008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e8c108f40 msgr2=0x7f6e8c0756c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:44.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.313+0000 7f6e919fb640 1 --2- 172.21.15.67:0/953127008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e8c108f40 0x7f6e8c0756c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:44.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.313+0000 7f6e919fb640 1 -- 172.21.15.67:0/953127008 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e8c1a81f0 con 0x7f6e8c103600 2024-07-27T03:12:44.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.313+0000 7f6e919fb640 1 --2- 172.21.15.67:0/953127008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e8c103600 0x7f6e8c075180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800d950 tx=0x7f6e8800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:44.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.313+0000 7f6e82ffd640 1 -- 172.21.15.67:0/953127008 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e88017030 con 0x7f6e8c103600 2024-07-27T03:12:44.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.313+0000 7f6e93c5f640 1 -- 172.21.15.67:0/953127008 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e8c1a84e0 con 0x7f6e8c103600 2024-07-27T03:12:44.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.313+0000 7f6e82ffd640 1 -- 172.21.15.67:0/953127008 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e88004020 con 0x7f6e8c103600 2024-07-27T03:12:44.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.313+0000 7f6e82ffd640 1 -- 172.21.15.67:0/953127008 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e880047a0 con 0x7f6e8c103600 2024-07-27T03:12:44.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.313+0000 7f6e93c5f640 1 -- 172.21.15.67:0/953127008 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e8c1a8a40 con 0x7f6e8c103600 2024-07-27T03:12:44.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.314+0000 7f6e93c5f640 1 -- 172.21.15.67:0/953127008 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e8c103a10 con 0x7f6e8c103600 2024-07-27T03:12:44.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.316+0000 7f6e82ffd640 1 -- 172.21.15.67:0/953127008 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f6e88009a30 con 0x7f6e8c103600 2024-07-27T03:12:44.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.317+0000 7f6e82ffd640 1 --2- 172.21.15.67:0/953127008 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f6e780b3bd0 0x7f6e780b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:44.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.317+0000 7f6e911fa640 1 --2- 172.21.15.67:0/953127008 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f6e780b3bd0 0x7f6e780b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:44.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.317+0000 7f6e82ffd640 1 -- 172.21.15.67:0/953127008 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8770+0+0 (secure 0 0 0) 0x7f6e880e3f50 con 0x7f6e8c103600 2024-07-27T03:12:44.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.318+0000 7f6e911fa640 1 --2- 172.21.15.67:0/953127008 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f6e780b3bd0 0x7f6e780b60e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c009230 tx=0x7f6e7c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:44.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.321+0000 7f6e82ffd640 1 -- 172.21.15.67:0/953127008 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f6e880ab380 con 0x7f6e8c103600 2024-07-27T03:12:44.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.502+0000 7f6e93c5f640 1 -- 172.21.15.67:0/953127008 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f6e8c076480 con 0x7f6e8c103600 2024-07-27T03:12:44.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.618+0000 7f6e82ffd640 1 -- 172.21.15.67:0/953127008 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v86) ==== 169+0+0 (secure 0 0 0) 0x7f6e880aac80 con 0x7f6e8c103600 2024-07-27T03:12:44.622 INFO:teuthology.orchestra.run.smithi067.stderr:set pool 4 allow_ec_overwrites to true 2024-07-27T03:12:44.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.621+0000 7f6e93c5f640 1 -- 172.21.15.67:0/953127008 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f6e780b3bd0 msgr2=0x7f6e780b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:44.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.621+0000 7f6e93c5f640 1 --2- 172.21.15.67:0/953127008 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f6e780b3bd0 0x7f6e780b60e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c009230 tx=0x7f6e7c031400 comp rx=0 tx=0).stop 2024-07-27T03:12:44.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.621+0000 7f6e93c5f640 1 -- 172.21.15.67:0/953127008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e8c103600 msgr2=0x7f6e8c075180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:44.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.621+0000 7f6e93c5f640 1 --2- 172.21.15.67:0/953127008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e8c103600 0x7f6e8c075180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800d950 tx=0x7f6e8800de10 comp rx=0 tx=0).stop 2024-07-27T03:12:44.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.621+0000 7f6e93c5f640 1 -- 172.21.15.67:0/953127008 shutdown_connections 2024-07-27T03:12:44.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.621+0000 7f6e93c5f640 1 --2- 172.21.15.67:0/953127008 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f6e780b3bd0 0x7f6e780b60e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:44.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.621+0000 7f6e93c5f640 1 --2- 172.21.15.67:0/953127008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e8c109890 0x7f6e8c075c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:44.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.621+0000 7f6e93c5f640 1 --2- 172.21.15.67:0/953127008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e8c108f40 0x7f6e8c0756c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:44.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.621+0000 7f6e93c5f640 1 --2- 172.21.15.67:0/953127008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6e8c103600 0x7f6e8c075180 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:44.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.621+0000 7f6e93c5f640 1 -- 172.21.15.67:0/953127008 >> 172.21.15.67:0/953127008 conn(0x7f6e8c0ff070 msgr2=0x7f6e8c101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:44.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.621+0000 7f6e93c5f640 1 -- 172.21.15.67:0/953127008 shutdown_connections 2024-07-27T03:12:44.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:44.621+0000 7f6e93c5f640 1 -- 172.21.15.67:0/953127008 wait complete. 2024-07-27T03:12:44.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:44 smithi175 ceph-mon[37125]: pgmap v308: 129 pgs: 1 creating+activating, 1 creating+peering, 29 unknown, 98 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-07-27T03:12:44.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:44 smithi175 ceph-mon[37125]: osdmap e85: 12 total, 12 up, 12 in 2024-07-27T03:12:44.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:44 smithi175 ceph-mon[37125]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-27T03:12:44.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:44 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/953127008' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-07-27T03:12:44.947 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:44 smithi067 ceph-mon[32814]: pgmap v308: 129 pgs: 1 creating+activating, 1 creating+peering, 29 unknown, 98 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-07-27T03:12:44.947 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:44 smithi067 ceph-mon[32814]: osdmap e85: 12 total, 12 up, 12 in 2024-07-27T03:12:44.947 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:44 smithi067 ceph-mon[32814]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-27T03:12:44.947 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:44 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/953127008' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-07-27T03:12:45.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:44 smithi149 ceph-mon[37173]: pgmap v308: 129 pgs: 1 creating+activating, 1 creating+peering, 29 unknown, 98 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-07-27T03:12:45.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:44 smithi149 ceph-mon[37173]: osdmap e85: 12 total, 12 up, 12 in 2024-07-27T03:12:45.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:44 smithi149 ceph-mon[37173]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-07-27T03:12:45.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:44 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/953127008' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-07-27T03:12:45.262 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-07-27T03:12:45.532 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:12:45.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:45 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/953127008' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-07-27T03:12:45.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:45 smithi175 ceph-mon[37125]: osdmap e86: 12 total, 12 up, 12 in 2024-07-27T03:12:45.910 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:45 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/953127008' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-07-27T03:12:45.910 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:45 smithi067 ceph-mon[32814]: osdmap e86: 12 total, 12 up, 12 in 2024-07-27T03:12:46.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:45 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/953127008' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-07-27T03:12:46.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:45 smithi149 ceph-mon[37173]: osdmap e86: 12 total, 12 up, 12 in 2024-07-27T03:12:46.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:46 smithi067 ceph-mon[32814]: pgmap v311: 129 pgs: 1 creating+activating, 1 creating+peering, 13 unknown, 114 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-07-27T03:12:47.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:46 smithi149 ceph-mon[37173]: pgmap v311: 129 pgs: 1 creating+activating, 1 creating+peering, 13 unknown, 114 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-07-27T03:12:47.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:46 smithi175 ceph-mon[37125]: pgmap v311: 129 pgs: 1 creating+activating, 1 creating+peering, 13 unknown, 114 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-07-27T03:12:47.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.210+0000 7f41cf166640 1 -- 172.21.15.67:0/1714791009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c8109880 msgr2=0x7f41c81100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:47.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.210+0000 7f41cf166640 1 --2- 172.21.15.67:0/1714791009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c8109880 0x7f41c81100b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009900 tx=0x7f41b002a410 comp rx=0 tx=0).stop 2024-07-27T03:12:47.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.210+0000 7f41cf166640 1 -- 172.21.15.67:0/1714791009 shutdown_connections 2024-07-27T03:12:47.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.210+0000 7f41cf166640 1 --2- 172.21.15.67:0/1714791009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c8109880 0x7f41c81100b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:47.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.210+0000 7f41cf166640 1 --2- 172.21.15.67:0/1714791009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41c8108f30 0x7f41c8109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:47.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.210+0000 7f41cf166640 1 --2- 172.21.15.67:0/1714791009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41c8101a50 0x7f41c8101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:47.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.210+0000 7f41cf166640 1 -- 172.21.15.67:0/1714791009 >> 172.21.15.67:0/1714791009 conn(0x7f41c8077630 msgr2=0x7f41c8077a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:47.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.210+0000 7f41cf166640 1 -- 172.21.15.67:0/1714791009 shutdown_connections 2024-07-27T03:12:47.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.210+0000 7f41cf166640 1 -- 172.21.15.67:0/1714791009 wait complete. 2024-07-27T03:12:47.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.211+0000 7f41cf166640 1 Processor -- start 2024-07-27T03:12:47.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.211+0000 7f41cf166640 1 -- start start 2024-07-27T03:12:47.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.211+0000 7f41cf166640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41c8101a50 0x7f41c81029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:47.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.211+0000 7f41cf166640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41c8108f30 0x7f41c8102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:47.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.211+0000 7f41cf166640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c8109880 0x7f41c8103470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:47.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.211+0000 7f41cf166640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f41c81135d0 con 0x7f41c8109880 2024-07-27T03:12:47.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.211+0000 7f41cf166640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f41c8113450 con 0x7f41c8101a50 2024-07-27T03:12:47.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.211+0000 7f41cf166640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f41c8113750 con 0x7f41c8108f30 2024-07-27T03:12:47.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.211+0000 7f41cd703640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c8109880 0x7f41c8103470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:47.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.211+0000 7f41cd703640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c8109880 0x7f41c8103470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56194/0 (socket says 172.21.15.67:56194) 2024-07-27T03:12:47.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.211+0000 7f41cd703640 1 -- 172.21.15.67:0/3405426226 learned_addr learned my addr 172.21.15.67:0/3405426226 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:12:47.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.211+0000 7f41ccf02640 1 --2- 172.21.15.67:0/3405426226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41c8101a50 0x7f41c81029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:47.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.211+0000 7f41cd703640 1 -- 172.21.15.67:0/3405426226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41c8108f30 msgr2=0x7f41c8102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:47.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.211+0000 7f41c7fff640 1 --2- 172.21.15.67:0/3405426226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41c8108f30 0x7f41c8102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:47.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.211+0000 7f41cd703640 1 --2- 172.21.15.67:0/3405426226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41c8108f30 0x7f41c8102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:47.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.211+0000 7f41cd703640 1 -- 172.21.15.67:0/3405426226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41c8101a50 msgr2=0x7f41c81029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:47.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.211+0000 7f41cd703640 1 --2- 172.21.15.67:0/3405426226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41c8101a50 0x7f41c81029f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:47.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.211+0000 7f41cd703640 1 -- 172.21.15.67:0/3405426226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c81a82c0 con 0x7f41c8109880 2024-07-27T03:12:47.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.212+0000 7f41ccf02640 1 --2- 172.21.15.67:0/3405426226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41c8101a50 0x7f41c81029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:12:47.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.212+0000 7f41c7fff640 1 --2- 172.21.15.67:0/3405426226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41c8108f30 0x7f41c8102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:12:47.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.212+0000 7f41cd703640 1 --2- 172.21.15.67:0/3405426226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c8109880 0x7f41c8103470 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009a20 tx=0x7f41b0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:47.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.212+0000 7f41c5ffb640 1 -- 172.21.15.67:0/3405426226 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b003f030 con 0x7f41c8109880 2024-07-27T03:12:47.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.212+0000 7f41c5ffb640 1 -- 172.21.15.67:0/3405426226 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41b0003aa0 con 0x7f41c8109880 2024-07-27T03:12:47.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.212+0000 7f41cf166640 1 -- 172.21.15.67:0/3405426226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c81a8550 con 0x7f41c8109880 2024-07-27T03:12:47.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.212+0000 7f41c5ffb640 1 -- 172.21.15.67:0/3405426226 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b0035070 con 0x7f41c8109880 2024-07-27T03:12:47.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.212+0000 7f41cf166640 1 -- 172.21.15.67:0/3405426226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c81a8b00 con 0x7f41c8109880 2024-07-27T03:12:47.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.213+0000 7f41cf166640 1 -- 172.21.15.67:0/3405426226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c8101ee0 con 0x7f41c8109880 2024-07-27T03:12:47.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.215+0000 7f41c5ffb640 1 -- 172.21.15.67:0/3405426226 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f41b0042030 con 0x7f41c8109880 2024-07-27T03:12:47.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.215+0000 7f41c5ffb640 1 --2- 172.21.15.67:0/3405426226 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f41b40b3c50 0x7f41b40b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:47.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.216+0000 7f41ccf02640 1 --2- 172.21.15.67:0/3405426226 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f41b40b3c50 0x7f41b40b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:47.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.216+0000 7f41c5ffb640 1 -- 172.21.15.67:0/3405426226 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8770+0+0 (secure 0 0 0) 0x7f41b0104680 con 0x7f41c8109880 2024-07-27T03:12:47.219 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.216+0000 7f41ccf02640 1 --2- 172.21.15.67:0/3405426226 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f41b40b3c50 0x7f41b40b6160 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f41c0006fd0 tx=0x7f41c0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:47.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.220+0000 7f41c5ffb640 1 -- 172.21.15.67:0/3405426226 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f41b00cbab0 con 0x7f41c8109880 2024-07-27T03:12:47.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.446+0000 7f41cf166640 1 -- 172.21.15.67:0/3405426226 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f41c802e100 con 0x7f41c8109880 2024-07-27T03:12:47.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.635+0000 7f41c5ffb640 1 -- 172.21.15.67:0/3405426226 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f41b00cb3b0 con 0x7f41c8109880 2024-07-27T03:12:47.638 INFO:teuthology.orchestra.run.smithi067.stderr:added data pool 4 to fsmap 2024-07-27T03:12:47.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.638+0000 7f41cf166640 1 -- 172.21.15.67:0/3405426226 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f41b40b3c50 msgr2=0x7f41b40b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:47.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.638+0000 7f41cf166640 1 --2- 172.21.15.67:0/3405426226 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f41b40b3c50 0x7f41b40b6160 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f41c0006fd0 tx=0x7f41c0008040 comp rx=0 tx=0).stop 2024-07-27T03:12:47.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.638+0000 7f41cf166640 1 -- 172.21.15.67:0/3405426226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c8109880 msgr2=0x7f41c8103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:47.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.638+0000 7f41cf166640 1 --2- 172.21.15.67:0/3405426226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c8109880 0x7f41c8103470 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009a20 tx=0x7f41b0004270 comp rx=0 tx=0).stop 2024-07-27T03:12:47.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.638+0000 7f41cf166640 1 -- 172.21.15.67:0/3405426226 shutdown_connections 2024-07-27T03:12:47.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.638+0000 7f41cf166640 1 --2- 172.21.15.67:0/3405426226 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f41b40b3c50 0x7f41b40b6160 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:47.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.638+0000 7f41cf166640 1 --2- 172.21.15.67:0/3405426226 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c8109880 0x7f41c8103470 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:47.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.638+0000 7f41cf166640 1 --2- 172.21.15.67:0/3405426226 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41c8108f30 0x7f41c8102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:47.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.638+0000 7f41cf166640 1 --2- 172.21.15.67:0/3405426226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41c8101a50 0x7f41c81029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:47.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.638+0000 7f41cf166640 1 -- 172.21.15.67:0/3405426226 >> 172.21.15.67:0/3405426226 conn(0x7f41c8077630 msgr2=0x7f41c8107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:47.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.638+0000 7f41cf166640 1 -- 172.21.15.67:0/3405426226 shutdown_connections 2024-07-27T03:12:47.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:47.638+0000 7f41cf166640 1 -- 172.21.15.67:0/3405426226 wait complete. 2024-07-27T03:12:47.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:47 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3405426226' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-07-27T03:12:48.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:47 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3405426226' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-07-27T03:12:48.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:47 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3405426226' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-07-27T03:12:48.349 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:12:48.621 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:12:48.964 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:48 smithi067 ceph-mon[32814]: pgmap v312: 129 pgs: 1 creating+activating, 128 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:48.964 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:48 smithi067 ceph-mon[32814]: osdmap e87: 12 total, 12 up, 12 in 2024-07-27T03:12:48.964 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:48 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3405426226' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-07-27T03:12:48.964 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:48 smithi067 ceph-mon[32814]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-27T03:12:49.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:48 smithi149 ceph-mon[37173]: pgmap v312: 129 pgs: 1 creating+activating, 128 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:49.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:48 smithi149 ceph-mon[37173]: osdmap e87: 12 total, 12 up, 12 in 2024-07-27T03:12:49.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:48 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3405426226' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-07-27T03:12:49.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:48 smithi149 ceph-mon[37173]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-27T03:12:49.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:48 smithi175 ceph-mon[37125]: pgmap v312: 129 pgs: 1 creating+activating, 128 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:49.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:48 smithi175 ceph-mon[37125]: osdmap e87: 12 total, 12 up, 12 in 2024-07-27T03:12:49.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:48 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3405426226' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-07-27T03:12:49.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:48 smithi175 ceph-mon[37125]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-27T03:12:49.823 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:49 smithi067 ceph-mon[32814]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-07-27T03:12:49.823 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:49 smithi067 ceph-mon[32814]: Cluster is now healthy 2024-07-27T03:12:50.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:49 smithi149 ceph-mon[37173]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-07-27T03:12:50.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:49 smithi149 ceph-mon[37173]: Cluster is now healthy 2024-07-27T03:12:50.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:49 smithi175 ceph-mon[37125]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-07-27T03:12:50.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:49 smithi175 ceph-mon[37125]: Cluster is now healthy 2024-07-27T03:12:50.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.367+0000 7fc06e0ae640 1 -- 172.21.15.67:0/1590830812 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc068104310 msgr2=0x7fc068104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:50.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.367+0000 7fc06e0ae640 1 --2- 172.21.15.67:0/1590830812 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc068104310 0x7fc068104720 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc058009900 tx=0x7fc05802a410 comp rx=0 tx=0).stop 2024-07-27T03:12:50.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.367+0000 7fc06e0ae640 1 -- 172.21.15.67:0/1590830812 shutdown_connections 2024-07-27T03:12:50.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.367+0000 7fc06e0ae640 1 --2- 172.21.15.67:0/1590830812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc06810d8c0 0x7fc06810fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:50.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.367+0000 7fc06e0ae640 1 --2- 172.21.15.67:0/1590830812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc068104d40 0x7fc06810d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:50.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.367+0000 7fc06e0ae640 1 --2- 172.21.15.67:0/1590830812 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc068104310 0x7fc068104720 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:50.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.367+0000 7fc06e0ae640 1 -- 172.21.15.67:0/1590830812 >> 172.21.15.67:0/1590830812 conn(0x7fc0680791a0 msgr2=0x7fc0680795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:50.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.367+0000 7fc06e0ae640 1 -- 172.21.15.67:0/1590830812 shutdown_connections 2024-07-27T03:12:50.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.367+0000 7fc06e0ae640 1 -- 172.21.15.67:0/1590830812 wait complete. 2024-07-27T03:12:50.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.368+0000 7fc06e0ae640 1 Processor -- start 2024-07-27T03:12:50.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.368+0000 7fc06e0ae640 1 -- start start 2024-07-27T03:12:50.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.369+0000 7fc06e0ae640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc068104310 0x7fc068199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:50.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.369+0000 7fc06e0ae640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc068104d40 0x7fc06819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:50.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.369+0000 7fc06e0ae640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc06810d8c0 0x7fc06819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:50.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.369+0000 7fc06e0ae640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc0681134e0 con 0x7fc06810d8c0 2024-07-27T03:12:50.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.369+0000 7fc06e0ae640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc068113360 con 0x7fc068104310 2024-07-27T03:12:50.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.369+0000 7fc06e0ae640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc068113660 con 0x7fc068104d40 2024-07-27T03:12:50.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.369+0000 7fc067fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc06810d8c0 0x7fc06819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:50.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.369+0000 7fc067fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc06810d8c0 0x7fc06819f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56214/0 (socket says 172.21.15.67:56214) 2024-07-27T03:12:50.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.369+0000 7fc067fff640 1 -- 172.21.15.67:0/3753448563 learned_addr learned my addr 172.21.15.67:0/3753448563 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:12:50.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.369+0000 7fc0677fe640 1 --2- 172.21.15.67:0/3753448563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc068104310 0x7fc068199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:50.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.369+0000 7fc05ffff640 1 --2- 172.21.15.67:0/3753448563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc068104d40 0x7fc06819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:50.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.369+0000 7fc067fff640 1 -- 172.21.15.67:0/3753448563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc068104d40 msgr2=0x7fc06819a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:50.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.369+0000 7fc067fff640 1 --2- 172.21.15.67:0/3753448563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc068104d40 0x7fc06819a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:50.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.369+0000 7fc067fff640 1 -- 172.21.15.67:0/3753448563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc068104310 msgr2=0x7fc068199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:50.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.369+0000 7fc067fff640 1 --2- 172.21.15.67:0/3753448563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc068104310 0x7fc068199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:50.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.369+0000 7fc067fff640 1 -- 172.21.15.67:0/3753448563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06819ff20 con 0x7fc06810d8c0 2024-07-27T03:12:50.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.369+0000 7fc067fff640 1 --2- 172.21.15.67:0/3753448563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc06810d8c0 0x7fc06819f7f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc06000cad0 tx=0x7fc06000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:50.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.370+0000 7fc0657fa640 1 -- 172.21.15.67:0/3753448563 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06000ae20 con 0x7fc06810d8c0 2024-07-27T03:12:50.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.370+0000 7fc0657fa640 1 -- 172.21.15.67:0/3753448563 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc060005020 con 0x7fc06810d8c0 2024-07-27T03:12:50.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.370+0000 7fc06e0ae640 1 -- 172.21.15.67:0/3753448563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0681a0210 con 0x7fc06810d8c0 2024-07-27T03:12:50.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.370+0000 7fc0657fa640 1 -- 172.21.15.67:0/3753448563 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06000f6b0 con 0x7fc06810d8c0 2024-07-27T03:12:50.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.370+0000 7fc06e0ae640 1 -- 172.21.15.67:0/3753448563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0681a6970 con 0x7fc06810d8c0 2024-07-27T03:12:50.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.371+0000 7fc06e0ae640 1 -- 172.21.15.67:0/3753448563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc068104720 con 0x7fc06810d8c0 2024-07-27T03:12:50.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.373+0000 7fc0657fa640 1 -- 172.21.15.67:0/3753448563 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7fc0600051d0 con 0x7fc06810d8c0 2024-07-27T03:12:50.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.373+0000 7fc0657fa640 1 --2- 172.21.15.67:0/3753448563 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc0540b3c50 0x7fc0540b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:50.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.373+0000 7fc0677fe640 1 --2- 172.21.15.67:0/3753448563 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc0540b3c50 0x7fc0540b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:50.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.374+0000 7fc0657fa640 1 -- 172.21.15.67:0/3753448563 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7fc0600e40b0 con 0x7fc06810d8c0 2024-07-27T03:12:50.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.374+0000 7fc0677fe640 1 --2- 172.21.15.67:0/3753448563 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc0540b3c50 0x7fc0540b6160 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc058000c00 tx=0x7fc058031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:50.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.376+0000 7fc0657fa640 1 -- 172.21.15.67:0/3753448563 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fc060014030 con 0x7fc06810d8c0 2024-07-27T03:12:50.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.596+0000 7fc06e0ae640 1 -- 172.21.15.67:0/3753448563 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc06819b0e0 con 0x7fc06810d8c0 2024-07-27T03:12:50.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.597+0000 7fc0657fa640 1 -- 172.21.15.67:0/3753448563 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11848 (secure 0 0 0) 0x7fc0600aaef0 con 0x7fc06810d8c0 2024-07-27T03:12:50.600 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:12:50.600 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"btime":"2024-07-27T03:12:47:627263+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14679,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/252686369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":252686369},{"type":"v1","addr":"172.21.15.67:6835","nonce":252686369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/3358832762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3358832762},{"type":"v1","addr":"172.21.15.67:6837","nonce":3358832762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/2162708317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2162708317},{"type":"v1","addr":"172.21.15.67:6839","nonce":2162708317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/795704244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":795704244},{"type":"v1","addr":"172.21.15.67:6841","nonce":795704244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24526,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24533,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3868325215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3868325215},{"type":"v1","addr":"172.21.15.149:6833","nonce":3868325215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24562,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2462749027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2462749027},{"type":"v1","addr":"172.21.15.175:6835","nonce":2462749027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24563,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3730964806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3730964806},{"type":"v1","addr":"172.21.15.149:6835","nonce":3730964806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24581,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1304803502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1304803502},{"type":"v1","addr":"172.21.15.175:6837","nonce":1304803502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24583,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/1670568853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1670568853},{"type":"v1","addr":"172.21.15.175:6839","nonce":1670568853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24587,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2932951845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2932951845},{"type":"v1","addr":"172.21.15.149:6837","nonce":2932951845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-27T03:12:34.320208+0000","modified":"2024-07-27T03:12:47.448376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24620":{"gid":24620,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.149:6839/281746498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":281746498},{"type":"v1","addr":"172.21.15.149:6839","nonce":281746498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24620,"qdb_cluster":[24620]},"id":1}]} 2024-07-27T03:12:50.601 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 17 2024-07-27T03:12:50.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.599+0000 7fc06e0ae640 1 -- 172.21.15.67:0/3753448563 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc0540b3c50 msgr2=0x7fc0540b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:50.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.599+0000 7fc06e0ae640 1 --2- 172.21.15.67:0/3753448563 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc0540b3c50 0x7fc0540b6160 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc058000c00 tx=0x7fc058031400 comp rx=0 tx=0).stop 2024-07-27T03:12:50.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.599+0000 7fc06e0ae640 1 -- 172.21.15.67:0/3753448563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc06810d8c0 msgr2=0x7fc06819f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:50.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.599+0000 7fc06e0ae640 1 --2- 172.21.15.67:0/3753448563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc06810d8c0 0x7fc06819f7f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc06000cad0 tx=0x7fc06000cf90 comp rx=0 tx=0).stop 2024-07-27T03:12:50.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.599+0000 7fc06e0ae640 1 -- 172.21.15.67:0/3753448563 shutdown_connections 2024-07-27T03:12:50.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.599+0000 7fc06e0ae640 1 --2- 172.21.15.67:0/3753448563 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc0540b3c50 0x7fc0540b6160 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:50.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.599+0000 7fc06e0ae640 1 --2- 172.21.15.67:0/3753448563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc06810d8c0 0x7fc06819f7f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:50.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.599+0000 7fc06e0ae640 1 --2- 172.21.15.67:0/3753448563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc068104d40 0x7fc06819a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:50.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.599+0000 7fc06e0ae640 1 --2- 172.21.15.67:0/3753448563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc068104310 0x7fc068199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:50.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.599+0000 7fc06e0ae640 1 -- 172.21.15.67:0/3753448563 >> 172.21.15.67:0/3753448563 conn(0x7fc0680791a0 msgr2=0x7fc0680ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:50.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.599+0000 7fc06e0ae640 1 -- 172.21.15.67:0/3753448563 shutdown_connections 2024-07-27T03:12:50.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:50.599+0000 7fc06e0ae640 1 -- 172.21.15.67:0/3753448563 wait complete. 2024-07-27T03:12:50.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:50 smithi067 ceph-mon[32814]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:50.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:50 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3753448563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:12:51.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:50 smithi149 ceph-mon[37173]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:51.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:50 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3753448563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:12:51.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:50 smithi175 ceph-mon[37125]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:51.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:50 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3753448563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:12:51.254 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd dump --format=json 2024-07-27T03:12:51.524 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:12:52.885 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:52 smithi067 ceph-mon[32814]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:53.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:52 smithi149 ceph-mon[37173]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:53.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.100+0000 7f24063ca640 1 -- 172.21.15.67:0/1947602552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2400107e10 msgr2=0x7f2400108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:53.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.100+0000 7f24063ca640 1 --2- 172.21.15.67:0/1947602552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2400107e10 0x7f2400108220 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009900 tx=0x7f23f402a410 comp rx=0 tx=0).stop 2024-07-27T03:12:53.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.100+0000 7f24063ca640 1 -- 172.21.15.67:0/1947602552 shutdown_connections 2024-07-27T03:12:53.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.100+0000 7f24063ca640 1 --2- 172.21.15.67:0/1947602552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2400108760 0x7f240010ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:53.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.100+0000 7f24063ca640 1 --2- 172.21.15.67:0/1947602552 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2400107e10 0x7f2400108220 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:53.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.100+0000 7f24063ca640 1 --2- 172.21.15.67:0/1947602552 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24001024d0 0x7f24001028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:53.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.100+0000 7f24063ca640 1 -- 172.21.15.67:0/1947602552 >> 172.21.15.67:0/1947602552 conn(0x7f24000791a0 msgr2=0x7f24000795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:53.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.100+0000 7f24063ca640 1 -- 172.21.15.67:0/1947602552 shutdown_connections 2024-07-27T03:12:53.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.100+0000 7f24063ca640 1 -- 172.21.15.67:0/1947602552 wait complete. 2024-07-27T03:12:53.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.101+0000 7f24063ca640 1 Processor -- start 2024-07-27T03:12:53.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.101+0000 7f24063ca640 1 -- start start 2024-07-27T03:12:53.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.101+0000 7f24063ca640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24001024d0 0x7f2400104b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:53.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.101+0000 7f24063ca640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2400107e10 0x7f2400105090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:53.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.101+0000 7f24063ca640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2400108760 0x7f24001055d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:53.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.101+0000 7f24063ca640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2400112410 con 0x7f2400108760 2024-07-27T03:12:53.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.101+0000 7f24063ca640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2400112290 con 0x7f24001024d0 2024-07-27T03:12:53.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.101+0000 7f24063ca640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2400112590 con 0x7f2400107e10 2024-07-27T03:12:53.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.101+0000 7f2404967640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2400108760 0x7f24001055d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:53.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.102+0000 7f2404967640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2400108760 0x7f24001055d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56234/0 (socket says 172.21.15.67:56234) 2024-07-27T03:12:53.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.102+0000 7f2404967640 1 -- 172.21.15.67:0/597423569 learned_addr learned my addr 172.21.15.67:0/597423569 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:12:53.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.102+0000 7f23fffff640 1 --2- 172.21.15.67:0/597423569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24001024d0 0x7f2400104b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:53.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.102+0000 7f23ff7fe640 1 --2- 172.21.15.67:0/597423569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2400107e10 0x7f2400105090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:53.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.102+0000 7f2404967640 1 -- 172.21.15.67:0/597423569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2400107e10 msgr2=0x7f2400105090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:53.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.102+0000 7f2404967640 1 --2- 172.21.15.67:0/597423569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2400107e10 0x7f2400105090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:53.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.102+0000 7f2404967640 1 -- 172.21.15.67:0/597423569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24001024d0 msgr2=0x7f2400104b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:53.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.102+0000 7f2404967640 1 --2- 172.21.15.67:0/597423569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24001024d0 0x7f2400104b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:53.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.102+0000 7f2404967640 1 -- 172.21.15.67:0/597423569 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24001a81c0 con 0x7f2400108760 2024-07-27T03:12:53.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.102+0000 7f23ff7fe640 1 --2- 172.21.15.67:0/597423569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2400107e10 0x7f2400105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:12:53.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.102+0000 7f23fffff640 1 --2- 172.21.15.67:0/597423569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24001024d0 0x7f2400104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:12:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.102+0000 7f2404967640 1 --2- 172.21.15.67:0/597423569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2400108760 0x7f24001055d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f23f800b970 tx=0x7f23f800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.102+0000 7f23fd7fa640 1 -- 172.21.15.67:0/597423569 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f8019070 con 0x7f2400108760 2024-07-27T03:12:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.102+0000 7f24063ca640 1 -- 172.21.15.67:0/597423569 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24001a84b0 con 0x7f2400108760 2024-07-27T03:12:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.102+0000 7f23fd7fa640 1 -- 172.21.15.67:0/597423569 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23f8014440 con 0x7f2400108760 2024-07-27T03:12:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.102+0000 7f23fd7fa640 1 -- 172.21.15.67:0/597423569 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f80075d0 con 0x7f2400108760 2024-07-27T03:12:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.102+0000 7f24063ca640 1 -- 172.21.15.67:0/597423569 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24001a8a90 con 0x7f2400108760 2024-07-27T03:12:53.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.103+0000 7f24063ca640 1 -- 172.21.15.67:0/597423569 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24001028e0 con 0x7f2400108760 2024-07-27T03:12:53.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.105+0000 7f23fd7fa640 1 -- 172.21.15.67:0/597423569 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f23f801c070 con 0x7f2400108760 2024-07-27T03:12:53.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.106+0000 7f23fd7fa640 1 --2- 172.21.15.67:0/597423569 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f23e80b3bd0 0x7f23e80b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:53.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.106+0000 7f23fffff640 1 --2- 172.21.15.67:0/597423569 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f23e80b3bd0 0x7f23e80b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:53.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.106+0000 7f23fd7fa640 1 -- 172.21.15.67:0/597423569 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8802+0+0 (secure 0 0 0) 0x7f23f80e38a0 con 0x7f2400108760 2024-07-27T03:12:53.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.106+0000 7f23fffff640 1 --2- 172.21.15.67:0/597423569 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f23e80b3bd0 0x7f23e80b60e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f24001065a0 tx=0x7f23f000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:53.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.109+0000 7f23fd7fa640 1 -- 172.21.15.67:0/597423569 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f23f80aa660 con 0x7f2400108760 2024-07-27T03:12:53.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:52 smithi175 ceph-mon[37125]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:53.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.296+0000 7f24063ca640 1 -- 172.21.15.67:0/597423569 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2400105e50 con 0x7f2400108760 2024-07-27T03:12:53.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.297+0000 7f23fd7fa640 1 -- 172.21.15.67:0/597423569 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24164 (secure 0 0 0) 0x7f2400105e50 con 0x7f2400108760 2024-07-27T03:12:53.300 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:12:53.301 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":87,"fsid":"c0782a00-4bc4-11ef-bcc7-c7b262605968","created":"2024-07-27T03:04:08.096824+0000","modified":"2024-07-27T03:12:47.448187+0000","last_up_change":"2024-07-27T03:11:33.314800+0000","last_in_change":"2024-07-27T03:11:14.028095+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-27T03:08:01.042479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-27T03:12:27.565948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-27T03:12:31.353726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-27T03:12:40.599486+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9f614272-18e8-4562-812c-4f43e40efd2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6803","nonce":2648129674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6805","nonce":2648129674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6809","nonce":2648129674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6807","nonce":2648129674}]},"public_addr":"172.21.15.67:6803/2648129674","cluster_addr":"172.21.15.67:6805/2648129674","heartbeat_back_addr":"172.21.15.67:6809/2648129674","heartbeat_front_addr":"172.21.15.67:6807/2648129674","state":["exists","up"]},{"osd":1,"uuid":"f59f5879-163a-4269-bbe7-1200eea65341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6801","nonce":2561753152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6803","nonce":2561753152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6807","nonce":2561753152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6805","nonce":2561753152}]},"public_addr":"172.21.15.149:6801/2561753152","cluster_addr":"172.21.15.149:6803/2561753152","heartbeat_back_addr":"172.21.15.149:6807/2561753152","heartbeat_front_addr":"172.21.15.149:6805/2561753152","state":["exists","up"]},{"osd":2,"uuid":"444e0ebe-7ca3-4be9-8bd5-db859f824445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6801","nonce":3770940078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6803","nonce":3770940078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6807","nonce":3770940078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6805","nonce":3770940078}]},"public_addr":"172.21.15.175:6801/3770940078","cluster_addr":"172.21.15.175:6803/3770940078","heartbeat_back_addr":"172.21.15.175:6807/3770940078","heartbeat_front_addr":"172.21.15.175:6805/3770940078","state":["exists","up"]},{"osd":3,"uuid":"d554db3d-9604-427c-8736-b1bb379f8d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6811","nonce":3159144693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6813","nonce":3159144693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6817","nonce":3159144693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6815","nonce":3159144693}]},"public_addr":"172.21.15.67:6811/3159144693","cluster_addr":"172.21.15.67:6813/3159144693","heartbeat_back_addr":"172.21.15.67:6817/3159144693","heartbeat_front_addr":"172.21.15.67:6815/3159144693","state":["exists","up"]},{"osd":4,"uuid":"c317dbe5-3b49-47a6-8408-7042a3dac345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3393503532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3393503532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3393503532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3393503532}]},"public_addr":"172.21.15.149:6809/3393503532","cluster_addr":"172.21.15.149:6811/3393503532","heartbeat_back_addr":"172.21.15.149:6815/3393503532","heartbeat_front_addr":"172.21.15.149:6813/3393503532","state":["exists","up"]},{"osd":5,"uuid":"5c85abd6-2b42-4957-a9b7-59a9ee36b841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6809","nonce":2291204784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6811","nonce":2291204784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6815","nonce":2291204784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6813","nonce":2291204784}]},"public_addr":"172.21.15.175:6809/2291204784","cluster_addr":"172.21.15.175:6811/2291204784","heartbeat_back_addr":"172.21.15.175:6815/2291204784","heartbeat_front_addr":"172.21.15.175:6813/2291204784","state":["exists","up"]},{"osd":6,"uuid":"bf6c277d-ec24-4470-98de-cf6f28876ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6819","nonce":2660182879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6821","nonce":2660182879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6825","nonce":2660182879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6823","nonce":2660182879}]},"public_addr":"172.21.15.67:6819/2660182879","cluster_addr":"172.21.15.67:6821/2660182879","heartbeat_back_addr":"172.21.15.67:6825/2660182879","heartbeat_front_addr":"172.21.15.67:6823/2660182879","state":["exists","up"]},{"osd":7,"uuid":"fea2c7cf-d5cb-45ae-9b29-f6e56c903891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6817","nonce":2289624404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6819","nonce":2289624404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6823","nonce":2289624404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6821","nonce":2289624404}]},"public_addr":"172.21.15.149:6817/2289624404","cluster_addr":"172.21.15.149:6819/2289624404","heartbeat_back_addr":"172.21.15.149:6823/2289624404","heartbeat_front_addr":"172.21.15.149:6821/2289624404","state":["exists","up"]},{"osd":8,"uuid":"14d04d96-579a-4dc9-a4d4-50e4850b48e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6817","nonce":2286032466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6819","nonce":2286032466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6823","nonce":2286032466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6821","nonce":2286032466}]},"public_addr":"172.21.15.175:6817/2286032466","cluster_addr":"172.21.15.175:6819/2286032466","heartbeat_back_addr":"172.21.15.175:6823/2286032466","heartbeat_front_addr":"172.21.15.175:6821/2286032466","state":["exists","up"]},{"osd":9,"uuid":"b65dfb6b-1efe-4314-a8be-ce6a295e99fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6827","nonce":3411081513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6829","nonce":3411081513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6833","nonce":3411081513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6831","nonce":3411081513}]},"public_addr":"172.21.15.67:6827/3411081513","cluster_addr":"172.21.15.67:6829/3411081513","heartbeat_back_addr":"172.21.15.67:6833/3411081513","heartbeat_front_addr":"172.21.15.67:6831/3411081513","state":["exists","up"]},{"osd":10,"uuid":"e3dabae0-0d64-4222-984c-fc1cce883456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6825","nonce":2011495212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6827","nonce":2011495212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6831","nonce":2011495212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6829","nonce":2011495212}]},"public_addr":"172.21.15.149:6825/2011495212","cluster_addr":"172.21.15.149:6827/2011495212","heartbeat_back_addr":"172.21.15.149:6831/2011495212","heartbeat_front_addr":"172.21.15.149:6829/2011495212","state":["exists","up"]},{"osd":11,"uuid":"406c082c-9b5f-4781-8cbf-3c04fa49e410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6825","nonce":2565514213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6827","nonce":2565514213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6831","nonce":2565514213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6829","nonce":2565514213}]},"public_addr":"172.21.15.175:6825/2565514213","cluster_addr":"172.21.15.175:6827/2565514213","heartbeat_back_addr":"172.21.15.175:6831/2565514213","heartbeat_front_addr":"172.21.15.175:6829/2565514213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:11.399380+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:33.728518+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:56.411181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:19.871731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:42.994576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:06.712815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:31.321967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:55.070898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:18.686108+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:43.187795+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:07.318538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:31.374843+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/3477601235":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/4130679770":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6800/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/1029332005":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6801/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6801/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/284078926":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6800/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/2559121542":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/925406021":"2024-07-28T03:04:30.600273+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-27T03:12:53.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.299+0000 7f24063ca640 1 -- 172.21.15.67:0/597423569 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f23e80b3bd0 msgr2=0x7f23e80b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:53.302 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.299+0000 7f24063ca640 1 --2- 172.21.15.67:0/597423569 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f23e80b3bd0 0x7f23e80b60e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f24001065a0 tx=0x7f23f000b390 comp rx=0 tx=0).stop 2024-07-27T03:12:53.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.299+0000 7f24063ca640 1 -- 172.21.15.67:0/597423569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2400108760 msgr2=0x7f24001055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:53.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.299+0000 7f24063ca640 1 --2- 172.21.15.67:0/597423569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2400108760 0x7f24001055d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f23f800b970 tx=0x7f23f800be30 comp rx=0 tx=0).stop 2024-07-27T03:12:53.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.299+0000 7f24063ca640 1 -- 172.21.15.67:0/597423569 shutdown_connections 2024-07-27T03:12:53.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.299+0000 7f24063ca640 1 --2- 172.21.15.67:0/597423569 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f23e80b3bd0 0x7f23e80b60e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:53.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.299+0000 7f24063ca640 1 --2- 172.21.15.67:0/597423569 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2400108760 0x7f24001055d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:53.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.299+0000 7f24063ca640 1 --2- 172.21.15.67:0/597423569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2400107e10 0x7f2400105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:53.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.299+0000 7f24063ca640 1 --2- 172.21.15.67:0/597423569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24001024d0 0x7f2400104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:53.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.300+0000 7f24063ca640 1 -- 172.21.15.67:0/597423569 >> 172.21.15.67:0/597423569 conn(0x7f24000791a0 msgr2=0x7f240006acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:53.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.300+0000 7f24063ca640 1 -- 172.21.15.67:0/597423569 shutdown_connections 2024-07-27T03:12:53.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:53.300+0000 7f24063ca640 1 -- 172.21.15.67:0/597423569 wait complete. 2024-07-27T03:12:53.898 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:53 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/597423569' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:12:53.928 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd dump --format=json 2024-07-27T03:12:54.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:53 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/597423569' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:12:54.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:53 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/597423569' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:12:54.194 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:12:54.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:54 smithi067 ceph-mon[32814]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:55.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:54 smithi149 ceph-mon[37173]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:55.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:54 smithi175 ceph-mon[37125]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:55.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.858+0000 7f95bc29c640 1 -- 172.21.15.67:0/1030004508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95b4108780 msgr2=0x7f95b410efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:55.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.858+0000 7f95bc29c640 1 --2- 172.21.15.67:0/1030004508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95b4108780 0x7f95b410efb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f95ac009ca0 tx=0x7f95ac02a8c0 comp rx=0 tx=0).stop 2024-07-27T03:12:55.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.858+0000 7f95bc29c640 1 -- 172.21.15.67:0/1030004508 shutdown_connections 2024-07-27T03:12:55.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.858+0000 7f95bc29c640 1 --2- 172.21.15.67:0/1030004508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95b4108780 0x7f95b410efb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:55.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.858+0000 7f95bc29c640 1 --2- 172.21.15.67:0/1030004508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95b4107e30 0x7f95b4108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:55.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.858+0000 7f95bc29c640 1 --2- 172.21.15.67:0/1030004508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95b41024f0 0x7f95b4102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:55.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.858+0000 7f95bc29c640 1 -- 172.21.15.67:0/1030004508 >> 172.21.15.67:0/1030004508 conn(0x7f95b40791b0 msgr2=0x7f95b40795c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:55.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.858+0000 7f95bc29c640 1 -- 172.21.15.67:0/1030004508 shutdown_connections 2024-07-27T03:12:55.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.858+0000 7f95bc29c640 1 -- 172.21.15.67:0/1030004508 wait complete. 2024-07-27T03:12:55.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.859+0000 7f95bc29c640 1 Processor -- start 2024-07-27T03:12:55.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.859+0000 7f95bc29c640 1 -- start start 2024-07-27T03:12:55.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.859+0000 7f95bc29c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95b41024f0 0x7f95b419ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:55.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.859+0000 7f95bc29c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95b4107e30 0x7f95b419b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:55.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.859+0000 7f95bc29c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95b4108780 0x7f95b419b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:55.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.859+0000 7f95bc29c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f95b4112450 con 0x7f95b4108780 2024-07-27T03:12:55.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.859+0000 7f95bc29c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f95b41122d0 con 0x7f95b41024f0 2024-07-27T03:12:55.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.859+0000 7f95bc29c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f95b41125d0 con 0x7f95b4107e30 2024-07-27T03:12:55.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.860+0000 7f95ba839640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95b4108780 0x7f95b419b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:55.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.860+0000 7f95ba839640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95b4108780 0x7f95b419b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59528/0 (socket says 172.21.15.67:59528) 2024-07-27T03:12:55.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.860+0000 7f95ba839640 1 -- 172.21.15.67:0/1328145698 learned_addr learned my addr 172.21.15.67:0/1328145698 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:12:55.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.860+0000 7f95b9837640 1 --2- 172.21.15.67:0/1328145698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95b4107e30 0x7f95b419b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:55.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.860+0000 7f95ba038640 1 --2- 172.21.15.67:0/1328145698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95b41024f0 0x7f95b419ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:55.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.860+0000 7f95ba839640 1 -- 172.21.15.67:0/1328145698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95b4107e30 msgr2=0x7f95b419b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:55.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.860+0000 7f95ba839640 1 --2- 172.21.15.67:0/1328145698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95b4107e30 0x7f95b419b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:55.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.860+0000 7f95ba839640 1 -- 172.21.15.67:0/1328145698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95b41024f0 msgr2=0x7f95b419ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:55.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.860+0000 7f95ba839640 1 --2- 172.21.15.67:0/1328145698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95b41024f0 0x7f95b419ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:55.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.860+0000 7f95ba839640 1 -- 172.21.15.67:0/1328145698 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b41a5fc0 con 0x7f95b4108780 2024-07-27T03:12:55.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.860+0000 7f95ba839640 1 --2- 172.21.15.67:0/1328145698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95b4108780 0x7f95b419b780 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f95ac005cc0 tx=0x7f95ac02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:55.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.860+0000 7f95ab7fe640 1 -- 172.21.15.67:0/1328145698 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac003bf0 con 0x7f95b4108780 2024-07-27T03:12:55.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.860+0000 7f95ab7fe640 1 -- 172.21.15.67:0/1328145698 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95ac003d90 con 0x7f95b4108780 2024-07-27T03:12:55.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.860+0000 7f95bc29c640 1 -- 172.21.15.67:0/1328145698 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b41a6250 con 0x7f95b4108780 2024-07-27T03:12:55.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.860+0000 7f95ab7fe640 1 -- 172.21.15.67:0/1328145698 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac02fb20 con 0x7f95b4108780 2024-07-27T03:12:55.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.860+0000 7f95bc29c640 1 -- 172.21.15.67:0/1328145698 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b41a6810 con 0x7f95b4108780 2024-07-27T03:12:55.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.861+0000 7f95bc29c640 1 -- 172.21.15.67:0/1328145698 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95b4103770 con 0x7f95b4108780 2024-07-27T03:12:55.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.863+0000 7f95ab7fe640 1 -- 172.21.15.67:0/1328145698 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f95ac004580 con 0x7f95b4108780 2024-07-27T03:12:55.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.864+0000 7f95ab7fe640 1 --2- 172.21.15.67:0/1328145698 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f95a00b3bd0 0x7f95a00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:55.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.864+0000 7f95ba038640 1 --2- 172.21.15.67:0/1328145698 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f95a00b3bd0 0x7f95a00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:55.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.864+0000 7f95ab7fe640 1 -- 172.21.15.67:0/1328145698 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f95ac1055b0 con 0x7f95b4108780 2024-07-27T03:12:55.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.864+0000 7f95ba038640 1 --2- 172.21.15.67:0/1328145698 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f95a00b3bd0 0x7f95a00b60e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f95b406c130 tx=0x7f95b000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:55.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:55.867+0000 7f95ab7fe640 1 -- 172.21.15.67:0/1328145698 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f95ac0cc8a0 con 0x7f95b4108780 2024-07-27T03:12:55.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-07-27T03:12:55.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-07-27T03:12:55.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-07-27T03:12:55.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-07-27T03:12:55.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-07-27T03:12:55.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-07-27T03:12:55.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-07-27T03:12:55.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]: dispatch 2024-07-27T03:12:55.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-07-27T03:12:55.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:55 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-07-27T03:12:56.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:55 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-07-27T03:12:56.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:55 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-07-27T03:12:56.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:55 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-07-27T03:12:56.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:55 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-07-27T03:12:56.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:55 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-07-27T03:12:56.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:55 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-07-27T03:12:56.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:55 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-07-27T03:12:56.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:55 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]: dispatch 2024-07-27T03:12:56.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:55 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-07-27T03:12:56.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:55 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-07-27T03:12:56.058 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:56.055+0000 7f95bc29c640 1 -- 172.21.15.67:0/1328145698 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95b410a5d0 con 0x7f95b4108780 2024-07-27T03:12:56.059 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:56.056+0000 7f95ab7fe640 1 -- 172.21.15.67:0/1328145698 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24690 (secure 0 0 0) 0x7f95ac0cc1a0 con 0x7f95b4108780 2024-07-27T03:12:56.060 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:12:56.060 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":88,"fsid":"c0782a00-4bc4-11ef-bcc7-c7b262605968","created":"2024-07-27T03:04:08.096824+0000","modified":"2024-07-27T03:12:55.701429+0000","last_up_change":"2024-07-27T03:11:33.314800+0000","last_in_change":"2024-07-27T03:11:14.028095+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-27T03:08:01.042479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-27T03:12:27.565948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-27T03:12:31.353726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-27T03:12:40.599486+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9f614272-18e8-4562-812c-4f43e40efd2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6803","nonce":2648129674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6805","nonce":2648129674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6809","nonce":2648129674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6807","nonce":2648129674}]},"public_addr":"172.21.15.67:6803/2648129674","cluster_addr":"172.21.15.67:6805/2648129674","heartbeat_back_addr":"172.21.15.67:6809/2648129674","heartbeat_front_addr":"172.21.15.67:6807/2648129674","state":["exists","up"]},{"osd":1,"uuid":"f59f5879-163a-4269-bbe7-1200eea65341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6801","nonce":2561753152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6803","nonce":2561753152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6807","nonce":2561753152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6805","nonce":2561753152}]},"public_addr":"172.21.15.149:6801/2561753152","cluster_addr":"172.21.15.149:6803/2561753152","heartbeat_back_addr":"172.21.15.149:6807/2561753152","heartbeat_front_addr":"172.21.15.149:6805/2561753152","state":["exists","up"]},{"osd":2,"uuid":"444e0ebe-7ca3-4be9-8bd5-db859f824445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6801","nonce":3770940078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6803","nonce":3770940078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6807","nonce":3770940078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6805","nonce":3770940078}]},"public_addr":"172.21.15.175:6801/3770940078","cluster_addr":"172.21.15.175:6803/3770940078","heartbeat_back_addr":"172.21.15.175:6807/3770940078","heartbeat_front_addr":"172.21.15.175:6805/3770940078","state":["exists","up"]},{"osd":3,"uuid":"d554db3d-9604-427c-8736-b1bb379f8d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6811","nonce":3159144693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6813","nonce":3159144693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6817","nonce":3159144693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6815","nonce":3159144693}]},"public_addr":"172.21.15.67:6811/3159144693","cluster_addr":"172.21.15.67:6813/3159144693","heartbeat_back_addr":"172.21.15.67:6817/3159144693","heartbeat_front_addr":"172.21.15.67:6815/3159144693","state":["exists","up"]},{"osd":4,"uuid":"c317dbe5-3b49-47a6-8408-7042a3dac345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3393503532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3393503532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3393503532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3393503532}]},"public_addr":"172.21.15.149:6809/3393503532","cluster_addr":"172.21.15.149:6811/3393503532","heartbeat_back_addr":"172.21.15.149:6815/3393503532","heartbeat_front_addr":"172.21.15.149:6813/3393503532","state":["exists","up"]},{"osd":5,"uuid":"5c85abd6-2b42-4957-a9b7-59a9ee36b841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6809","nonce":2291204784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6811","nonce":2291204784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6815","nonce":2291204784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6813","nonce":2291204784}]},"public_addr":"172.21.15.175:6809/2291204784","cluster_addr":"172.21.15.175:6811/2291204784","heartbeat_back_addr":"172.21.15.175:6815/2291204784","heartbeat_front_addr":"172.21.15.175:6813/2291204784","state":["exists","up"]},{"osd":6,"uuid":"bf6c277d-ec24-4470-98de-cf6f28876ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6819","nonce":2660182879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6821","nonce":2660182879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6825","nonce":2660182879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6823","nonce":2660182879}]},"public_addr":"172.21.15.67:6819/2660182879","cluster_addr":"172.21.15.67:6821/2660182879","heartbeat_back_addr":"172.21.15.67:6825/2660182879","heartbeat_front_addr":"172.21.15.67:6823/2660182879","state":["exists","up"]},{"osd":7,"uuid":"fea2c7cf-d5cb-45ae-9b29-f6e56c903891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6817","nonce":2289624404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6819","nonce":2289624404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6823","nonce":2289624404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6821","nonce":2289624404}]},"public_addr":"172.21.15.149:6817/2289624404","cluster_addr":"172.21.15.149:6819/2289624404","heartbeat_back_addr":"172.21.15.149:6823/2289624404","heartbeat_front_addr":"172.21.15.149:6821/2289624404","state":["exists","up"]},{"osd":8,"uuid":"14d04d96-579a-4dc9-a4d4-50e4850b48e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6817","nonce":2286032466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6819","nonce":2286032466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6823","nonce":2286032466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6821","nonce":2286032466}]},"public_addr":"172.21.15.175:6817/2286032466","cluster_addr":"172.21.15.175:6819/2286032466","heartbeat_back_addr":"172.21.15.175:6823/2286032466","heartbeat_front_addr":"172.21.15.175:6821/2286032466","state":["exists","up"]},{"osd":9,"uuid":"b65dfb6b-1efe-4314-a8be-ce6a295e99fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6827","nonce":3411081513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6829","nonce":3411081513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6833","nonce":3411081513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6831","nonce":3411081513}]},"public_addr":"172.21.15.67:6827/3411081513","cluster_addr":"172.21.15.67:6829/3411081513","heartbeat_back_addr":"172.21.15.67:6833/3411081513","heartbeat_front_addr":"172.21.15.67:6831/3411081513","state":["exists","up"]},{"osd":10,"uuid":"e3dabae0-0d64-4222-984c-fc1cce883456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6825","nonce":2011495212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6827","nonce":2011495212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6831","nonce":2011495212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6829","nonce":2011495212}]},"public_addr":"172.21.15.149:6825/2011495212","cluster_addr":"172.21.15.149:6827/2011495212","heartbeat_back_addr":"172.21.15.149:6831/2011495212","heartbeat_front_addr":"172.21.15.149:6829/2011495212","state":["exists","up"]},{"osd":11,"uuid":"406c082c-9b5f-4781-8cbf-3c04fa49e410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6825","nonce":2565514213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6827","nonce":2565514213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6831","nonce":2565514213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6829","nonce":2565514213}]},"public_addr":"172.21.15.175:6825/2565514213","cluster_addr":"172.21.15.175:6827/2565514213","heartbeat_back_addr":"172.21.15.175:6831/2565514213","heartbeat_front_addr":"172.21.15.175:6829/2565514213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:11.399380+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:33.728518+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:56.411181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:19.871731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:42.994576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:06.712815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:31.321967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:55.070898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:18.686108+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:43.187795+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:07.318538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:31.374843+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/3477601235":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/4130679770":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6800/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/1029332005":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6801/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6801/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/284078926":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6800/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/2559121542":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/925406021":"2024-07-28T03:04:30.600273+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-27T03:12:56.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:56.058+0000 7f95bc29c640 1 -- 172.21.15.67:0/1328145698 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f95a00b3bd0 msgr2=0x7f95a00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:56.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:56.058+0000 7f95bc29c640 1 --2- 172.21.15.67:0/1328145698 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f95a00b3bd0 0x7f95a00b60e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f95b406c130 tx=0x7f95b000b390 comp rx=0 tx=0).stop 2024-07-27T03:12:56.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:56.058+0000 7f95bc29c640 1 -- 172.21.15.67:0/1328145698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95b4108780 msgr2=0x7f95b419b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:56.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:56.058+0000 7f95bc29c640 1 --2- 172.21.15.67:0/1328145698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95b4108780 0x7f95b419b780 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f95ac005cc0 tx=0x7f95ac02f380 comp rx=0 tx=0).stop 2024-07-27T03:12:56.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:56.058+0000 7f95bc29c640 1 -- 172.21.15.67:0/1328145698 shutdown_connections 2024-07-27T03:12:56.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:56.058+0000 7f95bc29c640 1 --2- 172.21.15.67:0/1328145698 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f95a00b3bd0 0x7f95a00b60e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:56.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:56.058+0000 7f95bc29c640 1 --2- 172.21.15.67:0/1328145698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f95b4108780 0x7f95b419b780 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:56.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:56.058+0000 7f95bc29c640 1 --2- 172.21.15.67:0/1328145698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95b4107e30 0x7f95b419b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:56.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:56.058+0000 7f95bc29c640 1 --2- 172.21.15.67:0/1328145698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95b41024f0 0x7f95b419ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:56.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:56.058+0000 7f95bc29c640 1 -- 172.21.15.67:0/1328145698 >> 172.21.15.67:0/1328145698 conn(0x7f95b40791b0 msgr2=0x7f95b4104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:56.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:56.059+0000 7f95bc29c640 1 -- 172.21.15.67:0/1328145698 shutdown_connections 2024-07-27T03:12:56.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:56.059+0000 7f95bc29c640 1 -- 172.21.15.67:0/1328145698 wait complete. 2024-07-27T03:12:56.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:55 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-07-27T03:12:56.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:55 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-07-27T03:12:56.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:55 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-07-27T03:12:56.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:55 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-07-27T03:12:56.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:55 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-07-27T03:12:56.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:55 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-07-27T03:12:56.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:55 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-07-27T03:12:56.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:55 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]: dispatch 2024-07-27T03:12:56.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:55 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-07-27T03:12:56.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:55 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-07-27T03:12:56.860 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:12:57.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:56 smithi149 ceph-mon[37173]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:57.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:56 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-07-27T03:12:57.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:56 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-07-27T03:12:57.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:56 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-07-27T03:12:57.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:56 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-07-27T03:12:57.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:56 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-07-27T03:12:57.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:56 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-07-27T03:12:57.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:56 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-07-27T03:12:57.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:56 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]': finished 2024-07-27T03:12:57.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:56 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-07-27T03:12:57.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:56 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-07-27T03:12:57.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:56 smithi149 ceph-mon[37173]: osdmap e88: 12 total, 12 up, 12 in 2024-07-27T03:12:57.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:56 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1328145698' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:12:57.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:56 smithi175 ceph-mon[37125]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:57.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:56 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-07-27T03:12:57.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:56 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-07-27T03:12:57.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:56 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-07-27T03:12:57.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:56 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-07-27T03:12:57.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:56 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-07-27T03:12:57.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:56 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-07-27T03:12:57.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:56 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-07-27T03:12:57.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:56 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]': finished 2024-07-27T03:12:57.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:56 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-07-27T03:12:57.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:56 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-07-27T03:12:57.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:56 smithi175 ceph-mon[37125]: osdmap e88: 12 total, 12 up, 12 in 2024-07-27T03:12:57.126 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:56 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1328145698' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:12:57.130 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:12:57.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:56 smithi067 ceph-mon[32814]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:57.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:56 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-07-27T03:12:57.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:56 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-07-27T03:12:57.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:56 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-07-27T03:12:57.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:56 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-07-27T03:12:57.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:56 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-07-27T03:12:57.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:56 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-07-27T03:12:57.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:56 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-07-27T03:12:57.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:56 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]': finished 2024-07-27T03:12:57.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:56 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-07-27T03:12:57.221 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:56 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-07-27T03:12:57.222 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:56 smithi067 ceph-mon[32814]: osdmap e88: 12 total, 12 up, 12 in 2024-07-27T03:12:57.222 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:56 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1328145698' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:12:57.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:57 smithi067 ceph-mon[32814]: osdmap e89: 12 total, 12 up, 12 in 2024-07-27T03:12:58.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:57 smithi149 ceph-mon[37173]: osdmap e89: 12 total, 12 up, 12 in 2024-07-27T03:12:58.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:57 smithi175 ceph-mon[37125]: osdmap e89: 12 total, 12 up, 12 in 2024-07-27T03:12:58.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.908+0000 7fd70a4ae640 1 -- 172.21.15.67:0/3244444213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd704108760 msgr2=0x7fd70410ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:58.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.908+0000 7fd70a4ae640 1 --2- 172.21.15.67:0/3244444213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd704108760 0x7fd70410ef90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc009ca0 tx=0x7fd6fc02a8c0 comp rx=0 tx=0).stop 2024-07-27T03:12:58.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.908+0000 7fd70a4ae640 1 -- 172.21.15.67:0/3244444213 shutdown_connections 2024-07-27T03:12:58.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.908+0000 7fd70a4ae640 1 --2- 172.21.15.67:0/3244444213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd704108760 0x7fd70410ef90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:58.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.908+0000 7fd70a4ae640 1 --2- 172.21.15.67:0/3244444213 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd704107e10 0x7fd704108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:58.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.908+0000 7fd70a4ae640 1 --2- 172.21.15.67:0/3244444213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7041024d0 0x7fd7041028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:58.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.908+0000 7fd70a4ae640 1 -- 172.21.15.67:0/3244444213 >> 172.21.15.67:0/3244444213 conn(0x7fd7040791a0 msgr2=0x7fd7040795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:58.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.908+0000 7fd70a4ae640 1 -- 172.21.15.67:0/3244444213 shutdown_connections 2024-07-27T03:12:58.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.908+0000 7fd70a4ae640 1 -- 172.21.15.67:0/3244444213 wait complete. 2024-07-27T03:12:58.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.909+0000 7fd70a4ae640 1 Processor -- start 2024-07-27T03:12:58.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.909+0000 7fd70a4ae640 1 -- start start 2024-07-27T03:12:58.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.909+0000 7fd70a4ae640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7041024d0 0x7fd70419c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:58.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.909+0000 7fd70a4ae640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd704107e10 0x7fd70419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:58.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.909+0000 7fd70a4ae640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd704108760 0x7fd7041a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:58.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.909+0000 7fd70a4ae640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd7041134f0 con 0x7fd704108760 2024-07-27T03:12:58.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.909+0000 7fd70a4ae640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd704113370 con 0x7fd7041024d0 2024-07-27T03:12:58.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.909+0000 7fd70a4ae640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd704113670 con 0x7fd704107e10 2024-07-27T03:12:58.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.909+0000 7fd708a4b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd704108760 0x7fd7041a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:58.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.909+0000 7fd708a4b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd704108760 0x7fd7041a18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59546/0 (socket says 172.21.15.67:59546) 2024-07-27T03:12:58.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.909+0000 7fd708a4b640 1 -- 172.21.15.67:0/4090317853 learned_addr learned my addr 172.21.15.67:0/4090317853 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:12:58.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.909+0000 7fd703fff640 1 --2- 172.21.15.67:0/4090317853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7041024d0 0x7fd70419c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:58.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.910+0000 7fd6fbfff640 1 --2- 172.21.15.67:0/4090317853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd704107e10 0x7fd70419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:58.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.910+0000 7fd708a4b640 1 -- 172.21.15.67:0/4090317853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd704107e10 msgr2=0x7fd70419c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:58.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.910+0000 7fd708a4b640 1 --2- 172.21.15.67:0/4090317853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd704107e10 0x7fd70419c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:58.921 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.910+0000 7fd708a4b640 1 -- 172.21.15.67:0/4090317853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7041024d0 msgr2=0x7fd70419c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:58.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.910+0000 7fd708a4b640 1 --2- 172.21.15.67:0/4090317853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7041024d0 0x7fd70419c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:58.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.910+0000 7fd708a4b640 1 -- 172.21.15.67:0/4090317853 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7041a1f50 con 0x7fd704108760 2024-07-27T03:12:58.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.910+0000 7fd703fff640 1 --2- 172.21.15.67:0/4090317853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7041024d0 0x7fd70419c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:12:58.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.910+0000 7fd708a4b640 1 --2- 172.21.15.67:0/4090317853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd704108760 0x7fd7041a18e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc005cc0 tx=0x7fd6fc02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:58.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.910+0000 7fd701ffb640 1 -- 172.21.15.67:0/4090317853 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc003bf0 con 0x7fd704108760 2024-07-27T03:12:58.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.910+0000 7fd701ffb640 1 -- 172.21.15.67:0/4090317853 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6fc003d90 con 0x7fd704108760 2024-07-27T03:12:58.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.910+0000 7fd70a4ae640 1 -- 172.21.15.67:0/4090317853 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7041a21e0 con 0x7fd704108760 2024-07-27T03:12:58.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.910+0000 7fd701ffb640 1 -- 172.21.15.67:0/4090317853 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc02fbc0 con 0x7fd704108760 2024-07-27T03:12:58.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.910+0000 7fd70a4ae640 1 -- 172.21.15.67:0/4090317853 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7041a8a60 con 0x7fd704108760 2024-07-27T03:12:58.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.911+0000 7fd70a4ae640 1 -- 172.21.15.67:0/4090317853 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7041028e0 con 0x7fd704108760 2024-07-27T03:12:58.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.913+0000 7fd701ffb640 1 -- 172.21.15.67:0/4090317853 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7fd6fc02fd60 con 0x7fd704108760 2024-07-27T03:12:58.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.914+0000 7fd701ffb640 1 --2- 172.21.15.67:0/4090317853 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd6f00b3c80 0x7fd6f00b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:12:58.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.914+0000 7fd703fff640 1 --2- 172.21.15.67:0/4090317853 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd6f00b3c80 0x7fd6f00b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:12:58.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.914+0000 7fd701ffb640 1 -- 172.21.15.67:0/4090317853 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fd6fc105680 con 0x7fd704108760 2024-07-27T03:12:58.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.914+0000 7fd703fff640 1 --2- 172.21.15.67:0/4090317853 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd6f00b3c80 0x7fd6f00b6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400abe0 tx=0x7fd6f400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:12:58.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:58.917+0000 7fd701ffb640 1 -- 172.21.15.67:0/4090317853 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fd6fc0cc970 con 0x7fd704108760 2024-07-27T03:12:58.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:58 smithi067 ceph-mon[32814]: pgmap v320: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:59.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:58 smithi149 ceph-mon[37173]: pgmap v320: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:59.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:58 smithi175 ceph-mon[37125]: pgmap v320: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:12:59.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:59.137+0000 7fd70a4ae640 1 -- 172.21.15.67:0/4090317853 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd70419d1d0 con 0x7fd704108760 2024-07-27T03:12:59.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:59.138+0000 7fd701ffb640 1 -- 172.21.15.67:0/4090317853 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11848 (secure 0 0 0) 0x7fd70419d1d0 con 0x7fd704108760 2024-07-27T03:12:59.141 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:12:59.142 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"btime":"2024-07-27T03:12:47:627263+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14679,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/252686369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":252686369},{"type":"v1","addr":"172.21.15.67:6835","nonce":252686369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/3358832762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3358832762},{"type":"v1","addr":"172.21.15.67:6837","nonce":3358832762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14736,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6839/2162708317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2162708317},{"type":"v1","addr":"172.21.15.67:6839","nonce":2162708317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6841/795704244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":795704244},{"type":"v1","addr":"172.21.15.67:6841","nonce":795704244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24526,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24533,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/3868325215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3868325215},{"type":"v1","addr":"172.21.15.149:6833","nonce":3868325215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24562,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2462749027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2462749027},{"type":"v1","addr":"172.21.15.175:6835","nonce":2462749027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24563,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/3730964806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3730964806},{"type":"v1","addr":"172.21.15.149:6835","nonce":3730964806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24581,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1304803502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1304803502},{"type":"v1","addr":"172.21.15.175:6837","nonce":1304803502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24583,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6839/1670568853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1670568853},{"type":"v1","addr":"172.21.15.175:6839","nonce":1670568853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24587,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2932951845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2932951845},{"type":"v1","addr":"172.21.15.149:6837","nonce":2932951845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-07-27T03:12:34.320208+0000","modified":"2024-07-27T03:12:47.448376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24620":{"gid":24620,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.149:6839/281746498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":281746498},{"type":"v1","addr":"172.21.15.149:6839","nonce":281746498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24620,"qdb_cluster":[24620]},"id":1}]} 2024-07-27T03:12:59.142 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 17 2024-07-27T03:12:59.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:59.140+0000 7fd70a4ae640 1 -- 172.21.15.67:0/4090317853 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd6f00b3c80 msgr2=0x7fd6f00b6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:59.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:59.140+0000 7fd70a4ae640 1 --2- 172.21.15.67:0/4090317853 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd6f00b3c80 0x7fd6f00b6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400abe0 tx=0x7fd6f400a5c0 comp rx=0 tx=0).stop 2024-07-27T03:12:59.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:59.140+0000 7fd70a4ae640 1 -- 172.21.15.67:0/4090317853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd704108760 msgr2=0x7fd7041a18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:12:59.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:59.140+0000 7fd70a4ae640 1 --2- 172.21.15.67:0/4090317853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd704108760 0x7fd7041a18e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc005cc0 tx=0x7fd6fc02f380 comp rx=0 tx=0).stop 2024-07-27T03:12:59.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:59.140+0000 7fd70a4ae640 1 -- 172.21.15.67:0/4090317853 shutdown_connections 2024-07-27T03:12:59.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:59.140+0000 7fd70a4ae640 1 --2- 172.21.15.67:0/4090317853 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd6f00b3c80 0x7fd6f00b6190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:59.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:59.140+0000 7fd70a4ae640 1 --2- 172.21.15.67:0/4090317853 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd704108760 0x7fd7041a18e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:59.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:59.140+0000 7fd70a4ae640 1 --2- 172.21.15.67:0/4090317853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd704107e10 0x7fd70419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:59.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:59.140+0000 7fd70a4ae640 1 --2- 172.21.15.67:0/4090317853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7041024d0 0x7fd70419c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:12:59.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:59.140+0000 7fd70a4ae640 1 -- 172.21.15.67:0/4090317853 >> 172.21.15.67:0/4090317853 conn(0x7fd7040791a0 msgr2=0x7fd704104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:12:59.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:59.140+0000 7fd70a4ae640 1 -- 172.21.15.67:0/4090317853 shutdown_connections 2024-07-27T03:12:59.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:12:59.140+0000 7fd70a4ae640 1 -- 172.21.15.67:0/4090317853 wait complete. 2024-07-27T03:12:59.990 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd dump --format=json 2024-07-27T03:13:00.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:12:59 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/4090317853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:13:00.014 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:12:59 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/4090317853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:13:00.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:12:59 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/4090317853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:13:00.257 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:13:00.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:00 smithi067 ceph-mon[32814]: pgmap v321: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:13:01.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:00 smithi149 ceph-mon[37173]: pgmap v321: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:13:01.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:00 smithi175 ceph-mon[37125]: pgmap v321: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:13:02.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.000+0000 7f3313797640 1 -- 172.21.15.67:0/3885666357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f330c107e10 msgr2=0x7f330c108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:02.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.000+0000 7f3313797640 1 --2- 172.21.15.67:0/3885666357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f330c107e10 0x7f330c108220 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3300009900 tx=0x7f330002a410 comp rx=0 tx=0).stop 2024-07-27T03:13:02.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.000+0000 7f3313797640 1 -- 172.21.15.67:0/3885666357 shutdown_connections 2024-07-27T03:13:02.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.000+0000 7f3313797640 1 --2- 172.21.15.67:0/3885666357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f330c108760 0x7f330c10ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:02.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.000+0000 7f3313797640 1 --2- 172.21.15.67:0/3885666357 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f330c107e10 0x7f330c108220 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:02.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.000+0000 7f3313797640 1 --2- 172.21.15.67:0/3885666357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f330c1024d0 0x7f330c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.000+0000 7f3313797640 1 -- 172.21.15.67:0/3885666357 >> 172.21.15.67:0/3885666357 conn(0x7f330c0791a0 msgr2=0x7f330c0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.000+0000 7f3313797640 1 -- 172.21.15.67:0/3885666357 shutdown_connections 2024-07-27T03:13:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.000+0000 7f3313797640 1 -- 172.21.15.67:0/3885666357 wait complete. 2024-07-27T03:13:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.001+0000 7f3313797640 1 Processor -- start 2024-07-27T03:13:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.001+0000 7f3313797640 1 -- start start 2024-07-27T03:13:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.001+0000 7f3313797640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f330c1024d0 0x7f330c19ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.001+0000 7f3313797640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f330c107e10 0x7f330c19b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.001+0000 7f3313797640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f330c108760 0x7f330c19b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.002+0000 7f3313797640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f330c112390 con 0x7f330c1024d0 2024-07-27T03:13:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.002+0000 7f3313797640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f330c112210 con 0x7f330c108760 2024-07-27T03:13:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.002+0000 7f3313797640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f330c112510 con 0x7f330c107e10 2024-07-27T03:13:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.002+0000 7f3311533640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f330c1024d0 0x7f330c19ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:02.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.002+0000 7f3311533640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f330c1024d0 0x7f330c19ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59572/0 (socket says 172.21.15.67:59572) 2024-07-27T03:13:02.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.002+0000 7f3311533640 1 -- 172.21.15.67:0/669429626 learned_addr learned my addr 172.21.15.67:0/669429626 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:13:02.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.002+0000 7f3311d34640 1 --2- 172.21.15.67:0/669429626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f330c108760 0x7f330c19b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:02.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.002+0000 7f3310d32640 1 --2- 172.21.15.67:0/669429626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f330c107e10 0x7f330c19b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:02.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.002+0000 7f3311533640 1 -- 172.21.15.67:0/669429626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f330c107e10 msgr2=0x7f330c19b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:02.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.002+0000 7f3311533640 1 --2- 172.21.15.67:0/669429626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f330c107e10 0x7f330c19b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:02.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.002+0000 7f3311533640 1 -- 172.21.15.67:0/669429626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f330c108760 msgr2=0x7f330c19b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:02.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.002+0000 7f3311533640 1 --2- 172.21.15.67:0/669429626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f330c108760 0x7f330c19b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:02.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.002+0000 7f3311533640 1 -- 172.21.15.67:0/669429626 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330c1a5fa0 con 0x7f330c1024d0 2024-07-27T03:13:02.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.002+0000 7f3311533640 1 --2- 172.21.15.67:0/669429626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f330c1024d0 0x7f330c19ace0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f330400d950 tx=0x7f330400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:02.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.002+0000 7f330a7fc640 1 -- 172.21.15.67:0/669429626 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304017070 con 0x7f330c1024d0 2024-07-27T03:13:02.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.002+0000 7f330a7fc640 1 -- 172.21.15.67:0/669429626 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3304009c30 con 0x7f330c1024d0 2024-07-27T03:13:02.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.002+0000 7f3313797640 1 -- 172.21.15.67:0/669429626 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f330c1a6290 con 0x7f330c1024d0 2024-07-27T03:13:02.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.002+0000 7f330a7fc640 1 -- 172.21.15.67:0/669429626 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330400f620 con 0x7f330c1024d0 2024-07-27T03:13:02.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.002+0000 7f3313797640 1 -- 172.21.15.67:0/669429626 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f330c1a67f0 con 0x7f330c1024d0 2024-07-27T03:13:02.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.003+0000 7f3313797640 1 -- 172.21.15.67:0/669429626 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f330c1028e0 con 0x7f330c1024d0 2024-07-27T03:13:02.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.006+0000 7f330a7fc640 1 -- 172.21.15.67:0/669429626 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f3304010460 con 0x7f330c1024d0 2024-07-27T03:13:02.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.007+0000 7f330a7fc640 1 --2- 172.21.15.67:0/669429626 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f32f40b3c50 0x7f32f40b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:02.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.007+0000 7f3310d32640 1 --2- 172.21.15.67:0/669429626 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f32f40b3c50 0x7f32f40b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:02.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.007+0000 7f330a7fc640 1 -- 172.21.15.67:0/669429626 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f33040e3bb0 con 0x7f330c1024d0 2024-07-27T03:13:02.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.007+0000 7f3310d32640 1 --2- 172.21.15.67:0/669429626 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f32f40b3c50 0x7f32f40b6160 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3300000c00 tx=0x7f3300031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:02.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.010+0000 7f330a7fc640 1 -- 172.21.15.67:0/669429626 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f33040aaf20 con 0x7f330c1024d0 2024-07-27T03:13:02.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.192+0000 7f3313797640 1 -- 172.21.15.67:0/669429626 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f330c02e100 con 0x7f330c1024d0 2024-07-27T03:13:02.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.194+0000 7f330a7fc640 1 -- 172.21.15.67:0/669429626 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24690 (secure 0 0 0) 0x7f3304009de0 con 0x7f330c1024d0 2024-07-27T03:13:02.198 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:13:02.198 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"c0782a00-4bc4-11ef-bcc7-c7b262605968","created":"2024-07-27T03:04:08.096824+0000","modified":"2024-07-27T03:12:56.708905+0000","last_up_change":"2024-07-27T03:11:33.314800+0000","last_in_change":"2024-07-27T03:11:14.028095+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-27T03:08:01.042479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-27T03:12:27.565948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-27T03:12:31.353726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-27T03:12:40.599486+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9f614272-18e8-4562-812c-4f43e40efd2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6803","nonce":2648129674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6805","nonce":2648129674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6809","nonce":2648129674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6807","nonce":2648129674}]},"public_addr":"172.21.15.67:6803/2648129674","cluster_addr":"172.21.15.67:6805/2648129674","heartbeat_back_addr":"172.21.15.67:6809/2648129674","heartbeat_front_addr":"172.21.15.67:6807/2648129674","state":["exists","up"]},{"osd":1,"uuid":"f59f5879-163a-4269-bbe7-1200eea65341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6801","nonce":2561753152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6803","nonce":2561753152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6807","nonce":2561753152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6805","nonce":2561753152}]},"public_addr":"172.21.15.149:6801/2561753152","cluster_addr":"172.21.15.149:6803/2561753152","heartbeat_back_addr":"172.21.15.149:6807/2561753152","heartbeat_front_addr":"172.21.15.149:6805/2561753152","state":["exists","up"]},{"osd":2,"uuid":"444e0ebe-7ca3-4be9-8bd5-db859f824445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6801","nonce":3770940078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6803","nonce":3770940078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6807","nonce":3770940078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6805","nonce":3770940078}]},"public_addr":"172.21.15.175:6801/3770940078","cluster_addr":"172.21.15.175:6803/3770940078","heartbeat_back_addr":"172.21.15.175:6807/3770940078","heartbeat_front_addr":"172.21.15.175:6805/3770940078","state":["exists","up"]},{"osd":3,"uuid":"d554db3d-9604-427c-8736-b1bb379f8d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6811","nonce":3159144693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6813","nonce":3159144693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6817","nonce":3159144693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6815","nonce":3159144693}]},"public_addr":"172.21.15.67:6811/3159144693","cluster_addr":"172.21.15.67:6813/3159144693","heartbeat_back_addr":"172.21.15.67:6817/3159144693","heartbeat_front_addr":"172.21.15.67:6815/3159144693","state":["exists","up"]},{"osd":4,"uuid":"c317dbe5-3b49-47a6-8408-7042a3dac345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3393503532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3393503532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3393503532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3393503532}]},"public_addr":"172.21.15.149:6809/3393503532","cluster_addr":"172.21.15.149:6811/3393503532","heartbeat_back_addr":"172.21.15.149:6815/3393503532","heartbeat_front_addr":"172.21.15.149:6813/3393503532","state":["exists","up"]},{"osd":5,"uuid":"5c85abd6-2b42-4957-a9b7-59a9ee36b841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6809","nonce":2291204784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6811","nonce":2291204784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6815","nonce":2291204784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6813","nonce":2291204784}]},"public_addr":"172.21.15.175:6809/2291204784","cluster_addr":"172.21.15.175:6811/2291204784","heartbeat_back_addr":"172.21.15.175:6815/2291204784","heartbeat_front_addr":"172.21.15.175:6813/2291204784","state":["exists","up"]},{"osd":6,"uuid":"bf6c277d-ec24-4470-98de-cf6f28876ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6819","nonce":2660182879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6821","nonce":2660182879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6825","nonce":2660182879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6823","nonce":2660182879}]},"public_addr":"172.21.15.67:6819/2660182879","cluster_addr":"172.21.15.67:6821/2660182879","heartbeat_back_addr":"172.21.15.67:6825/2660182879","heartbeat_front_addr":"172.21.15.67:6823/2660182879","state":["exists","up"]},{"osd":7,"uuid":"fea2c7cf-d5cb-45ae-9b29-f6e56c903891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6817","nonce":2289624404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6819","nonce":2289624404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6823","nonce":2289624404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6821","nonce":2289624404}]},"public_addr":"172.21.15.149:6817/2289624404","cluster_addr":"172.21.15.149:6819/2289624404","heartbeat_back_addr":"172.21.15.149:6823/2289624404","heartbeat_front_addr":"172.21.15.149:6821/2289624404","state":["exists","up"]},{"osd":8,"uuid":"14d04d96-579a-4dc9-a4d4-50e4850b48e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6817","nonce":2286032466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6819","nonce":2286032466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6823","nonce":2286032466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6821","nonce":2286032466}]},"public_addr":"172.21.15.175:6817/2286032466","cluster_addr":"172.21.15.175:6819/2286032466","heartbeat_back_addr":"172.21.15.175:6823/2286032466","heartbeat_front_addr":"172.21.15.175:6821/2286032466","state":["exists","up"]},{"osd":9,"uuid":"b65dfb6b-1efe-4314-a8be-ce6a295e99fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6827","nonce":3411081513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6829","nonce":3411081513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6833","nonce":3411081513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6831","nonce":3411081513}]},"public_addr":"172.21.15.67:6827/3411081513","cluster_addr":"172.21.15.67:6829/3411081513","heartbeat_back_addr":"172.21.15.67:6833/3411081513","heartbeat_front_addr":"172.21.15.67:6831/3411081513","state":["exists","up"]},{"osd":10,"uuid":"e3dabae0-0d64-4222-984c-fc1cce883456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6825","nonce":2011495212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6827","nonce":2011495212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6831","nonce":2011495212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6829","nonce":2011495212}]},"public_addr":"172.21.15.149:6825/2011495212","cluster_addr":"172.21.15.149:6827/2011495212","heartbeat_back_addr":"172.21.15.149:6831/2011495212","heartbeat_front_addr":"172.21.15.149:6829/2011495212","state":["exists","up"]},{"osd":11,"uuid":"406c082c-9b5f-4781-8cbf-3c04fa49e410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6825","nonce":2565514213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6827","nonce":2565514213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6831","nonce":2565514213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6829","nonce":2565514213}]},"public_addr":"172.21.15.175:6825/2565514213","cluster_addr":"172.21.15.175:6827/2565514213","heartbeat_back_addr":"172.21.15.175:6831/2565514213","heartbeat_front_addr":"172.21.15.175:6829/2565514213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:11.399380+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:33.728518+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:56.411181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:19.871731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:42.994576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:06.712815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:31.321967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:55.070898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:18.686108+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:43.187795+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:07.318538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:31.374843+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/3477601235":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/4130679770":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6800/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/1029332005":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6801/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6801/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/284078926":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6800/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/2559121542":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/925406021":"2024-07-28T03:04:30.600273+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-27T03:13:02.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.197+0000 7f3313797640 1 -- 172.21.15.67:0/669429626 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f32f40b3c50 msgr2=0x7f32f40b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:02.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.197+0000 7f3313797640 1 --2- 172.21.15.67:0/669429626 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f32f40b3c50 0x7f32f40b6160 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3300000c00 tx=0x7f3300031400 comp rx=0 tx=0).stop 2024-07-27T03:13:02.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.197+0000 7f3313797640 1 -- 172.21.15.67:0/669429626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f330c1024d0 msgr2=0x7f330c19ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:02.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.197+0000 7f3313797640 1 --2- 172.21.15.67:0/669429626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f330c1024d0 0x7f330c19ace0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f330400d950 tx=0x7f330400de10 comp rx=0 tx=0).stop 2024-07-27T03:13:02.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.197+0000 7f3313797640 1 -- 172.21.15.67:0/669429626 shutdown_connections 2024-07-27T03:13:02.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.197+0000 7f3313797640 1 --2- 172.21.15.67:0/669429626 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f32f40b3c50 0x7f32f40b6160 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:02.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.197+0000 7f3313797640 1 --2- 172.21.15.67:0/669429626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f330c108760 0x7f330c19b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:02.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.197+0000 7f3313797640 1 --2- 172.21.15.67:0/669429626 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f330c107e10 0x7f330c19b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:02.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.197+0000 7f3313797640 1 --2- 172.21.15.67:0/669429626 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f330c1024d0 0x7f330c19ace0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:02.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.197+0000 7f3313797640 1 -- 172.21.15.67:0/669429626 >> 172.21.15.67:0/669429626 conn(0x7f330c0791a0 msgr2=0x7f330c104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:02.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.197+0000 7f3313797640 1 -- 172.21.15.67:0/669429626 shutdown_connections 2024-07-27T03:13:02.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:02.197+0000 7f3313797640 1 -- 172.21.15.67:0/669429626 wait complete. 2024-07-27T03:13:02.781 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:02 smithi067 ceph-mon[32814]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-07-27T03:13:02.781 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:02 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/669429626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:13:02.811 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-07-27T03:12:34.320208+0000', 'modified': '2024-07-27T03:12:47.448376+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24620}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24620': {'gid': 24620, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.149:6839/281746498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 281746498}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 281746498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24620, 'qdb_cluster': [24620]} 2024-07-27T03:13:02.812 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-07-27T03:13:02.812 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.b status 2024-07-27T03:13:03.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:02 smithi149 ceph-mon[37173]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-07-27T03:13:03.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:02 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/669429626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:13:03.115 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:13:03.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:02 smithi175 ceph-mon[37125]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-07-27T03:13:03.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:02 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/669429626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:13:04.815 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:04 smithi067 ceph-mon[32814]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-07-27T03:13:04.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.811+0000 7f77a75c3640 1 -- 172.21.15.67:0/2059980198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a0108790 msgr2=0x7f77a010efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:04.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.811+0000 7f77a75c3640 1 --2- 172.21.15.67:0/2059980198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a0108790 0x7f77a010efc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7798009ca0 tx=0x7f779802a8c0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.811+0000 7f77a75c3640 1 -- 172.21.15.67:0/2059980198 shutdown_connections 2024-07-27T03:13:04.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.811+0000 7f77a75c3640 1 --2- 172.21.15.67:0/2059980198 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a0108790 0x7f77a010efc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.811+0000 7f77a75c3640 1 --2- 172.21.15.67:0/2059980198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a0107e40 0x7f77a0108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.811+0000 7f77a75c3640 1 --2- 172.21.15.67:0/2059980198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77a0102550 0x7f77a0102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.811+0000 7f77a75c3640 1 -- 172.21.15.67:0/2059980198 >> 172.21.15.67:0/2059980198 conn(0x7f77a00791d0 msgr2=0x7f77a00795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:04.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.811+0000 7f77a75c3640 1 -- 172.21.15.67:0/2059980198 shutdown_connections 2024-07-27T03:13:04.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.811+0000 7f77a75c3640 1 -- 172.21.15.67:0/2059980198 wait complete. 2024-07-27T03:13:04.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.811+0000 7f77a75c3640 1 Processor -- start 2024-07-27T03:13:04.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.811+0000 7f77a75c3640 1 -- start start 2024-07-27T03:13:04.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.812+0000 7f77a75c3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a0102550 0x7f77a0198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:04.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.812+0000 7f77a75c3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77a0107e40 0x7f77a01992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:04.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.812+0000 7f77a75c3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a0108790 0x7f77a019e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:04.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.812+0000 7f77a75c3640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f77a01123f0 con 0x7f77a0108790 2024-07-27T03:13:04.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.812+0000 7f77a75c3640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f77a0112270 con 0x7f77a0107e40 2024-07-27T03:13:04.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.812+0000 7f77a75c3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f77a0112570 con 0x7f77a0102550 2024-07-27T03:13:04.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.812+0000 7f77a5b60640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a0108790 0x7f77a019e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:04.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.812+0000 7f77a5b60640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a0108790 0x7f77a019e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59594/0 (socket says 172.21.15.67:59594) 2024-07-27T03:13:04.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.812+0000 7f77a5b60640 1 -- 172.21.15.67:0/4205551192 learned_addr learned my addr 172.21.15.67:0/4205551192 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:13:04.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.812+0000 7f77a535f640 1 --2- 172.21.15.67:0/4205551192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a0102550 0x7f77a0198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:04.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.812+0000 7f77a4b5e640 1 --2- 172.21.15.67:0/4205551192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77a0107e40 0x7f77a01992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:04.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.812+0000 7f77a5b60640 1 -- 172.21.15.67:0/4205551192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a0102550 msgr2=0x7f77a0198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:04.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.812+0000 7f77a5b60640 1 --2- 172.21.15.67:0/4205551192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a0102550 0x7f77a0198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.812+0000 7f77a5b60640 1 -- 172.21.15.67:0/4205551192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77a0107e40 msgr2=0x7f77a01992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:04.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.812+0000 7f77a5b60640 1 --2- 172.21.15.67:0/4205551192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77a0107e40 0x7f77a01992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.812+0000 7f77a5b60640 1 -- 172.21.15.67:0/4205551192 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a019ebb0 con 0x7f77a0108790 2024-07-27T03:13:04.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.813+0000 7f77a5b60640 1 --2- 172.21.15.67:0/4205551192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a0108790 0x7f77a019e540 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7798005cc0 tx=0x7f779802f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:04.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.813+0000 7f77927fc640 1 -- 172.21.15.67:0/4205551192 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779804c070 con 0x7f77a0108790 2024-07-27T03:13:04.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.813+0000 7f77927fc640 1 -- 172.21.15.67:0/4205551192 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f779803fc60 con 0x7f77a0108790 2024-07-27T03:13:04.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.813+0000 7f77a75c3640 1 -- 172.21.15.67:0/4205551192 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a019ee40 con 0x7f77a0108790 2024-07-27T03:13:04.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.813+0000 7f77927fc640 1 -- 172.21.15.67:0/4205551192 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77980047c0 con 0x7f77a0108790 2024-07-27T03:13:04.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.813+0000 7f77a75c3640 1 -- 172.21.15.67:0/4205551192 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a006b5b0 con 0x7f77a0108790 2024-07-27T03:13:04.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.816+0000 7f77adb48740 1 Processor -- start 2024-07-27T03:13:04.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.816+0000 7f77927fc640 1 -- 172.21.15.67:0/4205551192 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f779804f070 con 0x7f77a0108790 2024-07-27T03:13:04.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.816+0000 7f77adb48740 1 -- start start 2024-07-27T03:13:04.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.817+0000 7f77adb48740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f1d28f60 0x5648f20ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:04.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.817+0000 7f77adb48740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d000a0 0x5648f20ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:04.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.817+0000 7f77adb48740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d08280 0x5648f20ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:04.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.817+0000 7f77adb48740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5648f1fb62d0 con 0x5648f1d08280 2024-07-27T03:13:04.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.817+0000 7f77adb48740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5648f20448b0 con 0x5648f1d000a0 2024-07-27T03:13:04.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.817+0000 7f77adb48740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5648f2033db0 con 0x5648f1d28f60 2024-07-27T03:13:04.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.817+0000 7f77a4b5e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d08280 0x5648f20ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:04.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.817+0000 7f77a4b5e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d08280 0x5648f20ceca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59608/0 (socket says 172.21.15.67:59608) 2024-07-27T03:13:04.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.817+0000 7f77a4b5e640 1 -- 172.21.15.67:0/3626629584 learned_addr learned my addr 172.21.15.67:0/3626629584 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:13:04.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.817+0000 7f77a535f640 1 --2- 172.21.15.67:0/3626629584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d000a0 0x5648f20ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:04.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.817+0000 7f77a4b5e640 1 --2- 172.21.15.67:0/3626629584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f1d28f60 0x5648f20ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:04.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.818+0000 7f77927fc640 1 --2- 172.21.15.67:0/4205551192 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f778c0b3c80 0x7f778c0b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:04.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.818+0000 7f77a535f640 1 -- 172.21.15.67:0/3626629584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f1d28f60 msgr2=0x5648f20ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:13:04.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.818+0000 7f77a535f640 1 --2- 172.21.15.67:0/3626629584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f1d28f60 0x5648f20ce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.818+0000 7f77a535f640 1 -- 172.21.15.67:0/3626629584 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d08280 msgr2=0x5648f20ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:13:04.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.818+0000 7f77a535f640 1 --2- 172.21.15.67:0/3626629584 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d08280 0x5648f20ceca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.818+0000 7f77a535f640 1 -- 172.21.15.67:0/3626629584 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648f206db10 con 0x5648f1d000a0 2024-07-27T03:13:04.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.818+0000 7f77a4b5e640 1 --2- 172.21.15.67:0/3626629584 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d08280 0x5648f20ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:13:04.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.818+0000 7f77927fc640 1 -- 172.21.15.67:0/4205551192 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f7798108de0 con 0x7f77a0108790 2024-07-27T03:13:04.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.818+0000 7f77a5b60640 1 --2- 172.21.15.67:0/4205551192 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f778c0b3c80 0x7f778c0b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:04.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.818+0000 7f77a535f640 1 --2- 172.21.15.67:0/3626629584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d000a0 0x5648f20ce760 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f779c00abf0 tx=0x7f779c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe60eb32a062eccf server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:04.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.818+0000 7f778b7fe640 1 -- 172.21.15.67:0/3626629584 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c005220 con 0x5648f1d000a0 2024-07-27T03:13:04.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.818+0000 7f778b7fe640 1 -- 172.21.15.67:0/3626629584 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f779c004c50 con 0x5648f1d000a0 2024-07-27T03:13:04.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.818+0000 7f778b7fe640 1 -- 172.21.15.67:0/3626629584 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c018730 con 0x5648f1d000a0 2024-07-27T03:13:04.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.819+0000 7f77adb48740 1 -- 172.21.15.67:0/3626629584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d000a0 msgr2=0x5648f20ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:04.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.819+0000 7f77a5b60640 1 --2- 172.21.15.67:0/4205551192 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f778c0b3c80 0x7f778c0b6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f77980094e0 tx=0x7f77980095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:04.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.819+0000 7f77adb48740 1 --2- 172.21.15.67:0/3626629584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d000a0 0x5648f20ce760 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f779c00abf0 tx=0x7f779c00f790 comp rx=0 tx=0).stop 2024-07-27T03:13:04.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.819+0000 7f77adb48740 1 -- 172.21.15.67:0/3626629584 shutdown_connections 2024-07-27T03:13:04.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.819+0000 7f77adb48740 1 --2- 172.21.15.67:0/3626629584 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d08280 0x5648f20ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.819+0000 7f77adb48740 1 --2- 172.21.15.67:0/3626629584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d000a0 0x5648f20ce760 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.819+0000 7f77adb48740 1 --2- 172.21.15.67:0/3626629584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f1d28f60 0x5648f20ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.819+0000 7f77adb48740 1 -- 172.21.15.67:0/3626629584 >> 172.21.15.67:0/3626629584 conn(0x5648f1d1d350 msgr2=0x5648f20cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.819+0000 7f77adb48740 1 -- 172.21.15.67:0/3626629584 shutdown_connections 2024-07-27T03:13:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.819+0000 7f77adb48740 1 -- 172.21.15.67:0/3626629584 wait complete. 2024-07-27T03:13:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.820+0000 7f77adb48740 1 -- start start 2024-07-27T03:13:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.820+0000 7f77adb48740 1 Processor -- start 2024-07-27T03:13:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.820+0000 7f77adb48740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d000a0 0x5648f20e25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.820+0000 7f77adb48740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f1d08280 0x5648f20e2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.820+0000 7f77adb48740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d28f60 0x5648f20ee310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.820+0000 7f77adb48740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5648f1fd67d0 con 0x5648f1d000a0 2024-07-27T03:13:04.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.820+0000 7f77adb48740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5648f2037df0 con 0x5648f1d28f60 2024-07-27T03:13:04.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.820+0000 7f77adb48740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5648f1e975f0 con 0x5648f1d08280 2024-07-27T03:13:04.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.820+0000 7f77adb48740 10 client.0 _ll_register_callbacks cb 0x5648f1fde800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:13:04.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.821+0000 7f77a4b5e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d000a0 0x5648f20e25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:04.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.821+0000 7f77a4b5e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d000a0 0x5648f20e25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59624/0 (socket says 172.21.15.67:59624) 2024-07-27T03:13:04.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.821+0000 7f77a4b5e640 1 -- 172.21.15.67:0/2075069410 learned_addr learned my addr 172.21.15.67:0/2075069410 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:13:04.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.821+0000 7f77a4b5e640 1 --2- 172.21.15.67:0/2075069410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d28f60 0x5648f20ee310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:04.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.821+0000 7f77a535f640 1 --2- 172.21.15.67:0/2075069410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f1d08280 0x5648f20e2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:04.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.821+0000 7f77a4b5e640 1 -- 172.21.15.67:0/2075069410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f1d08280 msgr2=0x5648f20e2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:13:04.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.821+0000 7f77a4b5e640 1 --2- 172.21.15.67:0/2075069410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f1d08280 0x5648f20e2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.821+0000 7f77a4b5e640 1 -- 172.21.15.67:0/2075069410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d28f60 msgr2=0x5648f20ee310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:13:04.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.821+0000 7f77a4b5e640 1 --2- 172.21.15.67:0/2075069410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d28f60 0x5648f20ee310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.821+0000 7f77a4b5e640 1 -- 172.21.15.67:0/2075069410 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648f1fc9640 con 0x5648f1d000a0 2024-07-27T03:13:04.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.821+0000 7f77a4b5e640 1 --2- 172.21.15.67:0/2075069410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d000a0 0x5648f20e25b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f779400ae00 tx=0x7f7794004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9836e5cc17afde6 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:04.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.821+0000 7f7788ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:13:04.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.821+0000 7f7788ff9640 1 -- 172.21.15.67:0/2075069410 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779400d550 con 0x5648f1d000a0 2024-07-27T03:13:04.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.822+0000 7f77adb48740 1 -- 172.21.15.67:0/2075069410 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f779c004c50 con 0x5648f1d000a0 2024-07-27T03:13:04.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.822+0000 7f7788ff9640 1 -- 172.21.15.67:0/2075069410 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f7794019030 con 0x5648f1d000a0 2024-07-27T03:13:04.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.822+0000 7f7788ff9640 1 -- 172.21.15.67:0/2075069410 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f779402c050 con 0x5648f1d000a0 2024-07-27T03:13:04.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.822+0000 7f77adb48740 10 client.14850 fetch_fsmap learned FSMap version 17 2024-07-27T03:13:04.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.822+0000 7f77adb48740 1 -- 172.21.15.67:0/2075069410 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5648f1fdaac0 con 0x5648f1d000a0 2024-07-27T03:13:04.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.822+0000 7f7788ff9640 1 -- 172.21.15.67:0/2075069410 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f779c004c50 con 0x5648f1d000a0 2024-07-27T03:13:04.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.822+0000 7f7788ff9640 1 -- 172.21.15.67:0/2075069410 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f77940245e0 con 0x5648f1d000a0 2024-07-27T03:13:04.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.822+0000 7f77adb48740 10 client.14850 fetch_fsmap finished waiting for FSMap version 17 2024-07-27T03:13:04.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.822+0000 7f77adb48740 10 client.14850 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-27T03:13:04.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.822+0000 7f77adb48740 20 client.14850 populate_metadata read hostname 'smithi067' 2024-07-27T03:13:04.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.822+0000 7f77adb48740 1 --2- 172.21.15.67:0/2075069410 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5648f1ce78e0 0x5648f20f4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:04.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.822+0000 7f77adb48740 4 client.14850 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:13:04.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.822+0000 7f77adb48740 1 -- 172.21.15.67:0/2075069410 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648f2046800 con 0x5648f1ce78e0 2024-07-27T03:13:04.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.823+0000 7f77a535f640 1 --2- 172.21.15.67:0/2075069410 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5648f1ce78e0 0x5648f20f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:04.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.823+0000 7f7788ff9640 10 client.14850 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:13:04.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.823+0000 7f77a535f640 1 --2- 172.21.15.67:0/2075069410 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5648f1ce78e0 0x5648f20f4900 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9da929d5a3510720 server_cookie=a36286f0258a8628 in_seq=0 out_seq=0 2024-07-27T03:13:04.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.824+0000 7f7788ff9640 1 -- 172.21.15.67:0/2075069410 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5648f2046800 con 0x5648f1ce78e0 2024-07-27T03:13:04.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.824+0000 7f7788ff9640 10 client.14850 handle_command_reply: tid=0 2024-07-27T03:13:04.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.824+0000 7f77adb48740 1 client.14850 shutdown 2024-07-27T03:13:04.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.824+0000 7f77adb48740 1 -- 172.21.15.67:0/2075069410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d000a0 msgr2=0x5648f20e25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:04.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.824+0000 7f77adb48740 1 --2- 172.21.15.67:0/2075069410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d000a0 0x5648f20e25b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f779400ae00 tx=0x7f7794004730 comp rx=0 tx=0).stop 2024-07-27T03:13:04.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.824+0000 7f77adb48740 1 -- 172.21.15.67:0/2075069410 shutdown_connections 2024-07-27T03:13:04.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.824+0000 7f77adb48740 1 --2- 172.21.15.67:0/2075069410 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5648f1ce78e0 0x5648f20f4900 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.824+0000 7f77adb48740 1 --2- 172.21.15.67:0/2075069410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d28f60 0x5648f20ee310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.824+0000 7f77adb48740 1 --2- 172.21.15.67:0/2075069410 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f1d08280 0x5648f20e2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.824+0000 7f77adb48740 1 --2- 172.21.15.67:0/2075069410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d000a0 0x5648f20e25b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.824+0000 7f77adb48740 1 -- 172.21.15.67:0/2075069410 >> 172.21.15.67:0/2075069410 conn(0x5648f1d1d350 msgr2=0x5648f20ca710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:04.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.824+0000 7f7788ff9640 0 client.14850 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:13:04.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.825+0000 7f77adb48740 1 -- 172.21.15.67:0/2075069410 shutdown_connections 2024-07-27T03:13:04.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.825+0000 7f77adb48740 1 -- 172.21.15.67:0/2075069410 wait complete. 2024-07-27T03:13:04.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.825+0000 7f77adb48740 20 client.14850 trim_cache size 0 max 16384 2024-07-27T03:13:04.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.843+0000 7f77adb48740 1 Processor -- start 2024-07-27T03:13:04.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.843+0000 7f77adb48740 1 -- start start 2024-07-27T03:13:04.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.843+0000 7f77adb48740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d000a0 0x5648f20f3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:04.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.843+0000 7f77adb48740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d08280 0x5648f20ee280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:04.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.843+0000 7f77adb48740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f1d28f60 0x5648f20ee7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:04.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.843+0000 7f77adb48740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5648f20f2c40 con 0x5648f1d000a0 2024-07-27T03:13:04.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.843+0000 7f77adb48740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5648f20f3310 con 0x5648f1d08280 2024-07-27T03:13:04.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.843+0000 7f77adb48740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5648f20f39e0 con 0x5648f1d28f60 2024-07-27T03:13:04.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.843+0000 7f77a4b5e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d000a0 0x5648f20f3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:04.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.844+0000 7f77a4b5e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d000a0 0x5648f20f3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54088/0 (socket says 172.21.15.67:54088) 2024-07-27T03:13:04.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.844+0000 7f77a4b5e640 1 -- 172.21.15.67:0/3318847055 learned_addr learned my addr 172.21.15.67:0/3318847055 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:13:04.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.844+0000 7f77a535f640 1 --2- 172.21.15.67:0/3318847055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d08280 0x5648f20ee280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:04.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.844+0000 7f77a4b5e640 1 -- 172.21.15.67:0/3318847055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f1d28f60 msgr2=0x5648f20ee7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:13:04.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.844+0000 7f77a4b5e640 1 --2- 172.21.15.67:0/3318847055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f1d28f60 0x5648f20ee7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.844+0000 7f77a4b5e640 1 -- 172.21.15.67:0/3318847055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d08280 msgr2=0x5648f20ee280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:13:04.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.844+0000 7f77a4b5e640 1 --2- 172.21.15.67:0/3318847055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d08280 0x5648f20ee280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.844+0000 7f77a4b5e640 1 -- 172.21.15.67:0/3318847055 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648f1d94540 con 0x5648f1d000a0 2024-07-27T03:13:04.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.844+0000 7f77a535f640 1 --2- 172.21.15.67:0/3318847055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d08280 0x5648f20ee280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:13:04.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.844+0000 7f77a4b5e640 1 --2- 172.21.15.67:0/3318847055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d000a0 0x5648f20f3490 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7794005bd0 tx=0x7f7794011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1af1f8e99fcf3d server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:04.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.844+0000 7f778affd640 1 -- 172.21.15.67:0/3318847055 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779400a630 con 0x5648f1d000a0 2024-07-27T03:13:04.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.844+0000 7f778affd640 1 -- 172.21.15.67:0/3318847055 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7794024b80 con 0x5648f1d000a0 2024-07-27T03:13:04.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.844+0000 7f778affd640 1 -- 172.21.15.67:0/3318847055 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794005030 con 0x5648f1d000a0 2024-07-27T03:13:04.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.845+0000 7f77adb48740 1 -- 172.21.15.67:0/3318847055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d000a0 msgr2=0x5648f20f3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:04.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.845+0000 7f77adb48740 1 --2- 172.21.15.67:0/3318847055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d000a0 0x5648f20f3490 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7794005bd0 tx=0x7f7794011ec0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.845+0000 7f77adb48740 1 -- 172.21.15.67:0/3318847055 shutdown_connections 2024-07-27T03:13:04.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.845+0000 7f77adb48740 1 --2- 172.21.15.67:0/3318847055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f1d28f60 0x5648f20ee7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.845+0000 7f77adb48740 1 --2- 172.21.15.67:0/3318847055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d08280 0x5648f20ee280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.845+0000 7f77adb48740 1 --2- 172.21.15.67:0/3318847055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d000a0 0x5648f20f3490 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.845+0000 7f77adb48740 1 -- 172.21.15.67:0/3318847055 >> 172.21.15.67:0/3318847055 conn(0x5648f1d1d350 msgr2=0x5648f20dd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:04.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.845+0000 7f77adb48740 1 -- 172.21.15.67:0/3318847055 shutdown_connections 2024-07-27T03:13:04.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.845+0000 7f77adb48740 1 -- 172.21.15.67:0/3318847055 wait complete. 2024-07-27T03:13:04.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.845+0000 7f77adb48740 1 -- start start 2024-07-27T03:13:04.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.846+0000 7f77adb48740 1 Processor -- start 2024-07-27T03:13:04.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.846+0000 7f77adb48740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d000a0 0x5648f20fd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:04.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.846+0000 7f77adb48740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d08280 0x5648f20f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:04.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.846+0000 7f77adb48740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f1d28f60 0x5648f20f6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:04.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.846+0000 7f77adb48740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5648f20f4780 con 0x5648f1d08280 2024-07-27T03:13:04.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.846+0000 7f77adb48740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5648f20f40b0 con 0x5648f1d000a0 2024-07-27T03:13:04.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.846+0000 7f77adb48740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5648f20f4e50 con 0x5648f1d28f60 2024-07-27T03:13:04.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.846+0000 7f77adb48740 10 client.0 _ll_register_callbacks cb 0x7f779400ad50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:13:04.855 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.846+0000 7f77a535f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d08280 0x5648f20f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:04.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.846+0000 7f77a535f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d08280 0x5648f20f5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54104/0 (socket says 172.21.15.67:54104) 2024-07-27T03:13:04.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.846+0000 7f77a535f640 1 -- 172.21.15.67:0/4289928488 learned_addr learned my addr 172.21.15.67:0/4289928488 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:13:04.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.846+0000 7f77a4b5e640 1 --2- 172.21.15.67:0/4289928488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d000a0 0x5648f20fd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:04.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.846+0000 7f77a535f640 1 -- 172.21.15.67:0/4289928488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f1d28f60 msgr2=0x5648f20f6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:13:04.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.846+0000 7f77a4b5e640 1 --2- 172.21.15.67:0/4289928488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f1d28f60 0x5648f20f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:04.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.847+0000 7f77a535f640 1 --2- 172.21.15.67:0/4289928488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f1d28f60 0x5648f20f6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.847+0000 7f77a535f640 1 -- 172.21.15.67:0/4289928488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d000a0 msgr2=0x5648f20fd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:13:04.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.847+0000 7f77a535f640 1 --2- 172.21.15.67:0/4289928488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d000a0 0x5648f20fd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.847+0000 7f77a535f640 1 -- 172.21.15.67:0/4289928488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648f20f6780 con 0x5648f1d08280 2024-07-27T03:13:04.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.847+0000 7f77a4b5e640 1 --2- 172.21.15.67:0/4289928488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d000a0 0x5648f20fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:13:04.856 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.847+0000 7f77a535f640 1 --2- 172.21.15.67:0/4289928488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d08280 0x5648f20f5b00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f779c0058c0 tx=0x7f779c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616a5b79158f0825 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:04.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.847+0000 7f7788ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:13:04.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.847+0000 7f7788ff9640 1 -- 172.21.15.67:0/4289928488 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c031020 con 0x5648f1d08280 2024-07-27T03:13:04.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.847+0000 7f7788ff9640 1 -- 172.21.15.67:0/4289928488 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f779c005d00 con 0x5648f1d08280 2024-07-27T03:13:04.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.847+0000 7f77adb48740 1 -- 172.21.15.67:0/4289928488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7794024b80 con 0x5648f1d08280 2024-07-27T03:13:04.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.847+0000 7f7788ff9640 1 -- 172.21.15.67:0/4289928488 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f779c03c050 con 0x5648f1d08280 2024-07-27T03:13:04.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.847+0000 7f77adb48740 10 client.14856 fetch_fsmap learned FSMap version 17 2024-07-27T03:13:04.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.847+0000 7f77adb48740 1 -- 172.21.15.67:0/4289928488 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5648f215ca60 con 0x5648f1d08280 2024-07-27T03:13:04.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.847+0000 7f7788ff9640 1 -- 172.21.15.67:0/4289928488 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7794024b80 con 0x5648f1d08280 2024-07-27T03:13:04.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.848+0000 7f7788ff9640 1 -- 172.21.15.67:0/4289928488 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f779c025550 con 0x5648f1d08280 2024-07-27T03:13:04.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.848+0000 7f77adb48740 10 client.14856 fetch_fsmap finished waiting for FSMap version 17 2024-07-27T03:13:04.857 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.848+0000 7f77adb48740 10 client.14856 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-27T03:13:04.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.848+0000 7f77adb48740 20 client.14856 populate_metadata read hostname 'smithi067' 2024-07-27T03:13:04.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.848+0000 7f77adb48740 1 --2- 172.21.15.67:0/4289928488 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5648f1cbcc90 0x5648f2160000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:04.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.848+0000 7f77adb48740 4 client.14856 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-27T03:13:04.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.848+0000 7f77adb48740 1 -- 172.21.15.67:0/4289928488 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "status"}) -- 0x5648f20422a0 con 0x5648f1cbcc90 2024-07-27T03:13:04.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.848+0000 7f77a4b5e640 1 --2- 172.21.15.67:0/4289928488 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5648f1cbcc90 0x5648f2160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:04.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.848+0000 7f77a4b5e640 1 --2- 172.21.15.67:0/4289928488 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5648f1cbcc90 0x5648f2160000 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=504fd58a915af283 server_cookie=2510ad893f9fedeb in_seq=0 out_seq=0 2024-07-27T03:13:04.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.848+0000 7f7788ff9640 10 client.14856 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:13:04.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.849+0000 7f7788ff9640 1 -- 172.21.15.67:0/4289928488 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5648f20422a0 con 0x5648f1cbcc90 2024-07-27T03:13:04.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.849+0000 7f7788ff9640 10 client.14856 handle_command_reply: tid=0 2024-07-27T03:13:04.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.849+0000 7f77adb48740 1 client.14856 shutdown 2024-07-27T03:13:04.858 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.849+0000 7f77adb48740 1 -- 172.21.15.67:0/4289928488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d08280 msgr2=0x5648f20f5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:04.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.849+0000 7f77adb48740 1 --2- 172.21.15.67:0/4289928488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d08280 0x5648f20f5b00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f779c0058c0 tx=0x7f779c0046b0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.849+0000 7f77adb48740 1 -- 172.21.15.67:0/4289928488 shutdown_connections 2024-07-27T03:13:04.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.849+0000 7f77adb48740 1 --2- 172.21.15.67:0/4289928488 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5648f1cbcc90 0x5648f2160000 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.849+0000 7f77adb48740 1 --2- 172.21.15.67:0/4289928488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5648f1d28f60 0x5648f20f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.849+0000 7f77adb48740 1 --2- 172.21.15.67:0/4289928488 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5648f1d08280 0x5648f20f5b00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.849+0000 7f7788ff9640 0 client.14856 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:13:04.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.849+0000 7f77adb48740 1 --2- 172.21.15.67:0/4289928488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648f1d000a0 0x5648f20fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.849+0000 7f77adb48740 1 -- 172.21.15.67:0/4289928488 >> 172.21.15.67:0/4289928488 conn(0x5648f1d1d350 msgr2=0x5648f20e0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:04.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.849+0000 7f77adb48740 1 -- 172.21.15.67:0/4289928488 shutdown_connections 2024-07-27T03:13:04.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.849+0000 7f77adb48740 1 -- 172.21.15.67:0/4289928488 wait complete. 2024-07-27T03:13:04.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.849+0000 7f77adb48740 20 client.14856 trim_cache size 0 max 16384 2024-07-27T03:13:04.860 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:13:04.860 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "c0782a00-4bc4-11ef-bcc7-c7b262605968", 2024-07-27T03:13:04.860 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-07-27T03:13:04.860 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24620, 2024-07-27T03:13:04.860 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-27T03:13:04.860 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-27T03:13:04.860 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-27T03:13:04.860 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 30.285999303000001, 2024-07-27T03:13:04.860 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 17, 2024-07-27T03:13:04.860 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 89, 2024-07-27T03:13:04.860 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 87, 2024-07-27T03:13:04.861 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 44.353246681999998 2024-07-27T03:13:04.861 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:13:04.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.850+0000 7f77a75c3640 1 -- 172.21.15.67:0/4205551192 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f778c0b3c80 msgr2=0x7f778c0b6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:04.861 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.850+0000 7f77a75c3640 1 --2- 172.21.15.67:0/4205551192 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f778c0b3c80 0x7f778c0b6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f77980094e0 tx=0x7f77980095c0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.850+0000 7f77a75c3640 1 -- 172.21.15.67:0/4205551192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a0108790 msgr2=0x7f77a019e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:04.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.850+0000 7f77a75c3640 1 --2- 172.21.15.67:0/4205551192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a0108790 0x7f77a019e540 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7798005cc0 tx=0x7f779802f470 comp rx=0 tx=0).stop 2024-07-27T03:13:04.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.850+0000 7f77a75c3640 1 -- 172.21.15.67:0/4205551192 shutdown_connections 2024-07-27T03:13:04.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.850+0000 7f77a75c3640 1 --2- 172.21.15.67:0/4205551192 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f778c0b3c80 0x7f778c0b6190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.850+0000 7f77a75c3640 1 --2- 172.21.15.67:0/4205551192 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f77a0108790 0x7f77a019e540 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.850+0000 7f77a75c3640 1 --2- 172.21.15.67:0/4205551192 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77a0107e40 0x7f77a01992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.850+0000 7f77a75c3640 1 --2- 172.21.15.67:0/4205551192 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a0102550 0x7f77a0198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:04.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.850+0000 7f77a75c3640 1 -- 172.21.15.67:0/4205551192 >> 172.21.15.67:0/4205551192 conn(0x7f77a00791d0 msgr2=0x7f77a0104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:04.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.850+0000 7f77a75c3640 1 -- 172.21.15.67:0/4205551192 shutdown_connections 2024-07-27T03:13:04.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:04.850+0000 7f77a75c3640 1 -- 172.21.15.67:0/4205551192 wait complete. 2024-07-27T03:13:05.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:04 smithi175 ceph-mon[37125]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-07-27T03:13:05.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:04 smithi149 ceph-mon[37173]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-07-27T03:13:05.681 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-07-27T03:13:05.681 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-07-27T03:13:05.681 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs ls 2024-07-27T03:13:05.958 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:13:06.963 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:06 smithi067 ceph-mon[32814]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-07-27T03:13:07.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:06 smithi175 ceph-mon[37125]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-07-27T03:13:07.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:06 smithi149 ceph-mon[37173]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-07-27T03:13:07.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.549+0000 7fde21bbf640 1 -- 172.21.15.67:0/2007826984 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde1c108760 msgr2=0x7fde1c10ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:07.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.549+0000 7fde21bbf640 1 --2- 172.21.15.67:0/2007826984 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde1c108760 0x7fde1c10ef90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fde14009ca0 tx=0x7fde1402a8c0 comp rx=0 tx=0).stop 2024-07-27T03:13:07.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.549+0000 7fde21bbf640 1 -- 172.21.15.67:0/2007826984 shutdown_connections 2024-07-27T03:13:07.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.549+0000 7fde21bbf640 1 --2- 172.21.15.67:0/2007826984 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde1c108760 0x7fde1c10ef90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:07.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.549+0000 7fde21bbf640 1 --2- 172.21.15.67:0/2007826984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde1c107e10 0x7fde1c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:07.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.549+0000 7fde21bbf640 1 --2- 172.21.15.67:0/2007826984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde1c1024d0 0x7fde1c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:07.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.549+0000 7fde21bbf640 1 -- 172.21.15.67:0/2007826984 >> 172.21.15.67:0/2007826984 conn(0x7fde1c0791a0 msgr2=0x7fde1c0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:07.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.550+0000 7fde21bbf640 1 -- 172.21.15.67:0/2007826984 shutdown_connections 2024-07-27T03:13:07.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.550+0000 7fde21bbf640 1 -- 172.21.15.67:0/2007826984 wait complete. 2024-07-27T03:13:07.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.550+0000 7fde21bbf640 1 Processor -- start 2024-07-27T03:13:07.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.550+0000 7fde21bbf640 1 -- start start 2024-07-27T03:13:07.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.551+0000 7fde21bbf640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde1c1024d0 0x7fde1c19acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:07.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.551+0000 7fde21bbf640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde1c107e10 0x7fde1c19b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:07.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.551+0000 7fde21bbf640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde1c108760 0x7fde1c19b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:07.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.551+0000 7fde21bbf640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c1123c0 con 0x7fde1c1024d0 2024-07-27T03:13:07.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.551+0000 7fde21bbf640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c112240 con 0x7fde1c107e10 2024-07-27T03:13:07.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.551+0000 7fde21bbf640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c112540 con 0x7fde1c108760 2024-07-27T03:13:07.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.551+0000 7fde1b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde1c1024d0 0x7fde1c19acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:07.569 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.551+0000 7fde1b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde1c1024d0 0x7fde1c19acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54126/0 (socket says 172.21.15.67:54126) 2024-07-27T03:13:07.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.551+0000 7fde1b7fe640 1 -- 172.21.15.67:0/3963274029 learned_addr learned my addr 172.21.15.67:0/3963274029 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:13:07.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.551+0000 7fde1bfff640 1 --2- 172.21.15.67:0/3963274029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde1c108760 0x7fde1c19b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:07.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.551+0000 7fde1affd640 1 --2- 172.21.15.67:0/3963274029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde1c107e10 0x7fde1c19b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:07.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.551+0000 7fde1b7fe640 1 -- 172.21.15.67:0/3963274029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde1c108760 msgr2=0x7fde1c19b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:07.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.551+0000 7fde1b7fe640 1 --2- 172.21.15.67:0/3963274029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde1c108760 0x7fde1c19b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:07.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.551+0000 7fde1b7fe640 1 -- 172.21.15.67:0/3963274029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde1c107e10 msgr2=0x7fde1c19b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:07.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.551+0000 7fde1b7fe640 1 --2- 172.21.15.67:0/3963274029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde1c107e10 0x7fde1c19b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:07.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.551+0000 7fde1b7fe640 1 -- 172.21.15.67:0/3963274029 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde1c1a5f00 con 0x7fde1c1024d0 2024-07-27T03:13:07.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.551+0000 7fde1affd640 1 --2- 172.21.15.67:0/3963274029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde1c107e10 0x7fde1c19b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:13:07.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.551+0000 7fde1b7fe640 1 --2- 172.21.15.67:0/3963274029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde1c1024d0 0x7fde1c19acd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00c920 tx=0x7fde0c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:07.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.552+0000 7fde18ff9640 1 -- 172.21.15.67:0/3963274029 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c007c70 con 0x7fde1c1024d0 2024-07-27T03:13:07.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.552+0000 7fde18ff9640 1 -- 172.21.15.67:0/3963274029 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde0c007e10 con 0x7fde1c1024d0 2024-07-27T03:13:07.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.552+0000 7fde21bbf640 1 -- 172.21.15.67:0/3963274029 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde1c1a61f0 con 0x7fde1c1024d0 2024-07-27T03:13:07.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.552+0000 7fde18ff9640 1 -- 172.21.15.67:0/3963274029 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c018820 con 0x7fde1c1024d0 2024-07-27T03:13:07.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.552+0000 7fde21bbf640 1 -- 172.21.15.67:0/3963274029 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde1c1a6730 con 0x7fde1c1024d0 2024-07-27T03:13:07.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.553+0000 7fde21bbf640 1 -- 172.21.15.67:0/3963274029 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde1c1028e0 con 0x7fde1c1024d0 2024-07-27T03:13:07.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.554+0000 7fde18ff9640 1 -- 172.21.15.67:0/3963274029 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7fde0c00f450 con 0x7fde1c1024d0 2024-07-27T03:13:07.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.555+0000 7fde18ff9640 1 --2- 172.21.15.67:0/3963274029 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fde040b3bd0 0x7fde040b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:07.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.555+0000 7fde1affd640 1 --2- 172.21.15.67:0/3963274029 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fde040b3bd0 0x7fde040b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:07.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.556+0000 7fde18ff9640 1 -- 172.21.15.67:0/3963274029 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fde0c0e4390 con 0x7fde1c1024d0 2024-07-27T03:13:07.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.556+0000 7fde1affd640 1 --2- 172.21.15.67:0/3963274029 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fde040b3bd0 0x7fde040b60e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fde1c107550 tx=0x7fde10005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:07.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.559+0000 7fde18ff9640 1 -- 172.21.15.67:0/3963274029 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fde0c014030 con 0x7fde1c1024d0 2024-07-27T03:13:07.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.782+0000 7fde21bbf640 1 -- 172.21.15.67:0/3963274029 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fde1c06ad00 con 0x7fde1c1024d0 2024-07-27T03:13:07.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.782+0000 7fde18ff9640 1 -- 172.21.15.67:0/3963274029 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fde0c0ab6b0 con 0x7fde1c1024d0 2024-07-27T03:13:07.785 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-07-27T03:13:07.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.784+0000 7fde21bbf640 1 -- 172.21.15.67:0/3963274029 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fde040b3bd0 msgr2=0x7fde040b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:07.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.784+0000 7fde21bbf640 1 --2- 172.21.15.67:0/3963274029 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fde040b3bd0 0x7fde040b60e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fde1c107550 tx=0x7fde10005d30 comp rx=0 tx=0).stop 2024-07-27T03:13:07.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.784+0000 7fde21bbf640 1 -- 172.21.15.67:0/3963274029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde1c1024d0 msgr2=0x7fde1c19acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:07.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.784+0000 7fde21bbf640 1 --2- 172.21.15.67:0/3963274029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde1c1024d0 0x7fde1c19acd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00c920 tx=0x7fde0c00cde0 comp rx=0 tx=0).stop 2024-07-27T03:13:07.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.784+0000 7fde21bbf640 1 -- 172.21.15.67:0/3963274029 shutdown_connections 2024-07-27T03:13:07.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.784+0000 7fde21bbf640 1 --2- 172.21.15.67:0/3963274029 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fde040b3bd0 0x7fde040b60e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:07.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.784+0000 7fde21bbf640 1 --2- 172.21.15.67:0/3963274029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde1c108760 0x7fde1c19b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:07.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.784+0000 7fde21bbf640 1 --2- 172.21.15.67:0/3963274029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde1c107e10 0x7fde1c19b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:07.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.784+0000 7fde21bbf640 1 --2- 172.21.15.67:0/3963274029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde1c1024d0 0x7fde1c19acd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:07.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.784+0000 7fde21bbf640 1 -- 172.21.15.67:0/3963274029 >> 172.21.15.67:0/3963274029 conn(0x7fde1c0791a0 msgr2=0x7fde1c104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:07.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.784+0000 7fde21bbf640 1 -- 172.21.15.67:0/3963274029 shutdown_connections 2024-07-27T03:13:07.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:07.784+0000 7fde21bbf640 1 -- 172.21.15.67:0/3963274029 wait complete. 2024-07-27T03:13:08.555 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-07-27T03:13:08.555 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-27T03:13:08.555 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-07-27T03:13:08.556 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-07-27T03:13:08.556 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-07-27T03:13:08.556 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-07-27T03:13:08.556 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-27T03:13:08.556 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-27T03:13:08.556 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-07-27T03:13:08.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:13:08.556 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-07-27T03:13:08.575 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-27T03:13:08.575 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-27T03:13:08.575 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-07-27T03:13:08.575 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-07-27T03:13:08.575 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-07-27T03:13:08.575 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-07-27T03:13:08.575 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-27T03:13:08.575 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-07-27T03:13:08.575 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-07-27T03:13:08.575 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-27T03:13:08.575 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-07-27T03:13:08.576 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-07-27T03:13:08.576 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-27T03:13:08.576 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-07-27T03:13:08.576 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-27T03:13:08.576 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-07-27T03:13:08.576 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-07-27T03:13:08.576 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 42201sec preferred_lft 42201sec 2024-07-27T03:13:08.576 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-07-27T03:13:08.576 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-07-27T03:13:08.577 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-07-27T03:13:08.577 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-27T03:13:08.577 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-07-27T03:13:08.577 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add name ceph-brx type bridge 2024-07-27T03:13:08.577 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr flush dev ceph-brx 2024-07-27T03:13:08.577 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set ceph-brx up 2024-07-27T03:13:08.577 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-07-27T03:13:08.577 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-07-27T03:13:08.668 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-27T03:13:08.778 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:08 smithi067 ceph-mon[32814]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-07-27T03:13:08.778 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:08 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3963274029' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-27T03:13:08.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-27T03:13:08.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:13:08.788 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-27T03:13:08.816 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-07-27T03:13:08.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:13:08.817 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-07-27T03:13:08.884 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-07-27T03:13:08.884 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-27T03:13:08.885 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-07-27T03:13:08.885 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-07-27T03:13:08.885 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-27T03:13:08.885 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-27T03:13:08.885 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-07-27T03:13:08.885 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-07-27T03:13:08.885 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-07-27T03:13:08.885 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-07-27T03:13:08.886 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-07-27T03:13:08.976 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-27T03:13:09.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:08 smithi175 ceph-mon[37125]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-07-27T03:13:09.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:08 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3963274029' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-27T03:13:09.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-27T03:13:09.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:13:09.171 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-07-27T03:13:09.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:13:09.187 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list-id 2024-07-27T03:13:09.244 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-27T03:13:09.244 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-07-27T03:13:09.244 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-27T03:13:09.244 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-07-27T03:13:09.244 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-07-27T03:13:09.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:08 smithi149 ceph-mon[37173]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-07-27T03:13:09.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:08 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3963274029' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-27T03:13:09.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-27T03:13:09.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-27T03:13:09.364 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-07-27T03:13:09.364 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-27T03:13:09.365 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-07-27T03:13:09.365 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-07-27T03:13:09.365 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-07-27T03:13:09.365 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-07-27T03:13:09.365 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-07-27T03:13:09.365 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-07-27T03:13:09.365 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-07-27T03:13:09.415 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-27T03:13:09.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-27T03:13:09.493 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-27T03:13:09.493 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-07-27T03:13:09.493 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-07-27T03:13:09.493 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set dev brx.0 master ceph-brx 2024-07-27T03:13:09.493 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-07-27T03:13:09.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-27T03:13:09.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-27T03:13:09.574 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-07-27T03:13:09.574 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-27T03:13:09.574 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-07-27T03:13:09.598 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-07-27T03:13:09.598 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-27T03:13:09.598 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-07-27T03:13:09.654 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-07-27T03:13:09.721 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-07-27T03:13:09.779 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-07-27T03:13:09.779 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-07-27T03:13:09.779 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-07-27T03:13:09.779 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-07-27T03:13:09.779 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-07-27T03:13:09.779 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-07-27T03:13:09.779 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-07-27T03:13:09.779 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-07-27T03:13:09.779 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-07-27T03:13:09.780 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-07-27T03:13:09.780 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-07-27T03:13:09.780 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-07-27T03:13:09.780 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-07-27T03:13:09.780 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-07-27T03:13:09.780 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-07-27T03:13:09.780 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-07-27T03:13:09.780 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-07-27T03:13:09.780 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-07-27T03:13:09.780 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-07-27T03:13:09.780 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-07-27T03:13:09.780 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-27T03:13:09.780 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-27T03:13:09.780 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-07-27T03:13:09.781 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-07-27T03:13:09.781 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-07-27T03:13:09.781 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay 2024-07-27T03:13:09.781 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/12bcfc4d12ec0a04ec4c60fcff70d14f47c466d670d89b15a9f74e828ec54b23/merged 2024-07-27T03:13:09.781 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/674bfc8892a054f27e7aa4ede34f89129086cfe49790ac9804f6ef6506431cf9/merged 2024-07-27T03:13:09.781 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/0 2024-07-27T03:13:09.781 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-07-27T03:13:09.781 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/70789b57cd5c4d6eaf30237a2972d191e48a922ceedd8c943b786379b1d13b6c/merged 2024-07-27T03:13:09.781 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/6f3c9de00f6ae917a7c76ef47dc9ab8dc6f76e61a196bccedef3ca7f4706f9d2/merged 2024-07-27T03:13:09.781 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/f40c4da81a2247311fc4f6bfcdf14ab414446f413225781ac486830336b3034b/merged 2024-07-27T03:13:09.781 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/b35eb33d96823b9b6d2b36aa3e83e1d0094e4bd16f0c99046b2cafe2d9fe2aff/merged 2024-07-27T03:13:09.781 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/66d157c7abe04527f02430d1f3b695d5509aab24f1d8a45f2266325fd19dac06/merged 2024-07-27T03:13:09.781 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/50118fe2307939a64213443e190c2f88261c9f7463467574d2e7f0d72822ade5/merged 2024-07-27T03:13:09.782 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/8bd5cabbcf787f24ed6004034e53d3d940512dc8fe0cbeeba288c16c88c87211/merged 2024-07-27T03:13:09.782 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/cb40d4687fce95ded9315bfa3a4ddf2e61b2ffc7a83a8d87e72c9faa8031cf8a/merged 2024-07-27T03:13:09.782 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-07-27T03:13:09.782 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-27T03:13:09.782 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-27T03:13:09.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:13:09.782 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-07-27T03:13:09.835 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-07-27T03:13:09.836 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-07-27T03:13:09.878 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-07-27T03:13:09.903 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-07-27T03:13:09.960 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-07-27T03:13:09.960 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-07-27T03:13:09.960 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-07-27T03:13:09.961 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-07-27T03:13:09.961 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-07-27T03:13:09.961 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-07-27T03:13:09.961 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-07-27T03:13:09.961 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-07-27T03:13:09.961 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-07-27T03:13:09.961 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-07-27T03:13:09.961 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-07-27T03:13:09.961 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-07-27T03:13:09.961 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-07-27T03:13:09.961 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-07-27T03:13:09.962 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-07-27T03:13:09.962 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-07-27T03:13:09.962 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-07-27T03:13:09.962 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-07-27T03:13:09.962 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-07-27T03:13:09.962 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-07-27T03:13:09.962 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-27T03:13:09.962 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-27T03:13:09.962 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-07-27T03:13:09.962 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-07-27T03:13:09.962 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-07-27T03:13:09.962 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay 2024-07-27T03:13:09.962 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/12bcfc4d12ec0a04ec4c60fcff70d14f47c466d670d89b15a9f74e828ec54b23/merged 2024-07-27T03:13:09.963 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/674bfc8892a054f27e7aa4ede34f89129086cfe49790ac9804f6ef6506431cf9/merged 2024-07-27T03:13:09.963 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/0 2024-07-27T03:13:09.963 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-07-27T03:13:09.963 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/70789b57cd5c4d6eaf30237a2972d191e48a922ceedd8c943b786379b1d13b6c/merged 2024-07-27T03:13:09.963 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/6f3c9de00f6ae917a7c76ef47dc9ab8dc6f76e61a196bccedef3ca7f4706f9d2/merged 2024-07-27T03:13:09.963 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/f40c4da81a2247311fc4f6bfcdf14ab414446f413225781ac486830336b3034b/merged 2024-07-27T03:13:09.963 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/b35eb33d96823b9b6d2b36aa3e83e1d0094e4bd16f0c99046b2cafe2d9fe2aff/merged 2024-07-27T03:13:09.963 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/66d157c7abe04527f02430d1f3b695d5509aab24f1d8a45f2266325fd19dac06/merged 2024-07-27T03:13:09.963 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/50118fe2307939a64213443e190c2f88261c9f7463467574d2e7f0d72822ade5/merged 2024-07-27T03:13:09.963 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/8bd5cabbcf787f24ed6004034e53d3d940512dc8fe0cbeeba288c16c88c87211/merged 2024-07-27T03:13:09.963 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/cb40d4687fce95ded9315bfa3a4ddf2e61b2ffc7a83a8d87e72c9faa8031cf8a/merged 2024-07-27T03:13:09.963 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-07-27T03:13:09.963 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-27T03:13:09.963 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-27T03:13:09.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:13:09.964 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-07-27T03:13:10.033 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:2024-07-27T03:13:10.031+0000 7fcbea614480 -1 init, newargv = 0x7fcbd80545d0 newargc=15 2024-07-27T03:13:10.034 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:ceph-fuse[82213]: starting ceph client 2024-07-27T03:13:10.040 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:ceph-fuse[82213]: starting fuse 2024-07-27T03:13:11.118 DEBUG:teuthology.orchestra.run.smithi067:> sudo modprobe fuse 2024-07-27T03:13:11.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:10 smithi175 ceph-mon[37125]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1 B/s, 0 objects/s recovering 2024-07-27T03:13:11.146 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:10 smithi067 ceph-mon[32814]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1 B/s, 0 objects/s recovering 2024-07-27T03:13:11.152 DEBUG:teuthology.orchestra.run.smithi067:> cat /proc/self/mounts | awk '{print $2}' 2024-07-27T03:13:11.212 INFO:teuthology.orchestra.run.smithi067.stdout:/proc 2024-07-27T03:13:11.213 INFO:teuthology.orchestra.run.smithi067.stdout:/sys 2024-07-27T03:13:11.213 INFO:teuthology.orchestra.run.smithi067.stdout:/dev 2024-07-27T03:13:11.213 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/security 2024-07-27T03:13:11.213 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/shm 2024-07-27T03:13:11.213 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/pts 2024-07-27T03:13:11.213 INFO:teuthology.orchestra.run.smithi067.stdout:/run 2024-07-27T03:13:11.213 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/cgroup 2024-07-27T03:13:11.213 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/pstore 2024-07-27T03:13:11.213 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/bpf 2024-07-27T03:13:11.213 INFO:teuthology.orchestra.run.smithi067.stdout:/ 2024-07-27T03:13:11.213 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/selinux 2024-07-27T03:13:11.214 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-07-27T03:13:11.214 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/mqueue 2024-07-27T03:13:11.214 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/debug 2024-07-27T03:13:11.214 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/hugepages 2024-07-27T03:13:11.214 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/tracing 2024-07-27T03:13:11.214 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/fs/fuse/connections 2024-07-27T03:13:11.214 INFO:teuthology.orchestra.run.smithi067.stdout:/sys/kernel/config 2024-07-27T03:13:11.214 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-sysctl.service 2024-07-27T03:13:11.214 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-07-27T03:13:11.214 INFO:teuthology.orchestra.run.smithi067.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-07-27T03:13:11.214 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/nfs/rpc_pipefs 2024-07-27T03:13:11.214 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/1000 2024-07-27T03:13:11.215 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/ceph 2024-07-27T03:13:11.215 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay 2024-07-27T03:13:11.215 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/12bcfc4d12ec0a04ec4c60fcff70d14f47c466d670d89b15a9f74e828ec54b23/merged 2024-07-27T03:13:11.215 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/674bfc8892a054f27e7aa4ede34f89129086cfe49790ac9804f6ef6506431cf9/merged 2024-07-27T03:13:11.215 INFO:teuthology.orchestra.run.smithi067.stdout:/run/user/0 2024-07-27T03:13:11.215 INFO:teuthology.orchestra.run.smithi067.stdout:/proc/sys/fs/binfmt_misc 2024-07-27T03:13:11.215 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/70789b57cd5c4d6eaf30237a2972d191e48a922ceedd8c943b786379b1d13b6c/merged 2024-07-27T03:13:11.215 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/6f3c9de00f6ae917a7c76ef47dc9ab8dc6f76e61a196bccedef3ca7f4706f9d2/merged 2024-07-27T03:13:11.215 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/f40c4da81a2247311fc4f6bfcdf14ab414446f413225781ac486830336b3034b/merged 2024-07-27T03:13:11.215 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/b35eb33d96823b9b6d2b36aa3e83e1d0094e4bd16f0c99046b2cafe2d9fe2aff/merged 2024-07-27T03:13:11.215 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/66d157c7abe04527f02430d1f3b695d5509aab24f1d8a45f2266325fd19dac06/merged 2024-07-27T03:13:11.215 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/50118fe2307939a64213443e190c2f88261c9f7463467574d2e7f0d72822ade5/merged 2024-07-27T03:13:11.215 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/8bd5cabbcf787f24ed6004034e53d3d940512dc8fe0cbeeba288c16c88c87211/merged 2024-07-27T03:13:11.215 INFO:teuthology.orchestra.run.smithi067.stdout:/var/lib/containers/storage/overlay/cb40d4687fce95ded9315bfa3a4ddf2e61b2ffc7a83a8d87e72c9faa8031cf8a/merged 2024-07-27T03:13:11.216 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns 2024-07-27T03:13:11.216 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-27T03:13:11.216 INFO:teuthology.orchestra.run.smithi067.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-27T03:13:11.216 INFO:teuthology.orchestra.run.smithi067.stdout:/home/ubuntu/cephtest/mnt.admin 2024-07-27T03:13:11.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:13:11.216 DEBUG:teuthology.orchestra.run.smithi067:> ls /sys/fs/fuse/connections 2024-07-27T03:13:11.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:10 smithi149 ceph-mon[37173]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1 B/s, 0 objects/s recovering 2024-07-27T03:13:11.271 INFO:teuthology.orchestra.run.smithi067.stdout:119 2024-07-27T03:13:11.272 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-07-27T03:13:11.272 DEBUG:teuthology.orchestra.run.smithi067:> sudo stdin-killer -- python3 -c ' 2024-07-27T03:13:11.272 DEBUG:teuthology.orchestra.run.smithi067:> import glob 2024-07-27T03:13:11.272 DEBUG:teuthology.orchestra.run.smithi067:> import re 2024-07-27T03:13:11.272 DEBUG:teuthology.orchestra.run.smithi067:> import os 2024-07-27T03:13:11.272 DEBUG:teuthology.orchestra.run.smithi067:> import subprocess 2024-07-27T03:13:11.272 DEBUG:teuthology.orchestra.run.smithi067:> 2024-07-27T03:13:11.272 DEBUG:teuthology.orchestra.run.smithi067:> def _find_admin_socket(client_name): 2024-07-27T03:13:11.272 DEBUG:teuthology.orchestra.run.smithi067:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-07-27T03:13:11.272 DEBUG:teuthology.orchestra.run.smithi067:> files = glob.glob(asok_path) 2024-07-27T03:13:11.272 DEBUG:teuthology.orchestra.run.smithi067:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-07-27T03:13:11.273 DEBUG:teuthology.orchestra.run.smithi067:> 2024-07-27T03:13:11.273 DEBUG:teuthology.orchestra.run.smithi067:> # Given a non-glob path, it better be there 2024-07-27T03:13:11.273 DEBUG:teuthology.orchestra.run.smithi067:> if "*" not in asok_path: 2024-07-27T03:13:11.273 DEBUG:teuthology.orchestra.run.smithi067:> assert(len(files) == 1) 2024-07-27T03:13:11.273 DEBUG:teuthology.orchestra.run.smithi067:> return files[0] 2024-07-27T03:13:11.273 DEBUG:teuthology.orchestra.run.smithi067:> 2024-07-27T03:13:11.273 DEBUG:teuthology.orchestra.run.smithi067:> for f in files: 2024-07-27T03:13:11.273 DEBUG:teuthology.orchestra.run.smithi067:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-07-27T03:13:11.273 DEBUG:teuthology.orchestra.run.smithi067:> if os.path.exists("/proc/{0}".format(pid)): 2024-07-27T03:13:11.273 DEBUG:teuthology.orchestra.run.smithi067:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-07-27T03:13:11.273 DEBUG:teuthology.orchestra.run.smithi067:> contents = proc_f.read() 2024-07-27T03:13:11.273 DEBUG:teuthology.orchestra.run.smithi067:> if mountpoint in contents: 2024-07-27T03:13:11.273 DEBUG:teuthology.orchestra.run.smithi067:> return f 2024-07-27T03:13:11.273 DEBUG:teuthology.orchestra.run.smithi067:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-07-27T03:13:11.273 DEBUG:teuthology.orchestra.run.smithi067:> 2024-07-27T03:13:11.274 DEBUG:teuthology.orchestra.run.smithi067:> print(_find_admin_socket("client.admin")) 2024-07-27T03:13:11.274 DEBUG:teuthology.orchestra.run.smithi067:> ' 2024-07-27T03:13:11.404 INFO:teuthology.orchestra.run.smithi067.stdout:/var/run/ceph/ceph-client.admin.82213.asok 2024-07-27T03:13:11.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-27T03:13:11.416 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.82213.asok 2024-07-27T03:13:11.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:13:11.416 DEBUG:teuthology.orchestra.run.smithi067:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.82213.asok status 2024-07-27T03:13:11.566 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:13:11.567 INFO:teuthology.orchestra.run.smithi067.stdout: "metadata": { 2024-07-27T03:13:11.567 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_sha1": "af8fdbf079811fe158ce62927bd145c35b74d4b8", 2024-07-27T03:13:11.567 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 19.0.0-5358-gaf8fdbf0 (af8fdbf079811fe158ce62927bd145c35b74d4b8) squid (dev)", 2024-07-27T03:13:11.567 INFO:teuthology.orchestra.run.smithi067.stdout: "entity_id": "admin", 2024-07-27T03:13:11.567 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": "smithi067", 2024-07-27T03:13:11.567 INFO:teuthology.orchestra.run.smithi067.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-07-27T03:13:11.567 INFO:teuthology.orchestra.run.smithi067.stdout: "pid": "82213", 2024-07-27T03:13:11.567 INFO:teuthology.orchestra.run.smithi067.stdout: "root": "/" 2024-07-27T03:13:11.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:13:11.567 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_count": 0, 2024-07-27T03:13:11.567 INFO:teuthology.orchestra.run.smithi067.stdout: "dentry_pinned_count": 0, 2024-07-27T03:13:11.567 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 14868, 2024-07-27T03:13:11.568 INFO:teuthology.orchestra.run.smithi067.stdout: "inst": { 2024-07-27T03:13:11.568 INFO:teuthology.orchestra.run.smithi067.stdout: "name": { 2024-07-27T03:13:11.568 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "client", 2024-07-27T03:13:11.568 INFO:teuthology.orchestra.run.smithi067.stdout: "num": 14868 2024-07-27T03:13:11.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:13:11.568 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-07-27T03:13:11.568 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-07-27T03:13:11.568 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-07-27T03:13:11.568 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 964513425 2024-07-27T03:13:11.568 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:13:11.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:13:11.568 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": { 2024-07-27T03:13:11.568 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-07-27T03:13:11.569 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "192.168.0.1:0", 2024-07-27T03:13:11.569 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 964513425 2024-07-27T03:13:11.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:13:11.569 INFO:teuthology.orchestra.run.smithi067.stdout: "inst_str": "client.14868 192.168.0.1:0/964513425", 2024-07-27T03:13:11.569 INFO:teuthology.orchestra.run.smithi067.stdout: "addr_str": "192.168.0.1:0/964513425", 2024-07-27T03:13:11.569 INFO:teuthology.orchestra.run.smithi067.stdout: "inode_count": 1, 2024-07-27T03:13:11.569 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_epoch": 17, 2024-07-27T03:13:11.569 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch": 89, 2024-07-27T03:13:11.569 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_epoch_barrier": 0, 2024-07-27T03:13:11.569 INFO:teuthology.orchestra.run.smithi067.stdout: "blocklisted": false, 2024-07-27T03:13:11.569 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs" 2024-07-27T03:13:11.569 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:13:11.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:13:11.575 DEBUG:teuthology.orchestra.run.smithi067:> stat --file-system '--printf=%T 2024-07-27T03:13:11.575 DEBUG:teuthology.orchestra.run.smithi067:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-07-27T03:13:11.592 INFO:teuthology.orchestra.run.smithi067.stdout:fuseblk 2024-07-27T03:13:11.592 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-07-27T03:13:11.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:13:11.593 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-07-27T03:13:11.660 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-07-27T03:13:11.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-27T03:13:11.773 INFO:teuthology.orchestra.run.smithi067.stdout:# file: . 2024-07-27T03:13:11.773 INFO:teuthology.orchestra.run.smithi067.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-07-27T03:13:11.773 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:13:11.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-27T03:13:11.779 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-27T03:13:11.780 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-27T03:13:11.798 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd blocklist ls 2024-07-27T03:13:12.103 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:13:12.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:12 smithi067 ceph-mon[32814]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1 B/s, 0 objects/s recovering 2024-07-27T03:13:13.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:12 smithi175 ceph-mon[37125]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1 B/s, 0 objects/s recovering 2024-07-27T03:13:13.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:12 smithi149 ceph-mon[37173]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1 B/s, 0 objects/s recovering 2024-07-27T03:13:13.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.871+0000 7fb655c67640 1 -- 172.21.15.67:0/4248201197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6501087e0 msgr2=0x7fb65010f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:13.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.871+0000 7fb655c67640 1 --2- 172.21.15.67:0/4248201197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6501087e0 0x7fb65010f010 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb648009ca0 tx=0x7fb64802a8e0 comp rx=0 tx=0).stop 2024-07-27T03:13:13.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.872+0000 7fb655c67640 1 -- 172.21.15.67:0/4248201197 shutdown_connections 2024-07-27T03:13:13.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.872+0000 7fb655c67640 1 --2- 172.21.15.67:0/4248201197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6501087e0 0x7fb65010f010 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:13.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.872+0000 7fb655c67640 1 --2- 172.21.15.67:0/4248201197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb650107e90 0x7fb6501082a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:13.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.872+0000 7fb655c67640 1 --2- 172.21.15.67:0/4248201197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb650102550 0x7fb650102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:13.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.872+0000 7fb655c67640 1 -- 172.21.15.67:0/4248201197 >> 172.21.15.67:0/4248201197 conn(0x7fb6500791e0 msgr2=0x7fb6500795f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:13.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.872+0000 7fb655c67640 1 -- 172.21.15.67:0/4248201197 shutdown_connections 2024-07-27T03:13:13.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.872+0000 7fb655c67640 1 -- 172.21.15.67:0/4248201197 wait complete. 2024-07-27T03:13:13.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.872+0000 7fb655c67640 1 Processor -- start 2024-07-27T03:13:13.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.872+0000 7fb655c67640 1 -- start start 2024-07-27T03:13:13.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.873+0000 7fb655c67640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb650102550 0x7fb650199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:13.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.873+0000 7fb655c67640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb650107e90 0x7fb65019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:13.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.873+0000 7fb655c67640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6501087e0 0x7fb65019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:13.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.873+0000 7fb655c67640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb650112560 con 0x7fb650107e90 2024-07-27T03:13:13.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.873+0000 7fb655c67640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb6501123e0 con 0x7fb6501087e0 2024-07-27T03:13:13.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.873+0000 7fb655c67640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb6501126e0 con 0x7fb650102550 2024-07-27T03:13:13.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.873+0000 7fb64effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb650107e90 0x7fb65019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:13.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.873+0000 7fb64effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb650107e90 0x7fb65019a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54142/0 (socket says 172.21.15.67:54142) 2024-07-27T03:13:13.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.873+0000 7fb64effd640 1 -- 172.21.15.67:0/3362296938 learned_addr learned my addr 172.21.15.67:0/3362296938 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:13:13.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.873+0000 7fb64f7fe640 1 --2- 172.21.15.67:0/3362296938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb650102550 0x7fb650199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:13.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.873+0000 7fb64ffff640 1 --2- 172.21.15.67:0/3362296938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6501087e0 0x7fb65019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:13.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.873+0000 7fb64effd640 1 -- 172.21.15.67:0/3362296938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb650102550 msgr2=0x7fb650199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:13.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.873+0000 7fb64effd640 1 --2- 172.21.15.67:0/3362296938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb650102550 0x7fb650199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:13.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.873+0000 7fb64effd640 1 -- 172.21.15.67:0/3362296938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6501087e0 msgr2=0x7fb65019f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:13.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.873+0000 7fb64effd640 1 --2- 172.21.15.67:0/3362296938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6501087e0 0x7fb65019f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:13.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.873+0000 7fb64effd640 1 -- 172.21.15.67:0/3362296938 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65019ff50 con 0x7fb650107e90 2024-07-27T03:13:13.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.874+0000 7fb64effd640 1 --2- 172.21.15.67:0/3362296938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb650107e90 0x7fb65019a580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb644006fd0 tx=0x7fb64400b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:13.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.874+0000 7fb64cff9640 1 -- 172.21.15.67:0/3362296938 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644015020 con 0x7fb650107e90 2024-07-27T03:13:13.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.874+0000 7fb64cff9640 1 -- 172.21.15.67:0/3362296938 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb644011ce0 con 0x7fb650107e90 2024-07-27T03:13:13.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.874+0000 7fb655c67640 1 -- 172.21.15.67:0/3362296938 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6501a0240 con 0x7fb650107e90 2024-07-27T03:13:13.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.874+0000 7fb64cff9640 1 -- 172.21.15.67:0/3362296938 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644010d90 con 0x7fb650107e90 2024-07-27T03:13:13.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.874+0000 7fb655c67640 1 -- 172.21.15.67:0/3362296938 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6501a6a20 con 0x7fb650107e90 2024-07-27T03:13:13.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.875+0000 7fb655c67640 1 -- 172.21.15.67:0/3362296938 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6501037d0 con 0x7fb650107e90 2024-07-27T03:13:13.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.877+0000 7fb64cff9640 1 -- 172.21.15.67:0/3362296938 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7fb644018020 con 0x7fb650107e90 2024-07-27T03:13:13.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.877+0000 7fb64cff9640 1 --2- 172.21.15.67:0/3362296938 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb6380b3c50 0x7fb6380b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:13.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.877+0000 7fb64cff9640 1 -- 172.21.15.67:0/3362296938 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fb6440e3cb0 con 0x7fb650107e90 2024-07-27T03:13:13.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.877+0000 7fb64f7fe640 1 --2- 172.21.15.67:0/3362296938 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb6380b3c50 0x7fb6380b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:13.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.878+0000 7fb64f7fe640 1 --2- 172.21.15.67:0/3362296938 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb6380b3c50 0x7fb6380b6160 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb640009990 tx=0x7fb640009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:13.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:13.880+0000 7fb64cff9640 1 -- 172.21.15.67:0/3362296938 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fb64401f020 con 0x7fb650107e90 2024-07-27T03:13:14.079 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:14.077+0000 7fb655c67640 1 -- 172.21.15.67:0/3362296938 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb650102960 con 0x7fb650107e90 2024-07-27T03:13:14.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:14.077+0000 7fb64cff9640 1 -- 172.21.15.67:0/3362296938 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+590 (secure 0 0 0) 0x7fb650102960 con 0x7fb650107e90 2024-07-27T03:13:14.080 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3477601235 2024-07-28T03:04:54.947985+0000 2024-07-27T03:13:14.081 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/4130679770 2024-07-28T03:04:30.600273+0000 2024-07-27T03:13:14.081 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/1921856974 2024-07-28T03:04:54.947985+0000 2024-07-27T03:13:14.081 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/1029332005 2024-07-28T03:04:54.947985+0000 2024-07-27T03:13:14.081 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/2346308006 2024-07-28T03:04:30.600273+0000 2024-07-27T03:13:14.081 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/1921856974 2024-07-28T03:04:54.947985+0000 2024-07-27T03:13:14.081 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/284078926 2024-07-28T03:04:54.947985+0000 2024-07-27T03:13:14.081 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/2346308006 2024-07-28T03:04:30.600273+0000 2024-07-27T03:13:14.081 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2559121542 2024-07-28T03:04:30.600273+0000 2024-07-27T03:13:14.081 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/925406021 2024-07-28T03:04:30.600273+0000 2024-07-27T03:13:14.081 INFO:teuthology.orchestra.run.smithi067.stderr:listed 10 entries 2024-07-27T03:13:14.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:14.080+0000 7fb655c67640 1 -- 172.21.15.67:0/3362296938 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb6380b3c50 msgr2=0x7fb6380b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:14.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:14.080+0000 7fb655c67640 1 --2- 172.21.15.67:0/3362296938 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb6380b3c50 0x7fb6380b6160 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb640009990 tx=0x7fb640009110 comp rx=0 tx=0).stop 2024-07-27T03:13:14.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:14.080+0000 7fb655c67640 1 -- 172.21.15.67:0/3362296938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb650107e90 msgr2=0x7fb65019a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:14.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:14.080+0000 7fb655c67640 1 --2- 172.21.15.67:0/3362296938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb650107e90 0x7fb65019a580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb644006fd0 tx=0x7fb64400b820 comp rx=0 tx=0).stop 2024-07-27T03:13:14.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:14.080+0000 7fb655c67640 1 -- 172.21.15.67:0/3362296938 shutdown_connections 2024-07-27T03:13:14.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:14.080+0000 7fb655c67640 1 --2- 172.21.15.67:0/3362296938 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb6380b3c50 0x7fb6380b6160 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:14.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:14.080+0000 7fb655c67640 1 --2- 172.21.15.67:0/3362296938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6501087e0 0x7fb65019f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:14.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:14.080+0000 7fb655c67640 1 --2- 172.21.15.67:0/3362296938 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb650107e90 0x7fb65019a580 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:14.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:14.080+0000 7fb655c67640 1 --2- 172.21.15.67:0/3362296938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb650102550 0x7fb650199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:14.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:14.080+0000 7fb655c67640 1 -- 172.21.15.67:0/3362296938 >> 172.21.15.67:0/3362296938 conn(0x7fb6500791e0 msgr2=0x7fb6501014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:14.083 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:14.080+0000 7fb655c67640 1 -- 172.21.15.67:0/3362296938 shutdown_connections 2024-07-27T03:13:14.084 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:14.080+0000 7fb655c67640 1 -- 172.21.15.67:0/3362296938 wait complete. 2024-07-27T03:13:14.746 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-27T03:13:14.746 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-27T03:13:14.763 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd blocklist ls 2024-07-27T03:13:14.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:14 smithi067 ceph-mon[32814]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:13:14.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:14 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3362296938' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-27T03:13:15.071 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:13:15.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:14 smithi175 ceph-mon[37125]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:13:15.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:14 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3362296938' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-27T03:13:15.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:14 smithi149 ceph-mon[37173]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:13:15.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:14 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3362296938' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-27T03:13:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.678+0000 7fde8e3af640 1 -- 172.21.15.67:0/3628073491 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde8810d8e0 msgr2=0x7fde8810fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.678+0000 7fde8e3af640 1 --2- 172.21.15.67:0/3628073491 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde8810d8e0 0x7fde8810fdb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fde80009ca0 tx=0x7fde8002a8c0 comp rx=0 tx=0).stop 2024-07-27T03:13:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.679+0000 7fde8e3af640 1 -- 172.21.15.67:0/3628073491 shutdown_connections 2024-07-27T03:13:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.679+0000 7fde8e3af640 1 --2- 172.21.15.67:0/3628073491 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde8810d8e0 0x7fde8810fdb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.679+0000 7fde8e3af640 1 --2- 172.21.15.67:0/3628073491 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde88104d10 0x7fde8810d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.679+0000 7fde8e3af640 1 --2- 172.21.15.67:0/3628073491 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde88104330 0x7fde88104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.679+0000 7fde8e3af640 1 -- 172.21.15.67:0/3628073491 >> 172.21.15.67:0/3628073491 conn(0x7fde880791d0 msgr2=0x7fde880795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.679+0000 7fde8e3af640 1 -- 172.21.15.67:0/3628073491 shutdown_connections 2024-07-27T03:13:16.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.679+0000 7fde8e3af640 1 -- 172.21.15.67:0/3628073491 wait complete. 2024-07-27T03:13:16.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.679+0000 7fde8e3af640 1 Processor -- start 2024-07-27T03:13:16.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.679+0000 7fde8e3af640 1 -- start start 2024-07-27T03:13:16.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.680+0000 7fde8e3af640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde88104330 0x7fde88199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:16.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.680+0000 7fde8e3af640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde88104d10 0x7fde8819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:16.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.680+0000 7fde8e3af640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde8810d8e0 0x7fde8819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:16.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.680+0000 7fde8e3af640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fde88113540 con 0x7fde88104d10 2024-07-27T03:13:16.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.680+0000 7fde8e3af640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fde881133c0 con 0x7fde8810d8e0 2024-07-27T03:13:16.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.680+0000 7fde8e3af640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fde881136c0 con 0x7fde88104330 2024-07-27T03:13:16.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.680+0000 7fde7ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde88104d10 0x7fde8819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:16.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.680+0000 7fde7ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde88104d10 0x7fde8819a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45384/0 (socket says 172.21.15.67:45384) 2024-07-27T03:13:16.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.680+0000 7fde7ffff640 1 -- 172.21.15.67:0/2248523007 learned_addr learned my addr 172.21.15.67:0/2248523007 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:13:16.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.680+0000 7fde87fff640 1 --2- 172.21.15.67:0/2248523007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde88104330 0x7fde88199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:16.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.680+0000 7fde8c94c640 1 --2- 172.21.15.67:0/2248523007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde8810d8e0 0x7fde8819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:16.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.680+0000 7fde7ffff640 1 -- 172.21.15.67:0/2248523007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde88104330 msgr2=0x7fde88199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:16.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.680+0000 7fde7ffff640 1 --2- 172.21.15.67:0/2248523007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde88104330 0x7fde88199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:16.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.680+0000 7fde7ffff640 1 -- 172.21.15.67:0/2248523007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde8810d8e0 msgr2=0x7fde8819f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:16.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.680+0000 7fde7ffff640 1 --2- 172.21.15.67:0/2248523007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde8810d8e0 0x7fde8819f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:16.685 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.680+0000 7fde7ffff640 1 -- 172.21.15.67:0/2248523007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8819fd40 con 0x7fde88104d10 2024-07-27T03:13:16.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.681+0000 7fde7ffff640 1 --2- 172.21.15.67:0/2248523007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde88104d10 0x7fde8819a430 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fde7000c8c0 tx=0x7fde7000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:16.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.681+0000 7fde85ffb640 1 -- 172.21.15.67:0/2248523007 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde70007bb0 con 0x7fde88104d10 2024-07-27T03:13:16.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.681+0000 7fde8e3af640 1 -- 172.21.15.67:0/2248523007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde881a0030 con 0x7fde88104d10 2024-07-27T03:13:16.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.681+0000 7fde85ffb640 1 -- 172.21.15.67:0/2248523007 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde70007d50 con 0x7fde88104d10 2024-07-27T03:13:16.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.681+0000 7fde85ffb640 1 -- 172.21.15.67:0/2248523007 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde700187c0 con 0x7fde88104d10 2024-07-27T03:13:16.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.681+0000 7fde8e3af640 1 -- 172.21.15.67:0/2248523007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde881a6850 con 0x7fde88104d10 2024-07-27T03:13:16.686 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.682+0000 7fde8e3af640 1 -- 172.21.15.67:0/2248523007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde8806bc00 con 0x7fde88104d10 2024-07-27T03:13:16.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.684+0000 7fde85ffb640 1 -- 172.21.15.67:0/2248523007 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7fde7000fa20 con 0x7fde88104d10 2024-07-27T03:13:16.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.684+0000 7fde85ffb640 1 --2- 172.21.15.67:0/2248523007 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fde740b3c00 0x7fde740b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:16.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.684+0000 7fde85ffb640 1 -- 172.21.15.67:0/2248523007 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fde700e42a0 con 0x7fde88104d10 2024-07-27T03:13:16.687 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.684+0000 7fde87fff640 1 --2- 172.21.15.67:0/2248523007 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fde740b3c00 0x7fde740b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:16.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.685+0000 7fde87fff640 1 --2- 172.21.15.67:0/2248523007 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fde740b3c00 0x7fde740b6110 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fde780058c0 tx=0x7fde7800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:16.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.688+0000 7fde85ffb640 1 -- 172.21.15.67:0/2248523007 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fde70014030 con 0x7fde88104d10 2024-07-27T03:13:16.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.884+0000 7fde8e3af640 1 -- 172.21.15.67:0/2248523007 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fde88104740 con 0x7fde88104d10 2024-07-27T03:13:16.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.884+0000 7fde85ffb640 1 -- 172.21.15.67:0/2248523007 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+590 (secure 0 0 0) 0x7fde700ab620 con 0x7fde88104d10 2024-07-27T03:13:16.887 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3477601235 2024-07-28T03:04:54.947985+0000 2024-07-27T03:13:16.887 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/4130679770 2024-07-28T03:04:30.600273+0000 2024-07-27T03:13:16.887 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/1921856974 2024-07-28T03:04:54.947985+0000 2024-07-27T03:13:16.887 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/1029332005 2024-07-28T03:04:54.947985+0000 2024-07-27T03:13:16.887 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/2346308006 2024-07-28T03:04:30.600273+0000 2024-07-27T03:13:16.887 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/1921856974 2024-07-28T03:04:54.947985+0000 2024-07-27T03:13:16.888 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/284078926 2024-07-28T03:04:54.947985+0000 2024-07-27T03:13:16.888 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/2346308006 2024-07-28T03:04:30.600273+0000 2024-07-27T03:13:16.888 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2559121542 2024-07-28T03:04:30.600273+0000 2024-07-27T03:13:16.888 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/925406021 2024-07-28T03:04:30.600273+0000 2024-07-27T03:13:16.888 INFO:teuthology.orchestra.run.smithi067.stderr:listed 10 entries 2024-07-27T03:13:16.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.886+0000 7fde8e3af640 1 -- 172.21.15.67:0/2248523007 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fde740b3c00 msgr2=0x7fde740b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:16.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.886+0000 7fde8e3af640 1 --2- 172.21.15.67:0/2248523007 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fde740b3c00 0x7fde740b6110 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fde780058c0 tx=0x7fde7800a2d0 comp rx=0 tx=0).stop 2024-07-27T03:13:16.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.886+0000 7fde8e3af640 1 -- 172.21.15.67:0/2248523007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde88104d10 msgr2=0x7fde8819a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:16.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.886+0000 7fde8e3af640 1 --2- 172.21.15.67:0/2248523007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde88104d10 0x7fde8819a430 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fde7000c8c0 tx=0x7fde7000cd80 comp rx=0 tx=0).stop 2024-07-27T03:13:16.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.886+0000 7fde8e3af640 1 -- 172.21.15.67:0/2248523007 shutdown_connections 2024-07-27T03:13:16.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.886+0000 7fde8e3af640 1 --2- 172.21.15.67:0/2248523007 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fde740b3c00 0x7fde740b6110 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:16.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.886+0000 7fde8e3af640 1 --2- 172.21.15.67:0/2248523007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde8810d8e0 0x7fde8819f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:16.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.886+0000 7fde8e3af640 1 --2- 172.21.15.67:0/2248523007 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fde88104d10 0x7fde8819a430 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:16.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.886+0000 7fde8e3af640 1 --2- 172.21.15.67:0/2248523007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde88104330 0x7fde88199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:16.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.886+0000 7fde8e3af640 1 -- 172.21.15.67:0/2248523007 >> 172.21.15.67:0/2248523007 conn(0x7fde880791d0 msgr2=0x7fde88100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:16.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.887+0000 7fde8e3af640 1 -- 172.21.15.67:0/2248523007 shutdown_connections 2024-07-27T03:13:16.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:16.887+0000 7fde8e3af640 1 -- 172.21.15.67:0/2248523007 wait complete. 2024-07-27T03:13:17.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:16 smithi175 ceph-mon[37125]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-27T03:13:17.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:16 smithi067 ceph-mon[32814]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-27T03:13:17.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:16 smithi149 ceph-mon[37173]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-27T03:13:17.829 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi067.front.sepia.ceph.com... 2024-07-27T03:13:17.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:13:17.829 DEBUG:teuthology.orchestra.run.smithi067:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-07-27T03:13:17.859 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi067.stderr:ceph-fuse[82213]: fuse finished with error 0 and tester_r 0 2024-07-27T03:13:17.861 INFO:teuthology.orchestra.run:waiting for 300 2024-07-27T03:13:18.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:17 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2248523007' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-27T03:13:18.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:17 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2248523007' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-27T03:13:18.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:17 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2248523007' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-27T03:13:19.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:18 smithi175 ceph-mon[37125]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-27T03:13:19.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:18 smithi067 ceph-mon[32814]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-27T03:13:19.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:18 smithi149 ceph-mon[37173]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-27T03:13:21.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:20 smithi175 ceph-mon[37125]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-27T03:13:21.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:20 smithi067 ceph-mon[32814]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-27T03:13:21.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:20 smithi149 ceph-mon[37173]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-07-27T03:13:23.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:22 smithi175 ceph-mon[37125]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-07-27T03:13:23.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:22 smithi067 ceph-mon[32814]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-07-27T03:13:23.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:22 smithi149 ceph-mon[37173]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-07-27T03:13:23.963 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-07-27T03:13:23.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:13:23.964 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-07-27T03:13:23.994 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd dump --format=json 2024-07-27T03:13:24.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:23 smithi175 ceph-mon[37125]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-07-27T03:13:24.132 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:23 smithi067 ceph-mon[32814]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-07-27T03:13:24.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:23 smithi149 ceph-mon[37173]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-07-27T03:13:24.321 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:13:26.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.030+0000 7f06020e4640 1 -- 172.21.15.67:0/884406592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05fc109860 msgr2=0x7f05fc110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:26.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.030+0000 7f06020e4640 1 --2- 172.21.15.67:0/884406592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05fc109860 0x7f05fc110090 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009900 tx=0x7f05f402a410 comp rx=0 tx=0).stop 2024-07-27T03:13:26.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.030+0000 7f06020e4640 1 -- 172.21.15.67:0/884406592 shutdown_connections 2024-07-27T03:13:26.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.030+0000 7f06020e4640 1 --2- 172.21.15.67:0/884406592 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05fc109860 0x7f05fc110090 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:26.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.030+0000 7f06020e4640 1 --2- 172.21.15.67:0/884406592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05fc108f10 0x7f05fc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:26.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.030+0000 7f06020e4640 1 --2- 172.21.15.67:0/884406592 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05fc1035d0 0x7f05fc1039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:26.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.030+0000 7f06020e4640 1 -- 172.21.15.67:0/884406592 >> 172.21.15.67:0/884406592 conn(0x7f05fc07b2c0 msgr2=0x7f05fc07b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:26.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.030+0000 7f06020e4640 1 -- 172.21.15.67:0/884406592 shutdown_connections 2024-07-27T03:13:26.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.030+0000 7f06020e4640 1 -- 172.21.15.67:0/884406592 wait complete. 2024-07-27T03:13:26.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.031+0000 7f06020e4640 1 Processor -- start 2024-07-27T03:13:26.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.031+0000 7f06020e4640 1 -- start start 2024-07-27T03:13:26.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.031+0000 7f06020e4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05fc1035d0 0x7f05fc199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:26.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.031+0000 7f06020e4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05fc108f10 0x7f05fc19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:26.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.031+0000 7f06020e4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05fc109860 0x7f05fc19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:26.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.031+0000 7f06020e4640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc113510 con 0x7f05fc1035d0 2024-07-27T03:13:26.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.031+0000 7f06020e4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc113390 con 0x7f05fc109860 2024-07-27T03:13:26.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.031+0000 7f06020e4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc113690 con 0x7f05fc108f10 2024-07-27T03:13:26.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.031+0000 7f05fb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05fc1035d0 0x7f05fc199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:26.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.031+0000 7f05fb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05fc1035d0 0x7f05fc199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52606/0 (socket says 172.21.15.67:52606) 2024-07-27T03:13:26.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.031+0000 7f05fb7fe640 1 -- 172.21.15.67:0/689550109 learned_addr learned my addr 172.21.15.67:0/689550109 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:13:26.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.032+0000 7f05faffd640 1 --2- 172.21.15.67:0/689550109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05fc108f10 0x7f05fc19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:26.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.032+0000 7f05fbfff640 1 --2- 172.21.15.67:0/689550109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05fc109860 0x7f05fc19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:26.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.032+0000 7f05fb7fe640 1 -- 172.21.15.67:0/689550109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05fc108f10 msgr2=0x7f05fc19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:26.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.032+0000 7f05fb7fe640 1 --2- 172.21.15.67:0/689550109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05fc108f10 0x7f05fc19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:26.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.032+0000 7f05fb7fe640 1 -- 172.21.15.67:0/689550109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05fc109860 msgr2=0x7f05fc19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:26.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.032+0000 7f05fb7fe640 1 --2- 172.21.15.67:0/689550109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05fc109860 0x7f05fc19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:26.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.032+0000 7f05fb7fe640 1 -- 172.21.15.67:0/689550109 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05fc19fe30 con 0x7f05fc1035d0 2024-07-27T03:13:26.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.032+0000 7f05fb7fe640 1 --2- 172.21.15.67:0/689550109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05fc1035d0 0x7f05fc199ec0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00db80 tx=0x7f05ec00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:26.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.032+0000 7f05f8ff9640 1 -- 172.21.15.67:0/689550109 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec017070 con 0x7f05fc1035d0 2024-07-27T03:13:26.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.032+0000 7f05f8ff9640 1 -- 172.21.15.67:0/689550109 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05ec003b70 con 0x7f05fc1035d0 2024-07-27T03:13:26.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.032+0000 7f06020e4640 1 -- 172.21.15.67:0/689550109 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05fc1a0120 con 0x7f05fc1035d0 2024-07-27T03:13:26.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.032+0000 7f05f8ff9640 1 -- 172.21.15.67:0/689550109 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec0106c0 con 0x7f05fc1035d0 2024-07-27T03:13:26.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.032+0000 7f06020e4640 1 -- 172.21.15.67:0/689550109 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05fc1a6880 con 0x7f05fc1035d0 2024-07-27T03:13:26.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.033+0000 7f06020e4640 1 -- 172.21.15.67:0/689550109 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05fc1039e0 con 0x7f05fc1035d0 2024-07-27T03:13:26.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.035+0000 7f05f8ff9640 1 -- 172.21.15.67:0/689550109 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f05ec004020 con 0x7f05fc1035d0 2024-07-27T03:13:26.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.036+0000 7f05f8ff9640 1 --2- 172.21.15.67:0/689550109 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f05e40b3bd0 0x7f05e40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:26.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.036+0000 7f05faffd640 1 --2- 172.21.15.67:0/689550109 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f05e40b3bd0 0x7f05e40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:26.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.036+0000 7f05f8ff9640 1 -- 172.21.15.67:0/689550109 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f05ec0e42e0 con 0x7f05fc1035d0 2024-07-27T03:13:26.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.037+0000 7f05faffd640 1 --2- 172.21.15.67:0/689550109 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f05e40b3bd0 0x7f05e40b60e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f05fc108ab0 tx=0x7f05f000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:26.043 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.039+0000 7f05f8ff9640 1 -- 172.21.15.67:0/689550109 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f05ec0ab5d0 con 0x7f05fc1035d0 2024-07-27T03:13:26.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.227+0000 7f06020e4640 1 -- 172.21.15.67:0/689550109 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05fc02e100 con 0x7f05fc1035d0 2024-07-27T03:13:26.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.228+0000 7f05f8ff9640 1 -- 172.21.15.67:0/689550109 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24690 (secure 0 0 0) 0x7f05ec0aaed0 con 0x7f05fc1035d0 2024-07-27T03:13:26.232 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:13:26.232 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"c0782a00-4bc4-11ef-bcc7-c7b262605968","created":"2024-07-27T03:04:08.096824+0000","modified":"2024-07-27T03:12:56.708905+0000","last_up_change":"2024-07-27T03:11:33.314800+0000","last_in_change":"2024-07-27T03:11:14.028095+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-27T03:08:01.042479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-27T03:12:27.565948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-27T03:12:31.353726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-27T03:12:40.599486+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9f614272-18e8-4562-812c-4f43e40efd2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6803","nonce":2648129674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6805","nonce":2648129674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6809","nonce":2648129674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6807","nonce":2648129674}]},"public_addr":"172.21.15.67:6803/2648129674","cluster_addr":"172.21.15.67:6805/2648129674","heartbeat_back_addr":"172.21.15.67:6809/2648129674","heartbeat_front_addr":"172.21.15.67:6807/2648129674","state":["exists","up"]},{"osd":1,"uuid":"f59f5879-163a-4269-bbe7-1200eea65341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6801","nonce":2561753152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6803","nonce":2561753152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6807","nonce":2561753152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6805","nonce":2561753152}]},"public_addr":"172.21.15.149:6801/2561753152","cluster_addr":"172.21.15.149:6803/2561753152","heartbeat_back_addr":"172.21.15.149:6807/2561753152","heartbeat_front_addr":"172.21.15.149:6805/2561753152","state":["exists","up"]},{"osd":2,"uuid":"444e0ebe-7ca3-4be9-8bd5-db859f824445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6801","nonce":3770940078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6803","nonce":3770940078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6807","nonce":3770940078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6805","nonce":3770940078}]},"public_addr":"172.21.15.175:6801/3770940078","cluster_addr":"172.21.15.175:6803/3770940078","heartbeat_back_addr":"172.21.15.175:6807/3770940078","heartbeat_front_addr":"172.21.15.175:6805/3770940078","state":["exists","up"]},{"osd":3,"uuid":"d554db3d-9604-427c-8736-b1bb379f8d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6811","nonce":3159144693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6813","nonce":3159144693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6817","nonce":3159144693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6815","nonce":3159144693}]},"public_addr":"172.21.15.67:6811/3159144693","cluster_addr":"172.21.15.67:6813/3159144693","heartbeat_back_addr":"172.21.15.67:6817/3159144693","heartbeat_front_addr":"172.21.15.67:6815/3159144693","state":["exists","up"]},{"osd":4,"uuid":"c317dbe5-3b49-47a6-8408-7042a3dac345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3393503532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3393503532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3393503532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3393503532}]},"public_addr":"172.21.15.149:6809/3393503532","cluster_addr":"172.21.15.149:6811/3393503532","heartbeat_back_addr":"172.21.15.149:6815/3393503532","heartbeat_front_addr":"172.21.15.149:6813/3393503532","state":["exists","up"]},{"osd":5,"uuid":"5c85abd6-2b42-4957-a9b7-59a9ee36b841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6809","nonce":2291204784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6811","nonce":2291204784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6815","nonce":2291204784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6813","nonce":2291204784}]},"public_addr":"172.21.15.175:6809/2291204784","cluster_addr":"172.21.15.175:6811/2291204784","heartbeat_back_addr":"172.21.15.175:6815/2291204784","heartbeat_front_addr":"172.21.15.175:6813/2291204784","state":["exists","up"]},{"osd":6,"uuid":"bf6c277d-ec24-4470-98de-cf6f28876ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6819","nonce":2660182879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6821","nonce":2660182879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6825","nonce":2660182879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6823","nonce":2660182879}]},"public_addr":"172.21.15.67:6819/2660182879","cluster_addr":"172.21.15.67:6821/2660182879","heartbeat_back_addr":"172.21.15.67:6825/2660182879","heartbeat_front_addr":"172.21.15.67:6823/2660182879","state":["exists","up"]},{"osd":7,"uuid":"fea2c7cf-d5cb-45ae-9b29-f6e56c903891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6817","nonce":2289624404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6819","nonce":2289624404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6823","nonce":2289624404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6821","nonce":2289624404}]},"public_addr":"172.21.15.149:6817/2289624404","cluster_addr":"172.21.15.149:6819/2289624404","heartbeat_back_addr":"172.21.15.149:6823/2289624404","heartbeat_front_addr":"172.21.15.149:6821/2289624404","state":["exists","up"]},{"osd":8,"uuid":"14d04d96-579a-4dc9-a4d4-50e4850b48e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6817","nonce":2286032466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6819","nonce":2286032466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6823","nonce":2286032466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6821","nonce":2286032466}]},"public_addr":"172.21.15.175:6817/2286032466","cluster_addr":"172.21.15.175:6819/2286032466","heartbeat_back_addr":"172.21.15.175:6823/2286032466","heartbeat_front_addr":"172.21.15.175:6821/2286032466","state":["exists","up"]},{"osd":9,"uuid":"b65dfb6b-1efe-4314-a8be-ce6a295e99fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6827","nonce":3411081513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6829","nonce":3411081513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6833","nonce":3411081513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6831","nonce":3411081513}]},"public_addr":"172.21.15.67:6827/3411081513","cluster_addr":"172.21.15.67:6829/3411081513","heartbeat_back_addr":"172.21.15.67:6833/3411081513","heartbeat_front_addr":"172.21.15.67:6831/3411081513","state":["exists","up"]},{"osd":10,"uuid":"e3dabae0-0d64-4222-984c-fc1cce883456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6825","nonce":2011495212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6827","nonce":2011495212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6831","nonce":2011495212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6829","nonce":2011495212}]},"public_addr":"172.21.15.149:6825/2011495212","cluster_addr":"172.21.15.149:6827/2011495212","heartbeat_back_addr":"172.21.15.149:6831/2011495212","heartbeat_front_addr":"172.21.15.149:6829/2011495212","state":["exists","up"]},{"osd":11,"uuid":"406c082c-9b5f-4781-8cbf-3c04fa49e410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6825","nonce":2565514213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6827","nonce":2565514213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6831","nonce":2565514213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6829","nonce":2565514213}]},"public_addr":"172.21.15.175:6825/2565514213","cluster_addr":"172.21.15.175:6827/2565514213","heartbeat_back_addr":"172.21.15.175:6831/2565514213","heartbeat_front_addr":"172.21.15.175:6829/2565514213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:11.399380+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:33.728518+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:56.411181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:19.871731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:42.994576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:06.712815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:31.321967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:55.070898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:18.686108+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:43.187795+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:07.318538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:31.374843+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/3477601235":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/4130679770":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6800/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/1029332005":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6801/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6801/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/284078926":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6800/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/2559121542":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/925406021":"2024-07-28T03:04:30.600273+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-27T03:13:26.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.230+0000 7f06020e4640 1 -- 172.21.15.67:0/689550109 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f05e40b3bd0 msgr2=0x7f05e40b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:26.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.230+0000 7f06020e4640 1 --2- 172.21.15.67:0/689550109 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f05e40b3bd0 0x7f05e40b60e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f05fc108ab0 tx=0x7f05f000b390 comp rx=0 tx=0).stop 2024-07-27T03:13:26.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.230+0000 7f06020e4640 1 -- 172.21.15.67:0/689550109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05fc1035d0 msgr2=0x7f05fc199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:26.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.230+0000 7f06020e4640 1 --2- 172.21.15.67:0/689550109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05fc1035d0 0x7f05fc199ec0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00db80 tx=0x7f05ec00b630 comp rx=0 tx=0).stop 2024-07-27T03:13:26.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.230+0000 7f06020e4640 1 -- 172.21.15.67:0/689550109 shutdown_connections 2024-07-27T03:13:26.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.230+0000 7f06020e4640 1 --2- 172.21.15.67:0/689550109 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f05e40b3bd0 0x7f05e40b60e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:26.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.231+0000 7f06020e4640 1 --2- 172.21.15.67:0/689550109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05fc109860 0x7f05fc19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:26.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.231+0000 7f06020e4640 1 --2- 172.21.15.67:0/689550109 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05fc108f10 0x7f05fc19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:26.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.231+0000 7f06020e4640 1 --2- 172.21.15.67:0/689550109 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05fc1035d0 0x7f05fc199ec0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:26.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.231+0000 7f06020e4640 1 -- 172.21.15.67:0/689550109 >> 172.21.15.67:0/689550109 conn(0x7f05fc07b2c0 msgr2=0x7f05fc106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:26.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.231+0000 7f06020e4640 1 -- 172.21.15.67:0/689550109 shutdown_connections 2024-07-27T03:13:26.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:26.231+0000 7f06020e4640 1 -- 172.21.15.67:0/689550109 wait complete. 2024-07-27T03:13:26.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:26 smithi175 ceph-mon[37125]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-07-27T03:13:26.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:26 smithi067 ceph-mon[32814]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-07-27T03:13:26.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:26 smithi149 ceph-mon[37173]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-07-27T03:13:26.804 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd dump --format=json 2024-07-27T03:13:27.076 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:13:27.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:27 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/689550109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:13:27.455 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:27 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/689550109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:13:27.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:27 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/689550109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:13:28.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:28 smithi067 ceph-mon[32814]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-27T03:13:28.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:28 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:13:28.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:28 smithi175 ceph-mon[37125]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-27T03:13:28.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:28 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:13:28.401 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:28 smithi149 ceph-mon[37173]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-27T03:13:28.402 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:28 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:13:28.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.805+0000 7f6787bf9640 1 -- 172.21.15.67:0/4168640609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6780109860 msgr2=0x7f6780110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:28.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.805+0000 7f6787bf9640 1 --2- 172.21.15.67:0/4168640609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6780109860 0x7f6780110090 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6778009ca0 tx=0x7f677802a8c0 comp rx=0 tx=0).stop 2024-07-27T03:13:28.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.805+0000 7f6787bf9640 1 -- 172.21.15.67:0/4168640609 shutdown_connections 2024-07-27T03:13:28.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.805+0000 7f6787bf9640 1 --2- 172.21.15.67:0/4168640609 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6780109860 0x7f6780110090 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:28.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.805+0000 7f6787bf9640 1 --2- 172.21.15.67:0/4168640609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6780108f10 0x7f6780109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:28.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.805+0000 7f6787bf9640 1 --2- 172.21.15.67:0/4168640609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67801035d0 0x7f67801039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:28.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.805+0000 7f6787bf9640 1 -- 172.21.15.67:0/4168640609 >> 172.21.15.67:0/4168640609 conn(0x7f678007b2c0 msgr2=0x7f678007b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:28.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.806+0000 7f6787bf9640 1 -- 172.21.15.67:0/4168640609 shutdown_connections 2024-07-27T03:13:28.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.806+0000 7f6787bf9640 1 -- 172.21.15.67:0/4168640609 wait complete. 2024-07-27T03:13:28.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.806+0000 7f6787bf9640 1 Processor -- start 2024-07-27T03:13:28.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.807+0000 7f6787bf9640 1 -- start start 2024-07-27T03:13:28.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.807+0000 7f6787bf9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67801035d0 0x7f6780199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:28.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.807+0000 7f6787bf9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6780108f10 0x7f678019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:28.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.807+0000 7f6787bf9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6780109860 0x7f678019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:28.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.807+0000 7f6787bf9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f6780113530 con 0x7f6780108f10 2024-07-27T03:13:28.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.807+0000 7f6787bf9640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f67801133b0 con 0x7f67801035d0 2024-07-27T03:13:28.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.807+0000 7f6787bf9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f67801136b0 con 0x7f6780109860 2024-07-27T03:13:28.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.807+0000 7f6785194640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6780108f10 0x7f678019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:28.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.807+0000 7f6785194640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6780108f10 0x7f678019a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52626/0 (socket says 172.21.15.67:52626) 2024-07-27T03:13:28.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.807+0000 7f6785194640 1 -- 172.21.15.67:0/4128948147 learned_addr learned my addr 172.21.15.67:0/4128948147 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:13:28.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.807+0000 7f6785995640 1 --2- 172.21.15.67:0/4128948147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67801035d0 0x7f6780199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:28.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.807+0000 7f6786196640 1 --2- 172.21.15.67:0/4128948147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6780109860 0x7f678019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:28.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.807+0000 7f6785194640 1 -- 172.21.15.67:0/4128948147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6780109860 msgr2=0x7f678019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:28.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.807+0000 7f6785194640 1 --2- 172.21.15.67:0/4128948147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6780109860 0x7f678019f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:28.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.807+0000 7f6785194640 1 -- 172.21.15.67:0/4128948147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67801035d0 msgr2=0x7f6780199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:28.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.807+0000 7f6785194640 1 --2- 172.21.15.67:0/4128948147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67801035d0 0x7f6780199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:28.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.807+0000 7f6785194640 1 -- 172.21.15.67:0/4128948147 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678019fed0 con 0x7f6780108f10 2024-07-27T03:13:28.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.808+0000 7f6785194640 1 --2- 172.21.15.67:0/4128948147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6780108f10 0x7f678019a500 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f677000e9c0 tx=0x7f677000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:28.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.808+0000 7f6776ffd640 1 -- 172.21.15.67:0/4128948147 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677000ccc0 con 0x7f6780108f10 2024-07-27T03:13:28.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.808+0000 7f6776ffd640 1 -- 172.21.15.67:0/4128948147 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67700054d0 con 0x7f6780108f10 2024-07-27T03:13:28.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.808+0000 7f6787bf9640 1 -- 172.21.15.67:0/4128948147 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67801a01c0 con 0x7f6780108f10 2024-07-27T03:13:28.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.808+0000 7f6776ffd640 1 -- 172.21.15.67:0/4128948147 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770010430 con 0x7f6780108f10 2024-07-27T03:13:28.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.808+0000 7f6787bf9640 1 -- 172.21.15.67:0/4128948147 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67801a6920 con 0x7f6780108f10 2024-07-27T03:13:28.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.809+0000 7f6787bf9640 1 -- 172.21.15.67:0/4128948147 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67801039e0 con 0x7f6780108f10 2024-07-27T03:13:28.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.810+0000 7f6776ffd640 1 -- 172.21.15.67:0/4128948147 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f6770005000 con 0x7f6780108f10 2024-07-27T03:13:28.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.811+0000 7f6776ffd640 1 --2- 172.21.15.67:0/4128948147 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f676c0b3bd0 0x7f676c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:28.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.811+0000 7f6776ffd640 1 -- 172.21.15.67:0/4128948147 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f67700e4410 con 0x7f6780108f10 2024-07-27T03:13:28.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.811+0000 7f6785995640 1 --2- 172.21.15.67:0/4128948147 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f676c0b3bd0 0x7f676c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:28.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.812+0000 7f6785995640 1 --2- 172.21.15.67:0/4128948147 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f676c0b3bd0 0x7f676c0b60e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f677c009bd0 tx=0x7f677c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:28.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:28.814+0000 7f6776ffd640 1 -- 172.21.15.67:0/4128948147 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f6770055600 con 0x7f6780108f10 2024-07-27T03:13:29.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:29.007+0000 7f6787bf9640 1 -- 172.21.15.67:0/4128948147 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f678019b090 con 0x7f6780108f10 2024-07-27T03:13:29.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:29.008+0000 7f6776ffd640 1 -- 172.21.15.67:0/4128948147 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24690 (secure 0 0 0) 0x7f678019b090 con 0x7f6780108f10 2024-07-27T03:13:29.014 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:13:29.015 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":89,"fsid":"c0782a00-4bc4-11ef-bcc7-c7b262605968","created":"2024-07-27T03:04:08.096824+0000","modified":"2024-07-27T03:12:56.708905+0000","last_up_change":"2024-07-27T03:11:33.314800+0000","last_in_change":"2024-07-27T03:11:14.028095+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-27T03:08:01.042479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-27T03:12:27.565948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-27T03:12:31.353726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-27T03:12:40.599486+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9f614272-18e8-4562-812c-4f43e40efd2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6803","nonce":2648129674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6805","nonce":2648129674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6809","nonce":2648129674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6807","nonce":2648129674}]},"public_addr":"172.21.15.67:6803/2648129674","cluster_addr":"172.21.15.67:6805/2648129674","heartbeat_back_addr":"172.21.15.67:6809/2648129674","heartbeat_front_addr":"172.21.15.67:6807/2648129674","state":["exists","up"]},{"osd":1,"uuid":"f59f5879-163a-4269-bbe7-1200eea65341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6801","nonce":2561753152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6803","nonce":2561753152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6807","nonce":2561753152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6805","nonce":2561753152}]},"public_addr":"172.21.15.149:6801/2561753152","cluster_addr":"172.21.15.149:6803/2561753152","heartbeat_back_addr":"172.21.15.149:6807/2561753152","heartbeat_front_addr":"172.21.15.149:6805/2561753152","state":["exists","up"]},{"osd":2,"uuid":"444e0ebe-7ca3-4be9-8bd5-db859f824445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6801","nonce":3770940078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6803","nonce":3770940078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6807","nonce":3770940078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6805","nonce":3770940078}]},"public_addr":"172.21.15.175:6801/3770940078","cluster_addr":"172.21.15.175:6803/3770940078","heartbeat_back_addr":"172.21.15.175:6807/3770940078","heartbeat_front_addr":"172.21.15.175:6805/3770940078","state":["exists","up"]},{"osd":3,"uuid":"d554db3d-9604-427c-8736-b1bb379f8d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6811","nonce":3159144693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6813","nonce":3159144693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6817","nonce":3159144693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6815","nonce":3159144693}]},"public_addr":"172.21.15.67:6811/3159144693","cluster_addr":"172.21.15.67:6813/3159144693","heartbeat_back_addr":"172.21.15.67:6817/3159144693","heartbeat_front_addr":"172.21.15.67:6815/3159144693","state":["exists","up"]},{"osd":4,"uuid":"c317dbe5-3b49-47a6-8408-7042a3dac345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3393503532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3393503532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3393503532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3393503532}]},"public_addr":"172.21.15.149:6809/3393503532","cluster_addr":"172.21.15.149:6811/3393503532","heartbeat_back_addr":"172.21.15.149:6815/3393503532","heartbeat_front_addr":"172.21.15.149:6813/3393503532","state":["exists","up"]},{"osd":5,"uuid":"5c85abd6-2b42-4957-a9b7-59a9ee36b841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6809","nonce":2291204784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6811","nonce":2291204784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6815","nonce":2291204784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6813","nonce":2291204784}]},"public_addr":"172.21.15.175:6809/2291204784","cluster_addr":"172.21.15.175:6811/2291204784","heartbeat_back_addr":"172.21.15.175:6815/2291204784","heartbeat_front_addr":"172.21.15.175:6813/2291204784","state":["exists","up"]},{"osd":6,"uuid":"bf6c277d-ec24-4470-98de-cf6f28876ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6819","nonce":2660182879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6821","nonce":2660182879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6825","nonce":2660182879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6823","nonce":2660182879}]},"public_addr":"172.21.15.67:6819/2660182879","cluster_addr":"172.21.15.67:6821/2660182879","heartbeat_back_addr":"172.21.15.67:6825/2660182879","heartbeat_front_addr":"172.21.15.67:6823/2660182879","state":["exists","up"]},{"osd":7,"uuid":"fea2c7cf-d5cb-45ae-9b29-f6e56c903891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6817","nonce":2289624404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6819","nonce":2289624404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6823","nonce":2289624404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6821","nonce":2289624404}]},"public_addr":"172.21.15.149:6817/2289624404","cluster_addr":"172.21.15.149:6819/2289624404","heartbeat_back_addr":"172.21.15.149:6823/2289624404","heartbeat_front_addr":"172.21.15.149:6821/2289624404","state":["exists","up"]},{"osd":8,"uuid":"14d04d96-579a-4dc9-a4d4-50e4850b48e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6817","nonce":2286032466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6819","nonce":2286032466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6823","nonce":2286032466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6821","nonce":2286032466}]},"public_addr":"172.21.15.175:6817/2286032466","cluster_addr":"172.21.15.175:6819/2286032466","heartbeat_back_addr":"172.21.15.175:6823/2286032466","heartbeat_front_addr":"172.21.15.175:6821/2286032466","state":["exists","up"]},{"osd":9,"uuid":"b65dfb6b-1efe-4314-a8be-ce6a295e99fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6827","nonce":3411081513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6829","nonce":3411081513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6833","nonce":3411081513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6831","nonce":3411081513}]},"public_addr":"172.21.15.67:6827/3411081513","cluster_addr":"172.21.15.67:6829/3411081513","heartbeat_back_addr":"172.21.15.67:6833/3411081513","heartbeat_front_addr":"172.21.15.67:6831/3411081513","state":["exists","up"]},{"osd":10,"uuid":"e3dabae0-0d64-4222-984c-fc1cce883456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6825","nonce":2011495212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6827","nonce":2011495212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6831","nonce":2011495212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6829","nonce":2011495212}]},"public_addr":"172.21.15.149:6825/2011495212","cluster_addr":"172.21.15.149:6827/2011495212","heartbeat_back_addr":"172.21.15.149:6831/2011495212","heartbeat_front_addr":"172.21.15.149:6829/2011495212","state":["exists","up"]},{"osd":11,"uuid":"406c082c-9b5f-4781-8cbf-3c04fa49e410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6825","nonce":2565514213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6827","nonce":2565514213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6831","nonce":2565514213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6829","nonce":2565514213}]},"public_addr":"172.21.15.175:6825/2565514213","cluster_addr":"172.21.15.175:6827/2565514213","heartbeat_back_addr":"172.21.15.175:6831/2565514213","heartbeat_front_addr":"172.21.15.175:6829/2565514213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:11.399380+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:33.728518+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:56.411181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:19.871731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:42.994576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:06.712815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:31.321967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:55.070898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:18.686108+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:43.187795+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:07.318538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:31.374843+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/3477601235":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/4130679770":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6800/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/1029332005":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6801/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6801/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/284078926":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6800/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/2559121542":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/925406021":"2024-07-28T03:04:30.600273+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-27T03:13:29.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:29.010+0000 7f6787bf9640 1 -- 172.21.15.67:0/4128948147 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f676c0b3bd0 msgr2=0x7f676c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:29.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:29.010+0000 7f6787bf9640 1 --2- 172.21.15.67:0/4128948147 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f676c0b3bd0 0x7f676c0b60e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f677c009bd0 tx=0x7f677c005f90 comp rx=0 tx=0).stop 2024-07-27T03:13:29.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:29.010+0000 7f6787bf9640 1 -- 172.21.15.67:0/4128948147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6780108f10 msgr2=0x7f678019a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:29.010+0000 7f6787bf9640 1 --2- 172.21.15.67:0/4128948147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6780108f10 0x7f678019a500 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f677000e9c0 tx=0x7f677000ee80 comp rx=0 tx=0).stop 2024-07-27T03:13:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:29.010+0000 7f6787bf9640 1 -- 172.21.15.67:0/4128948147 shutdown_connections 2024-07-27T03:13:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:29.010+0000 7f6787bf9640 1 --2- 172.21.15.67:0/4128948147 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f676c0b3bd0 0x7f676c0b60e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:29.010+0000 7f6787bf9640 1 --2- 172.21.15.67:0/4128948147 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6780109860 0x7f678019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:29.010+0000 7f6787bf9640 1 --2- 172.21.15.67:0/4128948147 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6780108f10 0x7f678019a500 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:29.010+0000 7f6787bf9640 1 --2- 172.21.15.67:0/4128948147 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67801035d0 0x7f6780199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:29.010+0000 7f6787bf9640 1 -- 172.21.15.67:0/4128948147 >> 172.21.15.67:0/4128948147 conn(0x7f678007b2c0 msgr2=0x7f6780106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:29.010+0000 7f6787bf9640 1 -- 172.21.15.67:0/4128948147 shutdown_connections 2024-07-27T03:13:29.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:29.010+0000 7f6787bf9640 1 -- 172.21.15.67:0/4128948147 wait complete. 2024-07-27T03:13:29.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:29 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/4128948147' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:13:29.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:29 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:13:29.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:29 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:13:29.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:29 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/4128948147' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:13:29.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:29 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:13:29.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:29 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:13:29.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:29 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/4128948147' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:13:29.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:29 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:13:29.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:29 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:13:29.891 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-07-27T03:13:30.295 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:13:30.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:30 smithi067 ceph-mon[32814]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-27T03:13:30.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:13:30.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:30 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:13:30.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:30 smithi149 ceph-mon[37173]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-27T03:13:30.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:13:30.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:30 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:13:30.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:30 smithi175 ceph-mon[37125]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-27T03:13:30.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:13:30.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:30 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:13:31.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.946+0000 7fbe2c8aa640 1 -- 172.21.15.67:0/2498707682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe28101b10 msgr2=0x7fbe28101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:31.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.946+0000 7fbe2c8aa640 1 --2- 172.21.15.67:0/2498707682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe28101b10 0x7fbe28101f20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbe10009900 tx=0x7fbe1002a410 comp rx=0 tx=0).stop 2024-07-27T03:13:31.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.946+0000 7fbe2c8aa640 1 -- 172.21.15.67:0/2498707682 shutdown_connections 2024-07-27T03:13:31.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.946+0000 7fbe2c8aa640 1 --2- 172.21.15.67:0/2498707682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe281098b0 0x7fbe281100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:31.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.946+0000 7fbe2c8aa640 1 --2- 172.21.15.67:0/2498707682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe28108f60 0x7fbe28109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:31.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.946+0000 7fbe2c8aa640 1 --2- 172.21.15.67:0/2498707682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe28101b10 0x7fbe28101f20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:31.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.946+0000 7fbe2c8aa640 1 -- 172.21.15.67:0/2498707682 >> 172.21.15.67:0/2498707682 conn(0x7fbe280798b0 msgr2=0x7fbe28079cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:31.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.946+0000 7fbe2c8aa640 1 -- 172.21.15.67:0/2498707682 shutdown_connections 2024-07-27T03:13:31.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.946+0000 7fbe2c8aa640 1 -- 172.21.15.67:0/2498707682 wait complete. 2024-07-27T03:13:31.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.947+0000 7fbe2c8aa640 1 Processor -- start 2024-07-27T03:13:31.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.947+0000 7fbe2c8aa640 1 -- start start 2024-07-27T03:13:31.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.947+0000 7fbe2c8aa640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe28101b10 0x7fbe28102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:31.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.947+0000 7fbe2c8aa640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe28108f60 0x7fbe28102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:31.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.947+0000 7fbe2c8aa640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe281098b0 0x7fbe28103530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:31.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.947+0000 7fbe2c8aa640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbe28113560 con 0x7fbe28108f60 2024-07-27T03:13:31.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.947+0000 7fbe2c8aa640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbe281133e0 con 0x7fbe281098b0 2024-07-27T03:13:31.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.947+0000 7fbe2c8aa640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbe281136e0 con 0x7fbe28101b10 2024-07-27T03:13:31.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.947+0000 7fbe25d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe28108f60 0x7fbe28102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:31.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.947+0000 7fbe25d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe28108f60 0x7fbe28102ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52650/0 (socket says 172.21.15.67:52650) 2024-07-27T03:13:31.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.947+0000 7fbe25d9b640 1 -- 172.21.15.67:0/1851128029 learned_addr learned my addr 172.21.15.67:0/1851128029 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:13:31.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.947+0000 7fbe2659c640 1 --2- 172.21.15.67:0/1851128029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe28101b10 0x7fbe28102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:31.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.947+0000 7fbe26d9d640 1 --2- 172.21.15.67:0/1851128029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe281098b0 0x7fbe28103530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:31.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.948+0000 7fbe25d9b640 1 -- 172.21.15.67:0/1851128029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe28101b10 msgr2=0x7fbe28102ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:31.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.948+0000 7fbe25d9b640 1 --2- 172.21.15.67:0/1851128029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe28101b10 0x7fbe28102ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:31.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.948+0000 7fbe25d9b640 1 -- 172.21.15.67:0/1851128029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe281098b0 msgr2=0x7fbe28103530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:31.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.948+0000 7fbe25d9b640 1 --2- 172.21.15.67:0/1851128029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe281098b0 0x7fbe28103530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:31.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.948+0000 7fbe25d9b640 1 -- 172.21.15.67:0/1851128029 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe281a8270 con 0x7fbe28108f60 2024-07-27T03:13:31.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.948+0000 7fbe25d9b640 1 --2- 172.21.15.67:0/1851128029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe28108f60 0x7fbe28102ff0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800d950 tx=0x7fbe1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:31.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.948+0000 7fbe1f7fe640 1 -- 172.21.15.67:0/1851128029 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18017070 con 0x7fbe28108f60 2024-07-27T03:13:31.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.948+0000 7fbe1f7fe640 1 -- 172.21.15.67:0/1851128029 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe18009c30 con 0x7fbe28108f60 2024-07-27T03:13:31.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.948+0000 7fbe2c8aa640 1 -- 172.21.15.67:0/1851128029 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe281a8560 con 0x7fbe28108f60 2024-07-27T03:13:31.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.948+0000 7fbe1f7fe640 1 -- 172.21.15.67:0/1851128029 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe180047a0 con 0x7fbe28108f60 2024-07-27T03:13:31.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.948+0000 7fbe2c8aa640 1 -- 172.21.15.67:0/1851128029 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe281a8af0 con 0x7fbe28108f60 2024-07-27T03:13:31.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.949+0000 7fbe2c8aa640 1 -- 172.21.15.67:0/1851128029 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe28101f20 con 0x7fbe28108f60 2024-07-27T03:13:31.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.951+0000 7fbe1f7fe640 1 -- 172.21.15.67:0/1851128029 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7fbe18004020 con 0x7fbe28108f60 2024-07-27T03:13:31.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.952+0000 7fbe1f7fe640 1 --2- 172.21.15.67:0/1851128029 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbe140b3c50 0x7fbe140b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:31.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.952+0000 7fbe2659c640 1 --2- 172.21.15.67:0/1851128029 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbe140b3c50 0x7fbe140b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:31.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.952+0000 7fbe1f7fe640 1 -- 172.21.15.67:0/1851128029 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fbe180e3f70 con 0x7fbe28108f60 2024-07-27T03:13:31.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.953+0000 7fbe2659c640 1 --2- 172.21.15.67:0/1851128029 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbe140b3c50 0x7fbe140b6160 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbe10000c00 tx=0x7fbe10031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:31.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:31.955+0000 7fbe1f7fe640 1 -- 172.21.15.67:0/1851128029 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fbe18020020 con 0x7fbe28108f60 2024-07-27T03:13:32.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:32.191+0000 7fbe2c8aa640 1 -- 172.21.15.67:0/1851128029 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbe28103db0 con 0x7fbe28108f60 2024-07-27T03:13:32.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:32 smithi067 ceph-mon[32814]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-27T03:13:32.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:32 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:13:32.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:32 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:13:32.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:32 smithi149 ceph-mon[37173]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-27T03:13:32.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:32 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:13:32.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:32 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:13:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:32 smithi175 ceph-mon[37125]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-07-27T03:13:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:32 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:13:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:32 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:13:33.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:33.191+0000 7fbe1f7fe640 1 -- 172.21.15.67:0/1851128029 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7fbe180ab2e0 con 0x7fbe28108f60 2024-07-27T03:13:33.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:33.193+0000 7fbe2c8aa640 1 -- 172.21.15.67:0/1851128029 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbe140b3c50 msgr2=0x7fbe140b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:33.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:33.193+0000 7fbe2c8aa640 1 --2- 172.21.15.67:0/1851128029 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbe140b3c50 0x7fbe140b6160 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbe10000c00 tx=0x7fbe10031400 comp rx=0 tx=0).stop 2024-07-27T03:13:33.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:33.193+0000 7fbe2c8aa640 1 -- 172.21.15.67:0/1851128029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe28108f60 msgr2=0x7fbe28102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:33.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:33.193+0000 7fbe2c8aa640 1 --2- 172.21.15.67:0/1851128029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe28108f60 0x7fbe28102ff0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800d950 tx=0x7fbe1800de10 comp rx=0 tx=0).stop 2024-07-27T03:13:33.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:33.193+0000 7fbe2c8aa640 1 -- 172.21.15.67:0/1851128029 shutdown_connections 2024-07-27T03:13:33.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:33.193+0000 7fbe2c8aa640 1 --2- 172.21.15.67:0/1851128029 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbe140b3c50 0x7fbe140b6160 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:33.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:33.193+0000 7fbe2c8aa640 1 --2- 172.21.15.67:0/1851128029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe281098b0 0x7fbe28103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:33.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:33.193+0000 7fbe2c8aa640 1 --2- 172.21.15.67:0/1851128029 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbe28108f60 0x7fbe28102ff0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:33.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:33.193+0000 7fbe2c8aa640 1 --2- 172.21.15.67:0/1851128029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe28101b10 0x7fbe28102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:33.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:33.193+0000 7fbe2c8aa640 1 -- 172.21.15.67:0/1851128029 >> 172.21.15.67:0/1851128029 conn(0x7fbe280798b0 msgr2=0x7fbe28107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:33.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:33.193+0000 7fbe2c8aa640 1 -- 172.21.15.67:0/1851128029 shutdown_connections 2024-07-27T03:13:33.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:33.193+0000 7fbe2c8aa640 1 -- 172.21.15.67:0/1851128029 wait complete. 2024-07-27T03:13:33.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:33 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1851128029' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-27T03:13:33.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:33 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:13:33.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:33 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:13:33.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:33 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:13:33.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:33 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1851128029' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-27T03:13:33.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:33 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:13:33.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:33 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:13:33.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:33 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:13:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:33 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1851128029' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-07-27T03:13:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:33 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:13:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:33 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:13:33.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:33 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:13:34.025 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-07-27T03:13:34.026 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-07-27T03:13:34.295 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:13:34.418 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:34 smithi067 ceph-mon[32814]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:13:34.418 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:34 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1851128029' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-07-27T03:13:34.418 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:34 smithi067 ceph-mon[32814]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-27T03:13:34.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:34 smithi149 ceph-mon[37173]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:13:34.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:34 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1851128029' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-07-27T03:13:34.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:34 smithi149 ceph-mon[37173]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-27T03:13:34.526 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:34 smithi175 ceph-mon[37125]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:13:34.526 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:34 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1851128029' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-07-27T03:13:34.526 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:34 smithi175 ceph-mon[37125]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-27T03:13:35.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.881+0000 7f843717b640 1 -- 172.21.15.67:0/478186363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8430102550 msgr2=0x7f8430102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:35.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.881+0000 7f843717b640 1 --2- 172.21.15.67:0/478186363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8430102550 0x7f8430102960 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8424009900 tx=0x7f842402a3f0 comp rx=0 tx=0).stop 2024-07-27T03:13:35.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.881+0000 7f843717b640 1 -- 172.21.15.67:0/478186363 shutdown_connections 2024-07-27T03:13:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.881+0000 7f843717b640 1 --2- 172.21.15.67:0/478186363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8430108790 0x7f843010efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.881+0000 7f843717b640 1 --2- 172.21.15.67:0/478186363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8430107e40 0x7f8430108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.881+0000 7f843717b640 1 --2- 172.21.15.67:0/478186363 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8430102550 0x7f8430102960 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.881+0000 7f843717b640 1 -- 172.21.15.67:0/478186363 >> 172.21.15.67:0/478186363 conn(0x7f84300791d0 msgr2=0x7f84300795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.881+0000 7f843717b640 1 -- 172.21.15.67:0/478186363 shutdown_connections 2024-07-27T03:13:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.881+0000 7f843717b640 1 -- 172.21.15.67:0/478186363 wait complete. 2024-07-27T03:13:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.882+0000 7f843717b640 1 Processor -- start 2024-07-27T03:13:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.882+0000 7f843717b640 1 -- start start 2024-07-27T03:13:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.882+0000 7f843717b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8430102550 0x7f843019ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.882+0000 7f843717b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8430107e40 0x7f843019b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.882+0000 7f843717b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8430108790 0x7f843019b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:35.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.882+0000 7f843717b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f84301123f0 con 0x7f8430108790 2024-07-27T03:13:35.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.882+0000 7f843717b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8430112270 con 0x7f8430107e40 2024-07-27T03:13:35.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.882+0000 7f843717b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8430112570 con 0x7f8430102550 2024-07-27T03:13:35.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.882+0000 7f8435718640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8430108790 0x7f843019b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:35.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.882+0000 7f8435718640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8430108790 0x7f843019b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40914/0 (socket says 172.21.15.67:40914) 2024-07-27T03:13:35.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.882+0000 7f8435718640 1 -- 172.21.15.67:0/2415793289 learned_addr learned my addr 172.21.15.67:0/2415793289 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:13:35.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.882+0000 7f842ffff640 1 --2- 172.21.15.67:0/2415793289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8430107e40 0x7f843019b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:35.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.882+0000 7f8434f17640 1 --2- 172.21.15.67:0/2415793289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8430102550 0x7f843019ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:35.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.882+0000 7f8435718640 1 -- 172.21.15.67:0/2415793289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8430102550 msgr2=0x7f843019ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:35.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.882+0000 7f8435718640 1 --2- 172.21.15.67:0/2415793289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8430102550 0x7f843019ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:35.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.882+0000 7f8435718640 1 -- 172.21.15.67:0/2415793289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8430107e40 msgr2=0x7f843019b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:35.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.882+0000 7f8435718640 1 --2- 172.21.15.67:0/2415793289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8430107e40 0x7f843019b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:35.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.882+0000 7f8435718640 1 -- 172.21.15.67:0/2415793289 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8430198b50 con 0x7f8430108790 2024-07-27T03:13:35.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.883+0000 7f8435718640 1 --2- 172.21.15.67:0/2415793289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8430108790 0x7f843019b760 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f842800cce0 tx=0x7f842800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:35.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.883+0000 7f842dffb640 1 -- 172.21.15.67:0/2415793289 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842800f450 con 0x7f8430108790 2024-07-27T03:13:35.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.883+0000 7f843717b640 1 -- 172.21.15.67:0/2415793289 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8430198e40 con 0x7f8430108790 2024-07-27T03:13:35.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.883+0000 7f842dffb640 1 -- 172.21.15.67:0/2415793289 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f842800faa0 con 0x7f8430108790 2024-07-27T03:13:35.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.883+0000 7f842dffb640 1 -- 172.21.15.67:0/2415793289 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8428018b70 con 0x7f8430108790 2024-07-27T03:13:35.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.883+0000 7f843717b640 1 -- 172.21.15.67:0/2415793289 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8430199330 con 0x7f8430108790 2024-07-27T03:13:35.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.884+0000 7f843717b640 1 -- 172.21.15.67:0/2415793289 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8430103780 con 0x7f8430108790 2024-07-27T03:13:35.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.886+0000 7f842dffb640 1 -- 172.21.15.67:0/2415793289 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f8428018d10 con 0x7f8430108790 2024-07-27T03:13:35.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.886+0000 7f842dffb640 1 --2- 172.21.15.67:0/2415793289 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f84200b3bd0 0x7f84200b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:35.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.886+0000 7f842dffb640 1 -- 172.21.15.67:0/2415793289 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f84280e4400 con 0x7f8430108790 2024-07-27T03:13:35.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.886+0000 7f8434f17640 1 --2- 172.21.15.67:0/2415793289 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f84200b3bd0 0x7f84200b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:35.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.887+0000 7f8434f17640 1 --2- 172.21.15.67:0/2415793289 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f84200b3bd0 0x7f84200b60e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8424000c00 tx=0x7f8424031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:35.892 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:35.889+0000 7f842dffb640 1 -- 172.21.15.67:0/2415793289 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f8428014030 con 0x7f8430108790 2024-07-27T03:13:36.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:36.109+0000 7f843717b640 1 -- 172.21.15.67:0/2415793289 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f843006b9e0 con 0x7f8430108790 2024-07-27T03:13:36.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:36.195+0000 7f842dffb640 1 -- 172.21.15.67:0/2415793289 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f843006b9e0 con 0x7f8430108790 2024-07-27T03:13:36.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:36.197+0000 7f843717b640 1 -- 172.21.15.67:0/2415793289 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f84200b3bd0 msgr2=0x7f84200b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:36.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:36.197+0000 7f843717b640 1 --2- 172.21.15.67:0/2415793289 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f84200b3bd0 0x7f84200b60e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8424000c00 tx=0x7f8424031400 comp rx=0 tx=0).stop 2024-07-27T03:13:36.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:36.197+0000 7f843717b640 1 -- 172.21.15.67:0/2415793289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8430108790 msgr2=0x7f843019b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:36.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:36.197+0000 7f843717b640 1 --2- 172.21.15.67:0/2415793289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8430108790 0x7f843019b760 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f842800cce0 tx=0x7f842800a620 comp rx=0 tx=0).stop 2024-07-27T03:13:36.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:36.197+0000 7f843717b640 1 -- 172.21.15.67:0/2415793289 shutdown_connections 2024-07-27T03:13:36.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:36.197+0000 7f843717b640 1 --2- 172.21.15.67:0/2415793289 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f84200b3bd0 0x7f84200b60e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:36.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:36.197+0000 7f843717b640 1 --2- 172.21.15.67:0/2415793289 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8430108790 0x7f843019b760 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:36.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:36.197+0000 7f843717b640 1 --2- 172.21.15.67:0/2415793289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8430107e40 0x7f843019b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:36.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:36.197+0000 7f843717b640 1 --2- 172.21.15.67:0/2415793289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8430102550 0x7f843019ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:36.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:36.197+0000 7f843717b640 1 -- 172.21.15.67:0/2415793289 >> 172.21.15.67:0/2415793289 conn(0x7f84300791d0 msgr2=0x7f8430101010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:36.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:36.197+0000 7f843717b640 1 -- 172.21.15.67:0/2415793289 shutdown_connections 2024-07-27T03:13:36.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:36.197+0000 7f843717b640 1 -- 172.21.15.67:0/2415793289 wait complete. 2024-07-27T03:13:36.236 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:36 smithi067 ceph-mon[32814]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:13:36.236 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:36 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2415793289' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-07-27T03:13:36.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:36 smithi149 ceph-mon[37173]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:13:36.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:36 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2415793289' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-07-27T03:13:36.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:36 smithi175 ceph-mon[37125]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:13:36.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:36 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2415793289' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-07-27T03:13:36.831 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-07-27T03:13:37.107 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:13:37.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:37 smithi067 ceph-mon[32814]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-27T03:13:37.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:37 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2415793289' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-07-27T03:13:37.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:37 smithi067 ceph-mon[32814]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-07-27T03:13:37.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:37 smithi067 ceph-mon[32814]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-27T03:13:37.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:37 smithi067 ceph-mon[32814]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-07-27T03:13:37.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:37 smithi067 ceph-mon[32814]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-07-27T03:13:37.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:37 smithi149 ceph-mon[37173]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-27T03:13:37.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:37 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2415793289' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-07-27T03:13:37.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:37 smithi149 ceph-mon[37173]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-07-27T03:13:37.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:37 smithi149 ceph-mon[37173]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-27T03:13:37.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:37 smithi149 ceph-mon[37173]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-07-27T03:13:37.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:37 smithi149 ceph-mon[37173]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-07-27T03:13:37.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:37 smithi175 ceph-mon[37125]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-07-27T03:13:37.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:37 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2415793289' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-07-27T03:13:37.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:37 smithi175 ceph-mon[37125]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-07-27T03:13:37.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:37 smithi175 ceph-mon[37125]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-07-27T03:13:37.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:37 smithi175 ceph-mon[37125]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-07-27T03:13:37.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:37 smithi175 ceph-mon[37125]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-07-27T03:13:38.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:38 smithi067 ceph-mon[32814]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-07-27T03:13:38.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:38 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] up:active 2024-07-27T03:13:38.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:38 smithi067 ceph-mon[32814]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-07-27T03:13:38.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:38 smithi067 ceph-mon[32814]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-07-27T03:13:38.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:38 smithi067 ceph-mon[32814]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-07-27T03:13:38.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:38 smithi067 ceph-mon[32814]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-07-27T03:13:38.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:38 smithi149 ceph-mon[37173]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-07-27T03:13:38.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:38 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] up:active 2024-07-27T03:13:38.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:38 smithi149 ceph-mon[37173]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-07-27T03:13:38.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:38 smithi149 ceph-mon[37173]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-07-27T03:13:38.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:38 smithi149 ceph-mon[37173]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-07-27T03:13:38.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:38 smithi149 ceph-mon[37173]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-07-27T03:13:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:38 smithi175 ceph-mon[37125]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-07-27T03:13:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:38 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] up:active 2024-07-27T03:13:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:38 smithi175 ceph-mon[37125]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-07-27T03:13:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:38 smithi175 ceph-mon[37125]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-07-27T03:13:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:38 smithi175 ceph-mon[37125]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-07-27T03:13:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:38 smithi175 ceph-mon[37125]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-07-27T03:13:38.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.800+0000 7fc1ab26a640 1 -- 172.21.15.67:0/335574637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1a4103350 msgr2=0x7fc1a410ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:38.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.800+0000 7fc1ab26a640 1 --2- 172.21.15.67:0/335574637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1a4103350 0x7fc1a410ef00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009ca0 tx=0x7fc19c02a8c0 comp rx=0 tx=0).stop 2024-07-27T03:13:38.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.800+0000 7fc1ab26a640 1 -- 172.21.15.67:0/335574637 shutdown_connections 2024-07-27T03:13:38.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.800+0000 7fc1ab26a640 1 --2- 172.21.15.67:0/335574637 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1a4103350 0x7fc1a410ef00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:38.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.800+0000 7fc1ab26a640 1 --2- 172.21.15.67:0/335574637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1a4102a00 0x7fc1a4102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:38.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.800+0000 7fc1ab26a640 1 --2- 172.21.15.67:0/335574637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1a4107090 0x7fc1a41074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:38.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.800+0000 7fc1ab26a640 1 -- 172.21.15.67:0/335574637 >> 172.21.15.67:0/335574637 conn(0x7fc1a40798a0 msgr2=0x7fc1a4079cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:38.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.800+0000 7fc1ab26a640 1 -- 172.21.15.67:0/335574637 shutdown_connections 2024-07-27T03:13:38.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.800+0000 7fc1ab26a640 1 -- 172.21.15.67:0/335574637 wait complete. 2024-07-27T03:13:38.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.801+0000 7fc1ab26a640 1 Processor -- start 2024-07-27T03:13:38.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.801+0000 7fc1ab26a640 1 -- start start 2024-07-27T03:13:38.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.801+0000 7fc1ab26a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1a4102a00 0x7fc1a419c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:38.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.801+0000 7fc1ab26a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1a4103350 0x7fc1a419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:38.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.801+0000 7fc1ab26a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1a4107090 0x7fc1a41a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:38.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.801+0000 7fc1ab26a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a4112430 con 0x7fc1a4102a00 2024-07-27T03:13:38.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.801+0000 7fc1ab26a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a41122b0 con 0x7fc1a4107090 2024-07-27T03:13:38.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.801+0000 7fc1ab26a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a41125b0 con 0x7fc1a4103350 2024-07-27T03:13:38.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.801+0000 7fc1a9006640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1a4102a00 0x7fc1a419c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:38.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.802+0000 7fc1a9006640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1a4102a00 0x7fc1a419c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40944/0 (socket says 172.21.15.67:40944) 2024-07-27T03:13:38.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.802+0000 7fc1a9006640 1 -- 172.21.15.67:0/2528091597 learned_addr learned my addr 172.21.15.67:0/2528091597 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:13:38.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.802+0000 7fc1a9807640 1 --2- 172.21.15.67:0/2528091597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1a4107090 0x7fc1a41a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:38.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.802+0000 7fc1a8805640 1 --2- 172.21.15.67:0/2528091597 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1a4103350 0x7fc1a419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:38.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.802+0000 7fc1a9006640 1 -- 172.21.15.67:0/2528091597 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1a4103350 msgr2=0x7fc1a419c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:38.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.802+0000 7fc1a9006640 1 --2- 172.21.15.67:0/2528091597 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1a4103350 0x7fc1a419c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:38.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.802+0000 7fc1a9006640 1 -- 172.21.15.67:0/2528091597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1a4107090 msgr2=0x7fc1a41a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:38.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.802+0000 7fc1a9006640 1 --2- 172.21.15.67:0/2528091597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1a4107090 0x7fc1a41a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:38.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.802+0000 7fc1a9006640 1 -- 172.21.15.67:0/2528091597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a41a1ff0 con 0x7fc1a4102a00 2024-07-27T03:13:38.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.802+0000 7fc1a9006640 1 --2- 172.21.15.67:0/2528091597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1a4102a00 0x7fc1a419c080 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000e9f0 tx=0x7fc1a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:38.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.802+0000 7fc19a7fc640 1 -- 172.21.15.67:0/2528091597 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a000cd80 con 0x7fc1a4102a00 2024-07-27T03:13:38.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.802+0000 7fc19a7fc640 1 -- 172.21.15.67:0/2528091597 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a00108e0 con 0x7fc1a4102a00 2024-07-27T03:13:38.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.802+0000 7fc1ab26a640 1 -- 172.21.15.67:0/2528091597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a41a22e0 con 0x7fc1a4102a00 2024-07-27T03:13:38.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.802+0000 7fc19a7fc640 1 -- 172.21.15.67:0/2528091597 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0018980 con 0x7fc1a4102a00 2024-07-27T03:13:38.809 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.802+0000 7fc1ab26a640 1 -- 172.21.15.67:0/2528091597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a41a8a40 con 0x7fc1a4102a00 2024-07-27T03:13:38.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.803+0000 7fc1ab26a640 1 -- 172.21.15.67:0/2528091597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a4102e10 con 0x7fc1a4102a00 2024-07-27T03:13:38.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.806+0000 7fc19a7fc640 1 -- 172.21.15.67:0/2528091597 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7fc1a0010430 con 0x7fc1a4102a00 2024-07-27T03:13:38.810 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.807+0000 7fc19a7fc640 1 --2- 172.21.15.67:0/2528091597 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc1900b3bd0 0x7fc1900b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:38.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.807+0000 7fc1a8805640 1 --2- 172.21.15.67:0/2528091597 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc1900b3bd0 0x7fc1900b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:38.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.807+0000 7fc19a7fc640 1 -- 172.21.15.67:0/2528091597 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fc1a00e4790 con 0x7fc1a4102a00 2024-07-27T03:13:38.811 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.808+0000 7fc1a8805640 1 --2- 172.21.15.67:0/2528091597 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc1900b3bd0 0x7fc1900b60e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4106b60 tx=0x7fc19400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:38.813 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:38.810+0000 7fc19a7fc640 1 -- 172.21.15.67:0/2528091597 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fc1a00ab3d0 con 0x7fc1a4102a00 2024-07-27T03:13:39.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:39.030+0000 7fc1ab26a640 1 -- 172.21.15.67:0/2528091597 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fc1a41a8d20 con 0x7fc1a4102a00 2024-07-27T03:13:39.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:39.230+0000 7fc19a7fc640 1 -- 172.21.15.67:0/2528091597 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7fc1a00af920 con 0x7fc1a4102a00 2024-07-27T03:13:39.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:39.232+0000 7fc1ab26a640 1 -- 172.21.15.67:0/2528091597 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc1900b3bd0 msgr2=0x7fc1900b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:39.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:39.232+0000 7fc1ab26a640 1 --2- 172.21.15.67:0/2528091597 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc1900b3bd0 0x7fc1900b60e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4106b60 tx=0x7fc19400a2d0 comp rx=0 tx=0).stop 2024-07-27T03:13:39.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:39.232+0000 7fc1ab26a640 1 -- 172.21.15.67:0/2528091597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1a4102a00 msgr2=0x7fc1a419c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:39.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:39.232+0000 7fc1ab26a640 1 --2- 172.21.15.67:0/2528091597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1a4102a00 0x7fc1a419c080 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000e9f0 tx=0x7fc1a000eeb0 comp rx=0 tx=0).stop 2024-07-27T03:13:39.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:39.232+0000 7fc1ab26a640 1 -- 172.21.15.67:0/2528091597 shutdown_connections 2024-07-27T03:13:39.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:39.232+0000 7fc1ab26a640 1 --2- 172.21.15.67:0/2528091597 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc1900b3bd0 0x7fc1900b60e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:39.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:39.232+0000 7fc1ab26a640 1 --2- 172.21.15.67:0/2528091597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1a4107090 0x7fc1a41a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:39.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:39.232+0000 7fc1ab26a640 1 --2- 172.21.15.67:0/2528091597 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1a4103350 0x7fc1a419c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:39.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:39.232+0000 7fc1ab26a640 1 --2- 172.21.15.67:0/2528091597 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1a4102a00 0x7fc1a419c080 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:39.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:39.232+0000 7fc1ab26a640 1 -- 172.21.15.67:0/2528091597 >> 172.21.15.67:0/2528091597 conn(0x7fc1a40798a0 msgr2=0x7fc1a4103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:39.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:39.232+0000 7fc1ab26a640 1 -- 172.21.15.67:0/2528091597 shutdown_connections 2024-07-27T03:13:39.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:39.232+0000 7fc1ab26a640 1 -- 172.21.15.67:0/2528091597 wait complete. 2024-07-27T03:13:39.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:39 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] up:active 2024-07-27T03:13:39.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:39 smithi067 ceph-mon[32814]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-07-27T03:13:39.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:39 smithi067 ceph-mon[32814]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-07-27T03:13:39.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:39 smithi067 ceph-mon[32814]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-07-27T03:13:39.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:39 smithi067 ceph-mon[32814]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-07-27T03:13:39.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:39 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2528091597' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-27T03:13:39.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:39 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] up:active 2024-07-27T03:13:39.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:39 smithi149 ceph-mon[37173]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-07-27T03:13:39.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:39 smithi149 ceph-mon[37173]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-07-27T03:13:39.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:39 smithi149 ceph-mon[37173]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-07-27T03:13:39.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:39 smithi149 ceph-mon[37173]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-07-27T03:13:39.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:39 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2528091597' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-27T03:13:39.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:39 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] up:active 2024-07-27T03:13:39.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:39 smithi175 ceph-mon[37125]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-07-27T03:13:39.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:39 smithi175 ceph-mon[37125]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-07-27T03:13:39.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:39 smithi175 ceph-mon[37125]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-07-27T03:13:39.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:39 smithi175 ceph-mon[37125]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-07-27T03:13:39.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:39 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2528091597' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-07-27T03:13:39.858 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-07-27T03:13:40.125 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:13:40.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:40 smithi149 ceph-mon[37173]: pgmap v341: 129 pgs: 129 active+clean; 583 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 1 op/s 2024-07-27T03:13:40.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:40 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2528091597' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-07-27T03:13:40.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:40 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] up:active 2024-07-27T03:13:40.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:40 smithi149 ceph-mon[37173]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-07-27T03:13:40.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:40 smithi149 ceph-mon[37173]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-27T03:13:40.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:40 smithi149 ceph-mon[37173]: Cluster is now healthy 2024-07-27T03:13:40.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:40 smithi149 ceph-mon[37173]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-07-27T03:13:40.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:40 smithi149 ceph-mon[37173]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-07-27T03:13:40.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:40 smithi149 ceph-mon[37173]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-07-27T03:13:40.565 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:40 smithi067 ceph-mon[32814]: pgmap v341: 129 pgs: 129 active+clean; 583 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 1 op/s 2024-07-27T03:13:40.565 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:40 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2528091597' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-07-27T03:13:40.565 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:40 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] up:active 2024-07-27T03:13:40.565 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:40 smithi067 ceph-mon[32814]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-07-27T03:13:40.565 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:40 smithi067 ceph-mon[32814]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-27T03:13:40.566 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:40 smithi067 ceph-mon[32814]: Cluster is now healthy 2024-07-27T03:13:40.566 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:40 smithi067 ceph-mon[32814]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-07-27T03:13:40.566 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:40 smithi067 ceph-mon[32814]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-07-27T03:13:40.566 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:40 smithi067 ceph-mon[32814]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-07-27T03:13:40.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:40 smithi175 ceph-mon[37125]: pgmap v341: 129 pgs: 129 active+clean; 583 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 1 op/s 2024-07-27T03:13:40.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:40 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2528091597' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-07-27T03:13:40.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:40 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] up:active 2024-07-27T03:13:40.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:40 smithi175 ceph-mon[37125]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-07-27T03:13:40.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:40 smithi175 ceph-mon[37125]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-07-27T03:13:40.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:40 smithi175 ceph-mon[37125]: Cluster is now healthy 2024-07-27T03:13:40.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:40 smithi175 ceph-mon[37125]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-07-27T03:13:40.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:40 smithi175 ceph-mon[37125]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-07-27T03:13:40.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:40 smithi175 ceph-mon[37125]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-07-27T03:13:41.000 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:13:40 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-b[67306]: 2024-07-27T03:13:40.540+0000 7f3a9fa3c640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-07-27T03:13:41.000 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:13:40 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-b[67306]: 2024-07-27T03:13:40.540+0000 7f3a9fa3c640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-07-27T03:13:41.000 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:13:40 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-b[67306]: 2024-07-27T03:13:40.540+0000 7f3a9fa3c640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-07-27T03:13:41.001 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:13:40 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-b[67306]: 2024-07-27T03:13:40.540+0000 7f3a9fa3c640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-07-27T03:13:41.502 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:41 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] up:active 2024-07-27T03:13:41.502 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:41 smithi149 ceph-mon[37173]: fsmap cephfs:5 7 up:standby 5 up:active 2024-07-27T03:13:41.502 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:41 smithi149 ceph-mon[37173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-27T03:13:41.568 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:41 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] up:active 2024-07-27T03:13:41.568 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:41 smithi067 ceph-mon[32814]: fsmap cephfs:5 7 up:standby 5 up:active 2024-07-27T03:13:41.568 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:41 smithi067 ceph-mon[32814]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-27T03:13:41.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:41 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] up:active 2024-07-27T03:13:41.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:41 smithi175 ceph-mon[37125]: fsmap cephfs:5 7 up:standby 5 up:active 2024-07-27T03:13:41.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:41 smithi175 ceph-mon[37125]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-27T03:13:41.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.831+0000 7fa36ad3a640 1 -- 172.21.15.67:0/4098420887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa364107e10 msgr2=0x7fa364108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:41.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.831+0000 7fa36ad3a640 1 --2- 172.21.15.67:0/4098420887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa364107e10 0x7fa364108220 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa350009900 tx=0x7fa35002a410 comp rx=0 tx=0).stop 2024-07-27T03:13:41.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.831+0000 7fa36ad3a640 1 -- 172.21.15.67:0/4098420887 shutdown_connections 2024-07-27T03:13:41.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.831+0000 7fa36ad3a640 1 --2- 172.21.15.67:0/4098420887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa364108760 0x7fa36410ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:41.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.831+0000 7fa36ad3a640 1 --2- 172.21.15.67:0/4098420887 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa364107e10 0x7fa364108220 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:41.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.831+0000 7fa36ad3a640 1 --2- 172.21.15.67:0/4098420887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3641024d0 0x7fa3641028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:41.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.831+0000 7fa36ad3a640 1 -- 172.21.15.67:0/4098420887 >> 172.21.15.67:0/4098420887 conn(0x7fa3640791a0 msgr2=0x7fa3640795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:41.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.831+0000 7fa36ad3a640 1 -- 172.21.15.67:0/4098420887 shutdown_connections 2024-07-27T03:13:41.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.831+0000 7fa36ad3a640 1 -- 172.21.15.67:0/4098420887 wait complete. 2024-07-27T03:13:41.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.832+0000 7fa36ad3a640 1 Processor -- start 2024-07-27T03:13:41.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.832+0000 7fa36ad3a640 1 -- start start 2024-07-27T03:13:41.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.832+0000 7fa36ad3a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3641024d0 0x7fa36410fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:41.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.833+0000 7fa36ad3a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa364107e10 0x7fa36410ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:41.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.833+0000 7fa36ad3a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa364108760 0x7fa3641104a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:41.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.833+0000 7fa36ad3a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa364113530 con 0x7fa364107e10 2024-07-27T03:13:41.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.833+0000 7fa36ad3a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa3641133b0 con 0x7fa364108760 2024-07-27T03:13:41.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.833+0000 7fa36ad3a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa3641136b0 con 0x7fa3641024d0 2024-07-27T03:13:41.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.833+0000 7fa363fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa364107e10 0x7fa36410ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:41.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.833+0000 7fa363fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa364107e10 0x7fa36410ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40966/0 (socket says 172.21.15.67:40966) 2024-07-27T03:13:41.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.833+0000 7fa363fff640 1 -- 172.21.15.67:0/2566342525 learned_addr learned my addr 172.21.15.67:0/2566342525 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:13:41.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.833+0000 7fa368ad6640 1 --2- 172.21.15.67:0/2566342525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3641024d0 0x7fa36410fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:41.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.833+0000 7fa3692d7640 1 --2- 172.21.15.67:0/2566342525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa364108760 0x7fa3641104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:41.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.833+0000 7fa363fff640 1 -- 172.21.15.67:0/2566342525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3641024d0 msgr2=0x7fa36410fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:41.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.833+0000 7fa363fff640 1 --2- 172.21.15.67:0/2566342525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3641024d0 0x7fa36410fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:41.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.833+0000 7fa363fff640 1 -- 172.21.15.67:0/2566342525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa364108760 msgr2=0x7fa3641104a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:41.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.833+0000 7fa363fff640 1 --2- 172.21.15.67:0/2566342525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa364108760 0x7fa3641104a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:41.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.833+0000 7fa363fff640 1 -- 172.21.15.67:0/2566342525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3641a82c0 con 0x7fa364107e10 2024-07-27T03:13:41.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.833+0000 7fa3692d7640 1 --2- 172.21.15.67:0/2566342525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa364108760 0x7fa3641104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:13:41.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.833+0000 7fa363fff640 1 --2- 172.21.15.67:0/2566342525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa364107e10 0x7fa36410ff60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa350032fd0 tx=0x7fa350004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:41.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.833+0000 7fa361ffb640 1 -- 172.21.15.67:0/2566342525 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35003f030 con 0x7fa364107e10 2024-07-27T03:13:41.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.834+0000 7fa361ffb640 1 -- 172.21.15.67:0/2566342525 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa350004490 con 0x7fa364107e10 2024-07-27T03:13:41.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.834+0000 7fa36ad3a640 1 -- 172.21.15.67:0/2566342525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3641a8550 con 0x7fa364107e10 2024-07-27T03:13:41.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.834+0000 7fa361ffb640 1 -- 172.21.15.67:0/2566342525 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa350035070 con 0x7fa364107e10 2024-07-27T03:13:41.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.834+0000 7fa36ad3a640 1 -- 172.21.15.67:0/2566342525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3641a8a80 con 0x7fa364107e10 2024-07-27T03:13:41.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.835+0000 7fa36ad3a640 1 -- 172.21.15.67:0/2566342525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3641028e0 con 0x7fa364107e10 2024-07-27T03:13:41.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.836+0000 7fa361ffb640 1 -- 172.21.15.67:0/2566342525 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7fa350042030 con 0x7fa364107e10 2024-07-27T03:13:41.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.837+0000 7fa361ffb640 1 --2- 172.21.15.67:0/2566342525 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa3540b3c50 0x7fa3540b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:41.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.837+0000 7fa368ad6640 1 --2- 172.21.15.67:0/2566342525 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa3540b3c50 0x7fa3540b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:41.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.837+0000 7fa361ffb640 1 -- 172.21.15.67:0/2566342525 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fa3501047f0 con 0x7fa364107e10 2024-07-27T03:13:41.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.838+0000 7fa368ad6640 1 --2- 172.21.15.67:0/2566342525 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa3540b3c50 0x7fa3540b6160 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa364111470 tx=0x7fa358009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:41.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:41.840+0000 7fa361ffb640 1 -- 172.21.15.67:0/2566342525 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fa3500cbae0 con 0x7fa364107e10 2024-07-27T03:13:42.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:42.060+0000 7fa36ad3a640 1 -- 172.21.15.67:0/2566342525 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fa36410a5b0 con 0x7fa364107e10 2024-07-27T03:13:42.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:42.248+0000 7fa361ffb640 1 -- 172.21.15.67:0/2566342525 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v29) ==== 115+0+0 (secure 0 0 0) 0x7fa3500cb3e0 con 0x7fa364107e10 2024-07-27T03:13:42.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:42.250+0000 7fa36ad3a640 1 -- 172.21.15.67:0/2566342525 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa3540b3c50 msgr2=0x7fa3540b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:42.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:42.250+0000 7fa36ad3a640 1 --2- 172.21.15.67:0/2566342525 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa3540b3c50 0x7fa3540b6160 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa364111470 tx=0x7fa358009110 comp rx=0 tx=0).stop 2024-07-27T03:13:42.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:42.250+0000 7fa36ad3a640 1 -- 172.21.15.67:0/2566342525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa364107e10 msgr2=0x7fa36410ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:42.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:42.250+0000 7fa36ad3a640 1 --2- 172.21.15.67:0/2566342525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa364107e10 0x7fa36410ff60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa350032fd0 tx=0x7fa350004270 comp rx=0 tx=0).stop 2024-07-27T03:13:42.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:42.250+0000 7fa36ad3a640 1 -- 172.21.15.67:0/2566342525 shutdown_connections 2024-07-27T03:13:42.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:42.250+0000 7fa36ad3a640 1 --2- 172.21.15.67:0/2566342525 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa3540b3c50 0x7fa3540b6160 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:42.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:42.250+0000 7fa36ad3a640 1 --2- 172.21.15.67:0/2566342525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa364108760 0x7fa3641104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:42.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:42.250+0000 7fa36ad3a640 1 --2- 172.21.15.67:0/2566342525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa364107e10 0x7fa36410ff60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:42.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:42.250+0000 7fa36ad3a640 1 --2- 172.21.15.67:0/2566342525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3641024d0 0x7fa36410fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:42.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:42.250+0000 7fa36ad3a640 1 -- 172.21.15.67:0/2566342525 >> 172.21.15.67:0/2566342525 conn(0x7fa3640791a0 msgr2=0x7fa364104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:42.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:42.251+0000 7fa36ad3a640 1 -- 172.21.15.67:0/2566342525 shutdown_connections 2024-07-27T03:13:42.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:42.251+0000 7fa36ad3a640 1 -- 172.21.15.67:0/2566342525 wait complete. 2024-07-27T03:13:42.552 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:42 smithi067 ceph-mon[32814]: pgmap v342: 129 pgs: 129 active+clean; 586 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-07-27T03:13:42.552 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:42 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2566342525' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-07-27T03:13:42.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:42 smithi175 ceph-mon[37125]: pgmap v342: 129 pgs: 129 active+clean; 586 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-07-27T03:13:42.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:42 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2566342525' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-07-27T03:13:42.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:42 smithi149 ceph-mon[37173]: pgmap v342: 129 pgs: 129 active+clean; 586 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-07-27T03:13:42.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:42 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2566342525' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-07-27T03:13:42.886 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-07-27T03:13:42.886 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-07-27T03:13:43.157 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:13:43.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:43 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2566342525' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-07-27T03:13:43.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:43 smithi067 ceph-mon[32814]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-27T03:13:43.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:43 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2566342525' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-07-27T03:13:43.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:43 smithi175 ceph-mon[37125]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-27T03:13:43.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:43 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2566342525' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-07-27T03:13:43.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:43 smithi149 ceph-mon[37173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-27T03:13:44.574 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:44 smithi067 ceph-mon[32814]: pgmap v343: 129 pgs: 129 active+clean; 587 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3.4 KiB/s wr, 14 op/s 2024-07-27T03:13:44.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:44 smithi175 ceph-mon[37125]: pgmap v343: 129 pgs: 129 active+clean; 587 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3.4 KiB/s wr, 14 op/s 2024-07-27T03:13:44.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:44 smithi149 ceph-mon[37173]: pgmap v343: 129 pgs: 129 active+clean; 587 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3.4 KiB/s wr, 14 op/s 2024-07-27T03:13:44.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.800+0000 7f0b0c8c9640 1 -- 172.21.15.67:0/3056322155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b08102550 msgr2=0x7f0b08102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:44.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.800+0000 7f0b0c8c9640 1 --2- 172.21.15.67:0/3056322155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b08102550 0x7f0b08102960 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0af8009900 tx=0x7f0af802a410 comp rx=0 tx=0).stop 2024-07-27T03:13:44.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.800+0000 7f0b0c8c9640 1 -- 172.21.15.67:0/3056322155 shutdown_connections 2024-07-27T03:13:44.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.800+0000 7f0b0c8c9640 1 --2- 172.21.15.67:0/3056322155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b08108790 0x7f0b0810efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:44.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.800+0000 7f0b0c8c9640 1 --2- 172.21.15.67:0/3056322155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b08107e40 0x7f0b08108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:44.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.800+0000 7f0b0c8c9640 1 --2- 172.21.15.67:0/3056322155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b08102550 0x7f0b08102960 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:44.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.801+0000 7f0b0c8c9640 1 -- 172.21.15.67:0/3056322155 >> 172.21.15.67:0/3056322155 conn(0x7f0b080791d0 msgr2=0x7f0b080795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:44.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.801+0000 7f0b0c8c9640 1 -- 172.21.15.67:0/3056322155 shutdown_connections 2024-07-27T03:13:44.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.801+0000 7f0b0c8c9640 1 -- 172.21.15.67:0/3056322155 wait complete. 2024-07-27T03:13:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.801+0000 7f0b0c8c9640 1 Processor -- start 2024-07-27T03:13:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.802+0000 7f0b0c8c9640 1 -- start start 2024-07-27T03:13:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.802+0000 7f0b0c8c9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b08102550 0x7f0b0819c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.802+0000 7f0b0c8c9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b08107e40 0x7f0b0819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.802+0000 7f0b0c8c9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b08108790 0x7f0b081a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.802+0000 7f0b0c8c9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0b081123f0 con 0x7f0b08102550 2024-07-27T03:13:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.802+0000 7f0b0c8c9640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0b08112270 con 0x7f0b08107e40 2024-07-27T03:13:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.802+0000 7f0b0c8c9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b08112570 con 0x7f0b08108790 2024-07-27T03:13:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.802+0000 7f0b0659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b08102550 0x7f0b0819c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.802+0000 7f0b0659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b08102550 0x7f0b0819c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40986/0 (socket says 172.21.15.67:40986) 2024-07-27T03:13:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.802+0000 7f0b0659c640 1 -- 172.21.15.67:0/2371981721 learned_addr learned my addr 172.21.15.67:0/2371981721 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:13:44.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.802+0000 7f0b06d9d640 1 --2- 172.21.15.67:0/2371981721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b08108790 0x7f0b081a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.802+0000 7f0afdd9b640 1 --2- 172.21.15.67:0/2371981721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b08107e40 0x7f0b0819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.803+0000 7f0b0659c640 1 -- 172.21.15.67:0/2371981721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b08108790 msgr2=0x7f0b081a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.803+0000 7f0b0659c640 1 --2- 172.21.15.67:0/2371981721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b08108790 0x7f0b081a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.803+0000 7f0b0659c640 1 -- 172.21.15.67:0/2371981721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b08107e40 msgr2=0x7f0b0819c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.803+0000 7f0b0659c640 1 --2- 172.21.15.67:0/2371981721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b08107e40 0x7f0b0819c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.803+0000 7f0b0659c640 1 -- 172.21.15.67:0/2371981721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b081a20d0 con 0x7f0b08102550 2024-07-27T03:13:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.803+0000 7f0afdd9b640 1 --2- 172.21.15.67:0/2371981721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b08107e40 0x7f0b0819c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:13:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.803+0000 7f0b06d9d640 1 --2- 172.21.15.67:0/2371981721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b08108790 0x7f0b081a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:13:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.803+0000 7f0b0659c640 1 --2- 172.21.15.67:0/2371981721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b08102550 0x7f0b0819c160 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0af8005940 tx=0x7f0af80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.803+0000 7f0afffff640 1 -- 172.21.15.67:0/2371981721 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af803f030 con 0x7f0b08102550 2024-07-27T03:13:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.803+0000 7f0b0c8c9640 1 -- 172.21.15.67:0/2371981721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b081a2360 con 0x7f0b08102550 2024-07-27T03:13:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.803+0000 7f0afffff640 1 -- 172.21.15.67:0/2371981721 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0af8003cb0 con 0x7f0b08102550 2024-07-27T03:13:44.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.803+0000 7f0afffff640 1 -- 172.21.15.67:0/2371981721 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af8035070 con 0x7f0b08102550 2024-07-27T03:13:44.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.803+0000 7f0b0c8c9640 1 -- 172.21.15.67:0/2371981721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b081a8b20 con 0x7f0b08102550 2024-07-27T03:13:44.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.804+0000 7f0b0c8c9640 1 -- 172.21.15.67:0/2371981721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b08103780 con 0x7f0b08102550 2024-07-27T03:13:44.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.806+0000 7f0afffff640 1 -- 172.21.15.67:0/2371981721 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 16) ==== 151059+0+0 (secure 0 0 0) 0x7f0af8042030 con 0x7f0b08102550 2024-07-27T03:13:44.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.806+0000 7f0afffff640 1 --2- 172.21.15.67:0/2371981721 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0af40b3c50 0x7f0af40b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:44.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.807+0000 7f0afdd9b640 1 --2- 172.21.15.67:0/2371981721 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0af40b3c50 0x7f0af40b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:44.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.807+0000 7f0afffff640 1 -- 172.21.15.67:0/2371981721 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f0af8104ab0 con 0x7f0b08102550 2024-07-27T03:13:44.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.807+0000 7f0afdd9b640 1 --2- 172.21.15.67:0/2371981721 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0af40b3c50 0x7f0af40b6160 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0b081005e0 tx=0x7f0af0005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:44.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:44.810+0000 7f0afffff640 1 -- 172.21.15.67:0/2371981721 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f0af80cbda0 con 0x7f0b08102550 2024-07-27T03:13:45.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:45.017+0000 7f0b0c8c9640 1 -- 172.21.15.67:0/2371981721 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0b0806bd20 con 0x7f0af40b3c50 2024-07-27T03:13:45.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:45.050+0000 7f0afffff640 1 -- 172.21.15.67:0/2371981721 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0b0806bd20 con 0x7f0af40b3c50 2024-07-27T03:13:45.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:45.052+0000 7f0b0c8c9640 1 -- 172.21.15.67:0/2371981721 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0af40b3c50 msgr2=0x7f0af40b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:45.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:45.052+0000 7f0b0c8c9640 1 --2- 172.21.15.67:0/2371981721 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0af40b3c50 0x7f0af40b6160 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0b081005e0 tx=0x7f0af0005e50 comp rx=0 tx=0).stop 2024-07-27T03:13:45.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:45.052+0000 7f0b0c8c9640 1 -- 172.21.15.67:0/2371981721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b08102550 msgr2=0x7f0b0819c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:45.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:45.052+0000 7f0b0c8c9640 1 --2- 172.21.15.67:0/2371981721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b08102550 0x7f0b0819c160 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0af8005940 tx=0x7f0af80042f0 comp rx=0 tx=0).stop 2024-07-27T03:13:45.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:45.053+0000 7f0b0c8c9640 1 -- 172.21.15.67:0/2371981721 shutdown_connections 2024-07-27T03:13:45.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:45.053+0000 7f0b0c8c9640 1 --2- 172.21.15.67:0/2371981721 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0af40b3c50 0x7f0af40b6160 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:45.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:45.053+0000 7f0b0c8c9640 1 --2- 172.21.15.67:0/2371981721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b08108790 0x7f0b081a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:45.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:45.053+0000 7f0b0c8c9640 1 --2- 172.21.15.67:0/2371981721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b08107e40 0x7f0b0819c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:45.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:45.053+0000 7f0b0c8c9640 1 --2- 172.21.15.67:0/2371981721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0b08102550 0x7f0b0819c160 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:45.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:45.053+0000 7f0b0c8c9640 1 -- 172.21.15.67:0/2371981721 >> 172.21.15.67:0/2371981721 conn(0x7f0b080791d0 msgr2=0x7f0b0806b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:45.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:45.053+0000 7f0b0c8c9640 1 -- 172.21.15.67:0/2371981721 shutdown_connections 2024-07-27T03:13:45.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:45.053+0000 7f0b0c8c9640 1 -- 172.21.15.67:0/2371981721 wait complete. 2024-07-27T03:13:45.220 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:13:45 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: 2024-07-27T03:13:45.023+0000 7f448291b640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-27T03:13:45.220 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:13:45 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: 2024-07-27T03:13:45.023+0000 7f448291b640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-27T03:13:45.220 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:13:45 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: 2024-07-27T03:13:45.023+0000 7f448291b640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-27T03:13:45.220 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:13:45 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: 2024-07-27T03:13:45.023+0000 7f448291b640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-27T03:13:45.220 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:13:45 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: 2024-07-27T03:13:45.023+0000 7f448291b640 -1 client.0 error registering admin socket command: (17) File exists 2024-07-27T03:13:45.665 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-07-27T03:13:45.932 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:13:46.390 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:46 smithi067 ceph-mon[32814]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:13:46.391 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:46 smithi067 ceph-mon[32814]: pgmap v344: 129 pgs: 129 active+clean; 587 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-07-27T03:13:46.391 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:46 smithi067 ceph-mon[32814]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-07-27T03:13:46.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:46 smithi175 ceph-mon[37125]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:13:46.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:46 smithi175 ceph-mon[37125]: pgmap v344: 129 pgs: 129 active+clean; 587 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-07-27T03:13:46.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:46 smithi175 ceph-mon[37125]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-07-27T03:13:46.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:46 smithi149 ceph-mon[37173]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:13:46.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:46 smithi149 ceph-mon[37173]: pgmap v344: 129 pgs: 129 active+clean; 587 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-07-27T03:13:46.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:46 smithi149 ceph-mon[37173]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-07-27T03:13:47.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.502+0000 7fe91ebf6640 1 -- 172.21.15.67:0/568358895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe918109860 msgr2=0x7fe918110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:47.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.502+0000 7fe91ebf6640 1 --2- 172.21.15.67:0/568358895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe918109860 0x7fe918110090 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe910009ca0 tx=0x7fe91002a8c0 comp rx=0 tx=0).stop 2024-07-27T03:13:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.502+0000 7fe91ebf6640 1 -- 172.21.15.67:0/568358895 shutdown_connections 2024-07-27T03:13:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.502+0000 7fe91ebf6640 1 --2- 172.21.15.67:0/568358895 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe918109860 0x7fe918110090 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.502+0000 7fe91ebf6640 1 --2- 172.21.15.67:0/568358895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe918108f10 0x7fe918109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.502+0000 7fe91ebf6640 1 --2- 172.21.15.67:0/568358895 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9181035d0 0x7fe9181039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.502+0000 7fe91ebf6640 1 -- 172.21.15.67:0/568358895 >> 172.21.15.67:0/568358895 conn(0x7fe91807b2c0 msgr2=0x7fe91807b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.502+0000 7fe91ebf6640 1 -- 172.21.15.67:0/568358895 shutdown_connections 2024-07-27T03:13:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.502+0000 7fe91ebf6640 1 -- 172.21.15.67:0/568358895 wait complete. 2024-07-27T03:13:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.503+0000 7fe91ebf6640 1 Processor -- start 2024-07-27T03:13:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.503+0000 7fe91ebf6640 1 -- start start 2024-07-27T03:13:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.503+0000 7fe91ebf6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9181035d0 0x7fe918199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:47.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.503+0000 7fe91ebf6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe918108f10 0x7fe91819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:47.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.503+0000 7fe91ebf6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe918109860 0x7fe91819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:47.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.503+0000 7fe91ebf6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe918113510 con 0x7fe918108f10 2024-07-27T03:13:47.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.503+0000 7fe91ebf6640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe918113390 con 0x7fe918109860 2024-07-27T03:13:47.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.503+0000 7fe91ebf6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe918113690 con 0x7fe9181035d0 2024-07-27T03:13:47.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.504+0000 7fe917fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe918108f10 0x7fe91819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:47.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.504+0000 7fe917fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe918108f10 0x7fe91819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32790/0 (socket says 172.21.15.67:32790) 2024-07-27T03:13:47.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.504+0000 7fe917fff640 1 -- 172.21.15.67:0/3007365202 learned_addr learned my addr 172.21.15.67:0/3007365202 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:13:47.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.504+0000 7fe91d193640 1 --2- 172.21.15.67:0/3007365202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe918109860 0x7fe91819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:47.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.504+0000 7fe91c992640 1 --2- 172.21.15.67:0/3007365202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9181035d0 0x7fe918199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:47.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.504+0000 7fe917fff640 1 -- 172.21.15.67:0/3007365202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9181035d0 msgr2=0x7fe918199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:47.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.504+0000 7fe917fff640 1 --2- 172.21.15.67:0/3007365202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9181035d0 0x7fe918199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:47.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.504+0000 7fe917fff640 1 -- 172.21.15.67:0/3007365202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe918109860 msgr2=0x7fe91819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:47.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.504+0000 7fe917fff640 1 --2- 172.21.15.67:0/3007365202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe918109860 0x7fe91819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:47.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.504+0000 7fe917fff640 1 -- 172.21.15.67:0/3007365202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91819fd70 con 0x7fe918108f10 2024-07-27T03:13:47.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.504+0000 7fe91c992640 1 --2- 172.21.15.67:0/3007365202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9181035d0 0x7fe918199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:13:47.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.504+0000 7fe917fff640 1 --2- 172.21.15.67:0/3007365202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe918108f10 0x7fe91819a460 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe90400d740 tx=0x7fe90400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:47.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.504+0000 7fe915ffb640 1 -- 172.21.15.67:0/3007365202 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe904017070 con 0x7fe918108f10 2024-07-27T03:13:47.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.504+0000 7fe91ebf6640 1 -- 172.21.15.67:0/3007365202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9181a0060 con 0x7fe918108f10 2024-07-27T03:13:47.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.505+0000 7fe915ffb640 1 -- 172.21.15.67:0/3007365202 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe90400de50 con 0x7fe918108f10 2024-07-27T03:13:47.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.505+0000 7fe915ffb640 1 -- 172.21.15.67:0/3007365202 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90400f620 con 0x7fe918108f10 2024-07-27T03:13:47.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.505+0000 7fe91ebf6640 1 -- 172.21.15.67:0/3007365202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9181a6880 con 0x7fe918108f10 2024-07-27T03:13:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.506+0000 7fe91ebf6640 1 -- 172.21.15.67:0/3007365202 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9181039e0 con 0x7fe918108f10 2024-07-27T03:13:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.507+0000 7fe915ffb640 1 -- 172.21.15.67:0/3007365202 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7fe90401a070 con 0x7fe918108f10 2024-07-27T03:13:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.508+0000 7fe915ffb640 1 --2- 172.21.15.67:0/3007365202 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe9080b3d10 0x7fe9080b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.508+0000 7fe91c992640 1 --2- 172.21.15.67:0/3007365202 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe9080b3d10 0x7fe9080b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:47.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.508+0000 7fe915ffb640 1 -- 172.21.15.67:0/3007365202 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7fe9040e3a20 con 0x7fe918108f10 2024-07-27T03:13:47.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.509+0000 7fe91c992640 1 --2- 172.21.15.67:0/3007365202 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe9080b3d10 0x7fe9080b6220 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00abe0 tx=0x7fe90c00a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:47.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.513+0000 7fe915ffb640 1 -- 172.21.15.67:0/3007365202 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fe9040aabb0 con 0x7fe918108f10 2024-07-27T03:13:47.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.720+0000 7fe91ebf6640 1 -- 172.21.15.67:0/3007365202 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe91810b6b0 con 0x7fe9080b3d10 2024-07-27T03:13:47.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.747+0000 7fe915ffb640 1 -- 172.21.15.67:0/3007365202 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe91810b6b0 con 0x7fe9080b3d10 2024-07-27T03:13:47.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.749+0000 7fe91ebf6640 1 -- 172.21.15.67:0/3007365202 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe9080b3d10 msgr2=0x7fe9080b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:47.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.749+0000 7fe91ebf6640 1 --2- 172.21.15.67:0/3007365202 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe9080b3d10 0x7fe9080b6220 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00abe0 tx=0x7fe90c00a5c0 comp rx=0 tx=0).stop 2024-07-27T03:13:47.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.749+0000 7fe91ebf6640 1 -- 172.21.15.67:0/3007365202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe918108f10 msgr2=0x7fe91819a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:47.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.749+0000 7fe91ebf6640 1 --2- 172.21.15.67:0/3007365202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe918108f10 0x7fe91819a460 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe90400d740 tx=0x7fe90400dc00 comp rx=0 tx=0).stop 2024-07-27T03:13:47.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.750+0000 7fe91ebf6640 1 -- 172.21.15.67:0/3007365202 shutdown_connections 2024-07-27T03:13:47.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.750+0000 7fe91ebf6640 1 --2- 172.21.15.67:0/3007365202 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe9080b3d10 0x7fe9080b6220 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:47.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.750+0000 7fe91ebf6640 1 --2- 172.21.15.67:0/3007365202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe918109860 0x7fe91819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:47.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.750+0000 7fe91ebf6640 1 --2- 172.21.15.67:0/3007365202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe918108f10 0x7fe91819a460 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:47.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.750+0000 7fe91ebf6640 1 --2- 172.21.15.67:0/3007365202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9181035d0 0x7fe918199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:47.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.750+0000 7fe91ebf6640 1 -- 172.21.15.67:0/3007365202 >> 172.21.15.67:0/3007365202 conn(0x7fe91807b2c0 msgr2=0x7fe918106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:47.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.750+0000 7fe91ebf6640 1 -- 172.21.15.67:0/3007365202 shutdown_connections 2024-07-27T03:13:47.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:47.750+0000 7fe91ebf6640 1 -- 172.21.15.67:0/3007365202 wait complete. 2024-07-27T03:13:48.339 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:48 smithi067 ceph-mon[32814]: pgmap v345: 129 pgs: 129 active+clean; 587 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-07-27T03:13:48.339 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:48 smithi067 ceph-mon[32814]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:13:48.340 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-07-27T03:13:48.607 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:13:48.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:48 smithi175 ceph-mon[37125]: pgmap v345: 129 pgs: 129 active+clean; 587 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-07-27T03:13:48.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:48 smithi175 ceph-mon[37125]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:13:48.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:48 smithi149 ceph-mon[37173]: pgmap v345: 129 pgs: 129 active+clean; 587 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-07-27T03:13:48.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:48 smithi149 ceph-mon[37173]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:13:50.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.310+0000 7f2566c93640 1 -- 172.21.15.67:0/490652300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25601024d0 msgr2=0x7f25601028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:50.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.310+0000 7f2566c93640 1 --2- 172.21.15.67:0/490652300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25601024d0 0x7f25601028e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2554009900 tx=0x7f255402a410 comp rx=0 tx=0).stop 2024-07-27T03:13:50.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.310+0000 7f2566c93640 1 -- 172.21.15.67:0/490652300 shutdown_connections 2024-07-27T03:13:50.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.310+0000 7f2566c93640 1 --2- 172.21.15.67:0/490652300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2560108760 0x7f256010ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:50.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.310+0000 7f2566c93640 1 --2- 172.21.15.67:0/490652300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2560107e10 0x7f2560108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:50.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.310+0000 7f2566c93640 1 --2- 172.21.15.67:0/490652300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25601024d0 0x7f25601028e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:50.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.310+0000 7f2566c93640 1 -- 172.21.15.67:0/490652300 >> 172.21.15.67:0/490652300 conn(0x7f25600791a0 msgr2=0x7f25600795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:50.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.310+0000 7f2566c93640 1 -- 172.21.15.67:0/490652300 shutdown_connections 2024-07-27T03:13:50.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.310+0000 7f2566c93640 1 -- 172.21.15.67:0/490652300 wait complete. 2024-07-27T03:13:50.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.311+0000 7f2566c93640 1 Processor -- start 2024-07-27T03:13:50.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.311+0000 7f2566c93640 1 -- start start 2024-07-27T03:13:50.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.312+0000 7f2566c93640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25601024d0 0x7f256019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:50.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.312+0000 7f2566c93640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2560107e10 0x7f256019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:50.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.312+0000 7f2566c93640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2560108760 0x7f256019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:50.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.312+0000 7f2566c93640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2560112430 con 0x7f25601024d0 2024-07-27T03:13:50.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.312+0000 7f2566c93640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f25601122b0 con 0x7f2560107e10 2024-07-27T03:13:50.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.312+0000 7f2566c93640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f25601125b0 con 0x7f2560108760 2024-07-27T03:13:50.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.312+0000 7f2564a2f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25601024d0 0x7f256019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:50.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.312+0000 7f2564a2f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25601024d0 0x7f256019a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32808/0 (socket says 172.21.15.67:32808) 2024-07-27T03:13:50.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.312+0000 7f2564a2f640 1 -- 172.21.15.67:0/2586264928 learned_addr learned my addr 172.21.15.67:0/2586264928 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:13:50.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.312+0000 7f2565230640 1 --2- 172.21.15.67:0/2586264928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2560108760 0x7f256019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:50.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.312+0000 7f255ffff640 1 --2- 172.21.15.67:0/2586264928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2560107e10 0x7f256019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:50.317 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.312+0000 7f2564a2f640 1 -- 172.21.15.67:0/2586264928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2560108760 msgr2=0x7f256019f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:50.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.312+0000 7f2564a2f640 1 --2- 172.21.15.67:0/2586264928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2560108760 0x7f256019f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:50.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.312+0000 7f2564a2f640 1 -- 172.21.15.67:0/2586264928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2560107e10 msgr2=0x7f256019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:50.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.312+0000 7f2564a2f640 1 --2- 172.21.15.67:0/2586264928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2560107e10 0x7f256019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:50.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.312+0000 7f2564a2f640 1 -- 172.21.15.67:0/2586264928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256019ffa0 con 0x7f25601024d0 2024-07-27T03:13:50.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.312+0000 7f255ffff640 1 --2- 172.21.15.67:0/2586264928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2560107e10 0x7f256019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:13:50.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.312+0000 7f2565230640 1 --2- 172.21.15.67:0/2586264928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2560108760 0x7f256019f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:13:50.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.312+0000 7f2564a2f640 1 --2- 172.21.15.67:0/2586264928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25601024d0 0x7f256019a030 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2554005940 tx=0x7f25540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:50.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.312+0000 7f255dffb640 1 -- 172.21.15.67:0/2586264928 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255403f030 con 0x7f25601024d0 2024-07-27T03:13:50.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.313+0000 7f2566c93640 1 -- 172.21.15.67:0/2586264928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25601a0230 con 0x7f25601024d0 2024-07-27T03:13:50.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.313+0000 7f255dffb640 1 -- 172.21.15.67:0/2586264928 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2554003cb0 con 0x7f25601024d0 2024-07-27T03:13:50.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.313+0000 7f255dffb640 1 -- 172.21.15.67:0/2586264928 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2554035070 con 0x7f25601024d0 2024-07-27T03:13:50.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.313+0000 7f2566c93640 1 -- 172.21.15.67:0/2586264928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25601a69f0 con 0x7f25601024d0 2024-07-27T03:13:50.318 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.314+0000 7f2566c93640 1 -- 172.21.15.67:0/2586264928 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25601028e0 con 0x7f25601024d0 2024-07-27T03:13:50.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.315+0000 7f255dffb640 1 -- 172.21.15.67:0/2586264928 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f2554042030 con 0x7f25601024d0 2024-07-27T03:13:50.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.316+0000 7f255dffb640 1 --2- 172.21.15.67:0/2586264928 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f25500b3c60 0x7f25500b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:50.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.316+0000 7f255ffff640 1 --2- 172.21.15.67:0/2586264928 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f25500b3c60 0x7f25500b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:50.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.317+0000 7f255dffb640 1 -- 172.21.15.67:0/2586264928 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f2554104a40 con 0x7f25601024d0 2024-07-27T03:13:50.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.317+0000 7f255ffff640 1 --2- 172.21.15.67:0/2586264928 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f25500b3c60 0x7f25500b6170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f254c006fd0 tx=0x7f254c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:50.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.321+0000 7f255dffb640 1 -- 172.21.15.67:0/2586264928 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f25540cbbd0 con 0x7f25601024d0 2024-07-27T03:13:50.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:50 smithi067 ceph-mon[32814]: pgmap v346: 129 pgs: 129 active+clean; 587 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-07-27T03:13:50.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.533+0000 7f2566c93640 1 -- 172.21.15.67:0/2586264928 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f2560105db0 con 0x7f25500b3c60 2024-07-27T03:13:50.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.538+0000 7f255dffb640 1 -- 172.21.15.67:0/2586264928 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f2560105db0 con 0x7f25500b3c60 2024-07-27T03:13:50.541 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/qa/sv_0/4f21fe6d-dcab-40cc-9cd3-700ee904cb4e 2024-07-27T03:13:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.540+0000 7f2566c93640 1 -- 172.21.15.67:0/2586264928 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f25500b3c60 msgr2=0x7f25500b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.540+0000 7f2566c93640 1 --2- 172.21.15.67:0/2586264928 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f25500b3c60 0x7f25500b6170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f254c006fd0 tx=0x7f254c008040 comp rx=0 tx=0).stop 2024-07-27T03:13:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.540+0000 7f2566c93640 1 -- 172.21.15.67:0/2586264928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25601024d0 msgr2=0x7f256019a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.540+0000 7f2566c93640 1 --2- 172.21.15.67:0/2586264928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25601024d0 0x7f256019a030 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2554005940 tx=0x7f25540042f0 comp rx=0 tx=0).stop 2024-07-27T03:13:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.540+0000 7f2566c93640 1 -- 172.21.15.67:0/2586264928 shutdown_connections 2024-07-27T03:13:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.540+0000 7f2566c93640 1 --2- 172.21.15.67:0/2586264928 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f25500b3c60 0x7f25500b6170 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.540+0000 7f2566c93640 1 --2- 172.21.15.67:0/2586264928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2560108760 0x7f256019f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.540+0000 7f2566c93640 1 --2- 172.21.15.67:0/2586264928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2560107e10 0x7f256019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.540+0000 7f2566c93640 1 --2- 172.21.15.67:0/2586264928 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25601024d0 0x7f256019a030 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.540+0000 7f2566c93640 1 -- 172.21.15.67:0/2586264928 >> 172.21.15.67:0/2586264928 conn(0x7f25600791a0 msgr2=0x7f2560101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.540+0000 7f2566c93640 1 -- 172.21.15.67:0/2586264928 shutdown_connections 2024-07-27T03:13:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:50.540+0000 7f2566c93640 1 -- 172.21.15.67:0/2586264928 wait complete. 2024-07-27T03:13:50.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:50 smithi175 ceph-mon[37125]: pgmap v346: 129 pgs: 129 active+clean; 587 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-07-27T03:13:50.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:50 smithi149 ceph-mon[37173]: pgmap v346: 129 pgs: 129 active+clean; 587 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.2 KiB/s wr, 20 op/s 2024-07-27T03:13:51.107 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-07-27T03:13:51.375 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:13:51.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:51 smithi067 ceph-mon[32814]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:13:51.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:51 smithi175 ceph-mon[37125]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:13:51.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:51 smithi149 ceph-mon[37173]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:13:52.583 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:52 smithi067 ceph-mon[32814]: pgmap v347: 129 pgs: 129 active+clean; 608 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.2 KiB/s wr, 23 op/s 2024-07-27T03:13:52.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:52 smithi175 ceph-mon[37125]: pgmap v347: 129 pgs: 129 active+clean; 608 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.2 KiB/s wr, 23 op/s 2024-07-27T03:13:52.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:52 smithi149 ceph-mon[37173]: pgmap v347: 129 pgs: 129 active+clean; 608 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.2 KiB/s wr, 23 op/s 2024-07-27T03:13:53.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.002+0000 7f808a2a5640 1 -- 172.21.15.67:0/668409428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8084108770 msgr2=0x7f808410efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:53.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.002+0000 7f808a2a5640 1 --2- 172.21.15.67:0/668409428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8084108770 0x7f808410efa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f807c009ca0 tx=0x7f807c02a8c0 comp rx=0 tx=0).stop 2024-07-27T03:13:53.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.002+0000 7f808a2a5640 1 -- 172.21.15.67:0/668409428 shutdown_connections 2024-07-27T03:13:53.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.002+0000 7f808a2a5640 1 --2- 172.21.15.67:0/668409428 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8084108770 0x7f808410efa0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:53.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.002+0000 7f808a2a5640 1 --2- 172.21.15.67:0/668409428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8084107e20 0x7f8084108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:53.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.002+0000 7f808a2a5640 1 --2- 172.21.15.67:0/668409428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80841024e0 0x7f80841028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:53.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.002+0000 7f808a2a5640 1 -- 172.21.15.67:0/668409428 >> 172.21.15.67:0/668409428 conn(0x7f80840791a0 msgr2=0x7f80840795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:53.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.002+0000 7f808a2a5640 1 -- 172.21.15.67:0/668409428 shutdown_connections 2024-07-27T03:13:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.002+0000 7f808a2a5640 1 -- 172.21.15.67:0/668409428 wait complete. 2024-07-27T03:13:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.003+0000 7f808a2a5640 1 Processor -- start 2024-07-27T03:13:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.003+0000 7f808a2a5640 1 -- start start 2024-07-27T03:13:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.003+0000 7f808a2a5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80841024e0 0x7f8084198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.003+0000 7f808a2a5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8084107e20 0x7f8084199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.003+0000 7f808a2a5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8084108770 0x7f808419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.003+0000 7f808a2a5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f80841123f0 con 0x7f8084107e20 2024-07-27T03:13:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.003+0000 7f808a2a5640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8084112270 con 0x7f8084108770 2024-07-27T03:13:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.003+0000 7f808a2a5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8084112570 con 0x7f80841024e0 2024-07-27T03:13:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.003+0000 7f80837fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8084107e20 0x7f8084199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.003+0000 7f80837fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8084107e20 0x7f8084199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:32824/0 (socket says 172.21.15.67:32824) 2024-07-27T03:13:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.003+0000 7f80837fe640 1 -- 172.21.15.67:0/1896316095 learned_addr learned my addr 172.21.15.67:0/1896316095 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:13:53.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.003+0000 7f8088842640 1 --2- 172.21.15.67:0/1896316095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8084108770 0x7f808419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.003+0000 7f8083fff640 1 --2- 172.21.15.67:0/1896316095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80841024e0 0x7f8084198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.004+0000 7f80837fe640 1 -- 172.21.15.67:0/1896316095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80841024e0 msgr2=0x7f8084198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.004+0000 7f80837fe640 1 --2- 172.21.15.67:0/1896316095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80841024e0 0x7f8084198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.004+0000 7f80837fe640 1 -- 172.21.15.67:0/1896316095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8084108770 msgr2=0x7f808419e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.004+0000 7f80837fe640 1 --2- 172.21.15.67:0/1896316095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8084108770 0x7f808419e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.004+0000 7f80837fe640 1 -- 172.21.15.67:0/1896316095 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808419ed10 con 0x7f8084107e20 2024-07-27T03:13:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.004+0000 7f8083fff640 1 --2- 172.21.15.67:0/1896316095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80841024e0 0x7f8084198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:13:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.004+0000 7f80837fe640 1 --2- 172.21.15.67:0/1896316095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8084107e20 0x7f8084199340 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8078009860 tx=0x7f8078009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.004+0000 7f80817fa640 1 -- 172.21.15.67:0/1896316095 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807801d070 con 0x7f8084107e20 2024-07-27T03:13:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.004+0000 7f80817fa640 1 -- 172.21.15.67:0/1896316095 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8078007c50 con 0x7f8084107e20 2024-07-27T03:13:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.004+0000 7f808a2a5640 1 -- 172.21.15.67:0/1896316095 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808419f000 con 0x7f8084107e20 2024-07-27T03:13:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.004+0000 7f80817fa640 1 -- 172.21.15.67:0/1896316095 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078018d30 con 0x7f8084107e20 2024-07-27T03:13:53.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.004+0000 7f808a2a5640 1 -- 172.21.15.67:0/1896316095 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f808406b550 con 0x7f8084107e20 2024-07-27T03:13:53.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.005+0000 7f808a2a5640 1 -- 172.21.15.67:0/1896316095 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8084103760 con 0x7f8084107e20 2024-07-27T03:13:53.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.007+0000 7f80817fa640 1 -- 172.21.15.67:0/1896316095 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f8078020070 con 0x7f8084107e20 2024-07-27T03:13:53.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.008+0000 7f80817fa640 1 --2- 172.21.15.67:0/1896316095 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f806c0b3c60 0x7f806c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:53.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.008+0000 7f8083fff640 1 --2- 172.21.15.67:0/1896316095 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f806c0b3c60 0x7f806c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:53.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.008+0000 7f80817fa640 1 -- 172.21.15.67:0/1896316095 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f80780e5310 con 0x7f8084107e20 2024-07-27T03:13:53.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.008+0000 7f8083fff640 1 --2- 172.21.15.67:0/1896316095 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f806c0b3c60 0x7f806c0b6170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8074005fd0 tx=0x7f8074005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:53.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.012+0000 7f80817fa640 1 -- 172.21.15.67:0/1896316095 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f8078007e00 con 0x7f8084107e20 2024-07-27T03:13:53.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.222+0000 7f808a2a5640 1 -- 172.21.15.67:0/1896316095 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f808406c000 con 0x7f806c0b3c60 2024-07-27T03:13:53.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.247+0000 7f80817fa640 1 -- 172.21.15.67:0/1896316095 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f808406c000 con 0x7f806c0b3c60 2024-07-27T03:13:53.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.249+0000 7f808a2a5640 1 -- 172.21.15.67:0/1896316095 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f806c0b3c60 msgr2=0x7f806c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:53.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.250+0000 7f808a2a5640 1 --2- 172.21.15.67:0/1896316095 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f806c0b3c60 0x7f806c0b6170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8074005fd0 tx=0x7f8074005ea0 comp rx=0 tx=0).stop 2024-07-27T03:13:53.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.250+0000 7f808a2a5640 1 -- 172.21.15.67:0/1896316095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8084107e20 msgr2=0x7f8084199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:53.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.250+0000 7f808a2a5640 1 --2- 172.21.15.67:0/1896316095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8084107e20 0x7f8084199340 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8078009860 tx=0x7f8078009d20 comp rx=0 tx=0).stop 2024-07-27T03:13:53.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.250+0000 7f808a2a5640 1 -- 172.21.15.67:0/1896316095 shutdown_connections 2024-07-27T03:13:53.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.250+0000 7f808a2a5640 1 --2- 172.21.15.67:0/1896316095 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f806c0b3c60 0x7f806c0b6170 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:53.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.250+0000 7f808a2a5640 1 --2- 172.21.15.67:0/1896316095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8084108770 0x7f808419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:53.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.250+0000 7f808a2a5640 1 --2- 172.21.15.67:0/1896316095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8084107e20 0x7f8084199340 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:53.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.250+0000 7f808a2a5640 1 --2- 172.21.15.67:0/1896316095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80841024e0 0x7f8084198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:53.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.250+0000 7f808a2a5640 1 -- 172.21.15.67:0/1896316095 >> 172.21.15.67:0/1896316095 conn(0x7f80840791a0 msgr2=0x7f8084104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:53.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.250+0000 7f808a2a5640 1 -- 172.21.15.67:0/1896316095 shutdown_connections 2024-07-27T03:13:53.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:53.250+0000 7f808a2a5640 1 -- 172.21.15.67:0/1896316095 wait complete. 2024-07-27T03:13:53.892 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-07-27T03:13:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:13:54.535 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:54 smithi067 ceph-mon[32814]: pgmap v348: 129 pgs: 129 active+clean; 608 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.2 KiB/s wr, 15 op/s 2024-07-27T03:13:54.536 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:54 smithi067 ceph-mon[32814]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:13:54.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:54 smithi175 ceph-mon[37125]: pgmap v348: 129 pgs: 129 active+clean; 608 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.2 KiB/s wr, 15 op/s 2024-07-27T03:13:54.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:54 smithi175 ceph-mon[37125]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:13:54.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:54 smithi149 ceph-mon[37173]: pgmap v348: 129 pgs: 129 active+clean; 608 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.2 KiB/s wr, 15 op/s 2024-07-27T03:13:54.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:54 smithi149 ceph-mon[37173]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:13:55.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.878+0000 7f4321e1f640 1 -- 172.21.15.67:0/2927344099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f431c1024d0 msgr2=0x7f431c1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:55.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.878+0000 7f4321e1f640 1 --2- 172.21.15.67:0/2927344099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f431c1024d0 0x7f431c1028e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f430c009900 tx=0x7f430c02a410 comp rx=0 tx=0).stop 2024-07-27T03:13:55.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.878+0000 7f4321e1f640 1 -- 172.21.15.67:0/2927344099 shutdown_connections 2024-07-27T03:13:55.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.878+0000 7f4321e1f640 1 --2- 172.21.15.67:0/2927344099 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f431c108760 0x7f431c10ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:55.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.878+0000 7f4321e1f640 1 --2- 172.21.15.67:0/2927344099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c107e10 0x7f431c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:55.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.878+0000 7f4321e1f640 1 --2- 172.21.15.67:0/2927344099 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f431c1024d0 0x7f431c1028e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:55.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.878+0000 7f4321e1f640 1 -- 172.21.15.67:0/2927344099 >> 172.21.15.67:0/2927344099 conn(0x7f431c0791a0 msgr2=0x7f431c0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:55.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.878+0000 7f4321e1f640 1 -- 172.21.15.67:0/2927344099 shutdown_connections 2024-07-27T03:13:55.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.879+0000 7f4321e1f640 1 -- 172.21.15.67:0/2927344099 wait complete. 2024-07-27T03:13:55.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.879+0000 7f4321e1f640 1 Processor -- start 2024-07-27T03:13:55.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.880+0000 7f4321e1f640 1 -- start start 2024-07-27T03:13:55.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.880+0000 7f4321e1f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f431c1024d0 0x7f431c199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:55.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.880+0000 7f4321e1f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c107e10 0x7f431c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:55.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.880+0000 7f4321e1f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f431c108760 0x7f431c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:55.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.880+0000 7f4321e1f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f431c113530 con 0x7f431c108760 2024-07-27T03:13:55.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.880+0000 7f4321e1f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1133b0 con 0x7f431c107e10 2024-07-27T03:13:55.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.880+0000 7f4321e1f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1136b0 con 0x7f431c1024d0 2024-07-27T03:13:55.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.880+0000 7f431bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f431c108760 0x7f431c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:55.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.880+0000 7f431bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f431c108760 0x7f431c19f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39350/0 (socket says 172.21.15.67:39350) 2024-07-27T03:13:55.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.880+0000 7f431bfff640 1 -- 172.21.15.67:0/3841774591 learned_addr learned my addr 172.21.15.67:0/3841774591 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:13:55.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.880+0000 7f431affd640 1 --2- 172.21.15.67:0/3841774591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c107e10 0x7f431c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:55.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.880+0000 7f431b7fe640 1 --2- 172.21.15.67:0/3841774591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f431c1024d0 0x7f431c199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:55.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.880+0000 7f431bfff640 1 -- 172.21.15.67:0/3841774591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f431c1024d0 msgr2=0x7f431c199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:55.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.880+0000 7f431bfff640 1 --2- 172.21.15.67:0/3841774591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f431c1024d0 0x7f431c199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:55.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.880+0000 7f431bfff640 1 -- 172.21.15.67:0/3841774591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c107e10 msgr2=0x7f431c19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:55.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.881+0000 7f431bfff640 1 --2- 172.21.15.67:0/3841774591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c107e10 0x7f431c19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:55.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.881+0000 7f431bfff640 1 -- 172.21.15.67:0/3841774591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c19ff10 con 0x7f431c108760 2024-07-27T03:13:55.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.881+0000 7f431bfff640 1 --2- 172.21.15.67:0/3841774591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f431c108760 0x7f431c19f7e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f431400b970 tx=0x7f431400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:55.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.881+0000 7f4318ff9640 1 -- 172.21.15.67:0/3841774591 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314019070 con 0x7f431c108760 2024-07-27T03:13:55.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.881+0000 7f4318ff9640 1 -- 172.21.15.67:0/3841774591 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4314014440 con 0x7f431c108760 2024-07-27T03:13:55.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.881+0000 7f4321e1f640 1 -- 172.21.15.67:0/3841774591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431c1a0200 con 0x7f431c108760 2024-07-27T03:13:55.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.881+0000 7f4318ff9640 1 -- 172.21.15.67:0/3841774591 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43140075d0 con 0x7f431c108760 2024-07-27T03:13:55.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.881+0000 7f4321e1f640 1 -- 172.21.15.67:0/3841774591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f431c1a6960 con 0x7f431c108760 2024-07-27T03:13:55.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.882+0000 7f4321e1f640 1 -- 172.21.15.67:0/3841774591 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f431c1028e0 con 0x7f431c108760 2024-07-27T03:13:55.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.883+0000 7f4318ff9640 1 -- 172.21.15.67:0/3841774591 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f431401c070 con 0x7f431c108760 2024-07-27T03:13:55.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.884+0000 7f4318ff9640 1 --2- 172.21.15.67:0/3841774591 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f43040b3ce0 0x7f43040b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:55.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.884+0000 7f431b7fe640 1 --2- 172.21.15.67:0/3841774591 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f43040b3ce0 0x7f43040b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:55.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.884+0000 7f4318ff9640 1 -- 172.21.15.67:0/3841774591 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9092+0+0 (secure 0 0 0) 0x7f43140e3550 con 0x7f431c108760 2024-07-27T03:13:55.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.885+0000 7f431b7fe640 1 --2- 172.21.15.67:0/3841774591 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f43040b3ce0 0x7f43040b61f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f430c031440 tx=0x7f430c005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:55.891 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:55.888+0000 7f4318ff9640 1 -- 172.21.15.67:0/3841774591 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f43140aa1a0 con 0x7f431c108760 2024-07-27T03:13:56.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:56.101+0000 7f4321e1f640 1 -- 172.21.15.67:0/3841774591 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f431c063270 con 0x7f43040b3ce0 2024-07-27T03:13:56.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:56.106+0000 7f4318ff9640 1 -- 172.21.15.67:0/3841774591 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f431c063270 con 0x7f43040b3ce0 2024-07-27T03:13:56.110 INFO:teuthology.orchestra.run.smithi067.stdout:/volumes/qa/sv_1/15d1cbe2-ea58-4f14-900a-340b84819fc4 2024-07-27T03:13:56.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:56.108+0000 7f4321e1f640 1 -- 172.21.15.67:0/3841774591 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f43040b3ce0 msgr2=0x7f43040b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:56.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:56.108+0000 7f4321e1f640 1 --2- 172.21.15.67:0/3841774591 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f43040b3ce0 0x7f43040b61f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f430c031440 tx=0x7f430c005d50 comp rx=0 tx=0).stop 2024-07-27T03:13:56.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:56.108+0000 7f4321e1f640 1 -- 172.21.15.67:0/3841774591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f431c108760 msgr2=0x7f431c19f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:56.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:56.108+0000 7f4321e1f640 1 --2- 172.21.15.67:0/3841774591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f431c108760 0x7f431c19f7e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f431400b970 tx=0x7f431400be30 comp rx=0 tx=0).stop 2024-07-27T03:13:56.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:56.108+0000 7f4321e1f640 1 -- 172.21.15.67:0/3841774591 shutdown_connections 2024-07-27T03:13:56.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:56.108+0000 7f4321e1f640 1 --2- 172.21.15.67:0/3841774591 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f43040b3ce0 0x7f43040b61f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:56.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:56.108+0000 7f4321e1f640 1 --2- 172.21.15.67:0/3841774591 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f431c108760 0x7f431c19f7e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:56.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:56.108+0000 7f4321e1f640 1 --2- 172.21.15.67:0/3841774591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f431c107e10 0x7f431c19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:56.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:56.108+0000 7f4321e1f640 1 --2- 172.21.15.67:0/3841774591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f431c1024d0 0x7f431c199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:56.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:56.108+0000 7f4321e1f640 1 -- 172.21.15.67:0/3841774591 >> 172.21.15.67:0/3841774591 conn(0x7f431c0791a0 msgr2=0x7f431c100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:56.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:56.109+0000 7f4321e1f640 1 -- 172.21.15.67:0/3841774591 shutdown_connections 2024-07-27T03:13:56.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:56.109+0000 7f4321e1f640 1 -- 172.21.15.67:0/3841774591 wait complete. 2024-07-27T03:13:56.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:56 smithi067 ceph-mon[32814]: pgmap v349: 129 pgs: 129 active+clean; 627 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 KiB/s wr, 15 op/s 2024-07-27T03:13:56.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:56 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-07-27T03:13:56.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:56 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-07-27T03:13:56.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:56 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 8]}]: dispatch 2024-07-27T03:13:56.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:56 smithi175 ceph-mon[37125]: pgmap v349: 129 pgs: 129 active+clean; 627 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 KiB/s wr, 15 op/s 2024-07-27T03:13:56.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:56 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-07-27T03:13:56.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:56 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-07-27T03:13:56.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:56 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 8]}]: dispatch 2024-07-27T03:13:56.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:56 smithi149 ceph-mon[37173]: pgmap v349: 129 pgs: 129 active+clean; 627 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 KiB/s wr, 15 op/s 2024-07-27T03:13:56.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:56 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-07-27T03:13:56.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:56 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-07-27T03:13:56.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:56 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 8]}]: dispatch 2024-07-27T03:13:56.926 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:13:57.200 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:13:57.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:57 smithi067 ceph-mon[32814]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:13:57.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-07-27T03:13:57.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-07-27T03:13:57.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:57 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 8]}]': finished 2024-07-27T03:13:57.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:57 smithi067 ceph-mon[32814]: osdmap e90: 12 total, 12 up, 12 in 2024-07-27T03:13:57.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:57 smithi175 ceph-mon[37125]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:13:57.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-07-27T03:13:57.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-07-27T03:13:57.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:57 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 8]}]': finished 2024-07-27T03:13:57.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:57 smithi175 ceph-mon[37125]: osdmap e90: 12 total, 12 up, 12 in 2024-07-27T03:13:57.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:57 smithi149 ceph-mon[37173]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:13:57.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-07-27T03:13:57.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-07-27T03:13:57.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:57 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 8]}]': finished 2024-07-27T03:13:57.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:57 smithi149 ceph-mon[37173]: osdmap e90: 12 total, 12 up, 12 in 2024-07-27T03:13:58.582 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:58 smithi067 ceph-mon[32814]: pgmap v351: 129 pgs: 129 active+clean; 627 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-07-27T03:13:58.582 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:58 smithi067 ceph-mon[32814]: osdmap e91: 12 total, 12 up, 12 in 2024-07-27T03:13:58.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:58 smithi175 ceph-mon[37125]: pgmap v351: 129 pgs: 129 active+clean; 627 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-07-27T03:13:58.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:58 smithi175 ceph-mon[37125]: osdmap e91: 12 total, 12 up, 12 in 2024-07-27T03:13:58.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:58 smithi149 ceph-mon[37173]: pgmap v351: 129 pgs: 129 active+clean; 627 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-07-27T03:13:58.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:58 smithi149 ceph-mon[37173]: osdmap e91: 12 total, 12 up, 12 in 2024-07-27T03:13:58.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.807+0000 7fcd496a6640 1 -- 172.21.15.67:0/2678195355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd441024d0 msgr2=0x7fcd441028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:58.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.807+0000 7fcd496a6640 1 --2- 172.21.15.67:0/2678195355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd441024d0 0x7fcd441028e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009ca0 tx=0x7fcd3c02a8c0 comp rx=0 tx=0).stop 2024-07-27T03:13:58.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.807+0000 7fcd496a6640 1 -- 172.21.15.67:0/2678195355 shutdown_connections 2024-07-27T03:13:58.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.807+0000 7fcd496a6640 1 --2- 172.21.15.67:0/2678195355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd44108760 0x7fcd4410ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:58.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.807+0000 7fcd496a6640 1 --2- 172.21.15.67:0/2678195355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd44107e10 0x7fcd44108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:58.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.807+0000 7fcd496a6640 1 --2- 172.21.15.67:0/2678195355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd441024d0 0x7fcd441028e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:58.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.807+0000 7fcd496a6640 1 -- 172.21.15.67:0/2678195355 >> 172.21.15.67:0/2678195355 conn(0x7fcd440791a0 msgr2=0x7fcd440795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:58.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.807+0000 7fcd496a6640 1 -- 172.21.15.67:0/2678195355 shutdown_connections 2024-07-27T03:13:58.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.807+0000 7fcd496a6640 1 -- 172.21.15.67:0/2678195355 wait complete. 2024-07-27T03:13:58.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.808+0000 7fcd496a6640 1 Processor -- start 2024-07-27T03:13:58.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.808+0000 7fcd496a6640 1 -- start start 2024-07-27T03:13:58.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.808+0000 7fcd496a6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd441024d0 0x7fcd44103cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:58.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.808+0000 7fcd496a6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd44107e10 0x7fcd44104200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:58.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.808+0000 7fcd496a6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd44108760 0x7fcd44104740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:58.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.808+0000 7fcd496a6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcd441123e0 con 0x7fcd44107e10 2024-07-27T03:13:58.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.808+0000 7fcd496a6640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcd44112260 con 0x7fcd44108760 2024-07-27T03:13:58.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.808+0000 7fcd496a6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcd44112560 con 0x7fcd441024d0 2024-07-27T03:13:58.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.809+0000 7fcd427fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd44107e10 0x7fcd44104200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:58.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.809+0000 7fcd427fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd44107e10 0x7fcd44104200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39372/0 (socket says 172.21.15.67:39372) 2024-07-27T03:13:58.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.809+0000 7fcd427fc640 1 -- 172.21.15.67:0/1154362407 learned_addr learned my addr 172.21.15.67:0/1154362407 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:13:58.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.809+0000 7fcd42ffd640 1 --2- 172.21.15.67:0/1154362407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd441024d0 0x7fcd44103cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:58.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.809+0000 7fcd427fc640 1 -- 172.21.15.67:0/1154362407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd441024d0 msgr2=0x7fcd44103cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:58.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.809+0000 7fcd427fc640 1 --2- 172.21.15.67:0/1154362407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd441024d0 0x7fcd44103cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:58.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.809+0000 7fcd427fc640 1 -- 172.21.15.67:0/1154362407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd44108760 msgr2=0x7fcd44104740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:58.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.809+0000 7fcd437fe640 1 --2- 172.21.15.67:0/1154362407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd44108760 0x7fcd44104740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:58.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.809+0000 7fcd427fc640 1 --2- 172.21.15.67:0/1154362407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd44108760 0x7fcd44104740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:58.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.809+0000 7fcd427fc640 1 -- 172.21.15.67:0/1154362407 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd441a8180 con 0x7fcd44107e10 2024-07-27T03:13:58.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.809+0000 7fcd42ffd640 1 --2- 172.21.15.67:0/1154362407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd441024d0 0x7fcd44103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:13:58.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.809+0000 7fcd437fe640 1 --2- 172.21.15.67:0/1154362407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd44108760 0x7fcd44104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:13:58.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.809+0000 7fcd427fc640 1 --2- 172.21.15.67:0/1154362407 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd44107e10 0x7fcd44104200 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00d740 tx=0x7fcd2c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:58.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.809+0000 7fcd3bfff640 1 -- 172.21.15.67:0/1154362407 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c017070 con 0x7fcd44107e10 2024-07-27T03:13:58.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.809+0000 7fcd3bfff640 1 -- 172.21.15.67:0/1154362407 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd2c00de50 con 0x7fcd44107e10 2024-07-27T03:13:58.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.809+0000 7fcd496a6640 1 -- 172.21.15.67:0/1154362407 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd441a8470 con 0x7fcd44107e10 2024-07-27T03:13:58.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.809+0000 7fcd3bfff640 1 -- 172.21.15.67:0/1154362407 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c004ce0 con 0x7fcd44107e10 2024-07-27T03:13:58.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.809+0000 7fcd496a6640 1 -- 172.21.15.67:0/1154362407 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd441a8960 con 0x7fcd44107e10 2024-07-27T03:13:58.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.810+0000 7fcd496a6640 1 -- 172.21.15.67:0/1154362407 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd441028e0 con 0x7fcd44107e10 2024-07-27T03:13:58.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.812+0000 7fcd3bfff640 1 -- 172.21.15.67:0/1154362407 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7fcd2c004e80 con 0x7fcd44107e10 2024-07-27T03:13:58.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.813+0000 7fcd3bfff640 1 --2- 172.21.15.67:0/1154362407 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcd300b3c60 0x7fcd300b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:13:58.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.813+0000 7fcd3bfff640 1 -- 172.21.15.67:0/1154362407 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fcd2c0e3fc0 con 0x7fcd44107e10 2024-07-27T03:13:58.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.813+0000 7fcd42ffd640 1 --2- 172.21.15.67:0/1154362407 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcd300b3c60 0x7fcd300b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:13:58.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.813+0000 7fcd42ffd640 1 --2- 172.21.15.67:0/1154362407 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcd300b3c60 0x7fcd300b6170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c0094e0 tx=0x7fcd3c0093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:13:58.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:58.816+0000 7fcd3bfff640 1 -- 172.21.15.67:0/1154362407 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fcd2c0ab100 con 0x7fcd44107e10 2024-07-27T03:13:59.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:59.035+0000 7fcd496a6640 1 -- 172.21.15.67:0/1154362407 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd441a8cd0 con 0x7fcd44107e10 2024-07-27T03:13:59.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:59.036+0000 7fcd3bfff640 1 -- 172.21.15.67:0/1154362407 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11983 (secure 0 0 0) 0x7fcd2c0aaa00 con 0x7fcd44107e10 2024-07-27T03:13:59.039 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:13:59.039 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":29,"btime":"2024-07-27T03:13:42:247804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14679,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/252686369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":252686369},{"type":"v1","addr":"172.21.15.67:6835","nonce":252686369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/3358832762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3358832762},{"type":"v1","addr":"172.21.15.67:6837","nonce":3358832762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-27T03:12:34.320208+0000","modified":"2024-07-27T03:13:42.062021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24620,"mds_1":24587,"mds_2":24583,"mds_3":24581,"mds_4":24563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14736":{"gid":14736,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/2162708317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2162708317},{"type":"v1","addr":"172.21.15.67:6839","nonce":2162708317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14748":{"gid":14748,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/795704244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":795704244},{"type":"v1","addr":"172.21.15.67:6841","nonce":795704244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/3868325215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3868325215},{"type":"v1","addr":"172.21.15.149:6833","nonce":3868325215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2462749027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2462749027},{"type":"v1","addr":"172.21.15.175:6835","nonce":2462749027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.149:6835/3730964806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3730964806},{"type":"v1","addr":"172.21.15.149:6835","nonce":3730964806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.175:6837/1304803502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1304803502},{"type":"v1","addr":"172.21.15.175:6837","nonce":1304803502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.175:6839/1670568853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1670568853},{"type":"v1","addr":"172.21.15.175:6839","nonce":1670568853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24587":{"gid":24587,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.149:6837/2932951845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2932951845},{"type":"v1","addr":"172.21.15.149:6837","nonce":2932951845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.149:6839/281746498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":281746498},{"type":"v1","addr":"172.21.15.149:6839","nonce":281746498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24620,"qdb_cluster":[24563,24581,24620,24583,24587]},"id":1}]} 2024-07-27T03:13:59.039 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-07-27T03:13:59.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:59.038+0000 7fcd496a6640 1 -- 172.21.15.67:0/1154362407 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcd300b3c60 msgr2=0x7fcd300b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:59.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:59.038+0000 7fcd496a6640 1 --2- 172.21.15.67:0/1154362407 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcd300b3c60 0x7fcd300b6170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c0094e0 tx=0x7fcd3c0093f0 comp rx=0 tx=0).stop 2024-07-27T03:13:59.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:59.038+0000 7fcd496a6640 1 -- 172.21.15.67:0/1154362407 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd44107e10 msgr2=0x7fcd44104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:13:59.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:59.038+0000 7fcd496a6640 1 --2- 172.21.15.67:0/1154362407 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd44107e10 0x7fcd44104200 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00d740 tx=0x7fcd2c00dc00 comp rx=0 tx=0).stop 2024-07-27T03:13:59.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:59.038+0000 7fcd496a6640 1 -- 172.21.15.67:0/1154362407 shutdown_connections 2024-07-27T03:13:59.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:59.038+0000 7fcd496a6640 1 --2- 172.21.15.67:0/1154362407 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcd300b3c60 0x7fcd300b6170 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:59.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:59.038+0000 7fcd496a6640 1 --2- 172.21.15.67:0/1154362407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd44108760 0x7fcd44104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:59.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:59.038+0000 7fcd496a6640 1 --2- 172.21.15.67:0/1154362407 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcd44107e10 0x7fcd44104200 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:59.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:59.038+0000 7fcd496a6640 1 --2- 172.21.15.67:0/1154362407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd441024d0 0x7fcd44103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:13:59.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:59.038+0000 7fcd496a6640 1 -- 172.21.15.67:0/1154362407 >> 172.21.15.67:0/1154362407 conn(0x7fcd440791a0 msgr2=0x7fcd44101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:13:59.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:59.038+0000 7fcd496a6640 1 -- 172.21.15.67:0/1154362407 shutdown_connections 2024-07-27T03:13:59.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:13:59.038+0000 7fcd496a6640 1 -- 172.21.15.67:0/1154362407 wait complete. 2024-07-27T03:13:59.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:13:59 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1154362407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:13:59.636 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:13:59 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1154362407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:13:59.664 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd dump --format=json 2024-07-27T03:13:59.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:13:59 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1154362407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:13:59.931 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:00.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:00 smithi175 ceph-mon[37125]: pgmap v353: 129 pgs: 129 active+clean; 627 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.5 KiB/s wr, 10 op/s 2024-07-27T03:14:00.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:00 smithi067 ceph-mon[32814]: pgmap v353: 129 pgs: 129 active+clean; 627 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.5 KiB/s wr, 10 op/s 2024-07-27T03:14:00.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:00 smithi149 ceph-mon[37173]: pgmap v353: 129 pgs: 129 active+clean; 627 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.5 KiB/s wr, 10 op/s 2024-07-27T03:14:01.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.579+0000 7fad9ef8f640 1 -- 172.21.15.67:0/3196660520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad98108760 msgr2=0x7fad9810ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:01.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.579+0000 7fad9ef8f640 1 --2- 172.21.15.67:0/3196660520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad98108760 0x7fad9810ef90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fad90009ca0 tx=0x7fad9002a8c0 comp rx=0 tx=0).stop 2024-07-27T03:14:01.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.579+0000 7fad9ef8f640 1 -- 172.21.15.67:0/3196660520 shutdown_connections 2024-07-27T03:14:01.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.579+0000 7fad9ef8f640 1 --2- 172.21.15.67:0/3196660520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad98108760 0x7fad9810ef90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:01.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.579+0000 7fad9ef8f640 1 --2- 172.21.15.67:0/3196660520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad98107e10 0x7fad98108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:01.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.579+0000 7fad9ef8f640 1 --2- 172.21.15.67:0/3196660520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad981024d0 0x7fad981028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:01.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.579+0000 7fad9ef8f640 1 -- 172.21.15.67:0/3196660520 >> 172.21.15.67:0/3196660520 conn(0x7fad980791a0 msgr2=0x7fad980795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:01.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.579+0000 7fad9ef8f640 1 -- 172.21.15.67:0/3196660520 shutdown_connections 2024-07-27T03:14:01.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.579+0000 7fad9ef8f640 1 -- 172.21.15.67:0/3196660520 wait complete. 2024-07-27T03:14:01.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.580+0000 7fad9ef8f640 1 Processor -- start 2024-07-27T03:14:01.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.580+0000 7fad9ef8f640 1 -- start start 2024-07-27T03:14:01.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.580+0000 7fad9ef8f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad981024d0 0x7fad98100af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:01.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.580+0000 7fad9ef8f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad98107e10 0x7fad980ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:01.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.580+0000 7fad9ef8f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad98108760 0x7fad980ff620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:01.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.580+0000 7fad9ef8f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fad981123e0 con 0x7fad981024d0 2024-07-27T03:14:01.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.580+0000 7fad9ef8f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fad98112260 con 0x7fad98108760 2024-07-27T03:14:01.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.580+0000 7fad9ef8f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fad98112560 con 0x7fad98107e10 2024-07-27T03:14:01.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.580+0000 7fad9cd2b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad981024d0 0x7fad98100af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:01.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.580+0000 7fad9cd2b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad981024d0 0x7fad98100af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39386/0 (socket says 172.21.15.67:39386) 2024-07-27T03:14:01.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.580+0000 7fad9cd2b640 1 -- 172.21.15.67:0/3927023085 learned_addr learned my addr 172.21.15.67:0/3927023085 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:14:01.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.580+0000 7fad9d52c640 1 --2- 172.21.15.67:0/3927023085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad98108760 0x7fad980ff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:01.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.580+0000 7fad9cd2b640 1 -- 172.21.15.67:0/3927023085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad98107e10 msgr2=0x7fad980ff0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:01.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.580+0000 7fad9cd2b640 1 --2- 172.21.15.67:0/3927023085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad98107e10 0x7fad980ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:01.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.580+0000 7fad9cd2b640 1 -- 172.21.15.67:0/3927023085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad98108760 msgr2=0x7fad980ff620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:01.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.580+0000 7fad9cd2b640 1 --2- 172.21.15.67:0/3927023085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad98108760 0x7fad980ff620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:01.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.580+0000 7fad9cd2b640 1 -- 172.21.15.67:0/3927023085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad980ffe40 con 0x7fad981024d0 2024-07-27T03:14:01.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.581+0000 7fad9cd2b640 1 --2- 172.21.15.67:0/3927023085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad981024d0 0x7fad98100af0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fad8800d770 tx=0x7fad8800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:01.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.581+0000 7fad95ffb640 1 -- 172.21.15.67:0/3927023085 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad88017070 con 0x7fad981024d0 2024-07-27T03:14:01.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.581+0000 7fad95ffb640 1 -- 172.21.15.67:0/3927023085 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad880054d0 con 0x7fad981024d0 2024-07-27T03:14:01.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.581+0000 7fad9ef8f640 1 -- 172.21.15.67:0/3927023085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad981ac8e0 con 0x7fad981024d0 2024-07-27T03:14:01.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.581+0000 7fad95ffb640 1 -- 172.21.15.67:0/3927023085 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad88004c30 con 0x7fad981024d0 2024-07-27T03:14:01.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.581+0000 7fad9ef8f640 1 -- 172.21.15.67:0/3927023085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad981acdd0 con 0x7fad981024d0 2024-07-27T03:14:01.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.582+0000 7fad9ef8f640 1 -- 172.21.15.67:0/3927023085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad981028e0 con 0x7fad981024d0 2024-07-27T03:14:01.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.584+0000 7fad95ffb640 1 -- 172.21.15.67:0/3927023085 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7fad88004e10 con 0x7fad981024d0 2024-07-27T03:14:01.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.585+0000 7fad95ffb640 1 --2- 172.21.15.67:0/3927023085 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fad800b3c60 0x7fad800b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:01.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.585+0000 7fad97fff640 1 --2- 172.21.15.67:0/3927023085 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fad800b3c60 0x7fad800b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:01.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.585+0000 7fad95ffb640 1 -- 172.21.15.67:0/3927023085 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fad880e3d00 con 0x7fad981024d0 2024-07-27T03:14:01.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.585+0000 7fad97fff640 1 --2- 172.21.15.67:0/3927023085 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fad800b3c60 0x7fad800b6170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fad981059f0 tx=0x7fad8c005b60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:01.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.589+0000 7fad95ffb640 1 -- 172.21.15.67:0/3927023085 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fad880aae40 con 0x7fad981024d0 2024-07-27T03:14:01.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.771+0000 7fad9ef8f640 1 -- 172.21.15.67:0/3927023085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad981ad1e0 con 0x7fad981024d0 2024-07-27T03:14:01.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.772+0000 7fad95ffb640 1 -- 172.21.15.67:0/3927023085 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24829 (secure 0 0 0) 0x7fad880aa740 con 0x7fad981024d0 2024-07-27T03:14:01.776 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:14:01.776 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":91,"fsid":"c0782a00-4bc4-11ef-bcc7-c7b262605968","created":"2024-07-27T03:04:08.096824+0000","modified":"2024-07-27T03:13:57.294556+0000","last_up_change":"2024-07-27T03:11:33.314800+0000","last_in_change":"2024-07-27T03:11:14.028095+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-27T03:08:01.042479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-27T03:12:27.565948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-27T03:12:31.353726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-27T03:12:40.599486+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9f614272-18e8-4562-812c-4f43e40efd2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6803","nonce":2648129674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6805","nonce":2648129674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6809","nonce":2648129674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6807","nonce":2648129674}]},"public_addr":"172.21.15.67:6803/2648129674","cluster_addr":"172.21.15.67:6805/2648129674","heartbeat_back_addr":"172.21.15.67:6809/2648129674","heartbeat_front_addr":"172.21.15.67:6807/2648129674","state":["exists","up"]},{"osd":1,"uuid":"f59f5879-163a-4269-bbe7-1200eea65341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6801","nonce":2561753152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6803","nonce":2561753152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6807","nonce":2561753152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6805","nonce":2561753152}]},"public_addr":"172.21.15.149:6801/2561753152","cluster_addr":"172.21.15.149:6803/2561753152","heartbeat_back_addr":"172.21.15.149:6807/2561753152","heartbeat_front_addr":"172.21.15.149:6805/2561753152","state":["exists","up"]},{"osd":2,"uuid":"444e0ebe-7ca3-4be9-8bd5-db859f824445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6801","nonce":3770940078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6803","nonce":3770940078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6807","nonce":3770940078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6805","nonce":3770940078}]},"public_addr":"172.21.15.175:6801/3770940078","cluster_addr":"172.21.15.175:6803/3770940078","heartbeat_back_addr":"172.21.15.175:6807/3770940078","heartbeat_front_addr":"172.21.15.175:6805/3770940078","state":["exists","up"]},{"osd":3,"uuid":"d554db3d-9604-427c-8736-b1bb379f8d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6811","nonce":3159144693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6813","nonce":3159144693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6817","nonce":3159144693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6815","nonce":3159144693}]},"public_addr":"172.21.15.67:6811/3159144693","cluster_addr":"172.21.15.67:6813/3159144693","heartbeat_back_addr":"172.21.15.67:6817/3159144693","heartbeat_front_addr":"172.21.15.67:6815/3159144693","state":["exists","up"]},{"osd":4,"uuid":"c317dbe5-3b49-47a6-8408-7042a3dac345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3393503532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3393503532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3393503532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3393503532}]},"public_addr":"172.21.15.149:6809/3393503532","cluster_addr":"172.21.15.149:6811/3393503532","heartbeat_back_addr":"172.21.15.149:6815/3393503532","heartbeat_front_addr":"172.21.15.149:6813/3393503532","state":["exists","up"]},{"osd":5,"uuid":"5c85abd6-2b42-4957-a9b7-59a9ee36b841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6809","nonce":2291204784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6811","nonce":2291204784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6815","nonce":2291204784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6813","nonce":2291204784}]},"public_addr":"172.21.15.175:6809/2291204784","cluster_addr":"172.21.15.175:6811/2291204784","heartbeat_back_addr":"172.21.15.175:6815/2291204784","heartbeat_front_addr":"172.21.15.175:6813/2291204784","state":["exists","up"]},{"osd":6,"uuid":"bf6c277d-ec24-4470-98de-cf6f28876ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6819","nonce":2660182879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6821","nonce":2660182879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6825","nonce":2660182879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6823","nonce":2660182879}]},"public_addr":"172.21.15.67:6819/2660182879","cluster_addr":"172.21.15.67:6821/2660182879","heartbeat_back_addr":"172.21.15.67:6825/2660182879","heartbeat_front_addr":"172.21.15.67:6823/2660182879","state":["exists","up"]},{"osd":7,"uuid":"fea2c7cf-d5cb-45ae-9b29-f6e56c903891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6817","nonce":2289624404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6819","nonce":2289624404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6823","nonce":2289624404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6821","nonce":2289624404}]},"public_addr":"172.21.15.149:6817/2289624404","cluster_addr":"172.21.15.149:6819/2289624404","heartbeat_back_addr":"172.21.15.149:6823/2289624404","heartbeat_front_addr":"172.21.15.149:6821/2289624404","state":["exists","up"]},{"osd":8,"uuid":"14d04d96-579a-4dc9-a4d4-50e4850b48e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6817","nonce":2286032466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6819","nonce":2286032466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6823","nonce":2286032466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6821","nonce":2286032466}]},"public_addr":"172.21.15.175:6817/2286032466","cluster_addr":"172.21.15.175:6819/2286032466","heartbeat_back_addr":"172.21.15.175:6823/2286032466","heartbeat_front_addr":"172.21.15.175:6821/2286032466","state":["exists","up"]},{"osd":9,"uuid":"b65dfb6b-1efe-4314-a8be-ce6a295e99fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6827","nonce":3411081513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6829","nonce":3411081513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6833","nonce":3411081513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6831","nonce":3411081513}]},"public_addr":"172.21.15.67:6827/3411081513","cluster_addr":"172.21.15.67:6829/3411081513","heartbeat_back_addr":"172.21.15.67:6833/3411081513","heartbeat_front_addr":"172.21.15.67:6831/3411081513","state":["exists","up"]},{"osd":10,"uuid":"e3dabae0-0d64-4222-984c-fc1cce883456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6825","nonce":2011495212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6827","nonce":2011495212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6831","nonce":2011495212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6829","nonce":2011495212}]},"public_addr":"172.21.15.149:6825/2011495212","cluster_addr":"172.21.15.149:6827/2011495212","heartbeat_back_addr":"172.21.15.149:6831/2011495212","heartbeat_front_addr":"172.21.15.149:6829/2011495212","state":["exists","up"]},{"osd":11,"uuid":"406c082c-9b5f-4781-8cbf-3c04fa49e410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6825","nonce":2565514213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6827","nonce":2565514213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6831","nonce":2565514213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6829","nonce":2565514213}]},"public_addr":"172.21.15.175:6825/2565514213","cluster_addr":"172.21.15.175:6827/2565514213","heartbeat_back_addr":"172.21.15.175:6831/2565514213","heartbeat_front_addr":"172.21.15.175:6829/2565514213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:11.399380+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:33.728518+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:56.411181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:19.871731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:42.994576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:06.712815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:31.321967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:55.070898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:18.686108+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:43.187795+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:07.318538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:31.374843+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/3477601235":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/4130679770":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6800/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/1029332005":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6801/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6801/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/284078926":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6800/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/2559121542":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/925406021":"2024-07-28T03:04:30.600273+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-27T03:14:01.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.775+0000 7fad9ef8f640 1 -- 172.21.15.67:0/3927023085 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fad800b3c60 msgr2=0x7fad800b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:01.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.775+0000 7fad9ef8f640 1 --2- 172.21.15.67:0/3927023085 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fad800b3c60 0x7fad800b6170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fad981059f0 tx=0x7fad8c005b60 comp rx=0 tx=0).stop 2024-07-27T03:14:01.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.775+0000 7fad9ef8f640 1 -- 172.21.15.67:0/3927023085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad981024d0 msgr2=0x7fad98100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:01.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.775+0000 7fad9ef8f640 1 --2- 172.21.15.67:0/3927023085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad981024d0 0x7fad98100af0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fad8800d770 tx=0x7fad8800dc30 comp rx=0 tx=0).stop 2024-07-27T03:14:01.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.775+0000 7fad9ef8f640 1 -- 172.21.15.67:0/3927023085 shutdown_connections 2024-07-27T03:14:01.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.775+0000 7fad9ef8f640 1 --2- 172.21.15.67:0/3927023085 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fad800b3c60 0x7fad800b6170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:01.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.775+0000 7fad9ef8f640 1 --2- 172.21.15.67:0/3927023085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad98108760 0x7fad980ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:01.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.775+0000 7fad9ef8f640 1 --2- 172.21.15.67:0/3927023085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad98107e10 0x7fad980ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:01.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.775+0000 7fad9ef8f640 1 --2- 172.21.15.67:0/3927023085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fad981024d0 0x7fad98100af0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:01.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.775+0000 7fad9ef8f640 1 -- 172.21.15.67:0/3927023085 >> 172.21.15.67:0/3927023085 conn(0x7fad980791a0 msgr2=0x7fad981014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:01.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.775+0000 7fad9ef8f640 1 -- 172.21.15.67:0/3927023085 shutdown_connections 2024-07-27T03:14:01.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:01.775+0000 7fad9ef8f640 1 -- 172.21.15.67:0/3927023085 wait complete. 2024-07-27T03:14:02.397 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-07-27T03:14:02.397 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph pg dump --format=json 2024-07-27T03:14:02.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:02 smithi175 ceph-mon[37125]: pgmap v354: 129 pgs: 129 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.4 KiB/s wr, 13 op/s; 14 B/s, 0 objects/s recovering 2024-07-27T03:14:02.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:02 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3927023085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:14:02.665 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:02.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:02 smithi067 ceph-mon[32814]: pgmap v354: 129 pgs: 129 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.4 KiB/s wr, 13 op/s; 14 B/s, 0 objects/s recovering 2024-07-27T03:14:02.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:02 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3927023085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:14:02.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:02 smithi149 ceph-mon[37173]: pgmap v354: 129 pgs: 129 active+clean; 636 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.4 KiB/s wr, 13 op/s; 14 B/s, 0 objects/s recovering 2024-07-27T03:14:02.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:02 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3927023085' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:14:04.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.207+0000 7f5989777640 1 -- 172.21.15.67:0/1948611940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5984109870 msgr2=0x7f59841100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:04.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.207+0000 7f5989777640 1 --2- 172.21.15.67:0/1948611940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5984109870 0x7f59841100a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f597c009ca0 tx=0x7f597c02a8c0 comp rx=0 tx=0).stop 2024-07-27T03:14:04.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.207+0000 7f5989777640 1 -- 172.21.15.67:0/1948611940 shutdown_connections 2024-07-27T03:14:04.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.207+0000 7f5989777640 1 --2- 172.21.15.67:0/1948611940 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5984109870 0x7f59841100a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:04.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.207+0000 7f5989777640 1 --2- 172.21.15.67:0/1948611940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5984108f20 0x7f5984109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:04.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.207+0000 7f5989777640 1 --2- 172.21.15.67:0/1948611940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5984103630 0x7f5984103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:04.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.207+0000 7f5989777640 1 -- 172.21.15.67:0/1948611940 >> 172.21.15.67:0/1948611940 conn(0x7f598407b380 msgr2=0x7f598407b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:04.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.207+0000 7f5989777640 1 -- 172.21.15.67:0/1948611940 shutdown_connections 2024-07-27T03:14:04.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.207+0000 7f5989777640 1 -- 172.21.15.67:0/1948611940 wait complete. 2024-07-27T03:14:04.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.208+0000 7f5989777640 1 Processor -- start 2024-07-27T03:14:04.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.208+0000 7f5989777640 1 -- start start 2024-07-27T03:14:04.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.208+0000 7f5989777640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5984103630 0x7f5984199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:04.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.208+0000 7f5989777640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5984108f20 0x7f598419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:04.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.208+0000 7f5989777640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5984109870 0x7f598419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:04.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.208+0000 7f5989777640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5984113540 con 0x7f5984103630 2024-07-27T03:14:04.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.208+0000 7f5989777640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f59841133c0 con 0x7f5984108f20 2024-07-27T03:14:04.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.208+0000 7f5989777640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f59841136c0 con 0x7f5984109870 2024-07-27T03:14:04.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.208+0000 7f5982ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5984103630 0x7f5984199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:04.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.209+0000 7f5982ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5984103630 0x7f5984199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39406/0 (socket says 172.21.15.67:39406) 2024-07-27T03:14:04.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.209+0000 7f5982ffd640 1 -- 172.21.15.67:0/3055244161 learned_addr learned my addr 172.21.15.67:0/3055244161 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:14:04.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.209+0000 7f59827fc640 1 --2- 172.21.15.67:0/3055244161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5984108f20 0x7f598419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:04.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.209+0000 7f59837fe640 1 --2- 172.21.15.67:0/3055244161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5984109870 0x7f598419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:04.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.209+0000 7f5982ffd640 1 -- 172.21.15.67:0/3055244161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5984109870 msgr2=0x7f598419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:04.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.209+0000 7f5982ffd640 1 --2- 172.21.15.67:0/3055244161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5984109870 0x7f598419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:04.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.209+0000 7f5982ffd640 1 -- 172.21.15.67:0/3055244161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5984108f20 msgr2=0x7f598419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:04.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.209+0000 7f5982ffd640 1 --2- 172.21.15.67:0/3055244161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5984108f20 0x7f598419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:04.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.209+0000 7f5982ffd640 1 -- 172.21.15.67:0/3055244161 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f598419fee0 con 0x7f5984103630 2024-07-27T03:14:04.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.209+0000 7f59837fe640 1 --2- 172.21.15.67:0/3055244161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5984109870 0x7f598419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:14:04.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.209+0000 7f5982ffd640 1 --2- 172.21.15.67:0/3055244161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5984103630 0x7f5984199f70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f597400d770 tx=0x7f597400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:04.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.209+0000 7f597bfff640 1 -- 172.21.15.67:0/3055244161 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974017070 con 0x7f5984103630 2024-07-27T03:14:04.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.209+0000 7f5989777640 1 -- 172.21.15.67:0/3055244161 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59841a01d0 con 0x7f5984103630 2024-07-27T03:14:04.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.209+0000 7f597bfff640 1 -- 172.21.15.67:0/3055244161 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59740054d0 con 0x7f5984103630 2024-07-27T03:14:04.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.209+0000 7f597bfff640 1 -- 172.21.15.67:0/3055244161 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974004c30 con 0x7f5984103630 2024-07-27T03:14:04.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.209+0000 7f5989777640 1 -- 172.21.15.67:0/3055244161 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59841a6930 con 0x7f5984103630 2024-07-27T03:14:04.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.210+0000 7f5989777640 1 -- 172.21.15.67:0/3055244161 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5984103a40 con 0x7f5984103630 2024-07-27T03:14:04.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.212+0000 7f597bfff640 1 -- 172.21.15.67:0/3055244161 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f5974004dd0 con 0x7f5984103630 2024-07-27T03:14:04.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.213+0000 7f597bfff640 1 --2- 172.21.15.67:0/3055244161 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f59700b3c60 0x7f59700b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:04.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.213+0000 7f59827fc640 1 --2- 172.21.15.67:0/3055244161 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f59700b3c60 0x7f59700b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:04.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.213+0000 7f597bfff640 1 -- 172.21.15.67:0/3055244161 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f59740e3fe0 con 0x7f5984103630 2024-07-27T03:14:04.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.213+0000 7f59827fc640 1 --2- 172.21.15.67:0/3055244161 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f59700b3c60 0x7f59700b6170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f59841069c0 tx=0x7f596c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:04.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.216+0000 7f597bfff640 1 -- 172.21.15.67:0/3055244161 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f59740ab120 con 0x7f5984103630 2024-07-27T03:14:04.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.398+0000 7f5989777640 1 -- 172.21.15.67:0/3055244161 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5984063270 con 0x7f59700b3c60 2024-07-27T03:14:04.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.404+0000 7f597bfff640 1 -- 172.21.15.67:0/3055244161 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340413 (secure 0 0 0) 0x7f5984063270 con 0x7f59700b3c60 2024-07-27T03:14:04.412 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:14:04.414 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-07-27T03:14:04.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.407+0000 7f5989777640 1 -- 172.21.15.67:0/3055244161 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f59700b3c60 msgr2=0x7f59700b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:04.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.407+0000 7f5989777640 1 --2- 172.21.15.67:0/3055244161 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f59700b3c60 0x7f59700b6170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f59841069c0 tx=0x7f596c00a2d0 comp rx=0 tx=0).stop 2024-07-27T03:14:04.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.407+0000 7f5989777640 1 -- 172.21.15.67:0/3055244161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5984103630 msgr2=0x7f5984199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:04.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.407+0000 7f5989777640 1 --2- 172.21.15.67:0/3055244161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5984103630 0x7f5984199f70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f597400d770 tx=0x7f597400dc30 comp rx=0 tx=0).stop 2024-07-27T03:14:04.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.407+0000 7f5989777640 1 -- 172.21.15.67:0/3055244161 shutdown_connections 2024-07-27T03:14:04.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.407+0000 7f5989777640 1 --2- 172.21.15.67:0/3055244161 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f59700b3c60 0x7f59700b6170 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:04.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.407+0000 7f5989777640 1 --2- 172.21.15.67:0/3055244161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5984109870 0x7f598419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:04.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.407+0000 7f5989777640 1 --2- 172.21.15.67:0/3055244161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5984108f20 0x7f598419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:04.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.407+0000 7f5989777640 1 --2- 172.21.15.67:0/3055244161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5984103630 0x7f5984199f70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:04.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.407+0000 7f5989777640 1 -- 172.21.15.67:0/3055244161 >> 172.21.15.67:0/3055244161 conn(0x7f598407b380 msgr2=0x7f5984104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:04.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.408+0000 7f5989777640 1 -- 172.21.15.67:0/3055244161 shutdown_connections 2024-07-27T03:14:04.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:04.408+0000 7f5989777640 1 -- 172.21.15.67:0/3055244161 wait complete. 2024-07-27T03:14:04.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:04 smithi175 ceph-mon[37125]: pgmap v355: 129 pgs: 129 active+clean; 636 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 9 op/s; 14 B/s, 0 objects/s recovering 2024-07-27T03:14:04.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:04 smithi067 ceph-mon[32814]: pgmap v355: 129 pgs: 129 active+clean; 636 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 9 op/s; 14 B/s, 0 objects/s recovering 2024-07-27T03:14:04.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:04 smithi149 ceph-mon[37173]: pgmap v355: 129 pgs: 129 active+clean; 636 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.6 KiB/s wr, 9 op/s; 14 B/s, 0 objects/s recovering 2024-07-27T03:14:05.102 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-07-27T03:14:03.068719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650795,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":317,"num_read_kb":251,"num_write":368,"num_write_kb":2282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951979,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":303,"ondisk_log_size":303,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388624,"kb_used_data":14448,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553200,"statfs":{"total":1151940427776,"available":1151542476800,"internally_reserved":0,"allocated":14794752,"data_stored":8232293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8627,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":66,"num_read_kb":39,"num_write":13,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002389"},"pg_stats":[{"pgid":"2.1f","version":"89'13","reported_seq":79,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:59.281499+0000","last_change":"2024-07-27T03:12:29.541107+0000","last_active":"2024-07-27T03:13:59.281499+0000","last_peered":"2024-07-27T03:13:59.281499+0000","last_clean":"2024-07-27T03:13:59.281499+0000","last_became_active":"2024-07-27T03:12:29.540806+0000","last_became_peered":"2024-07-27T03:12:29.540806+0000","last_unstale":"2024-07-27T03:13:59.281499+0000","last_undegraded":"2024-07-27T03:13:59.281499+0000","last_fullsized":"2024-07-27T03:13:59.281499+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:38:24.349354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404603+0000","last_change":"2024-07-27T03:12:32.565515+0000","last_active":"2024-07-27T03:13:57.404603+0000","last_peered":"2024-07-27T03:13:57.404603+0000","last_clean":"2024-07-27T03:13:57.404603+0000","last_became_active":"2024-07-27T03:12:32.565134+0000","last_became_peered":"2024-07-27T03:12:32.565134+0000","last_unstale":"2024-07-27T03:13:57.404603+0000","last_undegraded":"2024-07-27T03:13:57.404603+0000","last_fullsized":"2024-07-27T03:13:57.404603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:11:56.793139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302675+0000","last_change":"2024-07-27T03:12:42.613432+0000","last_active":"2024-07-27T03:13:57.302675+0000","last_peered":"2024-07-27T03:13:57.302675+0000","last_clean":"2024-07-27T03:13:57.302675+0000","last_became_active":"2024-07-27T03:12:42.612986+0000","last_became_peered":"2024-07-27T03:12:42.612986+0000","last_unstale":"2024-07-27T03:13:57.302675+0000","last_undegraded":"2024-07-27T03:13:57.302675+0000","last_fullsized":"2024-07-27T03:13:57.302675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:25:37.505117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405571+0000","last_change":"2024-07-27T03:12:29.614229+0000","last_active":"2024-07-27T03:13:57.405571+0000","last_peered":"2024-07-27T03:13:57.405571+0000","last_clean":"2024-07-27T03:13:57.405571+0000","last_became_active":"2024-07-27T03:12:29.613844+0000","last_became_peered":"2024-07-27T03:12:29.613844+0000","last_unstale":"2024-07-27T03:13:57.405571+0000","last_undegraded":"2024-07-27T03:13:57.405571+0000","last_fullsized":"2024-07-27T03:13:57.405571+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:10:28.917080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405727+0000","last_change":"2024-07-27T03:12:32.563187+0000","last_active":"2024-07-27T03:13:57.405727+0000","last_peered":"2024-07-27T03:13:57.405727+0000","last_clean":"2024-07-27T03:13:57.405727+0000","last_became_active":"2024-07-27T03:12:32.562674+0000","last_became_peered":"2024-07-27T03:12:32.562674+0000","last_unstale":"2024-07-27T03:13:57.405727+0000","last_undegraded":"2024-07-27T03:13:57.405727+0000","last_fullsized":"2024-07-27T03:13:57.405727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:05:39.211514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302984+0000","last_change":"2024-07-27T03:12:42.611516+0000","last_active":"2024-07-27T03:13:57.302984+0000","last_peered":"2024-07-27T03:13:57.302984+0000","last_clean":"2024-07-27T03:13:57.302984+0000","last_became_active":"2024-07-27T03:12:42.610934+0000","last_became_peered":"2024-07-27T03:12:42.610934+0000","last_unstale":"2024-07-27T03:13:57.302984+0000","last_undegraded":"2024-07-27T03:13:57.302984+0000","last_fullsized":"2024-07-27T03:13:57.302984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:13:17.323063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305683+0000","last_change":"2024-07-27T03:12:29.539573+0000","last_active":"2024-07-27T03:13:57.305683+0000","last_peered":"2024-07-27T03:13:57.305683+0000","last_clean":"2024-07-27T03:13:57.305683+0000","last_became_active":"2024-07-27T03:12:29.539056+0000","last_became_peered":"2024-07-27T03:12:29.539056+0000","last_unstale":"2024-07-27T03:13:57.305683+0000","last_undegraded":"2024-07-27T03:13:57.305683+0000","last_fullsized":"2024-07-27T03:13:57.305683+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:04:37.513644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405607+0000","last_change":"2024-07-27T03:12:32.567729+0000","last_active":"2024-07-27T03:13:57.405607+0000","last_peered":"2024-07-27T03:13:57.405607+0000","last_clean":"2024-07-27T03:13:57.405607+0000","last_became_active":"2024-07-27T03:12:32.564827+0000","last_became_peered":"2024-07-27T03:12:32.564827+0000","last_unstale":"2024-07-27T03:13:57.405607+0000","last_undegraded":"2024-07-27T03:13:57.405607+0000","last_fullsized":"2024-07-27T03:13:57.405607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:10:55.314988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303275+0000","last_change":"2024-07-27T03:12:42.612776+0000","last_active":"2024-07-27T03:13:57.303275+0000","last_peered":"2024-07-27T03:13:57.303275+0000","last_clean":"2024-07-27T03:13:57.303275+0000","last_became_active":"2024-07-27T03:12:42.612258+0000","last_became_peered":"2024-07-27T03:12:42.612258+0000","last_unstale":"2024-07-27T03:13:57.303275+0000","last_undegraded":"2024-07-27T03:13:57.303275+0000","last_fullsized":"2024-07-27T03:13:57.303275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:48:06.516002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305873+0000","last_change":"2024-07-27T03:12:29.540070+0000","last_active":"2024-07-27T03:13:57.305873+0000","last_peered":"2024-07-27T03:13:57.305873+0000","last_clean":"2024-07-27T03:13:57.305873+0000","last_became_active":"2024-07-27T03:12:29.539657+0000","last_became_peered":"2024-07-27T03:12:29.539657+0000","last_unstale":"2024-07-27T03:13:57.305873+0000","last_undegraded":"2024-07-27T03:13:57.305873+0000","last_fullsized":"2024-07-27T03:13:57.305873+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:42:48.222613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404933+0000","last_change":"2024-07-27T03:12:32.574418+0000","last_active":"2024-07-27T03:13:57.404933+0000","last_peered":"2024-07-27T03:13:57.404933+0000","last_clean":"2024-07-27T03:13:57.404933+0000","last_became_active":"2024-07-27T03:12:32.573974+0000","last_became_peered":"2024-07-27T03:12:32.573974+0000","last_unstale":"2024-07-27T03:13:57.404933+0000","last_undegraded":"2024-07-27T03:13:57.404933+0000","last_fullsized":"2024-07-27T03:13:57.404933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:34:30.442833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303264+0000","last_change":"2024-07-27T03:12:42.605155+0000","last_active":"2024-07-27T03:13:57.303264+0000","last_peered":"2024-07-27T03:13:57.303264+0000","last_clean":"2024-07-27T03:13:57.303264+0000","last_became_active":"2024-07-27T03:12:42.604685+0000","last_became_peered":"2024-07-27T03:12:42.604685+0000","last_unstale":"2024-07-27T03:13:57.303264+0000","last_undegraded":"2024-07-27T03:13:57.303264+0000","last_fullsized":"2024-07-27T03:13:57.303264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:28:58.931408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305799+0000","last_change":"2024-07-27T03:12:32.574711+0000","last_active":"2024-07-27T03:13:57.305799+0000","last_peered":"2024-07-27T03:13:57.305799+0000","last_clean":"2024-07-27T03:13:57.305799+0000","last_became_active":"2024-07-27T03:12:32.574365+0000","last_became_peered":"2024-07-27T03:12:32.574365+0000","last_unstale":"2024-07-27T03:13:57.305799+0000","last_undegraded":"2024-07-27T03:13:57.305799+0000","last_fullsized":"2024-07-27T03:13:57.305799+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:50:56.621371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305282+0000","last_change":"2024-07-27T03:12:29.540835+0000","last_active":"2024-07-27T03:13:57.305282+0000","last_peered":"2024-07-27T03:13:57.305282+0000","last_clean":"2024-07-27T03:13:57.305282+0000","last_became_active":"2024-07-27T03:12:29.540461+0000","last_became_peered":"2024-07-27T03:12:29.540461+0000","last_unstale":"2024-07-27T03:13:57.305282+0000","last_undegraded":"2024-07-27T03:13:57.305282+0000","last_fullsized":"2024-07-27T03:13:57.305282+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:19:20.817038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303286+0000","last_change":"2024-07-27T03:12:42.718584+0000","last_active":"2024-07-27T03:13:57.303286+0000","last_peered":"2024-07-27T03:13:57.303286+0000","last_clean":"2024-07-27T03:13:57.303286+0000","last_became_active":"2024-07-27T03:12:42.717752+0000","last_became_peered":"2024-07-27T03:12:42.717752+0000","last_unstale":"2024-07-27T03:13:57.303286+0000","last_undegraded":"2024-07-27T03:13:57.303286+0000","last_fullsized":"2024-07-27T03:13:57.303286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:23:52.629794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404350+0000","last_change":"2024-07-27T03:12:32.560672+0000","last_active":"2024-07-27T03:13:57.404350+0000","last_peered":"2024-07-27T03:13:57.404350+0000","last_clean":"2024-07-27T03:13:57.404350+0000","last_became_active":"2024-07-27T03:12:32.559172+0000","last_became_peered":"2024-07-27T03:12:32.559172+0000","last_unstale":"2024-07-27T03:13:57.404350+0000","last_undegraded":"2024-07-27T03:13:57.404350+0000","last_fullsized":"2024-07-27T03:13:57.404350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:02:49.346983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405148+0000","last_change":"2024-07-27T03:12:57.434569+0000","last_active":"2024-07-27T03:13:57.405148+0000","last_peered":"2024-07-27T03:13:57.405148+0000","last_clean":"2024-07-27T03:13:57.405148+0000","last_became_active":"2024-07-27T03:12:57.434038+0000","last_became_peered":"2024-07-27T03:12:57.434038+0000","last_unstale":"2024-07-27T03:13:57.405148+0000","last_undegraded":"2024-07-27T03:13:57.405148+0000","last_fullsized":"2024-07-27T03:13:57.405148+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:43:24.462838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.411985+0000","last_change":"2024-07-27T03:13:57.411931+0000","last_active":"2024-07-27T03:13:57.411985+0000","last_peered":"2024-07-27T03:13:57.411985+0000","last_clean":"2024-07-27T03:13:57.411985+0000","last_became_active":"2024-07-27T03:13:57.407362+0000","last_became_peered":"2024-07-27T03:13:57.407362+0000","last_unstale":"2024-07-27T03:13:57.411985+0000","last_undegraded":"2024-07-27T03:13:57.411985+0000","last_fullsized":"2024-07-27T03:13:57.411985+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:52:08.444268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405119+0000","last_change":"2024-07-27T03:12:32.572129+0000","last_active":"2024-07-27T03:13:57.405119+0000","last_peered":"2024-07-27T03:13:57.405119+0000","last_clean":"2024-07-27T03:13:57.405119+0000","last_became_active":"2024-07-27T03:12:32.571774+0000","last_became_peered":"2024-07-27T03:12:32.571774+0000","last_unstale":"2024-07-27T03:13:57.405119+0000","last_undegraded":"2024-07-27T03:13:57.405119+0000","last_fullsized":"2024-07-27T03:13:57.405119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:52:23.316957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404381+0000","last_change":"2024-07-27T03:12:29.539768+0000","last_active":"2024-07-27T03:13:57.404381+0000","last_peered":"2024-07-27T03:13:57.404381+0000","last_clean":"2024-07-27T03:13:57.404381+0000","last_became_active":"2024-07-27T03:12:29.539389+0000","last_became_peered":"2024-07-27T03:12:29.539389+0000","last_unstale":"2024-07-27T03:13:57.404381+0000","last_undegraded":"2024-07-27T03:13:57.404381+0000","last_fullsized":"2024-07-27T03:13:57.404381+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:09:35.961328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302562+0000","last_change":"2024-07-27T03:12:42.617569+0000","last_active":"2024-07-27T03:13:57.302562+0000","last_peered":"2024-07-27T03:13:57.302562+0000","last_clean":"2024-07-27T03:13:57.302562+0000","last_became_active":"2024-07-27T03:12:42.616357+0000","last_became_peered":"2024-07-27T03:12:42.616357+0000","last_unstale":"2024-07-27T03:13:57.302562+0000","last_undegraded":"2024-07-27T03:13:57.302562+0000","last_fullsized":"2024-07-27T03:13:57.302562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:35:11.297631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304782+0000","last_change":"2024-07-27T03:12:32.566272+0000","last_active":"2024-07-27T03:13:57.304782+0000","last_peered":"2024-07-27T03:13:57.304782+0000","last_clean":"2024-07-27T03:13:57.304782+0000","last_became_active":"2024-07-27T03:12:32.565865+0000","last_became_peered":"2024-07-27T03:12:32.565865+0000","last_unstale":"2024-07-27T03:13:57.304782+0000","last_undegraded":"2024-07-27T03:13:57.304782+0000","last_fullsized":"2024-07-27T03:13:57.304782+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:42:09.539586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405304+0000","last_change":"2024-07-27T03:12:29.613733+0000","last_active":"2024-07-27T03:13:57.405304+0000","last_peered":"2024-07-27T03:13:57.405304+0000","last_clean":"2024-07-27T03:13:57.405304+0000","last_became_active":"2024-07-27T03:12:29.613395+0000","last_became_peered":"2024-07-27T03:12:29.613395+0000","last_unstale":"2024-07-27T03:13:57.405304+0000","last_undegraded":"2024-07-27T03:13:57.405304+0000","last_fullsized":"2024-07-27T03:13:57.405304+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:21:34.431900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.407883+0000","last_change":"2024-07-27T03:13:57.407798+0000","last_active":"2024-07-27T03:13:57.407883+0000","last_peered":"2024-07-27T03:13:57.407883+0000","last_clean":"2024-07-27T03:13:57.407883+0000","last_became_active":"2024-07-27T03:13:57.407275+0000","last_became_peered":"2024-07-27T03:13:57.407275+0000","last_unstale":"2024-07-27T03:13:57.407883+0000","last_undegraded":"2024-07-27T03:13:57.407883+0000","last_fullsized":"2024-07-27T03:13:57.407883+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:47:27.817092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,8],"acting":[4,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405475+0000","last_change":"2024-07-27T03:12:32.567006+0000","last_active":"2024-07-27T03:13:57.405475+0000","last_peered":"2024-07-27T03:13:57.405475+0000","last_clean":"2024-07-27T03:13:57.405475+0000","last_became_active":"2024-07-27T03:12:32.566205+0000","last_became_peered":"2024-07-27T03:12:32.566205+0000","last_unstale":"2024-07-27T03:13:57.405475+0000","last_undegraded":"2024-07-27T03:13:57.405475+0000","last_fullsized":"2024-07-27T03:13:57.405475+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:11:23.171341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303619+0000","last_change":"2024-07-27T03:12:29.533909+0000","last_active":"2024-07-27T03:13:57.303619+0000","last_peered":"2024-07-27T03:13:57.303619+0000","last_clean":"2024-07-27T03:13:57.303619+0000","last_became_active":"2024-07-27T03:12:29.533327+0000","last_became_peered":"2024-07-27T03:12:29.533327+0000","last_unstale":"2024-07-27T03:13:57.303619+0000","last_undegraded":"2024-07-27T03:13:57.303619+0000","last_fullsized":"2024-07-27T03:13:57.303619+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:35:24.705096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302850+0000","last_change":"2024-07-27T03:12:42.609066+0000","last_active":"2024-07-27T03:13:57.302850+0000","last_peered":"2024-07-27T03:13:57.302850+0000","last_clean":"2024-07-27T03:13:57.302850+0000","last_became_active":"2024-07-27T03:12:42.608561+0000","last_became_peered":"2024-07-27T03:12:42.608561+0000","last_unstale":"2024-07-27T03:13:57.302850+0000","last_undegraded":"2024-07-27T03:13:57.302850+0000","last_fullsized":"2024-07-27T03:13:57.302850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:27:53.657498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405605+0000","last_change":"2024-07-27T03:12:32.560931+0000","last_active":"2024-07-27T03:13:57.405605+0000","last_peered":"2024-07-27T03:13:57.405605+0000","last_clean":"2024-07-27T03:13:57.405605+0000","last_became_active":"2024-07-27T03:12:32.560549+0000","last_became_peered":"2024-07-27T03:12:32.560549+0000","last_unstale":"2024-07-27T03:13:57.405605+0000","last_undegraded":"2024-07-27T03:13:57.405605+0000","last_fullsized":"2024-07-27T03:13:57.405605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T15:11:25.136035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'14","reported_seq":58,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302690+0000","last_change":"2024-07-27T03:12:29.535365+0000","last_active":"2024-07-27T03:13:57.302690+0000","last_peered":"2024-07-27T03:13:57.302690+0000","last_clean":"2024-07-27T03:13:57.302690+0000","last_became_active":"2024-07-27T03:12:29.534952+0000","last_became_peered":"2024-07-27T03:12:29.534952+0000","last_unstale":"2024-07-27T03:13:57.302690+0000","last_undegraded":"2024-07-27T03:13:57.302690+0000","last_fullsized":"2024-07-27T03:13:57.302690+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:34:59.889488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404008+0000","last_change":"2024-07-27T03:12:42.718416+0000","last_active":"2024-07-27T03:13:57.404008+0000","last_peered":"2024-07-27T03:13:57.404008+0000","last_clean":"2024-07-27T03:13:57.404008+0000","last_became_active":"2024-07-27T03:12:42.717949+0000","last_became_peered":"2024-07-27T03:12:42.717949+0000","last_unstale":"2024-07-27T03:13:57.404008+0000","last_undegraded":"2024-07-27T03:13:57.404008+0000","last_fullsized":"2024-07-27T03:13:57.404008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:05:05.762577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348818+0000","last_change":"2024-07-27T03:12:32.559824+0000","last_active":"2024-07-27T03:13:57.348818+0000","last_peered":"2024-07-27T03:13:57.348818+0000","last_clean":"2024-07-27T03:13:57.348818+0000","last_became_active":"2024-07-27T03:12:32.559435+0000","last_became_peered":"2024-07-27T03:12:32.559435+0000","last_unstale":"2024-07-27T03:13:57.348818+0000","last_undegraded":"2024-07-27T03:13:57.348818+0000","last_fullsized":"2024-07-27T03:13:57.348818+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:16:44.514272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305808+0000","last_change":"2024-07-27T03:12:29.614296+0000","last_active":"2024-07-27T03:13:57.305808+0000","last_peered":"2024-07-27T03:13:57.305808+0000","last_clean":"2024-07-27T03:13:57.305808+0000","last_became_active":"2024-07-27T03:12:29.612987+0000","last_became_peered":"2024-07-27T03:12:29.612987+0000","last_unstale":"2024-07-27T03:13:57.305808+0000","last_undegraded":"2024-07-27T03:13:57.305808+0000","last_fullsized":"2024-07-27T03:13:57.305808+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:23:09.226523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405142+0000","last_change":"2024-07-27T03:12:42.616575+0000","last_active":"2024-07-27T03:13:57.405142+0000","last_peered":"2024-07-27T03:13:57.405142+0000","last_clean":"2024-07-27T03:13:57.405142+0000","last_became_active":"2024-07-27T03:12:42.615980+0000","last_became_peered":"2024-07-27T03:12:42.615980+0000","last_unstale":"2024-07-27T03:13:57.405142+0000","last_undegraded":"2024-07-27T03:13:57.405142+0000","last_fullsized":"2024-07-27T03:13:57.405142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:00:59.041910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405960+0000","last_change":"2024-07-27T03:12:32.556109+0000","last_active":"2024-07-27T03:13:57.405960+0000","last_peered":"2024-07-27T03:13:57.405960+0000","last_clean":"2024-07-27T03:13:57.405960+0000","last_became_active":"2024-07-27T03:12:32.555747+0000","last_became_peered":"2024-07-27T03:12:32.555747+0000","last_unstale":"2024-07-27T03:13:57.405960+0000","last_undegraded":"2024-07-27T03:13:57.405960+0000","last_fullsized":"2024-07-27T03:13:57.405960+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:27:58.286605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302484+0000","last_change":"2024-07-27T03:12:29.613523+0000","last_active":"2024-07-27T03:13:57.302484+0000","last_peered":"2024-07-27T03:13:57.302484+0000","last_clean":"2024-07-27T03:13:57.302484+0000","last_became_active":"2024-07-27T03:12:29.613172+0000","last_became_peered":"2024-07-27T03:12:29.613172+0000","last_unstale":"2024-07-27T03:13:57.302484+0000","last_undegraded":"2024-07-27T03:13:57.302484+0000","last_fullsized":"2024-07-27T03:13:57.302484+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:49:45.225221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304847+0000","last_change":"2024-07-27T03:12:42.718536+0000","last_active":"2024-07-27T03:13:57.304847+0000","last_peered":"2024-07-27T03:13:57.304847+0000","last_clean":"2024-07-27T03:13:57.304847+0000","last_became_active":"2024-07-27T03:12:42.718068+0000","last_became_peered":"2024-07-27T03:12:42.718068+0000","last_unstale":"2024-07-27T03:13:57.304847+0000","last_undegraded":"2024-07-27T03:13:57.304847+0000","last_fullsized":"2024-07-27T03:13:57.304847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:18:28.478700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406742+0000","last_change":"2024-07-27T03:12:32.558060+0000","last_active":"2024-07-27T03:13:57.406742+0000","last_peered":"2024-07-27T03:13:57.406742+0000","last_clean":"2024-07-27T03:13:57.406742+0000","last_became_active":"2024-07-27T03:12:32.557260+0000","last_became_peered":"2024-07-27T03:12:32.557260+0000","last_unstale":"2024-07-27T03:13:57.406742+0000","last_undegraded":"2024-07-27T03:13:57.406742+0000","last_fullsized":"2024-07-27T03:13:57.406742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:36:54.945761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":72,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:00.283764+0000","last_change":"2024-07-27T03:12:29.541080+0000","last_active":"2024-07-27T03:14:00.283764+0000","last_peered":"2024-07-27T03:14:00.283764+0000","last_clean":"2024-07-27T03:14:00.283764+0000","last_became_active":"2024-07-27T03:12:29.540628+0000","last_became_peered":"2024-07-27T03:12:29.540628+0000","last_unstale":"2024-07-27T03:14:00.283764+0000","last_undegraded":"2024-07-27T03:14:00.283764+0000","last_fullsized":"2024-07-27T03:14:00.283764+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:45:17.045124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348808+0000","last_change":"2024-07-27T03:12:42.611561+0000","last_active":"2024-07-27T03:13:57.348808+0000","last_peered":"2024-07-27T03:13:57.348808+0000","last_clean":"2024-07-27T03:13:57.348808+0000","last_became_active":"2024-07-27T03:12:42.610507+0000","last_became_peered":"2024-07-27T03:12:42.610507+0000","last_unstale":"2024-07-27T03:13:57.348808+0000","last_undegraded":"2024-07-27T03:13:57.348808+0000","last_fullsized":"2024-07-27T03:13:57.348808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:15:08.972807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404266+0000","last_change":"2024-07-27T03:12:32.562316+0000","last_active":"2024-07-27T03:13:57.404266+0000","last_peered":"2024-07-27T03:13:57.404266+0000","last_clean":"2024-07-27T03:13:57.404266+0000","last_became_active":"2024-07-27T03:12:32.561729+0000","last_became_peered":"2024-07-27T03:12:32.561729+0000","last_unstale":"2024-07-27T03:13:57.404266+0000","last_undegraded":"2024-07-27T03:13:57.404266+0000","last_fullsized":"2024-07-27T03:13:57.404266+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:01:19.960269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":70,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348398+0000","last_change":"2024-07-27T03:12:29.613459+0000","last_active":"2024-07-27T03:13:57.348398+0000","last_peered":"2024-07-27T03:13:57.348398+0000","last_clean":"2024-07-27T03:13:57.348398+0000","last_became_active":"2024-07-27T03:12:29.613009+0000","last_became_peered":"2024-07-27T03:12:29.613009+0000","last_unstale":"2024-07-27T03:13:57.348398+0000","last_undegraded":"2024-07-27T03:13:57.348398+0000","last_fullsized":"2024-07-27T03:13:57.348398+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:16:37.230471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348633+0000","last_change":"2024-07-27T03:12:42.718516+0000","last_active":"2024-07-27T03:13:57.348633+0000","last_peered":"2024-07-27T03:13:57.348633+0000","last_clean":"2024-07-27T03:13:57.348633+0000","last_became_active":"2024-07-27T03:12:42.718041+0000","last_became_peered":"2024-07-27T03:12:42.718041+0000","last_unstale":"2024-07-27T03:13:57.348633+0000","last_undegraded":"2024-07-27T03:13:57.348633+0000","last_fullsized":"2024-07-27T03:13:57.348633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:35:39.633977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348194+0000","last_change":"2024-07-27T03:12:32.560752+0000","last_active":"2024-07-27T03:13:57.348194+0000","last_peered":"2024-07-27T03:13:57.348194+0000","last_clean":"2024-07-27T03:13:57.348194+0000","last_became_active":"2024-07-27T03:12:32.560248+0000","last_became_peered":"2024-07-27T03:12:32.560248+0000","last_unstale":"2024-07-27T03:13:57.348194+0000","last_undegraded":"2024-07-27T03:13:57.348194+0000","last_fullsized":"2024-07-27T03:13:57.348194+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:59:48.056867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305388+0000","last_change":"2024-07-27T03:12:29.539591+0000","last_active":"2024-07-27T03:13:57.305388+0000","last_peered":"2024-07-27T03:13:57.305388+0000","last_clean":"2024-07-27T03:13:57.305388+0000","last_became_active":"2024-07-27T03:12:29.539165+0000","last_became_peered":"2024-07-27T03:12:29.539165+0000","last_unstale":"2024-07-27T03:13:57.305388+0000","last_undegraded":"2024-07-27T03:13:57.305388+0000","last_fullsized":"2024-07-27T03:13:57.305388+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:04:21.940511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.408418+0000","last_change":"2024-07-27T03:13:57.407921+0000","last_active":"2024-07-27T03:13:57.408418+0000","last_peered":"2024-07-27T03:13:57.408418+0000","last_clean":"2024-07-27T03:13:57.408418+0000","last_became_active":"2024-07-27T03:13:57.407382+0000","last_became_peered":"2024-07-27T03:13:57.407382+0000","last_unstale":"2024-07-27T03:13:57.408418+0000","last_undegraded":"2024-07-27T03:13:57.408418+0000","last_fullsized":"2024-07-27T03:13:57.408418+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:48:27.362759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404318+0000","last_change":"2024-07-27T03:12:32.570653+0000","last_active":"2024-07-27T03:13:57.404318+0000","last_peered":"2024-07-27T03:13:57.404318+0000","last_clean":"2024-07-27T03:13:57.404318+0000","last_became_active":"2024-07-27T03:12:32.569088+0000","last_became_peered":"2024-07-27T03:12:32.569088+0000","last_unstale":"2024-07-27T03:13:57.404318+0000","last_undegraded":"2024-07-27T03:13:57.404318+0000","last_fullsized":"2024-07-27T03:13:57.404318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:50:24.669431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303802+0000","last_change":"2024-07-27T03:12:29.541654+0000","last_active":"2024-07-27T03:13:57.303802+0000","last_peered":"2024-07-27T03:13:57.303802+0000","last_clean":"2024-07-27T03:13:57.303802+0000","last_became_active":"2024-07-27T03:12:29.541264+0000","last_became_peered":"2024-07-27T03:12:29.541264+0000","last_unstale":"2024-07-27T03:13:57.303802+0000","last_undegraded":"2024-07-27T03:13:57.303802+0000","last_fullsized":"2024-07-27T03:13:57.303802+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:48:05.179016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":122,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306449+0000","last_change":"2024-07-27T03:11:11.343199+0000","last_active":"2024-07-27T03:13:57.306449+0000","last_peered":"2024-07-27T03:13:57.306449+0000","last_clean":"2024-07-27T03:13:57.306449+0000","last_became_active":"2024-07-27T03:11:11.337635+0000","last_became_peered":"2024-07-27T03:11:11.337635+0000","last_unstale":"2024-07-27T03:13:57.306449+0000","last_undegraded":"2024-07-27T03:13:57.306449+0000","last_fullsized":"2024-07-27T03:13:57.306449+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:08:01.694830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:08:01.694830+0000","last_clean_scrub_stamp":"2024-07-27T03:08:01.694830+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:48:49.979610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305758+0000","last_change":"2024-07-27T03:12:42.614346+0000","last_active":"2024-07-27T03:13:57.305758+0000","last_peered":"2024-07-27T03:13:57.305758+0000","last_clean":"2024-07-27T03:13:57.305758+0000","last_became_active":"2024-07-27T03:12:42.613881+0000","last_became_peered":"2024-07-27T03:12:42.613881+0000","last_unstale":"2024-07-27T03:13:57.305758+0000","last_undegraded":"2024-07-27T03:13:57.305758+0000","last_fullsized":"2024-07-27T03:13:57.305758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:12:38.299348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306056+0000","last_change":"2024-07-27T03:12:32.574330+0000","last_active":"2024-07-27T03:13:57.306056+0000","last_peered":"2024-07-27T03:13:57.306056+0000","last_clean":"2024-07-27T03:13:57.306056+0000","last_became_active":"2024-07-27T03:12:32.573977+0000","last_became_peered":"2024-07-27T03:12:32.573977+0000","last_unstale":"2024-07-27T03:13:57.306056+0000","last_undegraded":"2024-07-27T03:13:57.306056+0000","last_fullsized":"2024-07-27T03:13:57.306056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:39:09.925794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":72,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:58.281478+0000","last_change":"2024-07-27T03:12:29.613848+0000","last_active":"2024-07-27T03:13:58.281478+0000","last_peered":"2024-07-27T03:13:58.281478+0000","last_clean":"2024-07-27T03:13:58.281478+0000","last_became_active":"2024-07-27T03:12:29.613473+0000","last_became_peered":"2024-07-27T03:12:29.613473+0000","last_unstale":"2024-07-27T03:13:58.281478+0000","last_undegraded":"2024-07-27T03:13:58.281478+0000","last_fullsized":"2024-07-27T03:13:58.281478+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:52:56.894962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405287+0000","last_change":"2024-07-27T03:12:42.611295+0000","last_active":"2024-07-27T03:13:57.405287+0000","last_peered":"2024-07-27T03:13:57.405287+0000","last_clean":"2024-07-27T03:13:57.405287+0000","last_became_active":"2024-07-27T03:12:42.610782+0000","last_became_peered":"2024-07-27T03:12:42.610782+0000","last_unstale":"2024-07-27T03:13:57.405287+0000","last_undegraded":"2024-07-27T03:13:57.405287+0000","last_fullsized":"2024-07-27T03:13:57.405287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:24:02.849882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305853+0000","last_change":"2024-07-27T03:12:56.725924+0000","last_active":"2024-07-27T03:13:57.305853+0000","last_peered":"2024-07-27T03:13:57.305853+0000","last_clean":"2024-07-27T03:13:57.305853+0000","last_became_active":"2024-07-27T03:12:56.725567+0000","last_became_peered":"2024-07-27T03:12:56.725567+0000","last_unstale":"2024-07-27T03:13:57.305853+0000","last_undegraded":"2024-07-27T03:13:57.305853+0000","last_fullsized":"2024-07-27T03:13:57.305853+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:32:23.771483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405166+0000","last_change":"2024-07-27T03:12:29.537863+0000","last_active":"2024-07-27T03:13:57.405166+0000","last_peered":"2024-07-27T03:13:57.405166+0000","last_clean":"2024-07-27T03:13:57.405166+0000","last_became_active":"2024-07-27T03:12:29.537444+0000","last_became_peered":"2024-07-27T03:12:29.537444+0000","last_unstale":"2024-07-27T03:13:57.405166+0000","last_undegraded":"2024-07-27T03:13:57.405166+0000","last_fullsized":"2024-07-27T03:13:57.405166+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:52:05.239809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405213+0000","last_change":"2024-07-27T03:12:42.622599+0000","last_active":"2024-07-27T03:13:57.405213+0000","last_peered":"2024-07-27T03:13:57.405213+0000","last_clean":"2024-07-27T03:13:57.405213+0000","last_became_active":"2024-07-27T03:12:42.622151+0000","last_became_peered":"2024-07-27T03:12:42.622151+0000","last_unstale":"2024-07-27T03:13:57.405213+0000","last_undegraded":"2024-07-27T03:13:57.405213+0000","last_fullsized":"2024-07-27T03:13:57.405213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:34:31.622281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306321+0000","last_change":"2024-07-27T03:12:32.562300+0000","last_active":"2024-07-27T03:13:57.306321+0000","last_peered":"2024-07-27T03:13:57.306321+0000","last_clean":"2024-07-27T03:13:57.306321+0000","last_became_active":"2024-07-27T03:12:32.561343+0000","last_became_peered":"2024-07-27T03:12:32.561343+0000","last_unstale":"2024-07-27T03:13:57.306321+0000","last_undegraded":"2024-07-27T03:13:57.306321+0000","last_fullsized":"2024-07-27T03:13:57.306321+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:01:28.626943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348783+0000","last_change":"2024-07-27T03:12:29.535177+0000","last_active":"2024-07-27T03:13:57.348783+0000","last_peered":"2024-07-27T03:13:57.348783+0000","last_clean":"2024-07-27T03:13:57.348783+0000","last_became_active":"2024-07-27T03:12:29.534662+0000","last_became_peered":"2024-07-27T03:12:29.534662+0000","last_unstale":"2024-07-27T03:13:57.348783+0000","last_undegraded":"2024-07-27T03:13:57.348783+0000","last_fullsized":"2024-07-27T03:13:57.348783+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:50:49.661992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404014+0000","last_change":"2024-07-27T03:12:42.609925+0000","last_active":"2024-07-27T03:13:57.404014+0000","last_peered":"2024-07-27T03:13:57.404014+0000","last_clean":"2024-07-27T03:13:57.404014+0000","last_became_active":"2024-07-27T03:12:42.609456+0000","last_became_peered":"2024-07-27T03:12:42.609456+0000","last_unstale":"2024-07-27T03:13:57.404014+0000","last_undegraded":"2024-07-27T03:13:57.404014+0000","last_fullsized":"2024-07-27T03:13:57.404014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:07:52.128381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405991+0000","last_change":"2024-07-27T03:12:56.726170+0000","last_active":"2024-07-27T03:13:57.405991+0000","last_peered":"2024-07-27T03:13:57.405991+0000","last_clean":"2024-07-27T03:13:57.405991+0000","last_became_active":"2024-07-27T03:12:56.725685+0000","last_became_peered":"2024-07-27T03:12:56.725685+0000","last_unstale":"2024-07-27T03:13:57.405991+0000","last_undegraded":"2024-07-27T03:13:57.405991+0000","last_fullsized":"2024-07-27T03:13:57.405991+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:53:41.143333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405934+0000","last_change":"2024-07-27T03:12:56.726059+0000","last_active":"2024-07-27T03:13:57.405934+0000","last_peered":"2024-07-27T03:13:57.405934+0000","last_clean":"2024-07-27T03:13:57.405934+0000","last_became_active":"2024-07-27T03:12:56.722294+0000","last_became_peered":"2024-07-27T03:12:56.722294+0000","last_unstale":"2024-07-27T03:13:57.405934+0000","last_undegraded":"2024-07-27T03:13:57.405934+0000","last_fullsized":"2024-07-27T03:13:57.405934+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:18:42.627768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404251+0000","last_change":"2024-07-27T03:12:42.611966+0000","last_active":"2024-07-27T03:13:57.404251+0000","last_peered":"2024-07-27T03:13:57.404251+0000","last_clean":"2024-07-27T03:13:57.404251+0000","last_became_active":"2024-07-27T03:12:42.611522+0000","last_became_peered":"2024-07-27T03:12:42.611522+0000","last_unstale":"2024-07-27T03:13:57.404251+0000","last_undegraded":"2024-07-27T03:13:57.404251+0000","last_fullsized":"2024-07-27T03:13:57.404251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:09:54.034597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.301908+0000","last_change":"2024-07-27T03:12:56.719638+0000","last_active":"2024-07-27T03:13:57.301908+0000","last_peered":"2024-07-27T03:13:57.301908+0000","last_clean":"2024-07-27T03:13:57.301908+0000","last_became_active":"2024-07-27T03:12:56.719227+0000","last_became_peered":"2024-07-27T03:12:56.719227+0000","last_unstale":"2024-07-27T03:13:57.301908+0000","last_undegraded":"2024-07-27T03:13:57.301908+0000","last_fullsized":"2024-07-27T03:13:57.301908+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:54:16.891518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304900+0000","last_change":"2024-07-27T03:12:29.614346+0000","last_active":"2024-07-27T03:13:57.304900+0000","last_peered":"2024-07-27T03:13:57.304900+0000","last_clean":"2024-07-27T03:13:57.304900+0000","last_became_active":"2024-07-27T03:12:29.613672+0000","last_became_peered":"2024-07-27T03:12:29.613672+0000","last_unstale":"2024-07-27T03:13:57.304900+0000","last_undegraded":"2024-07-27T03:13:57.304900+0000","last_fullsized":"2024-07-27T03:13:57.304900+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:46:35.505735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348654+0000","last_change":"2024-07-27T03:12:42.618364+0000","last_active":"2024-07-27T03:13:57.348654+0000","last_peered":"2024-07-27T03:13:57.348654+0000","last_clean":"2024-07-27T03:13:57.348654+0000","last_became_active":"2024-07-27T03:12:42.617988+0000","last_became_peered":"2024-07-27T03:12:42.617988+0000","last_unstale":"2024-07-27T03:13:57.348654+0000","last_undegraded":"2024-07-27T03:13:57.348654+0000","last_fullsized":"2024-07-27T03:13:57.348654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:40:21.725877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305293+0000","last_change":"2024-07-27T03:12:32.563809+0000","last_active":"2024-07-27T03:13:57.305293+0000","last_peered":"2024-07-27T03:13:57.305293+0000","last_clean":"2024-07-27T03:13:57.305293+0000","last_became_active":"2024-07-27T03:12:32.563479+0000","last_became_peered":"2024-07-27T03:12:32.563479+0000","last_unstale":"2024-07-27T03:13:57.305293+0000","last_undegraded":"2024-07-27T03:13:57.305293+0000","last_fullsized":"2024-07-27T03:13:57.305293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:45:47.719241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":75,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:59.283359+0000","last_change":"2024-07-27T03:12:29.543786+0000","last_active":"2024-07-27T03:13:59.283359+0000","last_peered":"2024-07-27T03:13:59.283359+0000","last_clean":"2024-07-27T03:13:59.283359+0000","last_became_active":"2024-07-27T03:12:29.543203+0000","last_became_peered":"2024-07-27T03:12:29.543203+0000","last_unstale":"2024-07-27T03:13:59.283359+0000","last_undegraded":"2024-07-27T03:13:59.283359+0000","last_fullsized":"2024-07-27T03:13:59.283359+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:47:06.742171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302660+0000","last_change":"2024-07-27T03:12:42.622142+0000","last_active":"2024-07-27T03:13:57.302660+0000","last_peered":"2024-07-27T03:13:57.302660+0000","last_clean":"2024-07-27T03:13:57.302660+0000","last_became_active":"2024-07-27T03:12:42.621681+0000","last_became_peered":"2024-07-27T03:12:42.621681+0000","last_unstale":"2024-07-27T03:13:57.302660+0000","last_undegraded":"2024-07-27T03:13:57.302660+0000","last_fullsized":"2024-07-27T03:13:57.302660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:05:08.168424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305949+0000","last_change":"2024-07-27T03:12:32.567119+0000","last_active":"2024-07-27T03:13:57.305949+0000","last_peered":"2024-07-27T03:13:57.305949+0000","last_clean":"2024-07-27T03:13:57.305949+0000","last_became_active":"2024-07-27T03:12:32.566757+0000","last_became_peered":"2024-07-27T03:12:32.566757+0000","last_unstale":"2024-07-27T03:13:57.305949+0000","last_undegraded":"2024-07-27T03:13:57.305949+0000","last_fullsized":"2024-07-27T03:13:57.305949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:31:12.972804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404661+0000","last_change":"2024-07-27T03:12:29.540453+0000","last_active":"2024-07-27T03:13:57.404661+0000","last_peered":"2024-07-27T03:13:57.404661+0000","last_clean":"2024-07-27T03:13:57.404661+0000","last_became_active":"2024-07-27T03:12:29.539195+0000","last_became_peered":"2024-07-27T03:12:29.539195+0000","last_unstale":"2024-07-27T03:13:57.404661+0000","last_undegraded":"2024-07-27T03:13:57.404661+0000","last_fullsized":"2024-07-27T03:13:57.404661+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:44:31.144845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405314+0000","last_change":"2024-07-27T03:12:42.616671+0000","last_active":"2024-07-27T03:13:57.405314+0000","last_peered":"2024-07-27T03:13:57.405314+0000","last_clean":"2024-07-27T03:13:57.405314+0000","last_became_active":"2024-07-27T03:12:42.616198+0000","last_became_peered":"2024-07-27T03:12:42.616198+0000","last_unstale":"2024-07-27T03:13:57.405314+0000","last_undegraded":"2024-07-27T03:13:57.405314+0000","last_fullsized":"2024-07-27T03:13:57.405314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:01:31.584686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348461+0000","last_change":"2024-07-27T03:12:32.562350+0000","last_active":"2024-07-27T03:13:57.348461+0000","last_peered":"2024-07-27T03:13:57.348461+0000","last_clean":"2024-07-27T03:13:57.348461+0000","last_became_active":"2024-07-27T03:12:32.562016+0000","last_became_peered":"2024-07-27T03:12:32.562016+0000","last_unstale":"2024-07-27T03:13:57.348461+0000","last_undegraded":"2024-07-27T03:13:57.348461+0000","last_fullsized":"2024-07-27T03:13:57.348461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:44:35.027329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303171+0000","last_change":"2024-07-27T03:12:29.541387+0000","last_active":"2024-07-27T03:13:57.303171+0000","last_peered":"2024-07-27T03:13:57.303171+0000","last_clean":"2024-07-27T03:13:57.303171+0000","last_became_active":"2024-07-27T03:12:29.541005+0000","last_became_peered":"2024-07-27T03:12:29.541005+0000","last_unstale":"2024-07-27T03:13:57.303171+0000","last_undegraded":"2024-07-27T03:13:57.303171+0000","last_fullsized":"2024-07-27T03:13:57.303171+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:22:16.268348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404762+0000","last_change":"2024-07-27T03:12:42.610029+0000","last_active":"2024-07-27T03:13:57.404762+0000","last_peered":"2024-07-27T03:13:57.404762+0000","last_clean":"2024-07-27T03:13:57.404762+0000","last_became_active":"2024-07-27T03:12:42.609592+0000","last_became_peered":"2024-07-27T03:12:42.609592+0000","last_unstale":"2024-07-27T03:13:57.404762+0000","last_undegraded":"2024-07-27T03:13:57.404762+0000","last_fullsized":"2024-07-27T03:13:57.404762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:47:15.254770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305492+0000","last_change":"2024-07-27T03:12:32.568941+0000","last_active":"2024-07-27T03:13:57.305492+0000","last_peered":"2024-07-27T03:13:57.305492+0000","last_clean":"2024-07-27T03:13:57.305492+0000","last_became_active":"2024-07-27T03:12:32.568531+0000","last_became_peered":"2024-07-27T03:12:32.568531+0000","last_unstale":"2024-07-27T03:13:57.305492+0000","last_undegraded":"2024-07-27T03:13:57.305492+0000","last_fullsized":"2024-07-27T03:13:57.305492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:06:22.020505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'31","reported_seq":122,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:59.283553+0000","last_change":"2024-07-27T03:12:29.545131+0000","last_active":"2024-07-27T03:13:59.283553+0000","last_peered":"2024-07-27T03:13:59.283553+0000","last_clean":"2024-07-27T03:13:59.283553+0000","last_became_active":"2024-07-27T03:12:29.544768+0000","last_became_peered":"2024-07-27T03:12:29.544768+0000","last_unstale":"2024-07-27T03:13:59.283553+0000","last_undegraded":"2024-07-27T03:13:59.283553+0000","last_fullsized":"2024-07-27T03:13:59.283553+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:20:57.556128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52989,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":75,"num_write":33,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302028+0000","last_change":"2024-07-27T03:12:42.617483+0000","last_active":"2024-07-27T03:13:57.302028+0000","last_peered":"2024-07-27T03:13:57.302028+0000","last_clean":"2024-07-27T03:13:57.302028+0000","last_became_active":"2024-07-27T03:12:42.617065+0000","last_became_peered":"2024-07-27T03:12:42.617065+0000","last_unstale":"2024-07-27T03:13:57.302028+0000","last_undegraded":"2024-07-27T03:13:57.302028+0000","last_fullsized":"2024-07-27T03:13:57.302028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:52:17.448919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405841+0000","last_change":"2024-07-27T03:12:56.722257+0000","last_active":"2024-07-27T03:13:57.405841+0000","last_peered":"2024-07-27T03:13:57.405841+0000","last_clean":"2024-07-27T03:13:57.405841+0000","last_became_active":"2024-07-27T03:12:56.721773+0000","last_became_peered":"2024-07-27T03:12:56.721773+0000","last_unstale":"2024-07-27T03:13:57.405841+0000","last_undegraded":"2024-07-27T03:13:57.405841+0000","last_fullsized":"2024-07-27T03:13:57.405841+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:30:44.310705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405141+0000","last_change":"2024-07-27T03:12:29.614171+0000","last_active":"2024-07-27T03:13:57.405141+0000","last_peered":"2024-07-27T03:13:57.405141+0000","last_clean":"2024-07-27T03:13:57.405141+0000","last_became_active":"2024-07-27T03:12:29.613270+0000","last_became_peered":"2024-07-27T03:12:29.613270+0000","last_unstale":"2024-07-27T03:13:57.405141+0000","last_undegraded":"2024-07-27T03:13:57.405141+0000","last_fullsized":"2024-07-27T03:13:57.405141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:39:32.283778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303742+0000","last_change":"2024-07-27T03:12:42.610375+0000","last_active":"2024-07-27T03:13:57.303742+0000","last_peered":"2024-07-27T03:13:57.303742+0000","last_clean":"2024-07-27T03:13:57.303742+0000","last_became_active":"2024-07-27T03:12:42.609957+0000","last_became_peered":"2024-07-27T03:12:42.609957+0000","last_unstale":"2024-07-27T03:13:57.303742+0000","last_undegraded":"2024-07-27T03:13:57.303742+0000","last_fullsized":"2024-07-27T03:13:57.303742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:11:02.707179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405871+0000","last_change":"2024-07-27T03:12:56.722332+0000","last_active":"2024-07-27T03:13:57.405871+0000","last_peered":"2024-07-27T03:13:57.405871+0000","last_clean":"2024-07-27T03:13:57.405871+0000","last_became_active":"2024-07-27T03:12:56.721972+0000","last_became_peered":"2024-07-27T03:12:56.721972+0000","last_unstale":"2024-07-27T03:13:57.405871+0000","last_undegraded":"2024-07-27T03:13:57.405871+0000","last_fullsized":"2024-07-27T03:13:57.405871+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:54:31.880137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306029+0000","last_change":"2024-07-27T03:12:29.541051+0000","last_active":"2024-07-27T03:13:57.306029+0000","last_peered":"2024-07-27T03:13:57.306029+0000","last_clean":"2024-07-27T03:13:57.306029+0000","last_became_active":"2024-07-27T03:12:29.540619+0000","last_became_peered":"2024-07-27T03:12:29.540619+0000","last_unstale":"2024-07-27T03:13:57.306029+0000","last_undegraded":"2024-07-27T03:13:57.306029+0000","last_fullsized":"2024-07-27T03:13:57.306029+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:23:27.587419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304742+0000","last_change":"2024-07-27T03:12:56.720496+0000","last_active":"2024-07-27T03:13:57.304742+0000","last_peered":"2024-07-27T03:13:57.304742+0000","last_clean":"2024-07-27T03:13:57.304742+0000","last_became_active":"2024-07-27T03:12:56.719784+0000","last_became_peered":"2024-07-27T03:12:56.719784+0000","last_unstale":"2024-07-27T03:13:57.304742+0000","last_undegraded":"2024-07-27T03:13:57.304742+0000","last_fullsized":"2024-07-27T03:13:57.304742+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:07:20.518139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406627+0000","last_change":"2024-07-27T03:12:32.568409+0000","last_active":"2024-07-27T03:13:57.406627+0000","last_peered":"2024-07-27T03:13:57.406627+0000","last_clean":"2024-07-27T03:13:57.406627+0000","last_became_active":"2024-07-27T03:12:32.568029+0000","last_became_peered":"2024-07-27T03:12:32.568029+0000","last_unstale":"2024-07-27T03:13:57.406627+0000","last_undegraded":"2024-07-27T03:13:57.406627+0000","last_fullsized":"2024-07-27T03:13:57.406627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:44:53.857986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":88,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:59.282260+0000","last_change":"2024-07-27T03:12:29.544286+0000","last_active":"2024-07-27T03:13:59.282260+0000","last_peered":"2024-07-27T03:13:59.282260+0000","last_clean":"2024-07-27T03:13:59.282260+0000","last_became_active":"2024-07-27T03:12:29.543879+0000","last_became_peered":"2024-07-27T03:12:29.543879+0000","last_unstale":"2024-07-27T03:13:59.282260+0000","last_undegraded":"2024-07-27T03:13:59.282260+0000","last_fullsized":"2024-07-27T03:13:59.282260+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:20:36.997298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404118+0000","last_change":"2024-07-27T03:12:42.609824+0000","last_active":"2024-07-27T03:13:57.404118+0000","last_peered":"2024-07-27T03:13:57.404118+0000","last_clean":"2024-07-27T03:13:57.404118+0000","last_became_active":"2024-07-27T03:12:42.608502+0000","last_became_peered":"2024-07-27T03:12:42.608502+0000","last_unstale":"2024-07-27T03:13:57.404118+0000","last_undegraded":"2024-07-27T03:13:57.404118+0000","last_fullsized":"2024-07-27T03:13:57.404118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:10:23.253921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405119+0000","last_change":"2024-07-27T03:12:42.610744+0000","last_active":"2024-07-27T03:13:57.405119+0000","last_peered":"2024-07-27T03:13:57.405119+0000","last_clean":"2024-07-27T03:13:57.405119+0000","last_became_active":"2024-07-27T03:12:42.610322+0000","last_became_peered":"2024-07-27T03:12:42.610322+0000","last_unstale":"2024-07-27T03:13:57.405119+0000","last_undegraded":"2024-07-27T03:13:57.405119+0000","last_fullsized":"2024-07-27T03:13:57.405119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:01:47.618013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302035+0000","last_change":"2024-07-27T03:12:32.558372+0000","last_active":"2024-07-27T03:13:57.302035+0000","last_peered":"2024-07-27T03:13:57.302035+0000","last_clean":"2024-07-27T03:13:57.302035+0000","last_became_active":"2024-07-27T03:12:32.558003+0000","last_became_peered":"2024-07-27T03:12:32.558003+0000","last_unstale":"2024-07-27T03:13:57.302035+0000","last_undegraded":"2024-07-27T03:13:57.302035+0000","last_fullsized":"2024-07-27T03:13:57.302035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:41:43.673807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404430+0000","last_change":"2024-07-27T03:12:29.543182+0000","last_active":"2024-07-27T03:13:57.404430+0000","last_peered":"2024-07-27T03:13:57.404430+0000","last_clean":"2024-07-27T03:13:57.404430+0000","last_became_active":"2024-07-27T03:12:29.542744+0000","last_became_peered":"2024-07-27T03:12:29.542744+0000","last_unstale":"2024-07-27T03:13:57.404430+0000","last_undegraded":"2024-07-27T03:13:57.404430+0000","last_fullsized":"2024-07-27T03:13:57.404430+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:01:17.276549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405910+0000","last_change":"2024-07-27T03:12:42.622297+0000","last_active":"2024-07-27T03:13:57.405910+0000","last_peered":"2024-07-27T03:13:57.405910+0000","last_clean":"2024-07-27T03:13:57.405910+0000","last_became_active":"2024-07-27T03:12:42.621947+0000","last_became_peered":"2024-07-27T03:12:42.621947+0000","last_unstale":"2024-07-27T03:13:57.405910+0000","last_undegraded":"2024-07-27T03:13:57.405910+0000","last_fullsized":"2024-07-27T03:13:57.405910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:15:59.480216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404143+0000","last_change":"2024-07-27T03:12:32.565987+0000","last_active":"2024-07-27T03:13:57.404143+0000","last_peered":"2024-07-27T03:13:57.404143+0000","last_clean":"2024-07-27T03:13:57.404143+0000","last_became_active":"2024-07-27T03:12:32.565603+0000","last_became_peered":"2024-07-27T03:12:32.565603+0000","last_unstale":"2024-07-27T03:13:57.404143+0000","last_undegraded":"2024-07-27T03:13:57.404143+0000","last_fullsized":"2024-07-27T03:13:57.404143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:30:48.954683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348204+0000","last_change":"2024-07-27T03:12:29.532149+0000","last_active":"2024-07-27T03:13:57.348204+0000","last_peered":"2024-07-27T03:13:57.348204+0000","last_clean":"2024-07-27T03:13:57.348204+0000","last_became_active":"2024-07-27T03:12:29.531761+0000","last_became_peered":"2024-07-27T03:12:29.531761+0000","last_unstale":"2024-07-27T03:13:57.348204+0000","last_undegraded":"2024-07-27T03:13:57.348204+0000","last_fullsized":"2024-07-27T03:13:57.348204+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:44:30.902538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405968+0000","last_change":"2024-07-27T03:12:29.543148+0000","last_active":"2024-07-27T03:13:57.405968+0000","last_peered":"2024-07-27T03:13:57.405968+0000","last_clean":"2024-07-27T03:13:57.405968+0000","last_became_active":"2024-07-27T03:12:29.542774+0000","last_became_peered":"2024-07-27T03:12:29.542774+0000","last_unstale":"2024-07-27T03:13:57.405968+0000","last_undegraded":"2024-07-27T03:13:57.405968+0000","last_fullsized":"2024-07-27T03:13:57.405968+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:20:06.786121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404541+0000","last_change":"2024-07-27T03:12:32.561516+0000","last_active":"2024-07-27T03:13:57.404541+0000","last_peered":"2024-07-27T03:13:57.404541+0000","last_clean":"2024-07-27T03:13:57.404541+0000","last_became_active":"2024-07-27T03:12:32.561145+0000","last_became_peered":"2024-07-27T03:12:32.561145+0000","last_unstale":"2024-07-27T03:13:57.404541+0000","last_undegraded":"2024-07-27T03:13:57.404541+0000","last_fullsized":"2024-07-27T03:13:57.404541+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:49:26.317206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302917+0000","last_change":"2024-07-27T03:12:42.616735+0000","last_active":"2024-07-27T03:13:57.302917+0000","last_peered":"2024-07-27T03:13:57.302917+0000","last_clean":"2024-07-27T03:13:57.302917+0000","last_became_active":"2024-07-27T03:12:42.616171+0000","last_became_peered":"2024-07-27T03:12:42.616171+0000","last_unstale":"2024-07-27T03:13:57.302917+0000","last_undegraded":"2024-07-27T03:13:57.302917+0000","last_fullsized":"2024-07-27T03:13:57.302917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:45:46.236613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":71,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404749+0000","last_change":"2024-07-27T03:12:29.539775+0000","last_active":"2024-07-27T03:13:57.404749+0000","last_peered":"2024-07-27T03:13:57.404749+0000","last_clean":"2024-07-27T03:13:57.404749+0000","last_became_active":"2024-07-27T03:12:29.539276+0000","last_became_peered":"2024-07-27T03:12:29.539276+0000","last_unstale":"2024-07-27T03:13:57.404749+0000","last_undegraded":"2024-07-27T03:13:57.404749+0000","last_fullsized":"2024-07-27T03:13:57.404749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:09:19.364679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305066+0000","last_change":"2024-07-27T03:12:32.562673+0000","last_active":"2024-07-27T03:13:57.305066+0000","last_peered":"2024-07-27T03:13:57.305066+0000","last_clean":"2024-07-27T03:13:57.305066+0000","last_became_active":"2024-07-27T03:12:32.561591+0000","last_became_peered":"2024-07-27T03:12:32.561591+0000","last_unstale":"2024-07-27T03:13:57.305066+0000","last_undegraded":"2024-07-27T03:13:57.305066+0000","last_fullsized":"2024-07-27T03:13:57.305066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:49:38.095004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303348+0000","last_change":"2024-07-27T03:12:42.611756+0000","last_active":"2024-07-27T03:13:57.303348+0000","last_peered":"2024-07-27T03:13:57.303348+0000","last_clean":"2024-07-27T03:13:57.303348+0000","last_became_active":"2024-07-27T03:12:42.611246+0000","last_became_peered":"2024-07-27T03:12:42.611246+0000","last_unstale":"2024-07-27T03:13:57.303348+0000","last_undegraded":"2024-07-27T03:13:57.303348+0000","last_fullsized":"2024-07-27T03:13:57.303348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:13:06.027887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405629+0000","last_change":"2024-07-27T03:12:32.571339+0000","last_active":"2024-07-27T03:13:57.405629+0000","last_peered":"2024-07-27T03:13:57.405629+0000","last_clean":"2024-07-27T03:13:57.405629+0000","last_became_active":"2024-07-27T03:12:32.570973+0000","last_became_peered":"2024-07-27T03:12:32.570973+0000","last_unstale":"2024-07-27T03:13:57.405629+0000","last_undegraded":"2024-07-27T03:13:57.405629+0000","last_fullsized":"2024-07-27T03:13:57.405629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:24:25.357833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303303+0000","last_change":"2024-07-27T03:12:32.561243+0000","last_active":"2024-07-27T03:13:57.303303+0000","last_peered":"2024-07-27T03:13:57.303303+0000","last_clean":"2024-07-27T03:13:57.303303+0000","last_became_active":"2024-07-27T03:12:32.560748+0000","last_became_peered":"2024-07-27T03:12:32.560748+0000","last_unstale":"2024-07-27T03:13:57.303303+0000","last_undegraded":"2024-07-27T03:13:57.303303+0000","last_fullsized":"2024-07-27T03:13:57.303303+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:47:40.430283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302860+0000","last_change":"2024-07-27T03:12:32.562545+0000","last_active":"2024-07-27T03:13:57.302860+0000","last_peered":"2024-07-27T03:13:57.302860+0000","last_clean":"2024-07-27T03:13:57.302860+0000","last_became_active":"2024-07-27T03:12:32.562202+0000","last_became_peered":"2024-07-27T03:12:32.562202+0000","last_unstale":"2024-07-27T03:13:57.302860+0000","last_undegraded":"2024-07-27T03:13:57.302860+0000","last_fullsized":"2024-07-27T03:13:57.302860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:05:20.229232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405409+0000","last_change":"2024-07-27T03:12:32.565162+0000","last_active":"2024-07-27T03:13:57.405409+0000","last_peered":"2024-07-27T03:13:57.405409+0000","last_clean":"2024-07-27T03:13:57.405409+0000","last_became_active":"2024-07-27T03:12:32.564798+0000","last_became_peered":"2024-07-27T03:12:32.564798+0000","last_unstale":"2024-07-27T03:13:57.405409+0000","last_undegraded":"2024-07-27T03:13:57.405409+0000","last_fullsized":"2024-07-27T03:13:57.405409+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:54:29.742624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348688+0000","last_change":"2024-07-27T03:12:32.556874+0000","last_active":"2024-07-27T03:13:57.348688+0000","last_peered":"2024-07-27T03:13:57.348688+0000","last_clean":"2024-07-27T03:13:57.348688+0000","last_became_active":"2024-07-27T03:12:32.556467+0000","last_became_peered":"2024-07-27T03:12:32.556467+0000","last_unstale":"2024-07-27T03:13:57.348688+0000","last_undegraded":"2024-07-27T03:13:57.348688+0000","last_fullsized":"2024-07-27T03:13:57.348688+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:51:20.806785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405956+0000","last_change":"2024-07-27T03:12:32.569136+0000","last_active":"2024-07-27T03:13:57.405956+0000","last_peered":"2024-07-27T03:13:57.405956+0000","last_clean":"2024-07-27T03:13:57.405956+0000","last_became_active":"2024-07-27T03:12:32.568778+0000","last_became_peered":"2024-07-27T03:12:32.568778+0000","last_unstale":"2024-07-27T03:13:57.405956+0000","last_undegraded":"2024-07-27T03:13:57.405956+0000","last_fullsized":"2024-07-27T03:13:57.405956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:26:22.654974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303473+0000","last_change":"2024-07-27T03:12:32.569507+0000","last_active":"2024-07-27T03:13:57.303473+0000","last_peered":"2024-07-27T03:13:57.303473+0000","last_clean":"2024-07-27T03:13:57.303473+0000","last_became_active":"2024-07-27T03:12:32.569109+0000","last_became_peered":"2024-07-27T03:12:32.569109+0000","last_unstale":"2024-07-27T03:13:57.303473+0000","last_undegraded":"2024-07-27T03:13:57.303473+0000","last_fullsized":"2024-07-27T03:13:57.303473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:35:12.944933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405538+0000","last_change":"2024-07-27T03:12:32.566479+0000","last_active":"2024-07-27T03:13:57.405538+0000","last_peered":"2024-07-27T03:13:57.405538+0000","last_clean":"2024-07-27T03:13:57.405538+0000","last_became_active":"2024-07-27T03:12:32.566114+0000","last_became_peered":"2024-07-27T03:12:32.566114+0000","last_unstale":"2024-07-27T03:13:57.405538+0000","last_undegraded":"2024-07-27T03:13:57.405538+0000","last_fullsized":"2024-07-27T03:13:57.405538+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:24:19.944031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404192+0000","last_change":"2024-07-27T03:12:56.726118+0000","last_active":"2024-07-27T03:13:57.404192+0000","last_peered":"2024-07-27T03:13:57.404192+0000","last_clean":"2024-07-27T03:13:57.404192+0000","last_became_active":"2024-07-27T03:12:56.725637+0000","last_became_peered":"2024-07-27T03:12:56.725637+0000","last_unstale":"2024-07-27T03:13:57.404192+0000","last_undegraded":"2024-07-27T03:13:57.404192+0000","last_fullsized":"2024-07-27T03:13:57.404192+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:36:41.725150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404347+0000","last_change":"2024-07-27T03:12:32.568232+0000","last_active":"2024-07-27T03:13:57.404347+0000","last_peered":"2024-07-27T03:13:57.404347+0000","last_clean":"2024-07-27T03:13:57.404347+0000","last_became_active":"2024-07-27T03:12:32.567834+0000","last_became_peered":"2024-07-27T03:12:32.567834+0000","last_unstale":"2024-07-27T03:13:57.404347+0000","last_undegraded":"2024-07-27T03:13:57.404347+0000","last_fullsized":"2024-07-27T03:13:57.404347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:40:28.234539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405550+0000","last_change":"2024-07-27T03:12:32.573451+0000","last_active":"2024-07-27T03:13:57.405550+0000","last_peered":"2024-07-27T03:13:57.405550+0000","last_clean":"2024-07-27T03:13:57.405550+0000","last_became_active":"2024-07-27T03:12:32.573066+0000","last_became_peered":"2024-07-27T03:12:32.573066+0000","last_unstale":"2024-07-27T03:13:57.405550+0000","last_undegraded":"2024-07-27T03:13:57.405550+0000","last_fullsized":"2024-07-27T03:13:57.405550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:13:29.104055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348631+0000","last_change":"2024-07-27T03:12:32.561067+0000","last_active":"2024-07-27T03:13:57.348631+0000","last_peered":"2024-07-27T03:13:57.348631+0000","last_clean":"2024-07-27T03:13:57.348631+0000","last_became_active":"2024-07-27T03:12:32.560358+0000","last_became_peered":"2024-07-27T03:12:32.560358+0000","last_unstale":"2024-07-27T03:13:57.348631+0000","last_undegraded":"2024-07-27T03:13:57.348631+0000","last_fullsized":"2024-07-27T03:13:57.348631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:41:41.026804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405482+0000","last_change":"2024-07-27T03:12:32.572290+0000","last_active":"2024-07-27T03:13:57.405482+0000","last_peered":"2024-07-27T03:13:57.405482+0000","last_clean":"2024-07-27T03:13:57.405482+0000","last_became_active":"2024-07-27T03:12:32.571877+0000","last_became_peered":"2024-07-27T03:12:32.571877+0000","last_unstale":"2024-07-27T03:13:57.405482+0000","last_undegraded":"2024-07-27T03:13:57.405482+0000","last_fullsized":"2024-07-27T03:13:57.405482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:34:40.559787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303421+0000","last_change":"2024-07-27T03:12:32.565495+0000","last_active":"2024-07-27T03:13:57.303421+0000","last_peered":"2024-07-27T03:13:57.303421+0000","last_clean":"2024-07-27T03:13:57.303421+0000","last_became_active":"2024-07-27T03:12:32.563136+0000","last_became_peered":"2024-07-27T03:12:32.563136+0000","last_unstale":"2024-07-27T03:13:57.303421+0000","last_undegraded":"2024-07-27T03:13:57.303421+0000","last_fullsized":"2024-07-27T03:13:57.303421+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:31:02.032776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302591+0000","last_change":"2024-07-27T03:12:32.569477+0000","last_active":"2024-07-27T03:13:57.302591+0000","last_peered":"2024-07-27T03:13:57.302591+0000","last_clean":"2024-07-27T03:13:57.302591+0000","last_became_active":"2024-07-27T03:12:32.569098+0000","last_became_peered":"2024-07-27T03:12:32.569098+0000","last_unstale":"2024-07-27T03:13:57.302591+0000","last_undegraded":"2024-07-27T03:13:57.302591+0000","last_fullsized":"2024-07-27T03:13:57.302591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:44:26.082539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406145+0000","last_change":"2024-07-27T03:12:32.574704+0000","last_active":"2024-07-27T03:13:57.406145+0000","last_peered":"2024-07-27T03:13:57.406145+0000","last_clean":"2024-07-27T03:13:57.406145+0000","last_became_active":"2024-07-27T03:12:32.574261+0000","last_became_peered":"2024-07-27T03:12:32.574261+0000","last_unstale":"2024-07-27T03:13:57.406145+0000","last_undegraded":"2024-07-27T03:13:57.406145+0000","last_fullsized":"2024-07-27T03:13:57.406145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:53:06.445267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405109+0000","last_change":"2024-07-27T03:12:32.573835+0000","last_active":"2024-07-27T03:13:57.405109+0000","last_peered":"2024-07-27T03:13:57.405109+0000","last_clean":"2024-07-27T03:13:57.405109+0000","last_became_active":"2024-07-27T03:12:32.573450+0000","last_became_peered":"2024-07-27T03:12:32.573450+0000","last_unstale":"2024-07-27T03:13:57.405109+0000","last_undegraded":"2024-07-27T03:13:57.405109+0000","last_fullsized":"2024-07-27T03:13:57.405109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:40:45.264000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404917+0000","last_change":"2024-07-27T03:12:32.568542+0000","last_active":"2024-07-27T03:13:57.404917+0000","last_peered":"2024-07-27T03:13:57.404917+0000","last_clean":"2024-07-27T03:13:57.404917+0000","last_became_active":"2024-07-27T03:12:32.568179+0000","last_became_peered":"2024-07-27T03:12:32.568179+0000","last_unstale":"2024-07-27T03:13:57.404917+0000","last_undegraded":"2024-07-27T03:13:57.404917+0000","last_fullsized":"2024-07-27T03:13:57.404917+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:32:10.005601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404879+0000","last_change":"2024-07-27T03:12:32.564063+0000","last_active":"2024-07-27T03:13:57.404879+0000","last_peered":"2024-07-27T03:13:57.404879+0000","last_clean":"2024-07-27T03:13:57.404879+0000","last_became_active":"2024-07-27T03:12:32.563705+0000","last_became_peered":"2024-07-27T03:12:32.563705+0000","last_unstale":"2024-07-27T03:13:57.404879+0000","last_undegraded":"2024-07-27T03:13:57.404879+0000","last_fullsized":"2024-07-27T03:13:57.404879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:40:19.234194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348574+0000","last_change":"2024-07-27T03:12:57.178963+0000","last_active":"2024-07-27T03:13:57.348574+0000","last_peered":"2024-07-27T03:13:57.348574+0000","last_clean":"2024-07-27T03:13:57.348574+0000","last_became_active":"2024-07-27T03:12:57.178550+0000","last_became_peered":"2024-07-27T03:12:57.178550+0000","last_unstale":"2024-07-27T03:13:57.348574+0000","last_undegraded":"2024-07-27T03:13:57.348574+0000","last_fullsized":"2024-07-27T03:13:57.348574+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:20:06.586957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304191+0000","last_change":"2024-07-27T03:12:32.569863+0000","last_active":"2024-07-27T03:13:57.304191+0000","last_peered":"2024-07-27T03:13:57.304191+0000","last_clean":"2024-07-27T03:13:57.304191+0000","last_became_active":"2024-07-27T03:12:32.566457+0000","last_became_peered":"2024-07-27T03:12:32.566457+0000","last_unstale":"2024-07-27T03:13:57.304191+0000","last_undegraded":"2024-07-27T03:13:57.304191+0000","last_fullsized":"2024-07-27T03:13:57.304191+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:28:13.141002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305935+0000","last_change":"2024-07-27T03:12:32.574490+0000","last_active":"2024-07-27T03:13:57.305935+0000","last_peered":"2024-07-27T03:13:57.305935+0000","last_clean":"2024-07-27T03:13:57.305935+0000","last_became_active":"2024-07-27T03:12:32.574130+0000","last_became_peered":"2024-07-27T03:12:32.574130+0000","last_unstale":"2024-07-27T03:13:57.305935+0000","last_undegraded":"2024-07-27T03:13:57.305935+0000","last_fullsized":"2024-07-27T03:13:57.305935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:44:22.696669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406481+0000","last_change":"2024-07-27T03:12:32.560106+0000","last_active":"2024-07-27T03:13:57.406481+0000","last_peered":"2024-07-27T03:13:57.406481+0000","last_clean":"2024-07-27T03:13:57.406481+0000","last_became_active":"2024-07-27T03:12:32.559751+0000","last_became_peered":"2024-07-27T03:12:32.559751+0000","last_unstale":"2024-07-27T03:13:57.406481+0000","last_undegraded":"2024-07-27T03:13:57.406481+0000","last_fullsized":"2024-07-27T03:13:57.406481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:16:37.647462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406736+0000","last_change":"2024-07-27T03:12:32.569352+0000","last_active":"2024-07-27T03:13:57.406736+0000","last_peered":"2024-07-27T03:13:57.406736+0000","last_clean":"2024-07-27T03:13:57.406736+0000","last_became_active":"2024-07-27T03:12:32.569004+0000","last_became_peered":"2024-07-27T03:12:32.569004+0000","last_unstale":"2024-07-27T03:13:57.406736+0000","last_undegraded":"2024-07-27T03:13:57.406736+0000","last_fullsized":"2024-07-27T03:13:57.406736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:21:25.999214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404059+0000","last_change":"2024-07-27T03:12:32.562775+0000","last_active":"2024-07-27T03:13:57.404059+0000","last_peered":"2024-07-27T03:13:57.404059+0000","last_clean":"2024-07-27T03:13:57.404059+0000","last_became_active":"2024-07-27T03:12:32.562403+0000","last_became_peered":"2024-07-27T03:12:32.562403+0000","last_unstale":"2024-07-27T03:13:57.404059+0000","last_undegraded":"2024-07-27T03:13:57.404059+0000","last_fullsized":"2024-07-27T03:13:57.404059+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:16:09.474505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305528+0000","last_change":"2024-07-27T03:12:32.563271+0000","last_active":"2024-07-27T03:13:57.305528+0000","last_peered":"2024-07-27T03:13:57.305528+0000","last_clean":"2024-07-27T03:13:57.305528+0000","last_became_active":"2024-07-27T03:12:32.562850+0000","last_became_peered":"2024-07-27T03:12:32.562850+0000","last_unstale":"2024-07-27T03:13:57.305528+0000","last_undegraded":"2024-07-27T03:13:57.305528+0000","last_fullsized":"2024-07-27T03:13:57.305528+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:40:27.688205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405933+0000","last_change":"2024-07-27T03:12:32.557580+0000","last_active":"2024-07-27T03:13:57.405933+0000","last_peered":"2024-07-27T03:13:57.405933+0000","last_clean":"2024-07-27T03:13:57.405933+0000","last_became_active":"2024-07-27T03:12:32.557190+0000","last_became_peered":"2024-07-27T03:12:32.557190+0000","last_unstale":"2024-07-27T03:13:57.405933+0000","last_undegraded":"2024-07-27T03:13:57.405933+0000","last_fullsized":"2024-07-27T03:13:57.405933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:58:17.609035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304839+0000","last_change":"2024-07-27T03:12:32.563669+0000","last_active":"2024-07-27T03:13:57.304839+0000","last_peered":"2024-07-27T03:13:57.304839+0000","last_clean":"2024-07-27T03:13:57.304839+0000","last_became_active":"2024-07-27T03:12:32.563200+0000","last_became_peered":"2024-07-27T03:12:32.563200+0000","last_unstale":"2024-07-27T03:13:57.304839+0000","last_undegraded":"2024-07-27T03:13:57.304839+0000","last_fullsized":"2024-07-27T03:13:57.304839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:14:05.624118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304116+0000","last_change":"2024-07-27T03:12:32.561793+0000","last_active":"2024-07-27T03:13:57.304116+0000","last_peered":"2024-07-27T03:13:57.304116+0000","last_clean":"2024-07-27T03:13:57.304116+0000","last_became_active":"2024-07-27T03:12:32.561452+0000","last_became_peered":"2024-07-27T03:12:32.561452+0000","last_unstale":"2024-07-27T03:13:57.304116+0000","last_undegraded":"2024-07-27T03:13:57.304116+0000","last_fullsized":"2024-07-27T03:13:57.304116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:22:07.204824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405250+0000","last_change":"2024-07-27T03:12:32.569811+0000","last_active":"2024-07-27T03:13:57.405250+0000","last_peered":"2024-07-27T03:13:57.405250+0000","last_clean":"2024-07-27T03:13:57.405250+0000","last_became_active":"2024-07-27T03:12:32.569474+0000","last_became_peered":"2024-07-27T03:12:32.569474+0000","last_unstale":"2024-07-27T03:13:57.405250+0000","last_undegraded":"2024-07-27T03:13:57.405250+0000","last_fullsized":"2024-07-27T03:13:57.405250+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:38:39.317362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405291+0000","last_change":"2024-07-27T03:12:32.565159+0000","last_active":"2024-07-27T03:13:57.405291+0000","last_peered":"2024-07-27T03:13:57.405291+0000","last_clean":"2024-07-27T03:13:57.405291+0000","last_became_active":"2024-07-27T03:12:32.564750+0000","last_became_peered":"2024-07-27T03:12:32.564750+0000","last_unstale":"2024-07-27T03:13:57.405291+0000","last_undegraded":"2024-07-27T03:13:57.405291+0000","last_fullsized":"2024-07-27T03:13:57.405291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:51:26.186510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406117+0000","last_change":"2024-07-27T03:12:32.574429+0000","last_active":"2024-07-27T03:13:57.406117+0000","last_peered":"2024-07-27T03:13:57.406117+0000","last_clean":"2024-07-27T03:13:57.406117+0000","last_became_active":"2024-07-27T03:12:32.573986+0000","last_became_peered":"2024-07-27T03:12:32.573986+0000","last_unstale":"2024-07-27T03:13:57.406117+0000","last_undegraded":"2024-07-27T03:13:57.406117+0000","last_fullsized":"2024-07-27T03:13:57.406117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:35:00.345561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60193,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":241,"num_read_kb":187,"num_write":243,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":185397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645344,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":525562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841573,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37040,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708112,"statfs":{"total":95995035648,"available":95957106688,"internally_reserved":0,"allocated":1740800,"data_stored":1169832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037802,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1077248,"data_stored":525470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234030,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":525826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430259,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1671168,"data_stored":1122736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593784,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":520974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790012,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1679360,"data_stored":1117927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953538,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1060864,"data_stored":525439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149767,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":520682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1130496,"data_stored":575604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":579811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1077248,"data_stored":522430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":59559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":55352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":55116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-27T03:14:05.108 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph pg dump --format=json 2024-07-27T03:14:05.381 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:05.603 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:05 smithi067 ceph-mon[32814]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:14:05.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:05 smithi175 ceph-mon[37125]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:14:05.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:05 smithi149 ceph-mon[37173]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:14:06.584 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:06 smithi067 ceph-mon[32814]: pgmap v356: 129 pgs: 129 active+clean; 640 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.9 KiB/s wr, 11 op/s; 13 B/s, 0 objects/s recovering 2024-07-27T03:14:06.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:06 smithi175 ceph-mon[37125]: pgmap v356: 129 pgs: 129 active+clean; 640 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.9 KiB/s wr, 11 op/s; 13 B/s, 0 objects/s recovering 2024-07-27T03:14:06.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:06 smithi149 ceph-mon[37173]: pgmap v356: 129 pgs: 129 active+clean; 640 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.9 KiB/s wr, 11 op/s; 13 B/s, 0 objects/s recovering 2024-07-27T03:14:07.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.148+0000 7f64307c6640 1 -- 172.21.15.67:0/4032382081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6428104d40 msgr2=0x7f642810d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:07.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.148+0000 7f64307c6640 1 --2- 172.21.15.67:0/4032382081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6428104d40 0x7f642810d380 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6418009900 tx=0x7f641802a410 comp rx=0 tx=0).stop 2024-07-27T03:14:07.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.148+0000 7f64307c6640 1 -- 172.21.15.67:0/4032382081 shutdown_connections 2024-07-27T03:14:07.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.148+0000 7f64307c6640 1 --2- 172.21.15.67:0/4032382081 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f642810d8c0 0x7f642810fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:07.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.148+0000 7f64307c6640 1 --2- 172.21.15.67:0/4032382081 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6428104d40 0x7f642810d380 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:07.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.148+0000 7f64307c6640 1 --2- 172.21.15.67:0/4032382081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6428104310 0x7f6428104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:07.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.148+0000 7f64307c6640 1 -- 172.21.15.67:0/4032382081 >> 172.21.15.67:0/4032382081 conn(0x7f64280791a0 msgr2=0x7f64280795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:07.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.148+0000 7f64307c6640 1 -- 172.21.15.67:0/4032382081 shutdown_connections 2024-07-27T03:14:07.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.148+0000 7f64307c6640 1 -- 172.21.15.67:0/4032382081 wait complete. 2024-07-27T03:14:07.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.149+0000 7f64307c6640 1 Processor -- start 2024-07-27T03:14:07.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.149+0000 7f64307c6640 1 -- start start 2024-07-27T03:14:07.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.149+0000 7f64307c6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6428104310 0x7f6428102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:07.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.149+0000 7f64307c6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6428104d40 0x7f6428102760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:07.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.149+0000 7f64307c6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f642810d8c0 0x7f6428102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:07.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.149+0000 7f64307c6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f64281134b0 con 0x7f6428104d40 2024-07-27T03:14:07.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.149+0000 7f64307c6640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6428113330 con 0x7f6428104310 2024-07-27T03:14:07.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.149+0000 7f64307c6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6428113630 con 0x7f642810d8c0 2024-07-27T03:14:07.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.149+0000 7f642dd61640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6428104d40 0x7f6428102760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:07.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.149+0000 7f642dd61640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6428104d40 0x7f6428102760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58656/0 (socket says 172.21.15.67:58656) 2024-07-27T03:14:07.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.149+0000 7f642dd61640 1 -- 172.21.15.67:0/3232096650 learned_addr learned my addr 172.21.15.67:0/3232096650 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:14:07.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.149+0000 7f642e562640 1 --2- 172.21.15.67:0/3232096650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6428104310 0x7f6428102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:07.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.149+0000 7f642ed63640 1 --2- 172.21.15.67:0/3232096650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f642810d8c0 0x7f6428102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:07.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.149+0000 7f642dd61640 1 -- 172.21.15.67:0/3232096650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f642810d8c0 msgr2=0x7f6428102ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:07.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.149+0000 7f642dd61640 1 --2- 172.21.15.67:0/3232096650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f642810d8c0 0x7f6428102ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:07.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.149+0000 7f642dd61640 1 -- 172.21.15.67:0/3232096650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6428104310 msgr2=0x7f6428102220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:07.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.149+0000 7f642dd61640 1 --2- 172.21.15.67:0/3232096650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6428104310 0x7f6428102220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:07.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.149+0000 7f642dd61640 1 -- 172.21.15.67:0/3232096650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64281a8220 con 0x7f6428104d40 2024-07-27T03:14:07.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.150+0000 7f642dd61640 1 --2- 172.21.15.67:0/3232096650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6428104d40 0x7f6428102760 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6418032fd0 tx=0x7f6418004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:07.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.150+0000 7f641f7fe640 1 -- 172.21.15.67:0/3232096650 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641803f030 con 0x7f6428104d40 2024-07-27T03:14:07.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.150+0000 7f641f7fe640 1 -- 172.21.15.67:0/3232096650 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6418004490 con 0x7f6428104d40 2024-07-27T03:14:07.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.150+0000 7f64307c6640 1 -- 172.21.15.67:0/3232096650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64281a84b0 con 0x7f6428104d40 2024-07-27T03:14:07.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.150+0000 7f641f7fe640 1 -- 172.21.15.67:0/3232096650 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6418035070 con 0x7f6428104d40 2024-07-27T03:14:07.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.150+0000 7f64307c6640 1 -- 172.21.15.67:0/3232096650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64281a89e0 con 0x7f6428104d40 2024-07-27T03:14:07.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.151+0000 7f64307c6640 1 -- 172.21.15.67:0/3232096650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6428104720 con 0x7f6428104d40 2024-07-27T03:14:07.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.153+0000 7f641f7fe640 1 -- 172.21.15.67:0/3232096650 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f6418042030 con 0x7f6428104d40 2024-07-27T03:14:07.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.153+0000 7f641f7fe640 1 --2- 172.21.15.67:0/3232096650 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f64140b3ce0 0x7f64140b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:07.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.154+0000 7f642e562640 1 --2- 172.21.15.67:0/3232096650 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f64140b3ce0 0x7f64140b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:07.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.154+0000 7f641f7fe640 1 -- 172.21.15.67:0/3232096650 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f6418104ae0 con 0x7f6428104d40 2024-07-27T03:14:07.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.154+0000 7f642e562640 1 --2- 172.21.15.67:0/3232096650 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f64140b3ce0 0x7f64140b61f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6424005fd0 tx=0x7f6424005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:07.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.157+0000 7f641f7fe640 1 -- 172.21.15.67:0/3232096650 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f64180cbc20 con 0x7f6428104d40 2024-07-27T03:14:07.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.338+0000 7f64307c6640 1 -- 172.21.15.67:0/3232096650 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6428108a30 con 0x7f64140b3ce0 2024-07-27T03:14:07.346 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.344+0000 7f641f7fe640 1 -- 172.21.15.67:0/3232096650 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340416 (secure 0 0 0) 0x7f6428108a30 con 0x7f64140b3ce0 2024-07-27T03:14:07.350 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:14:07.352 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-07-27T03:14:07.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.348+0000 7f64307c6640 1 -- 172.21.15.67:0/3232096650 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f64140b3ce0 msgr2=0x7f64140b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:07.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.348+0000 7f64307c6640 1 --2- 172.21.15.67:0/3232096650 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f64140b3ce0 0x7f64140b61f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6424005fd0 tx=0x7f6424005850 comp rx=0 tx=0).stop 2024-07-27T03:14:07.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.348+0000 7f64307c6640 1 -- 172.21.15.67:0/3232096650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6428104d40 msgr2=0x7f6428102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:07.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.348+0000 7f64307c6640 1 --2- 172.21.15.67:0/3232096650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6428104d40 0x7f6428102760 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6418032fd0 tx=0x7f6418004270 comp rx=0 tx=0).stop 2024-07-27T03:14:07.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.348+0000 7f64307c6640 1 -- 172.21.15.67:0/3232096650 shutdown_connections 2024-07-27T03:14:07.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.348+0000 7f64307c6640 1 --2- 172.21.15.67:0/3232096650 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f64140b3ce0 0x7f64140b61f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:07.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.348+0000 7f64307c6640 1 --2- 172.21.15.67:0/3232096650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f642810d8c0 0x7f6428102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:07.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.348+0000 7f64307c6640 1 --2- 172.21.15.67:0/3232096650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6428104d40 0x7f6428102760 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:07.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.348+0000 7f64307c6640 1 --2- 172.21.15.67:0/3232096650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6428104310 0x7f6428102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:07.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.348+0000 7f64307c6640 1 -- 172.21.15.67:0/3232096650 >> 172.21.15.67:0/3232096650 conn(0x7f64280791a0 msgr2=0x7f64280ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:07.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.348+0000 7f64307c6640 1 -- 172.21.15.67:0/3232096650 shutdown_connections 2024-07-27T03:14:07.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:07.348+0000 7f64307c6640 1 -- 172.21.15.67:0/3232096650 wait complete. 2024-07-27T03:14:08.147 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-07-27T03:14:07.069836+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655613,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":368,"num_read_kb":281,"num_write":373,"num_write_kb":2289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951979,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":307,"ondisk_log_size":307,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388624,"kb_used_data":14460,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553200,"statfs":{"total":1151940427776,"available":1151542476800,"internally_reserved":0,"allocated":14807040,"data_stored":8246194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13328,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":102,"num_read_kb":64,"num_write":12,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002973"},"pg_stats":[{"pgid":"2.1f","version":"89'13","reported_seq":84,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:04.289312+0000","last_change":"2024-07-27T03:12:29.541107+0000","last_active":"2024-07-27T03:14:04.289312+0000","last_peered":"2024-07-27T03:14:04.289312+0000","last_clean":"2024-07-27T03:14:04.289312+0000","last_became_active":"2024-07-27T03:12:29.540806+0000","last_became_peered":"2024-07-27T03:12:29.540806+0000","last_unstale":"2024-07-27T03:14:04.289312+0000","last_undegraded":"2024-07-27T03:14:04.289312+0000","last_fullsized":"2024-07-27T03:14:04.289312+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:38:24.349354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404603+0000","last_change":"2024-07-27T03:12:32.565515+0000","last_active":"2024-07-27T03:13:57.404603+0000","last_peered":"2024-07-27T03:13:57.404603+0000","last_clean":"2024-07-27T03:13:57.404603+0000","last_became_active":"2024-07-27T03:12:32.565134+0000","last_became_peered":"2024-07-27T03:12:32.565134+0000","last_unstale":"2024-07-27T03:13:57.404603+0000","last_undegraded":"2024-07-27T03:13:57.404603+0000","last_fullsized":"2024-07-27T03:13:57.404603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:11:56.793139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302675+0000","last_change":"2024-07-27T03:12:42.613432+0000","last_active":"2024-07-27T03:13:57.302675+0000","last_peered":"2024-07-27T03:13:57.302675+0000","last_clean":"2024-07-27T03:13:57.302675+0000","last_became_active":"2024-07-27T03:12:42.612986+0000","last_became_peered":"2024-07-27T03:12:42.612986+0000","last_unstale":"2024-07-27T03:13:57.302675+0000","last_undegraded":"2024-07-27T03:13:57.302675+0000","last_fullsized":"2024-07-27T03:13:57.302675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:25:37.505117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405571+0000","last_change":"2024-07-27T03:12:29.614229+0000","last_active":"2024-07-27T03:13:57.405571+0000","last_peered":"2024-07-27T03:13:57.405571+0000","last_clean":"2024-07-27T03:13:57.405571+0000","last_became_active":"2024-07-27T03:12:29.613844+0000","last_became_peered":"2024-07-27T03:12:29.613844+0000","last_unstale":"2024-07-27T03:13:57.405571+0000","last_undegraded":"2024-07-27T03:13:57.405571+0000","last_fullsized":"2024-07-27T03:13:57.405571+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:10:28.917080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405727+0000","last_change":"2024-07-27T03:12:32.563187+0000","last_active":"2024-07-27T03:13:57.405727+0000","last_peered":"2024-07-27T03:13:57.405727+0000","last_clean":"2024-07-27T03:13:57.405727+0000","last_became_active":"2024-07-27T03:12:32.562674+0000","last_became_peered":"2024-07-27T03:12:32.562674+0000","last_unstale":"2024-07-27T03:13:57.405727+0000","last_undegraded":"2024-07-27T03:13:57.405727+0000","last_fullsized":"2024-07-27T03:13:57.405727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:05:39.211514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302984+0000","last_change":"2024-07-27T03:12:42.611516+0000","last_active":"2024-07-27T03:13:57.302984+0000","last_peered":"2024-07-27T03:13:57.302984+0000","last_clean":"2024-07-27T03:13:57.302984+0000","last_became_active":"2024-07-27T03:12:42.610934+0000","last_became_peered":"2024-07-27T03:12:42.610934+0000","last_unstale":"2024-07-27T03:13:57.302984+0000","last_undegraded":"2024-07-27T03:13:57.302984+0000","last_fullsized":"2024-07-27T03:13:57.302984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:13:17.323063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305683+0000","last_change":"2024-07-27T03:12:29.539573+0000","last_active":"2024-07-27T03:13:57.305683+0000","last_peered":"2024-07-27T03:13:57.305683+0000","last_clean":"2024-07-27T03:13:57.305683+0000","last_became_active":"2024-07-27T03:12:29.539056+0000","last_became_peered":"2024-07-27T03:12:29.539056+0000","last_unstale":"2024-07-27T03:13:57.305683+0000","last_undegraded":"2024-07-27T03:13:57.305683+0000","last_fullsized":"2024-07-27T03:13:57.305683+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:04:37.513644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405607+0000","last_change":"2024-07-27T03:12:32.567729+0000","last_active":"2024-07-27T03:13:57.405607+0000","last_peered":"2024-07-27T03:13:57.405607+0000","last_clean":"2024-07-27T03:13:57.405607+0000","last_became_active":"2024-07-27T03:12:32.564827+0000","last_became_peered":"2024-07-27T03:12:32.564827+0000","last_unstale":"2024-07-27T03:13:57.405607+0000","last_undegraded":"2024-07-27T03:13:57.405607+0000","last_fullsized":"2024-07-27T03:13:57.405607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:10:55.314988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303275+0000","last_change":"2024-07-27T03:12:42.612776+0000","last_active":"2024-07-27T03:13:57.303275+0000","last_peered":"2024-07-27T03:13:57.303275+0000","last_clean":"2024-07-27T03:13:57.303275+0000","last_became_active":"2024-07-27T03:12:42.612258+0000","last_became_peered":"2024-07-27T03:12:42.612258+0000","last_unstale":"2024-07-27T03:13:57.303275+0000","last_undegraded":"2024-07-27T03:13:57.303275+0000","last_fullsized":"2024-07-27T03:13:57.303275+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:48:06.516002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305873+0000","last_change":"2024-07-27T03:12:29.540070+0000","last_active":"2024-07-27T03:13:57.305873+0000","last_peered":"2024-07-27T03:13:57.305873+0000","last_clean":"2024-07-27T03:13:57.305873+0000","last_became_active":"2024-07-27T03:12:29.539657+0000","last_became_peered":"2024-07-27T03:12:29.539657+0000","last_unstale":"2024-07-27T03:13:57.305873+0000","last_undegraded":"2024-07-27T03:13:57.305873+0000","last_fullsized":"2024-07-27T03:13:57.305873+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:42:48.222613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404933+0000","last_change":"2024-07-27T03:12:32.574418+0000","last_active":"2024-07-27T03:13:57.404933+0000","last_peered":"2024-07-27T03:13:57.404933+0000","last_clean":"2024-07-27T03:13:57.404933+0000","last_became_active":"2024-07-27T03:12:32.573974+0000","last_became_peered":"2024-07-27T03:12:32.573974+0000","last_unstale":"2024-07-27T03:13:57.404933+0000","last_undegraded":"2024-07-27T03:13:57.404933+0000","last_fullsized":"2024-07-27T03:13:57.404933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:34:30.442833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303264+0000","last_change":"2024-07-27T03:12:42.605155+0000","last_active":"2024-07-27T03:13:57.303264+0000","last_peered":"2024-07-27T03:13:57.303264+0000","last_clean":"2024-07-27T03:13:57.303264+0000","last_became_active":"2024-07-27T03:12:42.604685+0000","last_became_peered":"2024-07-27T03:12:42.604685+0000","last_unstale":"2024-07-27T03:13:57.303264+0000","last_undegraded":"2024-07-27T03:13:57.303264+0000","last_fullsized":"2024-07-27T03:13:57.303264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:28:58.931408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305799+0000","last_change":"2024-07-27T03:12:32.574711+0000","last_active":"2024-07-27T03:13:57.305799+0000","last_peered":"2024-07-27T03:13:57.305799+0000","last_clean":"2024-07-27T03:13:57.305799+0000","last_became_active":"2024-07-27T03:12:32.574365+0000","last_became_peered":"2024-07-27T03:12:32.574365+0000","last_unstale":"2024-07-27T03:13:57.305799+0000","last_undegraded":"2024-07-27T03:13:57.305799+0000","last_fullsized":"2024-07-27T03:13:57.305799+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:50:56.621371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305282+0000","last_change":"2024-07-27T03:12:29.540835+0000","last_active":"2024-07-27T03:13:57.305282+0000","last_peered":"2024-07-27T03:13:57.305282+0000","last_clean":"2024-07-27T03:13:57.305282+0000","last_became_active":"2024-07-27T03:12:29.540461+0000","last_became_peered":"2024-07-27T03:12:29.540461+0000","last_unstale":"2024-07-27T03:13:57.305282+0000","last_undegraded":"2024-07-27T03:13:57.305282+0000","last_fullsized":"2024-07-27T03:13:57.305282+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:19:20.817038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303286+0000","last_change":"2024-07-27T03:12:42.718584+0000","last_active":"2024-07-27T03:13:57.303286+0000","last_peered":"2024-07-27T03:13:57.303286+0000","last_clean":"2024-07-27T03:13:57.303286+0000","last_became_active":"2024-07-27T03:12:42.717752+0000","last_became_peered":"2024-07-27T03:12:42.717752+0000","last_unstale":"2024-07-27T03:13:57.303286+0000","last_undegraded":"2024-07-27T03:13:57.303286+0000","last_fullsized":"2024-07-27T03:13:57.303286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:23:52.629794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404350+0000","last_change":"2024-07-27T03:12:32.560672+0000","last_active":"2024-07-27T03:13:57.404350+0000","last_peered":"2024-07-27T03:13:57.404350+0000","last_clean":"2024-07-27T03:13:57.404350+0000","last_became_active":"2024-07-27T03:12:32.559172+0000","last_became_peered":"2024-07-27T03:12:32.559172+0000","last_unstale":"2024-07-27T03:13:57.404350+0000","last_undegraded":"2024-07-27T03:13:57.404350+0000","last_fullsized":"2024-07-27T03:13:57.404350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:02:49.346983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405148+0000","last_change":"2024-07-27T03:12:57.434569+0000","last_active":"2024-07-27T03:13:57.405148+0000","last_peered":"2024-07-27T03:13:57.405148+0000","last_clean":"2024-07-27T03:13:57.405148+0000","last_became_active":"2024-07-27T03:12:57.434038+0000","last_became_peered":"2024-07-27T03:12:57.434038+0000","last_unstale":"2024-07-27T03:13:57.405148+0000","last_undegraded":"2024-07-27T03:13:57.405148+0000","last_fullsized":"2024-07-27T03:13:57.405148+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:43:24.462838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.411985+0000","last_change":"2024-07-27T03:13:57.411931+0000","last_active":"2024-07-27T03:13:57.411985+0000","last_peered":"2024-07-27T03:13:57.411985+0000","last_clean":"2024-07-27T03:13:57.411985+0000","last_became_active":"2024-07-27T03:13:57.407362+0000","last_became_peered":"2024-07-27T03:13:57.407362+0000","last_unstale":"2024-07-27T03:13:57.411985+0000","last_undegraded":"2024-07-27T03:13:57.411985+0000","last_fullsized":"2024-07-27T03:13:57.411985+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:52:08.444268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405119+0000","last_change":"2024-07-27T03:12:32.572129+0000","last_active":"2024-07-27T03:13:57.405119+0000","last_peered":"2024-07-27T03:13:57.405119+0000","last_clean":"2024-07-27T03:13:57.405119+0000","last_became_active":"2024-07-27T03:12:32.571774+0000","last_became_peered":"2024-07-27T03:12:32.571774+0000","last_unstale":"2024-07-27T03:13:57.405119+0000","last_undegraded":"2024-07-27T03:13:57.405119+0000","last_fullsized":"2024-07-27T03:13:57.405119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:52:23.316957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404381+0000","last_change":"2024-07-27T03:12:29.539768+0000","last_active":"2024-07-27T03:13:57.404381+0000","last_peered":"2024-07-27T03:13:57.404381+0000","last_clean":"2024-07-27T03:13:57.404381+0000","last_became_active":"2024-07-27T03:12:29.539389+0000","last_became_peered":"2024-07-27T03:12:29.539389+0000","last_unstale":"2024-07-27T03:13:57.404381+0000","last_undegraded":"2024-07-27T03:13:57.404381+0000","last_fullsized":"2024-07-27T03:13:57.404381+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:09:35.961328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302562+0000","last_change":"2024-07-27T03:12:42.617569+0000","last_active":"2024-07-27T03:13:57.302562+0000","last_peered":"2024-07-27T03:13:57.302562+0000","last_clean":"2024-07-27T03:13:57.302562+0000","last_became_active":"2024-07-27T03:12:42.616357+0000","last_became_peered":"2024-07-27T03:12:42.616357+0000","last_unstale":"2024-07-27T03:13:57.302562+0000","last_undegraded":"2024-07-27T03:13:57.302562+0000","last_fullsized":"2024-07-27T03:13:57.302562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:35:11.297631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304782+0000","last_change":"2024-07-27T03:12:32.566272+0000","last_active":"2024-07-27T03:13:57.304782+0000","last_peered":"2024-07-27T03:13:57.304782+0000","last_clean":"2024-07-27T03:13:57.304782+0000","last_became_active":"2024-07-27T03:12:32.565865+0000","last_became_peered":"2024-07-27T03:12:32.565865+0000","last_unstale":"2024-07-27T03:13:57.304782+0000","last_undegraded":"2024-07-27T03:13:57.304782+0000","last_fullsized":"2024-07-27T03:13:57.304782+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:42:09.539586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405304+0000","last_change":"2024-07-27T03:12:29.613733+0000","last_active":"2024-07-27T03:13:57.405304+0000","last_peered":"2024-07-27T03:13:57.405304+0000","last_clean":"2024-07-27T03:13:57.405304+0000","last_became_active":"2024-07-27T03:12:29.613395+0000","last_became_peered":"2024-07-27T03:12:29.613395+0000","last_unstale":"2024-07-27T03:13:57.405304+0000","last_undegraded":"2024-07-27T03:13:57.405304+0000","last_fullsized":"2024-07-27T03:13:57.405304+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:21:34.431900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.407883+0000","last_change":"2024-07-27T03:13:57.407798+0000","last_active":"2024-07-27T03:13:57.407883+0000","last_peered":"2024-07-27T03:13:57.407883+0000","last_clean":"2024-07-27T03:13:57.407883+0000","last_became_active":"2024-07-27T03:13:57.407275+0000","last_became_peered":"2024-07-27T03:13:57.407275+0000","last_unstale":"2024-07-27T03:13:57.407883+0000","last_undegraded":"2024-07-27T03:13:57.407883+0000","last_fullsized":"2024-07-27T03:13:57.407883+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:47:27.817092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,8],"acting":[4,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405475+0000","last_change":"2024-07-27T03:12:32.567006+0000","last_active":"2024-07-27T03:13:57.405475+0000","last_peered":"2024-07-27T03:13:57.405475+0000","last_clean":"2024-07-27T03:13:57.405475+0000","last_became_active":"2024-07-27T03:12:32.566205+0000","last_became_peered":"2024-07-27T03:12:32.566205+0000","last_unstale":"2024-07-27T03:13:57.405475+0000","last_undegraded":"2024-07-27T03:13:57.405475+0000","last_fullsized":"2024-07-27T03:13:57.405475+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:11:23.171341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303619+0000","last_change":"2024-07-27T03:12:29.533909+0000","last_active":"2024-07-27T03:13:57.303619+0000","last_peered":"2024-07-27T03:13:57.303619+0000","last_clean":"2024-07-27T03:13:57.303619+0000","last_became_active":"2024-07-27T03:12:29.533327+0000","last_became_peered":"2024-07-27T03:12:29.533327+0000","last_unstale":"2024-07-27T03:13:57.303619+0000","last_undegraded":"2024-07-27T03:13:57.303619+0000","last_fullsized":"2024-07-27T03:13:57.303619+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:35:24.705096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302850+0000","last_change":"2024-07-27T03:12:42.609066+0000","last_active":"2024-07-27T03:13:57.302850+0000","last_peered":"2024-07-27T03:13:57.302850+0000","last_clean":"2024-07-27T03:13:57.302850+0000","last_became_active":"2024-07-27T03:12:42.608561+0000","last_became_peered":"2024-07-27T03:12:42.608561+0000","last_unstale":"2024-07-27T03:13:57.302850+0000","last_undegraded":"2024-07-27T03:13:57.302850+0000","last_fullsized":"2024-07-27T03:13:57.302850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:27:53.657498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405605+0000","last_change":"2024-07-27T03:12:32.560931+0000","last_active":"2024-07-27T03:13:57.405605+0000","last_peered":"2024-07-27T03:13:57.405605+0000","last_clean":"2024-07-27T03:13:57.405605+0000","last_became_active":"2024-07-27T03:12:32.560549+0000","last_became_peered":"2024-07-27T03:12:32.560549+0000","last_unstale":"2024-07-27T03:13:57.405605+0000","last_undegraded":"2024-07-27T03:13:57.405605+0000","last_fullsized":"2024-07-27T03:13:57.405605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T15:11:25.136035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'15","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:59.582932+0000","last_change":"2024-07-27T03:12:29.535365+0000","last_active":"2024-07-27T03:13:59.582932+0000","last_peered":"2024-07-27T03:13:59.582932+0000","last_clean":"2024-07-27T03:13:59.582932+0000","last_became_active":"2024-07-27T03:12:29.534952+0000","last_became_peered":"2024-07-27T03:12:29.534952+0000","last_unstale":"2024-07-27T03:13:59.582932+0000","last_undegraded":"2024-07-27T03:13:59.582932+0000","last_fullsized":"2024-07-27T03:13:59.582932+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:34:59.889488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404008+0000","last_change":"2024-07-27T03:12:42.718416+0000","last_active":"2024-07-27T03:13:57.404008+0000","last_peered":"2024-07-27T03:13:57.404008+0000","last_clean":"2024-07-27T03:13:57.404008+0000","last_became_active":"2024-07-27T03:12:42.717949+0000","last_became_peered":"2024-07-27T03:12:42.717949+0000","last_unstale":"2024-07-27T03:13:57.404008+0000","last_undegraded":"2024-07-27T03:13:57.404008+0000","last_fullsized":"2024-07-27T03:13:57.404008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:05:05.762577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348818+0000","last_change":"2024-07-27T03:12:32.559824+0000","last_active":"2024-07-27T03:13:57.348818+0000","last_peered":"2024-07-27T03:13:57.348818+0000","last_clean":"2024-07-27T03:13:57.348818+0000","last_became_active":"2024-07-27T03:12:32.559435+0000","last_became_peered":"2024-07-27T03:12:32.559435+0000","last_unstale":"2024-07-27T03:13:57.348818+0000","last_undegraded":"2024-07-27T03:13:57.348818+0000","last_fullsized":"2024-07-27T03:13:57.348818+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:16:44.514272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305808+0000","last_change":"2024-07-27T03:12:29.614296+0000","last_active":"2024-07-27T03:13:57.305808+0000","last_peered":"2024-07-27T03:13:57.305808+0000","last_clean":"2024-07-27T03:13:57.305808+0000","last_became_active":"2024-07-27T03:12:29.612987+0000","last_became_peered":"2024-07-27T03:12:29.612987+0000","last_unstale":"2024-07-27T03:13:57.305808+0000","last_undegraded":"2024-07-27T03:13:57.305808+0000","last_fullsized":"2024-07-27T03:13:57.305808+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:23:09.226523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405142+0000","last_change":"2024-07-27T03:12:42.616575+0000","last_active":"2024-07-27T03:13:57.405142+0000","last_peered":"2024-07-27T03:13:57.405142+0000","last_clean":"2024-07-27T03:13:57.405142+0000","last_became_active":"2024-07-27T03:12:42.615980+0000","last_became_peered":"2024-07-27T03:12:42.615980+0000","last_unstale":"2024-07-27T03:13:57.405142+0000","last_undegraded":"2024-07-27T03:13:57.405142+0000","last_fullsized":"2024-07-27T03:13:57.405142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:00:59.041910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405960+0000","last_change":"2024-07-27T03:12:32.556109+0000","last_active":"2024-07-27T03:13:57.405960+0000","last_peered":"2024-07-27T03:13:57.405960+0000","last_clean":"2024-07-27T03:13:57.405960+0000","last_became_active":"2024-07-27T03:12:32.555747+0000","last_became_peered":"2024-07-27T03:12:32.555747+0000","last_unstale":"2024-07-27T03:13:57.405960+0000","last_undegraded":"2024-07-27T03:13:57.405960+0000","last_fullsized":"2024-07-27T03:13:57.405960+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:27:58.286605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302484+0000","last_change":"2024-07-27T03:12:29.613523+0000","last_active":"2024-07-27T03:13:57.302484+0000","last_peered":"2024-07-27T03:13:57.302484+0000","last_clean":"2024-07-27T03:13:57.302484+0000","last_became_active":"2024-07-27T03:12:29.613172+0000","last_became_peered":"2024-07-27T03:12:29.613172+0000","last_unstale":"2024-07-27T03:13:57.302484+0000","last_undegraded":"2024-07-27T03:13:57.302484+0000","last_fullsized":"2024-07-27T03:13:57.302484+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:49:45.225221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304847+0000","last_change":"2024-07-27T03:12:42.718536+0000","last_active":"2024-07-27T03:13:57.304847+0000","last_peered":"2024-07-27T03:13:57.304847+0000","last_clean":"2024-07-27T03:13:57.304847+0000","last_became_active":"2024-07-27T03:12:42.718068+0000","last_became_peered":"2024-07-27T03:12:42.718068+0000","last_unstale":"2024-07-27T03:13:57.304847+0000","last_undegraded":"2024-07-27T03:13:57.304847+0000","last_fullsized":"2024-07-27T03:13:57.304847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:18:28.478700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406742+0000","last_change":"2024-07-27T03:12:32.558060+0000","last_active":"2024-07-27T03:13:57.406742+0000","last_peered":"2024-07-27T03:13:57.406742+0000","last_clean":"2024-07-27T03:13:57.406742+0000","last_became_active":"2024-07-27T03:12:32.557260+0000","last_became_peered":"2024-07-27T03:12:32.557260+0000","last_unstale":"2024-07-27T03:13:57.406742+0000","last_undegraded":"2024-07-27T03:13:57.406742+0000","last_fullsized":"2024-07-27T03:13:57.406742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:36:54.945761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":77,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:05.291436+0000","last_change":"2024-07-27T03:12:29.541080+0000","last_active":"2024-07-27T03:14:05.291436+0000","last_peered":"2024-07-27T03:14:05.291436+0000","last_clean":"2024-07-27T03:14:05.291436+0000","last_became_active":"2024-07-27T03:12:29.540628+0000","last_became_peered":"2024-07-27T03:12:29.540628+0000","last_unstale":"2024-07-27T03:14:05.291436+0000","last_undegraded":"2024-07-27T03:14:05.291436+0000","last_fullsized":"2024-07-27T03:14:05.291436+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:45:17.045124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348808+0000","last_change":"2024-07-27T03:12:42.611561+0000","last_active":"2024-07-27T03:13:57.348808+0000","last_peered":"2024-07-27T03:13:57.348808+0000","last_clean":"2024-07-27T03:13:57.348808+0000","last_became_active":"2024-07-27T03:12:42.610507+0000","last_became_peered":"2024-07-27T03:12:42.610507+0000","last_unstale":"2024-07-27T03:13:57.348808+0000","last_undegraded":"2024-07-27T03:13:57.348808+0000","last_fullsized":"2024-07-27T03:13:57.348808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:15:08.972807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404266+0000","last_change":"2024-07-27T03:12:32.562316+0000","last_active":"2024-07-27T03:13:57.404266+0000","last_peered":"2024-07-27T03:13:57.404266+0000","last_clean":"2024-07-27T03:13:57.404266+0000","last_became_active":"2024-07-27T03:12:32.561729+0000","last_became_peered":"2024-07-27T03:12:32.561729+0000","last_unstale":"2024-07-27T03:13:57.404266+0000","last_undegraded":"2024-07-27T03:13:57.404266+0000","last_fullsized":"2024-07-27T03:13:57.404266+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:01:19.960269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":75,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:02.287189+0000","last_change":"2024-07-27T03:12:29.613459+0000","last_active":"2024-07-27T03:14:02.287189+0000","last_peered":"2024-07-27T03:14:02.287189+0000","last_clean":"2024-07-27T03:14:02.287189+0000","last_became_active":"2024-07-27T03:12:29.613009+0000","last_became_peered":"2024-07-27T03:12:29.613009+0000","last_unstale":"2024-07-27T03:14:02.287189+0000","last_undegraded":"2024-07-27T03:14:02.287189+0000","last_fullsized":"2024-07-27T03:14:02.287189+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:16:37.230471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348633+0000","last_change":"2024-07-27T03:12:42.718516+0000","last_active":"2024-07-27T03:13:57.348633+0000","last_peered":"2024-07-27T03:13:57.348633+0000","last_clean":"2024-07-27T03:13:57.348633+0000","last_became_active":"2024-07-27T03:12:42.718041+0000","last_became_peered":"2024-07-27T03:12:42.718041+0000","last_unstale":"2024-07-27T03:13:57.348633+0000","last_undegraded":"2024-07-27T03:13:57.348633+0000","last_fullsized":"2024-07-27T03:13:57.348633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:35:39.633977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348194+0000","last_change":"2024-07-27T03:12:32.560752+0000","last_active":"2024-07-27T03:13:57.348194+0000","last_peered":"2024-07-27T03:13:57.348194+0000","last_clean":"2024-07-27T03:13:57.348194+0000","last_became_active":"2024-07-27T03:12:32.560248+0000","last_became_peered":"2024-07-27T03:12:32.560248+0000","last_unstale":"2024-07-27T03:13:57.348194+0000","last_undegraded":"2024-07-27T03:13:57.348194+0000","last_fullsized":"2024-07-27T03:13:57.348194+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:59:48.056867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305388+0000","last_change":"2024-07-27T03:12:29.539591+0000","last_active":"2024-07-27T03:13:57.305388+0000","last_peered":"2024-07-27T03:13:57.305388+0000","last_clean":"2024-07-27T03:13:57.305388+0000","last_became_active":"2024-07-27T03:12:29.539165+0000","last_became_peered":"2024-07-27T03:12:29.539165+0000","last_unstale":"2024-07-27T03:13:57.305388+0000","last_undegraded":"2024-07-27T03:13:57.305388+0000","last_fullsized":"2024-07-27T03:13:57.305388+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:04:21.940511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.408418+0000","last_change":"2024-07-27T03:13:57.407921+0000","last_active":"2024-07-27T03:13:57.408418+0000","last_peered":"2024-07-27T03:13:57.408418+0000","last_clean":"2024-07-27T03:13:57.408418+0000","last_became_active":"2024-07-27T03:13:57.407382+0000","last_became_peered":"2024-07-27T03:13:57.407382+0000","last_unstale":"2024-07-27T03:13:57.408418+0000","last_undegraded":"2024-07-27T03:13:57.408418+0000","last_fullsized":"2024-07-27T03:13:57.408418+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:48:27.362759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404318+0000","last_change":"2024-07-27T03:12:32.570653+0000","last_active":"2024-07-27T03:13:57.404318+0000","last_peered":"2024-07-27T03:13:57.404318+0000","last_clean":"2024-07-27T03:13:57.404318+0000","last_became_active":"2024-07-27T03:12:32.569088+0000","last_became_peered":"2024-07-27T03:12:32.569088+0000","last_unstale":"2024-07-27T03:13:57.404318+0000","last_undegraded":"2024-07-27T03:13:57.404318+0000","last_fullsized":"2024-07-27T03:13:57.404318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:50:24.669431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303802+0000","last_change":"2024-07-27T03:12:29.541654+0000","last_active":"2024-07-27T03:13:57.303802+0000","last_peered":"2024-07-27T03:13:57.303802+0000","last_clean":"2024-07-27T03:13:57.303802+0000","last_became_active":"2024-07-27T03:12:29.541264+0000","last_became_peered":"2024-07-27T03:12:29.541264+0000","last_unstale":"2024-07-27T03:13:57.303802+0000","last_undegraded":"2024-07-27T03:13:57.303802+0000","last_fullsized":"2024-07-27T03:13:57.303802+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:48:05.179016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":122,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306449+0000","last_change":"2024-07-27T03:11:11.343199+0000","last_active":"2024-07-27T03:13:57.306449+0000","last_peered":"2024-07-27T03:13:57.306449+0000","last_clean":"2024-07-27T03:13:57.306449+0000","last_became_active":"2024-07-27T03:11:11.337635+0000","last_became_peered":"2024-07-27T03:11:11.337635+0000","last_unstale":"2024-07-27T03:13:57.306449+0000","last_undegraded":"2024-07-27T03:13:57.306449+0000","last_fullsized":"2024-07-27T03:13:57.306449+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:08:01.694830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:08:01.694830+0000","last_clean_scrub_stamp":"2024-07-27T03:08:01.694830+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:48:49.979610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305758+0000","last_change":"2024-07-27T03:12:42.614346+0000","last_active":"2024-07-27T03:13:57.305758+0000","last_peered":"2024-07-27T03:13:57.305758+0000","last_clean":"2024-07-27T03:13:57.305758+0000","last_became_active":"2024-07-27T03:12:42.613881+0000","last_became_peered":"2024-07-27T03:12:42.613881+0000","last_unstale":"2024-07-27T03:13:57.305758+0000","last_undegraded":"2024-07-27T03:13:57.305758+0000","last_fullsized":"2024-07-27T03:13:57.305758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:12:38.299348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306056+0000","last_change":"2024-07-27T03:12:32.574330+0000","last_active":"2024-07-27T03:13:57.306056+0000","last_peered":"2024-07-27T03:13:57.306056+0000","last_clean":"2024-07-27T03:13:57.306056+0000","last_became_active":"2024-07-27T03:12:32.573977+0000","last_became_peered":"2024-07-27T03:12:32.573977+0000","last_unstale":"2024-07-27T03:13:57.306056+0000","last_undegraded":"2024-07-27T03:13:57.306056+0000","last_fullsized":"2024-07-27T03:13:57.306056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:39:09.925794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":77,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:03.289164+0000","last_change":"2024-07-27T03:12:29.613848+0000","last_active":"2024-07-27T03:14:03.289164+0000","last_peered":"2024-07-27T03:14:03.289164+0000","last_clean":"2024-07-27T03:14:03.289164+0000","last_became_active":"2024-07-27T03:12:29.613473+0000","last_became_peered":"2024-07-27T03:12:29.613473+0000","last_unstale":"2024-07-27T03:14:03.289164+0000","last_undegraded":"2024-07-27T03:14:03.289164+0000","last_fullsized":"2024-07-27T03:14:03.289164+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:52:56.894962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405287+0000","last_change":"2024-07-27T03:12:42.611295+0000","last_active":"2024-07-27T03:13:57.405287+0000","last_peered":"2024-07-27T03:13:57.405287+0000","last_clean":"2024-07-27T03:13:57.405287+0000","last_became_active":"2024-07-27T03:12:42.610782+0000","last_became_peered":"2024-07-27T03:12:42.610782+0000","last_unstale":"2024-07-27T03:13:57.405287+0000","last_undegraded":"2024-07-27T03:13:57.405287+0000","last_fullsized":"2024-07-27T03:13:57.405287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:24:02.849882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305853+0000","last_change":"2024-07-27T03:12:56.725924+0000","last_active":"2024-07-27T03:13:57.305853+0000","last_peered":"2024-07-27T03:13:57.305853+0000","last_clean":"2024-07-27T03:13:57.305853+0000","last_became_active":"2024-07-27T03:12:56.725567+0000","last_became_peered":"2024-07-27T03:12:56.725567+0000","last_unstale":"2024-07-27T03:13:57.305853+0000","last_undegraded":"2024-07-27T03:13:57.305853+0000","last_fullsized":"2024-07-27T03:13:57.305853+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:32:23.771483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405166+0000","last_change":"2024-07-27T03:12:29.537863+0000","last_active":"2024-07-27T03:13:57.405166+0000","last_peered":"2024-07-27T03:13:57.405166+0000","last_clean":"2024-07-27T03:13:57.405166+0000","last_became_active":"2024-07-27T03:12:29.537444+0000","last_became_peered":"2024-07-27T03:12:29.537444+0000","last_unstale":"2024-07-27T03:13:57.405166+0000","last_undegraded":"2024-07-27T03:13:57.405166+0000","last_fullsized":"2024-07-27T03:13:57.405166+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:52:05.239809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405213+0000","last_change":"2024-07-27T03:12:42.622599+0000","last_active":"2024-07-27T03:13:57.405213+0000","last_peered":"2024-07-27T03:13:57.405213+0000","last_clean":"2024-07-27T03:13:57.405213+0000","last_became_active":"2024-07-27T03:12:42.622151+0000","last_became_peered":"2024-07-27T03:12:42.622151+0000","last_unstale":"2024-07-27T03:13:57.405213+0000","last_undegraded":"2024-07-27T03:13:57.405213+0000","last_fullsized":"2024-07-27T03:13:57.405213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:34:31.622281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306321+0000","last_change":"2024-07-27T03:12:32.562300+0000","last_active":"2024-07-27T03:13:57.306321+0000","last_peered":"2024-07-27T03:13:57.306321+0000","last_clean":"2024-07-27T03:13:57.306321+0000","last_became_active":"2024-07-27T03:12:32.561343+0000","last_became_peered":"2024-07-27T03:12:32.561343+0000","last_unstale":"2024-07-27T03:13:57.306321+0000","last_undegraded":"2024-07-27T03:13:57.306321+0000","last_fullsized":"2024-07-27T03:13:57.306321+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:01:28.626943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348783+0000","last_change":"2024-07-27T03:12:29.535177+0000","last_active":"2024-07-27T03:13:57.348783+0000","last_peered":"2024-07-27T03:13:57.348783+0000","last_clean":"2024-07-27T03:13:57.348783+0000","last_became_active":"2024-07-27T03:12:29.534662+0000","last_became_peered":"2024-07-27T03:12:29.534662+0000","last_unstale":"2024-07-27T03:13:57.348783+0000","last_undegraded":"2024-07-27T03:13:57.348783+0000","last_fullsized":"2024-07-27T03:13:57.348783+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:50:49.661992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404014+0000","last_change":"2024-07-27T03:12:42.609925+0000","last_active":"2024-07-27T03:13:57.404014+0000","last_peered":"2024-07-27T03:13:57.404014+0000","last_clean":"2024-07-27T03:13:57.404014+0000","last_became_active":"2024-07-27T03:12:42.609456+0000","last_became_peered":"2024-07-27T03:12:42.609456+0000","last_unstale":"2024-07-27T03:13:57.404014+0000","last_undegraded":"2024-07-27T03:13:57.404014+0000","last_fullsized":"2024-07-27T03:13:57.404014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:07:52.128381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405991+0000","last_change":"2024-07-27T03:12:56.726170+0000","last_active":"2024-07-27T03:13:57.405991+0000","last_peered":"2024-07-27T03:13:57.405991+0000","last_clean":"2024-07-27T03:13:57.405991+0000","last_became_active":"2024-07-27T03:12:56.725685+0000","last_became_peered":"2024-07-27T03:12:56.725685+0000","last_unstale":"2024-07-27T03:13:57.405991+0000","last_undegraded":"2024-07-27T03:13:57.405991+0000","last_fullsized":"2024-07-27T03:13:57.405991+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:53:41.143333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405934+0000","last_change":"2024-07-27T03:12:56.726059+0000","last_active":"2024-07-27T03:13:57.405934+0000","last_peered":"2024-07-27T03:13:57.405934+0000","last_clean":"2024-07-27T03:13:57.405934+0000","last_became_active":"2024-07-27T03:12:56.722294+0000","last_became_peered":"2024-07-27T03:12:56.722294+0000","last_unstale":"2024-07-27T03:13:57.405934+0000","last_undegraded":"2024-07-27T03:13:57.405934+0000","last_fullsized":"2024-07-27T03:13:57.405934+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:18:42.627768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404251+0000","last_change":"2024-07-27T03:12:42.611966+0000","last_active":"2024-07-27T03:13:57.404251+0000","last_peered":"2024-07-27T03:13:57.404251+0000","last_clean":"2024-07-27T03:13:57.404251+0000","last_became_active":"2024-07-27T03:12:42.611522+0000","last_became_peered":"2024-07-27T03:12:42.611522+0000","last_unstale":"2024-07-27T03:13:57.404251+0000","last_undegraded":"2024-07-27T03:13:57.404251+0000","last_fullsized":"2024-07-27T03:13:57.404251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:09:54.034597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.301908+0000","last_change":"2024-07-27T03:12:56.719638+0000","last_active":"2024-07-27T03:13:57.301908+0000","last_peered":"2024-07-27T03:13:57.301908+0000","last_clean":"2024-07-27T03:13:57.301908+0000","last_became_active":"2024-07-27T03:12:56.719227+0000","last_became_peered":"2024-07-27T03:12:56.719227+0000","last_unstale":"2024-07-27T03:13:57.301908+0000","last_undegraded":"2024-07-27T03:13:57.301908+0000","last_fullsized":"2024-07-27T03:13:57.301908+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:54:16.891518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304900+0000","last_change":"2024-07-27T03:12:29.614346+0000","last_active":"2024-07-27T03:13:57.304900+0000","last_peered":"2024-07-27T03:13:57.304900+0000","last_clean":"2024-07-27T03:13:57.304900+0000","last_became_active":"2024-07-27T03:12:29.613672+0000","last_became_peered":"2024-07-27T03:12:29.613672+0000","last_unstale":"2024-07-27T03:13:57.304900+0000","last_undegraded":"2024-07-27T03:13:57.304900+0000","last_fullsized":"2024-07-27T03:13:57.304900+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:46:35.505735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348654+0000","last_change":"2024-07-27T03:12:42.618364+0000","last_active":"2024-07-27T03:13:57.348654+0000","last_peered":"2024-07-27T03:13:57.348654+0000","last_clean":"2024-07-27T03:13:57.348654+0000","last_became_active":"2024-07-27T03:12:42.617988+0000","last_became_peered":"2024-07-27T03:12:42.617988+0000","last_unstale":"2024-07-27T03:13:57.348654+0000","last_undegraded":"2024-07-27T03:13:57.348654+0000","last_fullsized":"2024-07-27T03:13:57.348654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:40:21.725877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305293+0000","last_change":"2024-07-27T03:12:32.563809+0000","last_active":"2024-07-27T03:13:57.305293+0000","last_peered":"2024-07-27T03:13:57.305293+0000","last_clean":"2024-07-27T03:13:57.305293+0000","last_became_active":"2024-07-27T03:12:32.563479+0000","last_became_peered":"2024-07-27T03:12:32.563479+0000","last_unstale":"2024-07-27T03:13:57.305293+0000","last_undegraded":"2024-07-27T03:13:57.305293+0000","last_fullsized":"2024-07-27T03:13:57.305293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:45:47.719241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":80,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:04.290624+0000","last_change":"2024-07-27T03:12:29.543786+0000","last_active":"2024-07-27T03:14:04.290624+0000","last_peered":"2024-07-27T03:14:04.290624+0000","last_clean":"2024-07-27T03:14:04.290624+0000","last_became_active":"2024-07-27T03:12:29.543203+0000","last_became_peered":"2024-07-27T03:12:29.543203+0000","last_unstale":"2024-07-27T03:14:04.290624+0000","last_undegraded":"2024-07-27T03:14:04.290624+0000","last_fullsized":"2024-07-27T03:14:04.290624+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:47:06.742171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302660+0000","last_change":"2024-07-27T03:12:42.622142+0000","last_active":"2024-07-27T03:13:57.302660+0000","last_peered":"2024-07-27T03:13:57.302660+0000","last_clean":"2024-07-27T03:13:57.302660+0000","last_became_active":"2024-07-27T03:12:42.621681+0000","last_became_peered":"2024-07-27T03:12:42.621681+0000","last_unstale":"2024-07-27T03:13:57.302660+0000","last_undegraded":"2024-07-27T03:13:57.302660+0000","last_fullsized":"2024-07-27T03:13:57.302660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:05:08.168424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305949+0000","last_change":"2024-07-27T03:12:32.567119+0000","last_active":"2024-07-27T03:13:57.305949+0000","last_peered":"2024-07-27T03:13:57.305949+0000","last_clean":"2024-07-27T03:13:57.305949+0000","last_became_active":"2024-07-27T03:12:32.566757+0000","last_became_peered":"2024-07-27T03:12:32.566757+0000","last_unstale":"2024-07-27T03:13:57.305949+0000","last_undegraded":"2024-07-27T03:13:57.305949+0000","last_fullsized":"2024-07-27T03:13:57.305949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:31:12.972804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404661+0000","last_change":"2024-07-27T03:12:29.540453+0000","last_active":"2024-07-27T03:13:57.404661+0000","last_peered":"2024-07-27T03:13:57.404661+0000","last_clean":"2024-07-27T03:13:57.404661+0000","last_became_active":"2024-07-27T03:12:29.539195+0000","last_became_peered":"2024-07-27T03:12:29.539195+0000","last_unstale":"2024-07-27T03:13:57.404661+0000","last_undegraded":"2024-07-27T03:13:57.404661+0000","last_fullsized":"2024-07-27T03:13:57.404661+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:44:31.144845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405314+0000","last_change":"2024-07-27T03:12:42.616671+0000","last_active":"2024-07-27T03:13:57.405314+0000","last_peered":"2024-07-27T03:13:57.405314+0000","last_clean":"2024-07-27T03:13:57.405314+0000","last_became_active":"2024-07-27T03:12:42.616198+0000","last_became_peered":"2024-07-27T03:12:42.616198+0000","last_unstale":"2024-07-27T03:13:57.405314+0000","last_undegraded":"2024-07-27T03:13:57.405314+0000","last_fullsized":"2024-07-27T03:13:57.405314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:01:31.584686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348461+0000","last_change":"2024-07-27T03:12:32.562350+0000","last_active":"2024-07-27T03:13:57.348461+0000","last_peered":"2024-07-27T03:13:57.348461+0000","last_clean":"2024-07-27T03:13:57.348461+0000","last_became_active":"2024-07-27T03:12:32.562016+0000","last_became_peered":"2024-07-27T03:12:32.562016+0000","last_unstale":"2024-07-27T03:13:57.348461+0000","last_undegraded":"2024-07-27T03:13:57.348461+0000","last_fullsized":"2024-07-27T03:13:57.348461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:44:35.027329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303171+0000","last_change":"2024-07-27T03:12:29.541387+0000","last_active":"2024-07-27T03:13:57.303171+0000","last_peered":"2024-07-27T03:13:57.303171+0000","last_clean":"2024-07-27T03:13:57.303171+0000","last_became_active":"2024-07-27T03:12:29.541005+0000","last_became_peered":"2024-07-27T03:12:29.541005+0000","last_unstale":"2024-07-27T03:13:57.303171+0000","last_undegraded":"2024-07-27T03:13:57.303171+0000","last_fullsized":"2024-07-27T03:13:57.303171+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:22:16.268348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404762+0000","last_change":"2024-07-27T03:12:42.610029+0000","last_active":"2024-07-27T03:13:57.404762+0000","last_peered":"2024-07-27T03:13:57.404762+0000","last_clean":"2024-07-27T03:13:57.404762+0000","last_became_active":"2024-07-27T03:12:42.609592+0000","last_became_peered":"2024-07-27T03:12:42.609592+0000","last_unstale":"2024-07-27T03:13:57.404762+0000","last_undegraded":"2024-07-27T03:13:57.404762+0000","last_fullsized":"2024-07-27T03:13:57.404762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:47:15.254770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305492+0000","last_change":"2024-07-27T03:12:32.568941+0000","last_active":"2024-07-27T03:13:57.305492+0000","last_peered":"2024-07-27T03:13:57.305492+0000","last_clean":"2024-07-27T03:13:57.305492+0000","last_became_active":"2024-07-27T03:12:32.568531+0000","last_became_peered":"2024-07-27T03:12:32.568531+0000","last_unstale":"2024-07-27T03:13:57.305492+0000","last_undegraded":"2024-07-27T03:13:57.305492+0000","last_fullsized":"2024-07-27T03:13:57.305492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:06:22.020505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'34","reported_seq":136,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:04.290538+0000","last_change":"2024-07-27T03:12:29.545131+0000","last_active":"2024-07-27T03:14:04.290538+0000","last_peered":"2024-07-27T03:14:04.290538+0000","last_clean":"2024-07-27T03:14:04.290538+0000","last_became_active":"2024-07-27T03:12:29.544768+0000","last_became_peered":"2024-07-27T03:12:29.544768+0000","last_unstale":"2024-07-27T03:14:04.290538+0000","last_undegraded":"2024-07-27T03:14:04.290538+0000","last_fullsized":"2024-07-27T03:14:04.290538+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:20:57.556128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57807,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":85,"num_write":36,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302028+0000","last_change":"2024-07-27T03:12:42.617483+0000","last_active":"2024-07-27T03:13:57.302028+0000","last_peered":"2024-07-27T03:13:57.302028+0000","last_clean":"2024-07-27T03:13:57.302028+0000","last_became_active":"2024-07-27T03:12:42.617065+0000","last_became_peered":"2024-07-27T03:12:42.617065+0000","last_unstale":"2024-07-27T03:13:57.302028+0000","last_undegraded":"2024-07-27T03:13:57.302028+0000","last_fullsized":"2024-07-27T03:13:57.302028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:52:17.448919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405841+0000","last_change":"2024-07-27T03:12:56.722257+0000","last_active":"2024-07-27T03:13:57.405841+0000","last_peered":"2024-07-27T03:13:57.405841+0000","last_clean":"2024-07-27T03:13:57.405841+0000","last_became_active":"2024-07-27T03:12:56.721773+0000","last_became_peered":"2024-07-27T03:12:56.721773+0000","last_unstale":"2024-07-27T03:13:57.405841+0000","last_undegraded":"2024-07-27T03:13:57.405841+0000","last_fullsized":"2024-07-27T03:13:57.405841+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:30:44.310705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405141+0000","last_change":"2024-07-27T03:12:29.614171+0000","last_active":"2024-07-27T03:13:57.405141+0000","last_peered":"2024-07-27T03:13:57.405141+0000","last_clean":"2024-07-27T03:13:57.405141+0000","last_became_active":"2024-07-27T03:12:29.613270+0000","last_became_peered":"2024-07-27T03:12:29.613270+0000","last_unstale":"2024-07-27T03:13:57.405141+0000","last_undegraded":"2024-07-27T03:13:57.405141+0000","last_fullsized":"2024-07-27T03:13:57.405141+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:39:32.283778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303742+0000","last_change":"2024-07-27T03:12:42.610375+0000","last_active":"2024-07-27T03:13:57.303742+0000","last_peered":"2024-07-27T03:13:57.303742+0000","last_clean":"2024-07-27T03:13:57.303742+0000","last_became_active":"2024-07-27T03:12:42.609957+0000","last_became_peered":"2024-07-27T03:12:42.609957+0000","last_unstale":"2024-07-27T03:13:57.303742+0000","last_undegraded":"2024-07-27T03:13:57.303742+0000","last_fullsized":"2024-07-27T03:13:57.303742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:11:02.707179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405871+0000","last_change":"2024-07-27T03:12:56.722332+0000","last_active":"2024-07-27T03:13:57.405871+0000","last_peered":"2024-07-27T03:13:57.405871+0000","last_clean":"2024-07-27T03:13:57.405871+0000","last_became_active":"2024-07-27T03:12:56.721972+0000","last_became_peered":"2024-07-27T03:12:56.721972+0000","last_unstale":"2024-07-27T03:13:57.405871+0000","last_undegraded":"2024-07-27T03:13:57.405871+0000","last_fullsized":"2024-07-27T03:13:57.405871+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:54:31.880137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306029+0000","last_change":"2024-07-27T03:12:29.541051+0000","last_active":"2024-07-27T03:13:57.306029+0000","last_peered":"2024-07-27T03:13:57.306029+0000","last_clean":"2024-07-27T03:13:57.306029+0000","last_became_active":"2024-07-27T03:12:29.540619+0000","last_became_peered":"2024-07-27T03:12:29.540619+0000","last_unstale":"2024-07-27T03:13:57.306029+0000","last_undegraded":"2024-07-27T03:13:57.306029+0000","last_fullsized":"2024-07-27T03:13:57.306029+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:23:27.587419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304742+0000","last_change":"2024-07-27T03:12:56.720496+0000","last_active":"2024-07-27T03:13:57.304742+0000","last_peered":"2024-07-27T03:13:57.304742+0000","last_clean":"2024-07-27T03:13:57.304742+0000","last_became_active":"2024-07-27T03:12:56.719784+0000","last_became_peered":"2024-07-27T03:12:56.719784+0000","last_unstale":"2024-07-27T03:13:57.304742+0000","last_undegraded":"2024-07-27T03:13:57.304742+0000","last_fullsized":"2024-07-27T03:13:57.304742+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:07:20.518139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406627+0000","last_change":"2024-07-27T03:12:32.568409+0000","last_active":"2024-07-27T03:13:57.406627+0000","last_peered":"2024-07-27T03:13:57.406627+0000","last_clean":"2024-07-27T03:13:57.406627+0000","last_became_active":"2024-07-27T03:12:32.568029+0000","last_became_peered":"2024-07-27T03:12:32.568029+0000","last_unstale":"2024-07-27T03:13:57.406627+0000","last_undegraded":"2024-07-27T03:13:57.406627+0000","last_fullsized":"2024-07-27T03:13:57.406627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:44:53.857986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":98,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:04.290022+0000","last_change":"2024-07-27T03:12:29.544286+0000","last_active":"2024-07-27T03:14:04.290022+0000","last_peered":"2024-07-27T03:14:04.290022+0000","last_clean":"2024-07-27T03:14:04.290022+0000","last_became_active":"2024-07-27T03:12:29.543879+0000","last_became_peered":"2024-07-27T03:12:29.543879+0000","last_unstale":"2024-07-27T03:14:04.290022+0000","last_undegraded":"2024-07-27T03:14:04.290022+0000","last_fullsized":"2024-07-27T03:14:04.290022+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:20:36.997298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404118+0000","last_change":"2024-07-27T03:12:42.609824+0000","last_active":"2024-07-27T03:13:57.404118+0000","last_peered":"2024-07-27T03:13:57.404118+0000","last_clean":"2024-07-27T03:13:57.404118+0000","last_became_active":"2024-07-27T03:12:42.608502+0000","last_became_peered":"2024-07-27T03:12:42.608502+0000","last_unstale":"2024-07-27T03:13:57.404118+0000","last_undegraded":"2024-07-27T03:13:57.404118+0000","last_fullsized":"2024-07-27T03:13:57.404118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:10:23.253921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405119+0000","last_change":"2024-07-27T03:12:42.610744+0000","last_active":"2024-07-27T03:13:57.405119+0000","last_peered":"2024-07-27T03:13:57.405119+0000","last_clean":"2024-07-27T03:13:57.405119+0000","last_became_active":"2024-07-27T03:12:42.610322+0000","last_became_peered":"2024-07-27T03:12:42.610322+0000","last_unstale":"2024-07-27T03:13:57.405119+0000","last_undegraded":"2024-07-27T03:13:57.405119+0000","last_fullsized":"2024-07-27T03:13:57.405119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:01:47.618013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302035+0000","last_change":"2024-07-27T03:12:32.558372+0000","last_active":"2024-07-27T03:13:57.302035+0000","last_peered":"2024-07-27T03:13:57.302035+0000","last_clean":"2024-07-27T03:13:57.302035+0000","last_became_active":"2024-07-27T03:12:32.558003+0000","last_became_peered":"2024-07-27T03:12:32.558003+0000","last_unstale":"2024-07-27T03:13:57.302035+0000","last_undegraded":"2024-07-27T03:13:57.302035+0000","last_fullsized":"2024-07-27T03:13:57.302035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:41:43.673807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404430+0000","last_change":"2024-07-27T03:12:29.543182+0000","last_active":"2024-07-27T03:13:57.404430+0000","last_peered":"2024-07-27T03:13:57.404430+0000","last_clean":"2024-07-27T03:13:57.404430+0000","last_became_active":"2024-07-27T03:12:29.542744+0000","last_became_peered":"2024-07-27T03:12:29.542744+0000","last_unstale":"2024-07-27T03:13:57.404430+0000","last_undegraded":"2024-07-27T03:13:57.404430+0000","last_fullsized":"2024-07-27T03:13:57.404430+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:01:17.276549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405910+0000","last_change":"2024-07-27T03:12:42.622297+0000","last_active":"2024-07-27T03:13:57.405910+0000","last_peered":"2024-07-27T03:13:57.405910+0000","last_clean":"2024-07-27T03:13:57.405910+0000","last_became_active":"2024-07-27T03:12:42.621947+0000","last_became_peered":"2024-07-27T03:12:42.621947+0000","last_unstale":"2024-07-27T03:13:57.405910+0000","last_undegraded":"2024-07-27T03:13:57.405910+0000","last_fullsized":"2024-07-27T03:13:57.405910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:15:59.480216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404143+0000","last_change":"2024-07-27T03:12:32.565987+0000","last_active":"2024-07-27T03:13:57.404143+0000","last_peered":"2024-07-27T03:13:57.404143+0000","last_clean":"2024-07-27T03:13:57.404143+0000","last_became_active":"2024-07-27T03:12:32.565603+0000","last_became_peered":"2024-07-27T03:12:32.565603+0000","last_unstale":"2024-07-27T03:13:57.404143+0000","last_undegraded":"2024-07-27T03:13:57.404143+0000","last_fullsized":"2024-07-27T03:13:57.404143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:30:48.954683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348204+0000","last_change":"2024-07-27T03:12:29.532149+0000","last_active":"2024-07-27T03:13:57.348204+0000","last_peered":"2024-07-27T03:13:57.348204+0000","last_clean":"2024-07-27T03:13:57.348204+0000","last_became_active":"2024-07-27T03:12:29.531761+0000","last_became_peered":"2024-07-27T03:12:29.531761+0000","last_unstale":"2024-07-27T03:13:57.348204+0000","last_undegraded":"2024-07-27T03:13:57.348204+0000","last_fullsized":"2024-07-27T03:13:57.348204+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:44:30.902538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405968+0000","last_change":"2024-07-27T03:12:29.543148+0000","last_active":"2024-07-27T03:13:57.405968+0000","last_peered":"2024-07-27T03:13:57.405968+0000","last_clean":"2024-07-27T03:13:57.405968+0000","last_became_active":"2024-07-27T03:12:29.542774+0000","last_became_peered":"2024-07-27T03:12:29.542774+0000","last_unstale":"2024-07-27T03:13:57.405968+0000","last_undegraded":"2024-07-27T03:13:57.405968+0000","last_fullsized":"2024-07-27T03:13:57.405968+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:20:06.786121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404541+0000","last_change":"2024-07-27T03:12:32.561516+0000","last_active":"2024-07-27T03:13:57.404541+0000","last_peered":"2024-07-27T03:13:57.404541+0000","last_clean":"2024-07-27T03:13:57.404541+0000","last_became_active":"2024-07-27T03:12:32.561145+0000","last_became_peered":"2024-07-27T03:12:32.561145+0000","last_unstale":"2024-07-27T03:13:57.404541+0000","last_undegraded":"2024-07-27T03:13:57.404541+0000","last_fullsized":"2024-07-27T03:13:57.404541+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:49:26.317206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302917+0000","last_change":"2024-07-27T03:12:42.616735+0000","last_active":"2024-07-27T03:13:57.302917+0000","last_peered":"2024-07-27T03:13:57.302917+0000","last_clean":"2024-07-27T03:13:57.302917+0000","last_became_active":"2024-07-27T03:12:42.616171+0000","last_became_peered":"2024-07-27T03:12:42.616171+0000","last_unstale":"2024-07-27T03:13:57.302917+0000","last_undegraded":"2024-07-27T03:13:57.302917+0000","last_fullsized":"2024-07-27T03:13:57.302917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:45:46.236613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":76,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:02.287813+0000","last_change":"2024-07-27T03:12:29.539775+0000","last_active":"2024-07-27T03:14:02.287813+0000","last_peered":"2024-07-27T03:14:02.287813+0000","last_clean":"2024-07-27T03:14:02.287813+0000","last_became_active":"2024-07-27T03:12:29.539276+0000","last_became_peered":"2024-07-27T03:12:29.539276+0000","last_unstale":"2024-07-27T03:14:02.287813+0000","last_undegraded":"2024-07-27T03:14:02.287813+0000","last_fullsized":"2024-07-27T03:14:02.287813+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:09:19.364679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305066+0000","last_change":"2024-07-27T03:12:32.562673+0000","last_active":"2024-07-27T03:13:57.305066+0000","last_peered":"2024-07-27T03:13:57.305066+0000","last_clean":"2024-07-27T03:13:57.305066+0000","last_became_active":"2024-07-27T03:12:32.561591+0000","last_became_peered":"2024-07-27T03:12:32.561591+0000","last_unstale":"2024-07-27T03:13:57.305066+0000","last_undegraded":"2024-07-27T03:13:57.305066+0000","last_fullsized":"2024-07-27T03:13:57.305066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:49:38.095004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303348+0000","last_change":"2024-07-27T03:12:42.611756+0000","last_active":"2024-07-27T03:13:57.303348+0000","last_peered":"2024-07-27T03:13:57.303348+0000","last_clean":"2024-07-27T03:13:57.303348+0000","last_became_active":"2024-07-27T03:12:42.611246+0000","last_became_peered":"2024-07-27T03:12:42.611246+0000","last_unstale":"2024-07-27T03:13:57.303348+0000","last_undegraded":"2024-07-27T03:13:57.303348+0000","last_fullsized":"2024-07-27T03:13:57.303348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:13:06.027887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405629+0000","last_change":"2024-07-27T03:12:32.571339+0000","last_active":"2024-07-27T03:13:57.405629+0000","last_peered":"2024-07-27T03:13:57.405629+0000","last_clean":"2024-07-27T03:13:57.405629+0000","last_became_active":"2024-07-27T03:12:32.570973+0000","last_became_peered":"2024-07-27T03:12:32.570973+0000","last_unstale":"2024-07-27T03:13:57.405629+0000","last_undegraded":"2024-07-27T03:13:57.405629+0000","last_fullsized":"2024-07-27T03:13:57.405629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:24:25.357833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303303+0000","last_change":"2024-07-27T03:12:32.561243+0000","last_active":"2024-07-27T03:13:57.303303+0000","last_peered":"2024-07-27T03:13:57.303303+0000","last_clean":"2024-07-27T03:13:57.303303+0000","last_became_active":"2024-07-27T03:12:32.560748+0000","last_became_peered":"2024-07-27T03:12:32.560748+0000","last_unstale":"2024-07-27T03:13:57.303303+0000","last_undegraded":"2024-07-27T03:13:57.303303+0000","last_fullsized":"2024-07-27T03:13:57.303303+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:47:40.430283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302860+0000","last_change":"2024-07-27T03:12:32.562545+0000","last_active":"2024-07-27T03:13:57.302860+0000","last_peered":"2024-07-27T03:13:57.302860+0000","last_clean":"2024-07-27T03:13:57.302860+0000","last_became_active":"2024-07-27T03:12:32.562202+0000","last_became_peered":"2024-07-27T03:12:32.562202+0000","last_unstale":"2024-07-27T03:13:57.302860+0000","last_undegraded":"2024-07-27T03:13:57.302860+0000","last_fullsized":"2024-07-27T03:13:57.302860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:05:20.229232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405409+0000","last_change":"2024-07-27T03:12:32.565162+0000","last_active":"2024-07-27T03:13:57.405409+0000","last_peered":"2024-07-27T03:13:57.405409+0000","last_clean":"2024-07-27T03:13:57.405409+0000","last_became_active":"2024-07-27T03:12:32.564798+0000","last_became_peered":"2024-07-27T03:12:32.564798+0000","last_unstale":"2024-07-27T03:13:57.405409+0000","last_undegraded":"2024-07-27T03:13:57.405409+0000","last_fullsized":"2024-07-27T03:13:57.405409+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:54:29.742624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348688+0000","last_change":"2024-07-27T03:12:32.556874+0000","last_active":"2024-07-27T03:13:57.348688+0000","last_peered":"2024-07-27T03:13:57.348688+0000","last_clean":"2024-07-27T03:13:57.348688+0000","last_became_active":"2024-07-27T03:12:32.556467+0000","last_became_peered":"2024-07-27T03:12:32.556467+0000","last_unstale":"2024-07-27T03:13:57.348688+0000","last_undegraded":"2024-07-27T03:13:57.348688+0000","last_fullsized":"2024-07-27T03:13:57.348688+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:51:20.806785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405956+0000","last_change":"2024-07-27T03:12:32.569136+0000","last_active":"2024-07-27T03:13:57.405956+0000","last_peered":"2024-07-27T03:13:57.405956+0000","last_clean":"2024-07-27T03:13:57.405956+0000","last_became_active":"2024-07-27T03:12:32.568778+0000","last_became_peered":"2024-07-27T03:12:32.568778+0000","last_unstale":"2024-07-27T03:13:57.405956+0000","last_undegraded":"2024-07-27T03:13:57.405956+0000","last_fullsized":"2024-07-27T03:13:57.405956+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:26:22.654974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303473+0000","last_change":"2024-07-27T03:12:32.569507+0000","last_active":"2024-07-27T03:13:57.303473+0000","last_peered":"2024-07-27T03:13:57.303473+0000","last_clean":"2024-07-27T03:13:57.303473+0000","last_became_active":"2024-07-27T03:12:32.569109+0000","last_became_peered":"2024-07-27T03:12:32.569109+0000","last_unstale":"2024-07-27T03:13:57.303473+0000","last_undegraded":"2024-07-27T03:13:57.303473+0000","last_fullsized":"2024-07-27T03:13:57.303473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:35:12.944933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405538+0000","last_change":"2024-07-27T03:12:32.566479+0000","last_active":"2024-07-27T03:13:57.405538+0000","last_peered":"2024-07-27T03:13:57.405538+0000","last_clean":"2024-07-27T03:13:57.405538+0000","last_became_active":"2024-07-27T03:12:32.566114+0000","last_became_peered":"2024-07-27T03:12:32.566114+0000","last_unstale":"2024-07-27T03:13:57.405538+0000","last_undegraded":"2024-07-27T03:13:57.405538+0000","last_fullsized":"2024-07-27T03:13:57.405538+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:24:19.944031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404192+0000","last_change":"2024-07-27T03:12:56.726118+0000","last_active":"2024-07-27T03:13:57.404192+0000","last_peered":"2024-07-27T03:13:57.404192+0000","last_clean":"2024-07-27T03:13:57.404192+0000","last_became_active":"2024-07-27T03:12:56.725637+0000","last_became_peered":"2024-07-27T03:12:56.725637+0000","last_unstale":"2024-07-27T03:13:57.404192+0000","last_undegraded":"2024-07-27T03:13:57.404192+0000","last_fullsized":"2024-07-27T03:13:57.404192+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:36:41.725150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404347+0000","last_change":"2024-07-27T03:12:32.568232+0000","last_active":"2024-07-27T03:13:57.404347+0000","last_peered":"2024-07-27T03:13:57.404347+0000","last_clean":"2024-07-27T03:13:57.404347+0000","last_became_active":"2024-07-27T03:12:32.567834+0000","last_became_peered":"2024-07-27T03:12:32.567834+0000","last_unstale":"2024-07-27T03:13:57.404347+0000","last_undegraded":"2024-07-27T03:13:57.404347+0000","last_fullsized":"2024-07-27T03:13:57.404347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:40:28.234539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405550+0000","last_change":"2024-07-27T03:12:32.573451+0000","last_active":"2024-07-27T03:13:57.405550+0000","last_peered":"2024-07-27T03:13:57.405550+0000","last_clean":"2024-07-27T03:13:57.405550+0000","last_became_active":"2024-07-27T03:12:32.573066+0000","last_became_peered":"2024-07-27T03:12:32.573066+0000","last_unstale":"2024-07-27T03:13:57.405550+0000","last_undegraded":"2024-07-27T03:13:57.405550+0000","last_fullsized":"2024-07-27T03:13:57.405550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:13:29.104055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348631+0000","last_change":"2024-07-27T03:12:32.561067+0000","last_active":"2024-07-27T03:13:57.348631+0000","last_peered":"2024-07-27T03:13:57.348631+0000","last_clean":"2024-07-27T03:13:57.348631+0000","last_became_active":"2024-07-27T03:12:32.560358+0000","last_became_peered":"2024-07-27T03:12:32.560358+0000","last_unstale":"2024-07-27T03:13:57.348631+0000","last_undegraded":"2024-07-27T03:13:57.348631+0000","last_fullsized":"2024-07-27T03:13:57.348631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:41:41.026804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405482+0000","last_change":"2024-07-27T03:12:32.572290+0000","last_active":"2024-07-27T03:13:57.405482+0000","last_peered":"2024-07-27T03:13:57.405482+0000","last_clean":"2024-07-27T03:13:57.405482+0000","last_became_active":"2024-07-27T03:12:32.571877+0000","last_became_peered":"2024-07-27T03:12:32.571877+0000","last_unstale":"2024-07-27T03:13:57.405482+0000","last_undegraded":"2024-07-27T03:13:57.405482+0000","last_fullsized":"2024-07-27T03:13:57.405482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:34:40.559787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303421+0000","last_change":"2024-07-27T03:12:32.565495+0000","last_active":"2024-07-27T03:13:57.303421+0000","last_peered":"2024-07-27T03:13:57.303421+0000","last_clean":"2024-07-27T03:13:57.303421+0000","last_became_active":"2024-07-27T03:12:32.563136+0000","last_became_peered":"2024-07-27T03:12:32.563136+0000","last_unstale":"2024-07-27T03:13:57.303421+0000","last_undegraded":"2024-07-27T03:13:57.303421+0000","last_fullsized":"2024-07-27T03:13:57.303421+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:31:02.032776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302591+0000","last_change":"2024-07-27T03:12:32.569477+0000","last_active":"2024-07-27T03:13:57.302591+0000","last_peered":"2024-07-27T03:13:57.302591+0000","last_clean":"2024-07-27T03:13:57.302591+0000","last_became_active":"2024-07-27T03:12:32.569098+0000","last_became_peered":"2024-07-27T03:12:32.569098+0000","last_unstale":"2024-07-27T03:13:57.302591+0000","last_undegraded":"2024-07-27T03:13:57.302591+0000","last_fullsized":"2024-07-27T03:13:57.302591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:44:26.082539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406145+0000","last_change":"2024-07-27T03:12:32.574704+0000","last_active":"2024-07-27T03:13:57.406145+0000","last_peered":"2024-07-27T03:13:57.406145+0000","last_clean":"2024-07-27T03:13:57.406145+0000","last_became_active":"2024-07-27T03:12:32.574261+0000","last_became_peered":"2024-07-27T03:12:32.574261+0000","last_unstale":"2024-07-27T03:13:57.406145+0000","last_undegraded":"2024-07-27T03:13:57.406145+0000","last_fullsized":"2024-07-27T03:13:57.406145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:53:06.445267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405109+0000","last_change":"2024-07-27T03:12:32.573835+0000","last_active":"2024-07-27T03:13:57.405109+0000","last_peered":"2024-07-27T03:13:57.405109+0000","last_clean":"2024-07-27T03:13:57.405109+0000","last_became_active":"2024-07-27T03:12:32.573450+0000","last_became_peered":"2024-07-27T03:12:32.573450+0000","last_unstale":"2024-07-27T03:13:57.405109+0000","last_undegraded":"2024-07-27T03:13:57.405109+0000","last_fullsized":"2024-07-27T03:13:57.405109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:40:45.264000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404917+0000","last_change":"2024-07-27T03:12:32.568542+0000","last_active":"2024-07-27T03:13:57.404917+0000","last_peered":"2024-07-27T03:13:57.404917+0000","last_clean":"2024-07-27T03:13:57.404917+0000","last_became_active":"2024-07-27T03:12:32.568179+0000","last_became_peered":"2024-07-27T03:12:32.568179+0000","last_unstale":"2024-07-27T03:13:57.404917+0000","last_undegraded":"2024-07-27T03:13:57.404917+0000","last_fullsized":"2024-07-27T03:13:57.404917+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:32:10.005601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404879+0000","last_change":"2024-07-27T03:12:32.564063+0000","last_active":"2024-07-27T03:13:57.404879+0000","last_peered":"2024-07-27T03:13:57.404879+0000","last_clean":"2024-07-27T03:13:57.404879+0000","last_became_active":"2024-07-27T03:12:32.563705+0000","last_became_peered":"2024-07-27T03:12:32.563705+0000","last_unstale":"2024-07-27T03:13:57.404879+0000","last_undegraded":"2024-07-27T03:13:57.404879+0000","last_fullsized":"2024-07-27T03:13:57.404879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:40:19.234194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348574+0000","last_change":"2024-07-27T03:12:57.178963+0000","last_active":"2024-07-27T03:13:57.348574+0000","last_peered":"2024-07-27T03:13:57.348574+0000","last_clean":"2024-07-27T03:13:57.348574+0000","last_became_active":"2024-07-27T03:12:57.178550+0000","last_became_peered":"2024-07-27T03:12:57.178550+0000","last_unstale":"2024-07-27T03:13:57.348574+0000","last_undegraded":"2024-07-27T03:13:57.348574+0000","last_fullsized":"2024-07-27T03:13:57.348574+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:20:06.586957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304191+0000","last_change":"2024-07-27T03:12:32.569863+0000","last_active":"2024-07-27T03:13:57.304191+0000","last_peered":"2024-07-27T03:13:57.304191+0000","last_clean":"2024-07-27T03:13:57.304191+0000","last_became_active":"2024-07-27T03:12:32.566457+0000","last_became_peered":"2024-07-27T03:12:32.566457+0000","last_unstale":"2024-07-27T03:13:57.304191+0000","last_undegraded":"2024-07-27T03:13:57.304191+0000","last_fullsized":"2024-07-27T03:13:57.304191+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:28:13.141002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305935+0000","last_change":"2024-07-27T03:12:32.574490+0000","last_active":"2024-07-27T03:13:57.305935+0000","last_peered":"2024-07-27T03:13:57.305935+0000","last_clean":"2024-07-27T03:13:57.305935+0000","last_became_active":"2024-07-27T03:12:32.574130+0000","last_became_peered":"2024-07-27T03:12:32.574130+0000","last_unstale":"2024-07-27T03:13:57.305935+0000","last_undegraded":"2024-07-27T03:13:57.305935+0000","last_fullsized":"2024-07-27T03:13:57.305935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:44:22.696669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406481+0000","last_change":"2024-07-27T03:12:32.560106+0000","last_active":"2024-07-27T03:13:57.406481+0000","last_peered":"2024-07-27T03:13:57.406481+0000","last_clean":"2024-07-27T03:13:57.406481+0000","last_became_active":"2024-07-27T03:12:32.559751+0000","last_became_peered":"2024-07-27T03:12:32.559751+0000","last_unstale":"2024-07-27T03:13:57.406481+0000","last_undegraded":"2024-07-27T03:13:57.406481+0000","last_fullsized":"2024-07-27T03:13:57.406481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:16:37.647462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406736+0000","last_change":"2024-07-27T03:12:32.569352+0000","last_active":"2024-07-27T03:13:57.406736+0000","last_peered":"2024-07-27T03:13:57.406736+0000","last_clean":"2024-07-27T03:13:57.406736+0000","last_became_active":"2024-07-27T03:12:32.569004+0000","last_became_peered":"2024-07-27T03:12:32.569004+0000","last_unstale":"2024-07-27T03:13:57.406736+0000","last_undegraded":"2024-07-27T03:13:57.406736+0000","last_fullsized":"2024-07-27T03:13:57.406736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:21:25.999214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404059+0000","last_change":"2024-07-27T03:12:32.562775+0000","last_active":"2024-07-27T03:13:57.404059+0000","last_peered":"2024-07-27T03:13:57.404059+0000","last_clean":"2024-07-27T03:13:57.404059+0000","last_became_active":"2024-07-27T03:12:32.562403+0000","last_became_peered":"2024-07-27T03:12:32.562403+0000","last_unstale":"2024-07-27T03:13:57.404059+0000","last_undegraded":"2024-07-27T03:13:57.404059+0000","last_fullsized":"2024-07-27T03:13:57.404059+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:16:09.474505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305528+0000","last_change":"2024-07-27T03:12:32.563271+0000","last_active":"2024-07-27T03:13:57.305528+0000","last_peered":"2024-07-27T03:13:57.305528+0000","last_clean":"2024-07-27T03:13:57.305528+0000","last_became_active":"2024-07-27T03:12:32.562850+0000","last_became_peered":"2024-07-27T03:12:32.562850+0000","last_unstale":"2024-07-27T03:13:57.305528+0000","last_undegraded":"2024-07-27T03:13:57.305528+0000","last_fullsized":"2024-07-27T03:13:57.305528+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:40:27.688205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405933+0000","last_change":"2024-07-27T03:12:32.557580+0000","last_active":"2024-07-27T03:13:57.405933+0000","last_peered":"2024-07-27T03:13:57.405933+0000","last_clean":"2024-07-27T03:13:57.405933+0000","last_became_active":"2024-07-27T03:12:32.557190+0000","last_became_peered":"2024-07-27T03:12:32.557190+0000","last_unstale":"2024-07-27T03:13:57.405933+0000","last_undegraded":"2024-07-27T03:13:57.405933+0000","last_fullsized":"2024-07-27T03:13:57.405933+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:58:17.609035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304839+0000","last_change":"2024-07-27T03:12:32.563669+0000","last_active":"2024-07-27T03:13:57.304839+0000","last_peered":"2024-07-27T03:13:57.304839+0000","last_clean":"2024-07-27T03:13:57.304839+0000","last_became_active":"2024-07-27T03:12:32.563200+0000","last_became_peered":"2024-07-27T03:12:32.563200+0000","last_unstale":"2024-07-27T03:13:57.304839+0000","last_undegraded":"2024-07-27T03:13:57.304839+0000","last_fullsized":"2024-07-27T03:13:57.304839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:14:05.624118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304116+0000","last_change":"2024-07-27T03:12:32.561793+0000","last_active":"2024-07-27T03:13:57.304116+0000","last_peered":"2024-07-27T03:13:57.304116+0000","last_clean":"2024-07-27T03:13:57.304116+0000","last_became_active":"2024-07-27T03:12:32.561452+0000","last_became_peered":"2024-07-27T03:12:32.561452+0000","last_unstale":"2024-07-27T03:13:57.304116+0000","last_undegraded":"2024-07-27T03:13:57.304116+0000","last_fullsized":"2024-07-27T03:13:57.304116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:22:07.204824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405250+0000","last_change":"2024-07-27T03:12:32.569811+0000","last_active":"2024-07-27T03:13:57.405250+0000","last_peered":"2024-07-27T03:13:57.405250+0000","last_clean":"2024-07-27T03:13:57.405250+0000","last_became_active":"2024-07-27T03:12:32.569474+0000","last_became_peered":"2024-07-27T03:12:32.569474+0000","last_unstale":"2024-07-27T03:13:57.405250+0000","last_undegraded":"2024-07-27T03:13:57.405250+0000","last_fullsized":"2024-07-27T03:13:57.405250+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:38:39.317362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405291+0000","last_change":"2024-07-27T03:12:32.565159+0000","last_active":"2024-07-27T03:13:57.405291+0000","last_peered":"2024-07-27T03:13:57.405291+0000","last_clean":"2024-07-27T03:13:57.405291+0000","last_became_active":"2024-07-27T03:12:32.564750+0000","last_became_peered":"2024-07-27T03:12:32.564750+0000","last_unstale":"2024-07-27T03:13:57.405291+0000","last_undegraded":"2024-07-27T03:13:57.405291+0000","last_fullsized":"2024-07-27T03:13:57.405291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:51:26.186510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406117+0000","last_change":"2024-07-27T03:12:32.574429+0000","last_active":"2024-07-27T03:13:57.406117+0000","last_peered":"2024-07-27T03:13:57.406117+0000","last_clean":"2024-07-27T03:13:57.406117+0000","last_became_active":"2024-07-27T03:12:32.573986+0000","last_became_peered":"2024-07-27T03:12:32.573986+0000","last_unstale":"2024-07-27T03:13:57.406117+0000","last_undegraded":"2024-07-27T03:13:57.406117+0000","last_fullsized":"2024-07-27T03:13:57.406117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:35:00.345561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65011,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":292,"num_read_kb":217,"num_write":248,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":199298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645345,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":525562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841574,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37040,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708112,"statfs":{"total":95995035648,"available":95957106688,"internally_reserved":0,"allocated":1744896,"data_stored":1174650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037803,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1077248,"data_stored":525470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234031,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":525826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430260,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1671168,"data_stored":1122736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593785,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":520974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790013,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1679360,"data_stored":1117927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953539,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1060864,"data_stored":525439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149767,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":520682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1134592,"data_stored":580422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1118208,"data_stored":584076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1077248,"data_stored":522430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":60170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":59934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-27T03:14:08.152 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-07-27T03:14:08.153 INFO:tasks.cephadm:Setting up client nodes... 2024-07-27T03:14:08.153 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-07-27T03:14:08.427 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:08.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:08 smithi067 ceph-mon[32814]: pgmap v357: 129 pgs: 129 active+clean; 640 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s; 11 B/s, 0 objects/s recovering 2024-07-27T03:14:08.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:08 smithi067 ceph-mon[32814]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:14:08.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:08 smithi175 ceph-mon[37125]: pgmap v357: 129 pgs: 129 active+clean; 640 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s; 11 B/s, 0 objects/s recovering 2024-07-27T03:14:08.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:08 smithi175 ceph-mon[37125]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:14:08.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:08 smithi149 ceph-mon[37173]: pgmap v357: 129 pgs: 129 active+clean; 640 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s; 11 B/s, 0 objects/s recovering 2024-07-27T03:14:08.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:08 smithi149 ceph-mon[37173]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:14:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.240+0000 7f9ad21b9640 1 -- 172.21.15.67:0/3754807170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9acc102170 msgr2=0x7f9acc1025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.240+0000 7f9ad21b9640 1 --2- 172.21.15.67:0/3754807170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9acc102170 0x7f9acc1025e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009900 tx=0x7f9abc02a480 comp rx=0 tx=0).stop 2024-07-27T03:14:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.240+0000 7f9ad21b9640 1 -- 172.21.15.67:0/3754807170 shutdown_connections 2024-07-27T03:14:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.240+0000 7f9ad21b9640 1 --2- 172.21.15.67:0/3754807170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9acc103ba0 0x7f9acc10a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.240+0000 7f9ad21b9640 1 --2- 172.21.15.67:0/3754807170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9acc103200 0x7f9acc103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.240+0000 7f9ad21b9640 1 --2- 172.21.15.67:0/3754807170 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9acc102170 0x7f9acc1025e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.240+0000 7f9ad21b9640 1 -- 172.21.15.67:0/3754807170 >> 172.21.15.67:0/3754807170 conn(0x7f9acc0fdb00 msgr2=0x7f9acc0fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:10.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.240+0000 7f9ad21b9640 1 -- 172.21.15.67:0/3754807170 shutdown_connections 2024-07-27T03:14:10.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.240+0000 7f9ad21b9640 1 -- 172.21.15.67:0/3754807170 wait complete. 2024-07-27T03:14:10.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.241+0000 7f9ad21b9640 1 Processor -- start 2024-07-27T03:14:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.241+0000 7f9ad21b9640 1 -- start start 2024-07-27T03:14:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.242+0000 7f9ad21b9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9acc102170 0x7f9acc19bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.242+0000 7f9ad21b9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9acc103200 0x7f9acc19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.242+0000 7f9ad21b9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9acc103ba0 0x7f9acc19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.242+0000 7f9ad21b9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc10dd30 con 0x7f9acc103200 2024-07-27T03:14:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.242+0000 7f9ad21b9640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc10dbb0 con 0x7f9acc103ba0 2024-07-27T03:14:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.242+0000 7f9ad21b9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc10deb0 con 0x7f9acc102170 2024-07-27T03:14:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.242+0000 7f9acaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9acc103200 0x7f9acc19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.242+0000 7f9acaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9acc103200 0x7f9acc19c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58662/0 (socket says 172.21.15.67:58662) 2024-07-27T03:14:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.242+0000 7f9acaffd640 1 -- 172.21.15.67:0/432191693 learned_addr learned my addr 172.21.15.67:0/432191693 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:14:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.242+0000 7f9acb7fe640 1 --2- 172.21.15.67:0/432191693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9acc102170 0x7f9acc19bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.242+0000 7f9acbfff640 1 --2- 172.21.15.67:0/432191693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9acc103ba0 0x7f9acc19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.242+0000 7f9acaffd640 1 -- 172.21.15.67:0/432191693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9acc102170 msgr2=0x7f9acc19bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.242+0000 7f9acaffd640 1 --2- 172.21.15.67:0/432191693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9acc102170 0x7f9acc19bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.242+0000 7f9acaffd640 1 -- 172.21.15.67:0/432191693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9acc103ba0 msgr2=0x7f9acc19ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.242+0000 7f9acaffd640 1 --2- 172.21.15.67:0/432191693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9acc103ba0 0x7f9acc19ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.242+0000 7f9acaffd640 1 -- 172.21.15.67:0/432191693 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9acc19d680 con 0x7f9acc103200 2024-07-27T03:14:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.242+0000 7f9acb7fe640 1 --2- 172.21.15.67:0/432191693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9acc102170 0x7f9acc19bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:14:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.242+0000 7f9acbfff640 1 --2- 172.21.15.67:0/432191693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9acc103ba0 0x7f9acc19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:14:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.242+0000 7f9acaffd640 1 --2- 172.21.15.67:0/432191693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9acc103200 0x7f9acc19c2c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000c970 tx=0x7f9ac000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.243+0000 7f9ac8ff9640 1 -- 172.21.15.67:0/432191693 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0007bb0 con 0x7f9acc103200 2024-07-27T03:14:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.243+0000 7f9ac8ff9640 1 -- 172.21.15.67:0/432191693 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ac0007d50 con 0x7f9acc103200 2024-07-27T03:14:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.243+0000 7f9ad21b9640 1 -- 172.21.15.67:0/432191693 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9acc19d970 con 0x7f9acc103200 2024-07-27T03:14:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.243+0000 7f9ac8ff9640 1 -- 172.21.15.67:0/432191693 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac000f450 con 0x7f9acc103200 2024-07-27T03:14:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.243+0000 7f9ad21b9640 1 -- 172.21.15.67:0/432191693 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9acc1a4070 con 0x7f9acc103200 2024-07-27T03:14:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.244+0000 7f9ad21b9640 1 -- 172.21.15.67:0/432191693 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9acc1025e0 con 0x7f9acc103200 2024-07-27T03:14:10.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.245+0000 7f9ac8ff9640 1 -- 172.21.15.67:0/432191693 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f9ac0004f00 con 0x7f9acc103200 2024-07-27T03:14:10.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.246+0000 7f9ac8ff9640 1 --2- 172.21.15.67:0/432191693 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f9ab40b3c60 0x7f9ab40b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:10.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.246+0000 7f9acb7fe640 1 --2- 172.21.15.67:0/432191693 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f9ab40b3c60 0x7f9ab40b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:10.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.246+0000 7f9ac8ff9640 1 -- 172.21.15.67:0/432191693 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f9ac00e4350 con 0x7f9acc103200 2024-07-27T03:14:10.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.247+0000 7f9acb7fe640 1 --2- 172.21.15.67:0/432191693 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f9ab40b3c60 0x7f9ab40b6170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9abc031440 tx=0x7f9abc005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:10.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.249+0000 7f9ac8ff9640 1 -- 172.21.15.67:0/432191693 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f9ac0014030 con 0x7f9acc103200 2024-07-27T03:14:10.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.504+0000 7f9ad21b9640 1 -- 172.21.15.67:0/432191693 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f9acc1a44a0 con 0x7f9acc103200 2024-07-27T03:14:10.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.507+0000 7f9ac8ff9640 1 -- 172.21.15.67:0/432191693 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f9ac00ab490 con 0x7f9acc103200 2024-07-27T03:14:10.510 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2024-07-27T03:14:10.510 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQACZqRmeaIlHhAAIic7TJ2nByEB7WRfHD0w9A== 2024-07-27T03:14:10.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.509+0000 7f9ad21b9640 1 -- 172.21.15.67:0/432191693 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f9ab40b3c60 msgr2=0x7f9ab40b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:10.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.509+0000 7f9ad21b9640 1 --2- 172.21.15.67:0/432191693 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f9ab40b3c60 0x7f9ab40b6170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9abc031440 tx=0x7f9abc005d50 comp rx=0 tx=0).stop 2024-07-27T03:14:10.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.509+0000 7f9ad21b9640 1 -- 172.21.15.67:0/432191693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9acc103200 msgr2=0x7f9acc19c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:10.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.509+0000 7f9ad21b9640 1 --2- 172.21.15.67:0/432191693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9acc103200 0x7f9acc19c2c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000c970 tx=0x7f9ac000ce30 comp rx=0 tx=0).stop 2024-07-27T03:14:10.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.509+0000 7f9ad21b9640 1 -- 172.21.15.67:0/432191693 shutdown_connections 2024-07-27T03:14:10.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.509+0000 7f9ad21b9640 1 --2- 172.21.15.67:0/432191693 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f9ab40b3c60 0x7f9ab40b6170 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:10.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.509+0000 7f9ad21b9640 1 --2- 172.21.15.67:0/432191693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9acc103ba0 0x7f9acc19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:10.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.509+0000 7f9ad21b9640 1 --2- 172.21.15.67:0/432191693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9acc103200 0x7f9acc19c2c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:10.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.509+0000 7f9ad21b9640 1 --2- 172.21.15.67:0/432191693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9acc102170 0x7f9acc19bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:10.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.509+0000 7f9ad21b9640 1 -- 172.21.15.67:0/432191693 >> 172.21.15.67:0/432191693 conn(0x7f9acc0fdb00 msgr2=0x7f9acc0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:10.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.510+0000 7f9ad21b9640 1 -- 172.21.15.67:0/432191693 shutdown_connections 2024-07-27T03:14:10.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:10.510+0000 7f9ad21b9640 1 -- 172.21.15.67:0/432191693 wait complete. 2024-07-27T03:14:10.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:10 smithi175 ceph-mon[37125]: pgmap v358: 129 pgs: 129 active+clean; 640 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s; 9 B/s, 0 objects/s recovering 2024-07-27T03:14:10.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:10 smithi067 ceph-mon[32814]: pgmap v358: 129 pgs: 129 active+clean; 640 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s; 9 B/s, 0 objects/s recovering 2024-07-27T03:14:10.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:10 smithi149 ceph-mon[37173]: pgmap v358: 129 pgs: 129 active+clean; 640 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 10 op/s; 9 B/s, 0 objects/s recovering 2024-07-27T03:14:11.149 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-27T03:14:11.149 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-07-27T03:14:11.150 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-07-27T03:14:11.191 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph config log 1 --format=json 2024-07-27T03:14:11.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:11 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/432191693' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-27T03:14:11.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:11 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/432191693' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-27T03:14:11.497 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:11.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:11 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/432191693' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-27T03:14:11.627 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:11 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/432191693' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-27T03:14:11.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:11 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/432191693' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-07-27T03:14:11.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:11 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/432191693' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-07-27T03:14:12.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:12 smithi067 ceph-mon[32814]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 12 op/s; 9 B/s, 0 objects/s recovering 2024-07-27T03:14:12.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:12 smithi175 ceph-mon[37125]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 12 op/s; 9 B/s, 0 objects/s recovering 2024-07-27T03:14:13.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:12 smithi149 ceph-mon[37173]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 12 op/s; 9 B/s, 0 objects/s recovering 2024-07-27T03:14:13.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.392+0000 7fc201699640 1 -- 172.21.15.67:0/3210599467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1fc102550 msgr2=0x7fc1fc102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:13.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.392+0000 7fc201699640 1 --2- 172.21.15.67:0/3210599467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1fc102550 0x7fc1fc102960 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009900 tx=0x7fc1ec02a410 comp rx=0 tx=0).stop 2024-07-27T03:14:13.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.393+0000 7fc201699640 1 -- 172.21.15.67:0/3210599467 shutdown_connections 2024-07-27T03:14:13.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.393+0000 7fc201699640 1 --2- 172.21.15.67:0/3210599467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1fc108790 0x7fc1fc10efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:13.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.393+0000 7fc201699640 1 --2- 172.21.15.67:0/3210599467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1fc107e40 0x7fc1fc108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:13.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.393+0000 7fc201699640 1 --2- 172.21.15.67:0/3210599467 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1fc102550 0x7fc1fc102960 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:13.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.393+0000 7fc201699640 1 -- 172.21.15.67:0/3210599467 >> 172.21.15.67:0/3210599467 conn(0x7fc1fc0791d0 msgr2=0x7fc1fc0795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:13.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.393+0000 7fc201699640 1 -- 172.21.15.67:0/3210599467 shutdown_connections 2024-07-27T03:14:13.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.393+0000 7fc201699640 1 -- 172.21.15.67:0/3210599467 wait complete. 2024-07-27T03:14:13.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.394+0000 7fc201699640 1 Processor -- start 2024-07-27T03:14:13.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.394+0000 7fc201699640 1 -- start start 2024-07-27T03:14:13.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.394+0000 7fc201699640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1fc102550 0x7fc1fc198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:13.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.394+0000 7fc201699640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1fc107e40 0x7fc1fc199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:13.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.394+0000 7fc201699640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1fc108790 0x7fc1fc19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:13.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.394+0000 7fc201699640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc112410 con 0x7fc1fc108790 2024-07-27T03:14:13.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.394+0000 7fc201699640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc112290 con 0x7fc1fc102550 2024-07-27T03:14:13.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.394+0000 7fc201699640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc112590 con 0x7fc1fc107e40 2024-07-27T03:14:13.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.395+0000 7fc1fb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1fc108790 0x7fc1fc19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:13.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.395+0000 7fc1fb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1fc108790 0x7fc1fc19e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58672/0 (socket says 172.21.15.67:58672) 2024-07-27T03:14:13.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.395+0000 7fc1fb7fe640 1 -- 172.21.15.67:0/361002778 learned_addr learned my addr 172.21.15.67:0/361002778 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:14:13.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.395+0000 7fc1faffd640 1 --2- 172.21.15.67:0/361002778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1fc102550 0x7fc1fc198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:13.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.395+0000 7fc1fa7fc640 1 --2- 172.21.15.67:0/361002778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1fc107e40 0x7fc1fc199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:13.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.395+0000 7fc1fb7fe640 1 -- 172.21.15.67:0/361002778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1fc107e40 msgr2=0x7fc1fc199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.395+0000 7fc1fb7fe640 1 --2- 172.21.15.67:0/361002778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1fc107e40 0x7fc1fc199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.395+0000 7fc1fb7fe640 1 -- 172.21.15.67:0/361002778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1fc102550 msgr2=0x7fc1fc198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.395+0000 7fc1fb7fe640 1 --2- 172.21.15.67:0/361002778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1fc102550 0x7fc1fc198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.395+0000 7fc1fb7fe640 1 -- 172.21.15.67:0/361002778 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1fc19ed30 con 0x7fc1fc108790 2024-07-27T03:14:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.395+0000 7fc1fa7fc640 1 --2- 172.21.15.67:0/361002778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1fc107e40 0x7fc1fc199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:14:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.395+0000 7fc1fb7fe640 1 --2- 172.21.15.67:0/361002778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1fc108790 0x7fc1fc19e600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400eaa0 tx=0x7fc1f400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.395+0000 7fc1e3fff640 1 -- 172.21.15.67:0/361002778 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f400cdb0 con 0x7fc1fc108790 2024-07-27T03:14:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.395+0000 7fc1e3fff640 1 -- 172.21.15.67:0/361002778 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f4005020 con 0x7fc1fc108790 2024-07-27T03:14:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.395+0000 7fc1e3fff640 1 -- 172.21.15.67:0/361002778 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f40116d0 con 0x7fc1fc108790 2024-07-27T03:14:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.396+0000 7fc201699640 1 -- 172.21.15.67:0/361002778 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1fc19f020 con 0x7fc1fc108790 2024-07-27T03:14:13.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.396+0000 7fc201699640 1 -- 172.21.15.67:0/361002778 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1fc1a5780 con 0x7fc1fc108790 2024-07-27T03:14:13.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.397+0000 7fc201699640 1 -- 172.21.15.67:0/361002778 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1fc103780 con 0x7fc1fc108790 2024-07-27T03:14:13.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.399+0000 7fc1e3fff640 1 -- 172.21.15.67:0/361002778 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7fc1f40051d0 con 0x7fc1fc108790 2024-07-27T03:14:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.400+0000 7fc1e3fff640 1 --2- 172.21.15.67:0/361002778 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc1e40b3c60 0x7fc1e40b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.400+0000 7fc1faffd640 1 --2- 172.21.15.67:0/361002778 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc1e40b3c60 0x7fc1e40b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.400+0000 7fc1e3fff640 1 -- 172.21.15.67:0/361002778 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fc1f40e46b0 con 0x7fc1fc108790 2024-07-27T03:14:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.401+0000 7fc1faffd640 1 --2- 172.21.15.67:0/361002778 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc1e40b3c60 0x7fc1e40b6170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec031440 tx=0x7fc1ec009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:13.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.403+0000 7fc1e3fff640 1 -- 172.21.15.67:0/361002778 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fc1f4016030 con 0x7fc1fc108790 2024-07-27T03:14:13.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.591+0000 7fc201699640 1 -- 172.21.15.67:0/361002778 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fc1fc102960 con 0x7fc1fc108790 2024-07-27T03:14:13.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.591+0000 7fc1e3fff640 1 -- 172.21.15.67:0/361002778 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v50) ==== 86+0+133 (secure 0 0 0) 0x7fc1fc102960 con 0x7fc1fc108790 2024-07-27T03:14:13.594 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:14:13.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.593+0000 7fc201699640 1 -- 172.21.15.67:0/361002778 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc1e40b3c60 msgr2=0x7fc1e40b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:13.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.593+0000 7fc201699640 1 --2- 172.21.15.67:0/361002778 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc1e40b3c60 0x7fc1e40b6170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec031440 tx=0x7fc1ec009560 comp rx=0 tx=0).stop 2024-07-27T03:14:13.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.593+0000 7fc201699640 1 -- 172.21.15.67:0/361002778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1fc108790 msgr2=0x7fc1fc19e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:13.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.593+0000 7fc201699640 1 --2- 172.21.15.67:0/361002778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1fc108790 0x7fc1fc19e600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400eaa0 tx=0x7fc1f400ef60 comp rx=0 tx=0).stop 2024-07-27T03:14:13.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.593+0000 7fc201699640 1 -- 172.21.15.67:0/361002778 shutdown_connections 2024-07-27T03:14:13.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.593+0000 7fc201699640 1 --2- 172.21.15.67:0/361002778 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc1e40b3c60 0x7fc1e40b6170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:13.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.593+0000 7fc201699640 1 --2- 172.21.15.67:0/361002778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1fc108790 0x7fc1fc19e600 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:13.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.593+0000 7fc201699640 1 --2- 172.21.15.67:0/361002778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1fc107e40 0x7fc1fc199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:13.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.593+0000 7fc201699640 1 --2- 172.21.15.67:0/361002778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1fc102550 0x7fc1fc198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:13.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.593+0000 7fc201699640 1 -- 172.21.15.67:0/361002778 >> 172.21.15.67:0/361002778 conn(0x7fc1fc0791d0 msgr2=0x7fc1fc100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:13.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.594+0000 7fc201699640 1 -- 172.21.15.67:0/361002778 shutdown_connections 2024-07-27T03:14:13.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:13.594+0000 7fc201699640 1 -- 172.21.15.67:0/361002778 wait complete. 2024-07-27T03:14:14.311 INFO:teuthology.orchestra.run.smithi067.stdout:[{"version":50,"timestamp":"2024-07-27T03:12:20.627630+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-07-27T03:14:14.312 INFO:tasks.ceph_manager:config epoch is 50 2024-07-27T03:14:14.312 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-07-27T03:14:14.312 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-07-27T03:14:14.312 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph mgr dump --format=json 2024-07-27T03:14:14.583 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:14.705 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:14 smithi067 ceph-mon[32814]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-27T03:14:14.705 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:14 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/361002778' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-07-27T03:14:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:14 smithi175 ceph-mon[37125]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-27T03:14:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:14 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/361002778' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-07-27T03:14:15.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:14 smithi149 ceph-mon[37173]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-27T03:14:15.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:14 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/361002778' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-07-27T03:14:16.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.271+0000 7fea61f5c640 1 -- 172.21.15.67:0/2813141065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea5c108f80 msgr2=0x7fea5c109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:16.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.271+0000 7fea61f5c640 1 --2- 172.21.15.67:0/2813141065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea5c108f80 0x7fea5c109390 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fea44009900 tx=0x7fea4402a410 comp rx=0 tx=0).stop 2024-07-27T03:14:16.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.272+0000 7fea61f5c640 1 -- 172.21.15.67:0/2813141065 shutdown_connections 2024-07-27T03:14:16.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.272+0000 7fea61f5c640 1 --2- 172.21.15.67:0/2813141065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea5c1098d0 0x7fea5c110190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:16.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.272+0000 7fea61f5c640 1 --2- 172.21.15.67:0/2813141065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea5c108f80 0x7fea5c109390 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:16.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.272+0000 7fea61f5c640 1 --2- 172.21.15.67:0/2813141065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea5c101ad0 0x7fea5c101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:16.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.272+0000 7fea61f5c640 1 -- 172.21.15.67:0/2813141065 >> 172.21.15.67:0/2813141065 conn(0x7fea5c079a00 msgr2=0x7fea5c079e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:16.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.272+0000 7fea61f5c640 1 -- 172.21.15.67:0/2813141065 shutdown_connections 2024-07-27T03:14:16.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.272+0000 7fea61f5c640 1 -- 172.21.15.67:0/2813141065 wait complete. 2024-07-27T03:14:16.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.272+0000 7fea61f5c640 1 Processor -- start 2024-07-27T03:14:16.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.273+0000 7fea61f5c640 1 -- start start 2024-07-27T03:14:16.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.273+0000 7fea61f5c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea5c101ad0 0x7fea5c199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:16.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.273+0000 7fea61f5c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea5c108f80 0x7fea5c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:16.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.273+0000 7fea61f5c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea5c1098d0 0x7fea5c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:16.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.273+0000 7fea61f5c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c113530 con 0x7fea5c101ad0 2024-07-27T03:14:16.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.273+0000 7fea61f5c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1133b0 con 0x7fea5c108f80 2024-07-27T03:14:16.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.273+0000 7fea61f5c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c1136b0 con 0x7fea5c1098d0 2024-07-27T03:14:16.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.273+0000 7fea5b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea5c101ad0 0x7fea5c199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:16.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.273+0000 7fea5b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea5c101ad0 0x7fea5c199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52286/0 (socket says 172.21.15.67:52286) 2024-07-27T03:14:16.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.273+0000 7fea5b7fe640 1 -- 172.21.15.67:0/1143723914 learned_addr learned my addr 172.21.15.67:0/1143723914 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:14:16.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.273+0000 7fea5affd640 1 --2- 172.21.15.67:0/1143723914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea5c108f80 0x7fea5c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:16.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.273+0000 7fea5bfff640 1 --2- 172.21.15.67:0/1143723914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea5c1098d0 0x7fea5c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:16.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.274+0000 7fea5b7fe640 1 -- 172.21.15.67:0/1143723914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea5c1098d0 msgr2=0x7fea5c19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:16.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.274+0000 7fea5b7fe640 1 --2- 172.21.15.67:0/1143723914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea5c1098d0 0x7fea5c19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:16.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.274+0000 7fea5b7fe640 1 -- 172.21.15.67:0/1143723914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea5c108f80 msgr2=0x7fea5c19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:16.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.274+0000 7fea5b7fe640 1 --2- 172.21.15.67:0/1143723914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea5c108f80 0x7fea5c19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:16.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.274+0000 7fea5b7fe640 1 -- 172.21.15.67:0/1143723914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c19ff10 con 0x7fea5c101ad0 2024-07-27T03:14:16.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.274+0000 7fea5b7fe640 1 --2- 172.21.15.67:0/1143723914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea5c101ad0 0x7fea5c199fa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fea4c00e9f0 tx=0x7fea4c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:16.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.274+0000 7fea58ff9640 1 -- 172.21.15.67:0/1143723914 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c00cd80 con 0x7fea5c101ad0 2024-07-27T03:14:16.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.274+0000 7fea58ff9640 1 -- 172.21.15.67:0/1143723914 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea4c0108e0 con 0x7fea5c101ad0 2024-07-27T03:14:16.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.274+0000 7fea61f5c640 1 -- 172.21.15.67:0/1143723914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5c1a0200 con 0x7fea5c101ad0 2024-07-27T03:14:16.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.274+0000 7fea58ff9640 1 -- 172.21.15.67:0/1143723914 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c018980 con 0x7fea5c101ad0 2024-07-27T03:14:16.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.274+0000 7fea61f5c640 1 -- 172.21.15.67:0/1143723914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea5c1a6960 con 0x7fea5c101ad0 2024-07-27T03:14:16.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.275+0000 7fea61f5c640 1 -- 172.21.15.67:0/1143723914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea5c101ee0 con 0x7fea5c101ad0 2024-07-27T03:14:16.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.277+0000 7fea58ff9640 1 -- 172.21.15.67:0/1143723914 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7fea4c010430 con 0x7fea5c101ad0 2024-07-27T03:14:16.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.277+0000 7fea58ff9640 1 --2- 172.21.15.67:0/1143723914 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fea480b3c60 0x7fea480b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:16.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.278+0000 7fea5affd640 1 --2- 172.21.15.67:0/1143723914 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fea480b3c60 0x7fea480b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:16.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.278+0000 7fea58ff9640 1 -- 172.21.15.67:0/1143723914 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fea4c0e4a80 con 0x7fea5c101ad0 2024-07-27T03:14:16.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.278+0000 7fea5affd640 1 --2- 172.21.15.67:0/1143723914 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fea480b3c60 0x7fea480b6170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fea44009230 tx=0x7fea44031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:16.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.281+0000 7fea58ff9640 1 -- 172.21.15.67:0/1143723914 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fea4c0abbc0 con 0x7fea5c101ad0 2024-07-27T03:14:16.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.512+0000 7fea61f5c640 1 -- 172.21.15.67:0/1143723914 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fea5c02e100 con 0x7fea5c101ad0 2024-07-27T03:14:16.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.517+0000 7fea58ff9640 1 -- 172.21.15.67:0/1143723914 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+291017 (secure 0 0 0) 0x7fea4c0ab4c0 con 0x7fea5c101ad0 2024-07-27T03:14:16.523 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:14:16.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.521+0000 7fea61f5c640 1 -- 172.21.15.67:0/1143723914 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fea480b3c60 msgr2=0x7fea480b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:16.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.521+0000 7fea61f5c640 1 --2- 172.21.15.67:0/1143723914 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fea480b3c60 0x7fea480b6170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fea44009230 tx=0x7fea44031400 comp rx=0 tx=0).stop 2024-07-27T03:14:16.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.521+0000 7fea61f5c640 1 -- 172.21.15.67:0/1143723914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea5c101ad0 msgr2=0x7fea5c199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:16.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.521+0000 7fea61f5c640 1 --2- 172.21.15.67:0/1143723914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea5c101ad0 0x7fea5c199fa0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fea4c00e9f0 tx=0x7fea4c00eeb0 comp rx=0 tx=0).stop 2024-07-27T03:14:16.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.521+0000 7fea61f5c640 1 -- 172.21.15.67:0/1143723914 shutdown_connections 2024-07-27T03:14:16.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.521+0000 7fea61f5c640 1 --2- 172.21.15.67:0/1143723914 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fea480b3c60 0x7fea480b6170 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:16.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.521+0000 7fea61f5c640 1 --2- 172.21.15.67:0/1143723914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea5c1098d0 0x7fea5c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:16.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.521+0000 7fea61f5c640 1 --2- 172.21.15.67:0/1143723914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea5c108f80 0x7fea5c19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:16.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.521+0000 7fea61f5c640 1 --2- 172.21.15.67:0/1143723914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fea5c101ad0 0x7fea5c199fa0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:16.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.521+0000 7fea61f5c640 1 -- 172.21.15.67:0/1143723914 >> 172.21.15.67:0/1143723914 conn(0x7fea5c079a00 msgr2=0x7fea5c107090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:16.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.521+0000 7fea61f5c640 1 -- 172.21.15.67:0/1143723914 shutdown_connections 2024-07-27T03:14:16.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:16.521+0000 7fea61f5c640 1 -- 172.21.15.67:0/1143723914 wait complete. 2024-07-27T03:14:16.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:16 smithi067 ceph-mon[32814]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 11 op/s 2024-07-27T03:14:16.527 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:16 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1143723914' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-07-27T03:14:16.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:16 smithi175 ceph-mon[37125]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 11 op/s 2024-07-27T03:14:16.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:16 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1143723914' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-07-27T03:14:17.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:16 smithi149 ceph-mon[37173]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 11 op/s 2024-07-27T03:14:17.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:16 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1143723914' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-07-27T03:14:17.200 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":1534270904},{"type":"v1","addr":"172.21.15.67:6801","nonce":1534270904}]},"active_addr":"172.21.15.67:6801/1534270904","active_change":"2024-07-27T03:04:54.948110+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24122,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.67:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":3926355641}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":138992728}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":247463021}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":1722417064}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":280701211}]}]} 2024-07-27T03:14:17.206 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-07-27T03:14:17.206 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-07-27T03:14:17.206 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd dump --format=json 2024-07-27T03:14:17.473 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:18.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:18 smithi067 ceph-mon[32814]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-07-27T03:14:18.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:18 smithi175 ceph-mon[37125]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-07-27T03:14:19.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:18 smithi149 ceph-mon[37173]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-07-27T03:14:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.399+0000 7fd9583a9640 1 -- 172.21.15.67:0/571515932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9501024f0 msgr2=0x7fd950102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.399+0000 7fd9583a9640 1 --2- 172.21.15.67:0/571515932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9501024f0 0x7fd950102900 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009900 tx=0x7fd94c02a3f0 comp rx=0 tx=0).stop 2024-07-27T03:14:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.399+0000 7fd9583a9640 1 -- 172.21.15.67:0/571515932 shutdown_connections 2024-07-27T03:14:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.399+0000 7fd9583a9640 1 --2- 172.21.15.67:0/571515932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd950108780 0x7fd95010efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.399+0000 7fd9583a9640 1 --2- 172.21.15.67:0/571515932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd950107e30 0x7fd950108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.399+0000 7fd9583a9640 1 --2- 172.21.15.67:0/571515932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9501024f0 0x7fd950102900 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.399+0000 7fd9583a9640 1 -- 172.21.15.67:0/571515932 >> 172.21.15.67:0/571515932 conn(0x7fd9500791b0 msgr2=0x7fd9500795c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.400+0000 7fd9583a9640 1 -- 172.21.15.67:0/571515932 shutdown_connections 2024-07-27T03:14:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.400+0000 7fd9583a9640 1 -- 172.21.15.67:0/571515932 wait complete. 2024-07-27T03:14:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.400+0000 7fd9583a9640 1 Processor -- start 2024-07-27T03:14:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.400+0000 7fd9583a9640 1 -- start start 2024-07-27T03:14:19.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.400+0000 7fd9583a9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9501024f0 0x7fd95019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:19.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.401+0000 7fd9583a9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd950107e30 0x7fd95019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:19.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.401+0000 7fd9583a9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd950108780 0x7fd95019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:19.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.401+0000 7fd9583a9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd950112460 con 0x7fd9501024f0 2024-07-27T03:14:19.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.401+0000 7fd9583a9640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501122e0 con 0x7fd950108780 2024-07-27T03:14:19.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.401+0000 7fd9583a9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501125e0 con 0x7fd950107e30 2024-07-27T03:14:19.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.401+0000 7fd956145640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9501024f0 0x7fd95019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:19.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.401+0000 7fd956145640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9501024f0 0x7fd95019ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52300/0 (socket says 172.21.15.67:52300) 2024-07-27T03:14:19.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.401+0000 7fd956145640 1 -- 172.21.15.67:0/773051581 learned_addr learned my addr 172.21.15.67:0/773051581 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:14:19.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.401+0000 7fd955944640 1 --2- 172.21.15.67:0/773051581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd950107e30 0x7fd95019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:19.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.401+0000 7fd956946640 1 --2- 172.21.15.67:0/773051581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd950108780 0x7fd95019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:19.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.401+0000 7fd956145640 1 -- 172.21.15.67:0/773051581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd950107e30 msgr2=0x7fd95019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:19.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.401+0000 7fd956145640 1 --2- 172.21.15.67:0/773051581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd950107e30 0x7fd95019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:19.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.401+0000 7fd956145640 1 -- 172.21.15.67:0/773051581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd950108780 msgr2=0x7fd95019b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:19.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.401+0000 7fd956145640 1 --2- 172.21.15.67:0/773051581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd950108780 0x7fd95019b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:19.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.401+0000 7fd956145640 1 -- 172.21.15.67:0/773051581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9501a5fa0 con 0x7fd9501024f0 2024-07-27T03:14:19.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.401+0000 7fd956145640 1 --2- 172.21.15.67:0/773051581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9501024f0 0x7fd95019ad70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd94c006010 tx=0x7fd94c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:19.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.401+0000 7fd9477fe640 1 -- 172.21.15.67:0/773051581 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c03f030 con 0x7fd9501024f0 2024-07-27T03:14:19.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.401+0000 7fd9477fe640 1 -- 172.21.15.67:0/773051581 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd94c003800 con 0x7fd9501024f0 2024-07-27T03:14:19.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.402+0000 7fd9583a9640 1 -- 172.21.15.67:0/773051581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9501a6230 con 0x7fd9501024f0 2024-07-27T03:14:19.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.402+0000 7fd9477fe640 1 -- 172.21.15.67:0/773051581 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c035070 con 0x7fd9501024f0 2024-07-27T03:14:19.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.402+0000 7fd9583a9640 1 -- 172.21.15.67:0/773051581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9501a6790 con 0x7fd9501024f0 2024-07-27T03:14:19.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.403+0000 7fd9583a9640 1 -- 172.21.15.67:0/773051581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd950103770 con 0x7fd9501024f0 2024-07-27T03:14:19.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.404+0000 7fd9477fe640 1 -- 172.21.15.67:0/773051581 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7fd94c003cb0 con 0x7fd9501024f0 2024-07-27T03:14:19.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.405+0000 7fd9477fe640 1 --2- 172.21.15.67:0/773051581 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd93c0b3c60 0x7fd93c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:19.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.405+0000 7fd955944640 1 --2- 172.21.15.67:0/773051581 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd93c0b3c60 0x7fd93c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:19.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.405+0000 7fd9477fe640 1 -- 172.21.15.67:0/773051581 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fd94c104cd0 con 0x7fd9501024f0 2024-07-27T03:14:19.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.405+0000 7fd955944640 1 --2- 172.21.15.67:0/773051581 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd93c0b3c60 0x7fd93c0b6170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd950100670 tx=0x7fd94000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:19.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.408+0000 7fd9477fe640 1 -- 172.21.15.67:0/773051581 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fd94c0cbe10 con 0x7fd9501024f0 2024-07-27T03:14:19.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.592+0000 7fd9583a9640 1 -- 172.21.15.67:0/773051581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd950102900 con 0x7fd9501024f0 2024-07-27T03:14:19.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.593+0000 7fd9477fe640 1 -- 172.21.15.67:0/773051581 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24829 (secure 0 0 0) 0x7fd94c0cb710 con 0x7fd9501024f0 2024-07-27T03:14:19.597 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:14:19.597 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":91,"fsid":"c0782a00-4bc4-11ef-bcc7-c7b262605968","created":"2024-07-27T03:04:08.096824+0000","modified":"2024-07-27T03:13:57.294556+0000","last_up_change":"2024-07-27T03:11:33.314800+0000","last_in_change":"2024-07-27T03:11:14.028095+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-27T03:08:01.042479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-27T03:12:27.565948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-27T03:12:31.353726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-27T03:12:40.599486+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9f614272-18e8-4562-812c-4f43e40efd2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6803","nonce":2648129674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6805","nonce":2648129674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6809","nonce":2648129674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6807","nonce":2648129674}]},"public_addr":"172.21.15.67:6803/2648129674","cluster_addr":"172.21.15.67:6805/2648129674","heartbeat_back_addr":"172.21.15.67:6809/2648129674","heartbeat_front_addr":"172.21.15.67:6807/2648129674","state":["exists","up"]},{"osd":1,"uuid":"f59f5879-163a-4269-bbe7-1200eea65341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6801","nonce":2561753152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6803","nonce":2561753152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6807","nonce":2561753152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6805","nonce":2561753152}]},"public_addr":"172.21.15.149:6801/2561753152","cluster_addr":"172.21.15.149:6803/2561753152","heartbeat_back_addr":"172.21.15.149:6807/2561753152","heartbeat_front_addr":"172.21.15.149:6805/2561753152","state":["exists","up"]},{"osd":2,"uuid":"444e0ebe-7ca3-4be9-8bd5-db859f824445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6801","nonce":3770940078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6803","nonce":3770940078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6807","nonce":3770940078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6805","nonce":3770940078}]},"public_addr":"172.21.15.175:6801/3770940078","cluster_addr":"172.21.15.175:6803/3770940078","heartbeat_back_addr":"172.21.15.175:6807/3770940078","heartbeat_front_addr":"172.21.15.175:6805/3770940078","state":["exists","up"]},{"osd":3,"uuid":"d554db3d-9604-427c-8736-b1bb379f8d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6811","nonce":3159144693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6813","nonce":3159144693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6817","nonce":3159144693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6815","nonce":3159144693}]},"public_addr":"172.21.15.67:6811/3159144693","cluster_addr":"172.21.15.67:6813/3159144693","heartbeat_back_addr":"172.21.15.67:6817/3159144693","heartbeat_front_addr":"172.21.15.67:6815/3159144693","state":["exists","up"]},{"osd":4,"uuid":"c317dbe5-3b49-47a6-8408-7042a3dac345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3393503532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3393503532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3393503532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3393503532}]},"public_addr":"172.21.15.149:6809/3393503532","cluster_addr":"172.21.15.149:6811/3393503532","heartbeat_back_addr":"172.21.15.149:6815/3393503532","heartbeat_front_addr":"172.21.15.149:6813/3393503532","state":["exists","up"]},{"osd":5,"uuid":"5c85abd6-2b42-4957-a9b7-59a9ee36b841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6809","nonce":2291204784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6811","nonce":2291204784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6815","nonce":2291204784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6813","nonce":2291204784}]},"public_addr":"172.21.15.175:6809/2291204784","cluster_addr":"172.21.15.175:6811/2291204784","heartbeat_back_addr":"172.21.15.175:6815/2291204784","heartbeat_front_addr":"172.21.15.175:6813/2291204784","state":["exists","up"]},{"osd":6,"uuid":"bf6c277d-ec24-4470-98de-cf6f28876ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6819","nonce":2660182879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6821","nonce":2660182879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6825","nonce":2660182879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6823","nonce":2660182879}]},"public_addr":"172.21.15.67:6819/2660182879","cluster_addr":"172.21.15.67:6821/2660182879","heartbeat_back_addr":"172.21.15.67:6825/2660182879","heartbeat_front_addr":"172.21.15.67:6823/2660182879","state":["exists","up"]},{"osd":7,"uuid":"fea2c7cf-d5cb-45ae-9b29-f6e56c903891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6817","nonce":2289624404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6819","nonce":2289624404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6823","nonce":2289624404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6821","nonce":2289624404}]},"public_addr":"172.21.15.149:6817/2289624404","cluster_addr":"172.21.15.149:6819/2289624404","heartbeat_back_addr":"172.21.15.149:6823/2289624404","heartbeat_front_addr":"172.21.15.149:6821/2289624404","state":["exists","up"]},{"osd":8,"uuid":"14d04d96-579a-4dc9-a4d4-50e4850b48e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6817","nonce":2286032466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6819","nonce":2286032466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6823","nonce":2286032466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6821","nonce":2286032466}]},"public_addr":"172.21.15.175:6817/2286032466","cluster_addr":"172.21.15.175:6819/2286032466","heartbeat_back_addr":"172.21.15.175:6823/2286032466","heartbeat_front_addr":"172.21.15.175:6821/2286032466","state":["exists","up"]},{"osd":9,"uuid":"b65dfb6b-1efe-4314-a8be-ce6a295e99fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6827","nonce":3411081513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6829","nonce":3411081513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6833","nonce":3411081513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6831","nonce":3411081513}]},"public_addr":"172.21.15.67:6827/3411081513","cluster_addr":"172.21.15.67:6829/3411081513","heartbeat_back_addr":"172.21.15.67:6833/3411081513","heartbeat_front_addr":"172.21.15.67:6831/3411081513","state":["exists","up"]},{"osd":10,"uuid":"e3dabae0-0d64-4222-984c-fc1cce883456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6825","nonce":2011495212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6827","nonce":2011495212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6831","nonce":2011495212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6829","nonce":2011495212}]},"public_addr":"172.21.15.149:6825/2011495212","cluster_addr":"172.21.15.149:6827/2011495212","heartbeat_back_addr":"172.21.15.149:6831/2011495212","heartbeat_front_addr":"172.21.15.149:6829/2011495212","state":["exists","up"]},{"osd":11,"uuid":"406c082c-9b5f-4781-8cbf-3c04fa49e410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6825","nonce":2565514213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6827","nonce":2565514213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6831","nonce":2565514213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6829","nonce":2565514213}]},"public_addr":"172.21.15.175:6825/2565514213","cluster_addr":"172.21.15.175:6827/2565514213","heartbeat_back_addr":"172.21.15.175:6831/2565514213","heartbeat_front_addr":"172.21.15.175:6829/2565514213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:11.399380+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:33.728518+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:56.411181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:19.871731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:42.994576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:06.712815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:31.321967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:55.070898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:18.686108+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:43.187795+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:07.318538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:31.374843+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/3477601235":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/4130679770":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6800/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/1029332005":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6801/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6801/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/284078926":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6800/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/2559121542":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/925406021":"2024-07-28T03:04:30.600273+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-27T03:14:19.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.595+0000 7fd9583a9640 1 -- 172.21.15.67:0/773051581 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd93c0b3c60 msgr2=0x7fd93c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:19.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.595+0000 7fd9583a9640 1 --2- 172.21.15.67:0/773051581 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd93c0b3c60 0x7fd93c0b6170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd950100670 tx=0x7fd94000a2d0 comp rx=0 tx=0).stop 2024-07-27T03:14:19.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.595+0000 7fd9583a9640 1 -- 172.21.15.67:0/773051581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9501024f0 msgr2=0x7fd95019ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:19.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.595+0000 7fd9583a9640 1 --2- 172.21.15.67:0/773051581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9501024f0 0x7fd95019ad70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd94c006010 tx=0x7fd94c0042f0 comp rx=0 tx=0).stop 2024-07-27T03:14:19.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.595+0000 7fd9583a9640 1 -- 172.21.15.67:0/773051581 shutdown_connections 2024-07-27T03:14:19.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.595+0000 7fd9583a9640 1 --2- 172.21.15.67:0/773051581 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd93c0b3c60 0x7fd93c0b6170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:19.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.595+0000 7fd9583a9640 1 --2- 172.21.15.67:0/773051581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd950108780 0x7fd95019b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:19.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.595+0000 7fd9583a9640 1 --2- 172.21.15.67:0/773051581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd950107e30 0x7fd95019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:19.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.595+0000 7fd9583a9640 1 --2- 172.21.15.67:0/773051581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd9501024f0 0x7fd95019ad70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:19.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.595+0000 7fd9583a9640 1 -- 172.21.15.67:0/773051581 >> 172.21.15.67:0/773051581 conn(0x7fd9500791b0 msgr2=0x7fd950100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:19.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.596+0000 7fd9583a9640 1 -- 172.21.15.67:0/773051581 shutdown_connections 2024-07-27T03:14:19.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:19.596+0000 7fd9583a9640 1 -- 172.21.15.67:0/773051581 wait complete. 2024-07-27T03:14:20.389 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-07-27T03:14:20.389 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd dump --format=json 2024-07-27T03:14:20.657 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:20 smithi067 ceph-mon[32814]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 8 op/s 2024-07-27T03:14:20.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:20 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/773051581' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:14:20.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:20 smithi175 ceph-mon[37125]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 8 op/s 2024-07-27T03:14:20.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:20 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/773051581' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:14:21.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:20 smithi149 ceph-mon[37173]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 426 B/s wr, 8 op/s 2024-07-27T03:14:21.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:20 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/773051581' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:14:22.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.366+0000 7f594781e640 1 -- 172.21.15.67:0/2743747711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5940107100 msgr2=0x7f5940107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:22.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.366+0000 7f594781e640 1 --2- 172.21.15.67:0/2743747711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5940107100 0x7f5940107510 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f593c009900 tx=0x7f593c02a410 comp rx=0 tx=0).stop 2024-07-27T03:14:22.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.367+0000 7f594781e640 1 -- 172.21.15.67:0/2743747711 shutdown_connections 2024-07-27T03:14:22.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.367+0000 7f594781e640 1 --2- 172.21.15.67:0/2743747711 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59401033c0 0x7f594010ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:22.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.367+0000 7f594781e640 1 --2- 172.21.15.67:0/2743747711 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5940102a70 0x7f5940102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:22.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.367+0000 7f594781e640 1 --2- 172.21.15.67:0/2743747711 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5940107100 0x7f5940107510 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:22.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.367+0000 7f594781e640 1 -- 172.21.15.67:0/2743747711 >> 172.21.15.67:0/2743747711 conn(0x7f5940079910 msgr2=0x7f5940079d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:22.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.367+0000 7f594781e640 1 -- 172.21.15.67:0/2743747711 shutdown_connections 2024-07-27T03:14:22.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.367+0000 7f594781e640 1 -- 172.21.15.67:0/2743747711 wait complete. 2024-07-27T03:14:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.367+0000 7f594781e640 1 Processor -- start 2024-07-27T03:14:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.367+0000 7f594781e640 1 -- start start 2024-07-27T03:14:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.368+0000 7f594781e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5940102a70 0x7f594010fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.368+0000 7f594781e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59401033c0 0x7f5940110040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.368+0000 7f594781e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5940107100 0x7f5940110580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.368+0000 7f594781e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f59401135d0 con 0x7f5940102a70 2024-07-27T03:14:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.368+0000 7f594781e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5940113450 con 0x7f59401033c0 2024-07-27T03:14:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.368+0000 7f594781e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5940113750 con 0x7f5940107100 2024-07-27T03:14:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.368+0000 7f59455ba640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5940102a70 0x7f594010fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.368+0000 7f59455ba640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5940102a70 0x7f594010fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52318/0 (socket says 172.21.15.67:52318) 2024-07-27T03:14:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.368+0000 7f59455ba640 1 -- 172.21.15.67:0/1615393065 learned_addr learned my addr 172.21.15.67:0/1615393065 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:14:22.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.368+0000 7f5945dbb640 1 --2- 172.21.15.67:0/1615393065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5940107100 0x7f5940110580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.368+0000 7f5944db9640 1 --2- 172.21.15.67:0/1615393065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59401033c0 0x7f5940110040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.368+0000 7f59455ba640 1 -- 172.21.15.67:0/1615393065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5940107100 msgr2=0x7f5940110580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.368+0000 7f59455ba640 1 --2- 172.21.15.67:0/1615393065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5940107100 0x7f5940110580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.368+0000 7f59455ba640 1 -- 172.21.15.67:0/1615393065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59401033c0 msgr2=0x7f5940110040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.368+0000 7f59455ba640 1 --2- 172.21.15.67:0/1615393065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59401033c0 0x7f5940110040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.368+0000 7f59455ba640 1 -- 172.21.15.67:0/1615393065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59401a8260 con 0x7f5940102a70 2024-07-27T03:14:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.368+0000 7f5944db9640 1 --2- 172.21.15.67:0/1615393065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59401033c0 0x7f5940110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:14:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.369+0000 7f59455ba640 1 --2- 172.21.15.67:0/1615393065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5940102a70 0x7f594010fb00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f593c005940 tx=0x7f593c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.369+0000 7f59367fc640 1 -- 172.21.15.67:0/1615393065 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c03f030 con 0x7f5940102a70 2024-07-27T03:14:22.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.369+0000 7f594781e640 1 -- 172.21.15.67:0/1615393065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59401a84f0 con 0x7f5940102a70 2024-07-27T03:14:22.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.369+0000 7f59367fc640 1 -- 172.21.15.67:0/1615393065 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f593c003cb0 con 0x7f5940102a70 2024-07-27T03:14:22.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.369+0000 7f59367fc640 1 -- 172.21.15.67:0/1615393065 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c035070 con 0x7f5940102a70 2024-07-27T03:14:22.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.369+0000 7f594781e640 1 -- 172.21.15.67:0/1615393065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59401a8a20 con 0x7f5940102a70 2024-07-27T03:14:22.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.370+0000 7f594781e640 1 -- 172.21.15.67:0/1615393065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5940102e80 con 0x7f5940102a70 2024-07-27T03:14:22.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.372+0000 7f59367fc640 1 -- 172.21.15.67:0/1615393065 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f593c042030 con 0x7f5940102a70 2024-07-27T03:14:22.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.372+0000 7f59367fc640 1 --2- 172.21.15.67:0/1615393065 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f592c0b3ce0 0x7f592c0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:22.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.373+0000 7f5944db9640 1 --2- 172.21.15.67:0/1615393065 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f592c0b3ce0 0x7f592c0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:22.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.373+0000 7f59367fc640 1 -- 172.21.15.67:0/1615393065 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f593c104ce0 con 0x7f5940102a70 2024-07-27T03:14:22.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.373+0000 7f5944db9640 1 --2- 172.21.15.67:0/1615393065 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f592c0b3ce0 0x7f592c0b61f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5940104a60 tx=0x7f5930005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:22.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.376+0000 7f59367fc640 1 -- 172.21.15.67:0/1615393065 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f593c0cbe20 con 0x7f5940102a70 2024-07-27T03:14:22.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.560+0000 7f594781e640 1 -- 172.21.15.67:0/1615393065 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5940110e00 con 0x7f5940102a70 2024-07-27T03:14:22.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.561+0000 7f59367fc640 1 -- 172.21.15.67:0/1615393065 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24829 (secure 0 0 0) 0x7f593c0cb720 con 0x7f5940102a70 2024-07-27T03:14:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:14:22.565 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":91,"fsid":"c0782a00-4bc4-11ef-bcc7-c7b262605968","created":"2024-07-27T03:04:08.096824+0000","modified":"2024-07-27T03:13:57.294556+0000","last_up_change":"2024-07-27T03:11:33.314800+0000","last_in_change":"2024-07-27T03:11:14.028095+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-27T03:08:01.042479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-27T03:12:27.565948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-27T03:12:31.353726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-27T03:12:40.599486+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9f614272-18e8-4562-812c-4f43e40efd2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6803","nonce":2648129674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6805","nonce":2648129674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6809","nonce":2648129674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6807","nonce":2648129674}]},"public_addr":"172.21.15.67:6803/2648129674","cluster_addr":"172.21.15.67:6805/2648129674","heartbeat_back_addr":"172.21.15.67:6809/2648129674","heartbeat_front_addr":"172.21.15.67:6807/2648129674","state":["exists","up"]},{"osd":1,"uuid":"f59f5879-163a-4269-bbe7-1200eea65341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6801","nonce":2561753152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6803","nonce":2561753152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6807","nonce":2561753152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6805","nonce":2561753152}]},"public_addr":"172.21.15.149:6801/2561753152","cluster_addr":"172.21.15.149:6803/2561753152","heartbeat_back_addr":"172.21.15.149:6807/2561753152","heartbeat_front_addr":"172.21.15.149:6805/2561753152","state":["exists","up"]},{"osd":2,"uuid":"444e0ebe-7ca3-4be9-8bd5-db859f824445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6801","nonce":3770940078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6803","nonce":3770940078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6807","nonce":3770940078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6805","nonce":3770940078}]},"public_addr":"172.21.15.175:6801/3770940078","cluster_addr":"172.21.15.175:6803/3770940078","heartbeat_back_addr":"172.21.15.175:6807/3770940078","heartbeat_front_addr":"172.21.15.175:6805/3770940078","state":["exists","up"]},{"osd":3,"uuid":"d554db3d-9604-427c-8736-b1bb379f8d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6811","nonce":3159144693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6813","nonce":3159144693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6817","nonce":3159144693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6815","nonce":3159144693}]},"public_addr":"172.21.15.67:6811/3159144693","cluster_addr":"172.21.15.67:6813/3159144693","heartbeat_back_addr":"172.21.15.67:6817/3159144693","heartbeat_front_addr":"172.21.15.67:6815/3159144693","state":["exists","up"]},{"osd":4,"uuid":"c317dbe5-3b49-47a6-8408-7042a3dac345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3393503532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3393503532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3393503532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3393503532}]},"public_addr":"172.21.15.149:6809/3393503532","cluster_addr":"172.21.15.149:6811/3393503532","heartbeat_back_addr":"172.21.15.149:6815/3393503532","heartbeat_front_addr":"172.21.15.149:6813/3393503532","state":["exists","up"]},{"osd":5,"uuid":"5c85abd6-2b42-4957-a9b7-59a9ee36b841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6809","nonce":2291204784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6811","nonce":2291204784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6815","nonce":2291204784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6813","nonce":2291204784}]},"public_addr":"172.21.15.175:6809/2291204784","cluster_addr":"172.21.15.175:6811/2291204784","heartbeat_back_addr":"172.21.15.175:6815/2291204784","heartbeat_front_addr":"172.21.15.175:6813/2291204784","state":["exists","up"]},{"osd":6,"uuid":"bf6c277d-ec24-4470-98de-cf6f28876ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6819","nonce":2660182879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6821","nonce":2660182879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6825","nonce":2660182879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6823","nonce":2660182879}]},"public_addr":"172.21.15.67:6819/2660182879","cluster_addr":"172.21.15.67:6821/2660182879","heartbeat_back_addr":"172.21.15.67:6825/2660182879","heartbeat_front_addr":"172.21.15.67:6823/2660182879","state":["exists","up"]},{"osd":7,"uuid":"fea2c7cf-d5cb-45ae-9b29-f6e56c903891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6817","nonce":2289624404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6819","nonce":2289624404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6823","nonce":2289624404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6821","nonce":2289624404}]},"public_addr":"172.21.15.149:6817/2289624404","cluster_addr":"172.21.15.149:6819/2289624404","heartbeat_back_addr":"172.21.15.149:6823/2289624404","heartbeat_front_addr":"172.21.15.149:6821/2289624404","state":["exists","up"]},{"osd":8,"uuid":"14d04d96-579a-4dc9-a4d4-50e4850b48e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6817","nonce":2286032466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6819","nonce":2286032466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6823","nonce":2286032466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6821","nonce":2286032466}]},"public_addr":"172.21.15.175:6817/2286032466","cluster_addr":"172.21.15.175:6819/2286032466","heartbeat_back_addr":"172.21.15.175:6823/2286032466","heartbeat_front_addr":"172.21.15.175:6821/2286032466","state":["exists","up"]},{"osd":9,"uuid":"b65dfb6b-1efe-4314-a8be-ce6a295e99fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6827","nonce":3411081513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6829","nonce":3411081513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6833","nonce":3411081513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6831","nonce":3411081513}]},"public_addr":"172.21.15.67:6827/3411081513","cluster_addr":"172.21.15.67:6829/3411081513","heartbeat_back_addr":"172.21.15.67:6833/3411081513","heartbeat_front_addr":"172.21.15.67:6831/3411081513","state":["exists","up"]},{"osd":10,"uuid":"e3dabae0-0d64-4222-984c-fc1cce883456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6825","nonce":2011495212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6827","nonce":2011495212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6831","nonce":2011495212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6829","nonce":2011495212}]},"public_addr":"172.21.15.149:6825/2011495212","cluster_addr":"172.21.15.149:6827/2011495212","heartbeat_back_addr":"172.21.15.149:6831/2011495212","heartbeat_front_addr":"172.21.15.149:6829/2011495212","state":["exists","up"]},{"osd":11,"uuid":"406c082c-9b5f-4781-8cbf-3c04fa49e410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6825","nonce":2565514213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6827","nonce":2565514213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6831","nonce":2565514213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6829","nonce":2565514213}]},"public_addr":"172.21.15.175:6825/2565514213","cluster_addr":"172.21.15.175:6827/2565514213","heartbeat_back_addr":"172.21.15.175:6831/2565514213","heartbeat_front_addr":"172.21.15.175:6829/2565514213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:11.399380+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:33.728518+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:56.411181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:19.871731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:42.994576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:06.712815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:31.321967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:55.070898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:18.686108+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:43.187795+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:07.318538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:31.374843+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/3477601235":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/4130679770":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6800/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/1029332005":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6801/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6801/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/284078926":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6800/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/2559121542":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/925406021":"2024-07-28T03:04:30.600273+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-27T03:14:22.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.563+0000 7f594781e640 1 -- 172.21.15.67:0/1615393065 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f592c0b3ce0 msgr2=0x7f592c0b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:22.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.563+0000 7f594781e640 1 --2- 172.21.15.67:0/1615393065 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f592c0b3ce0 0x7f592c0b61f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5940104a60 tx=0x7f5930005e70 comp rx=0 tx=0).stop 2024-07-27T03:14:22.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.563+0000 7f594781e640 1 -- 172.21.15.67:0/1615393065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5940102a70 msgr2=0x7f594010fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:22.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.563+0000 7f594781e640 1 --2- 172.21.15.67:0/1615393065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5940102a70 0x7f594010fb00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f593c005940 tx=0x7f593c0042f0 comp rx=0 tx=0).stop 2024-07-27T03:14:22.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.563+0000 7f594781e640 1 -- 172.21.15.67:0/1615393065 shutdown_connections 2024-07-27T03:14:22.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.563+0000 7f594781e640 1 --2- 172.21.15.67:0/1615393065 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f592c0b3ce0 0x7f592c0b61f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:22.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.564+0000 7f594781e640 1 --2- 172.21.15.67:0/1615393065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5940107100 0x7f5940110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:22.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.564+0000 7f594781e640 1 --2- 172.21.15.67:0/1615393065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59401033c0 0x7f5940110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:22.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.564+0000 7f594781e640 1 --2- 172.21.15.67:0/1615393065 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5940102a70 0x7f594010fb00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:22.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.564+0000 7f594781e640 1 -- 172.21.15.67:0/1615393065 >> 172.21.15.67:0/1615393065 conn(0x7f5940079910 msgr2=0x7f5940105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:22.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.564+0000 7f594781e640 1 -- 172.21.15.67:0/1615393065 shutdown_connections 2024-07-27T03:14:22.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:22.564+0000 7f594781e640 1 -- 172.21.15.67:0/1615393065 wait complete. 2024-07-27T03:14:22.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:22 smithi175 ceph-mon[37125]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-07-27T03:14:22.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:22 smithi067 ceph-mon[32814]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-07-27T03:14:23.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:22 smithi149 ceph-mon[37173]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-07-27T03:14:23.353 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-07-27T03:14:23.353 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-07-27T03:14:23.354 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-07-27T03:14:23.354 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-07-27T03:14:23.355 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-07-27T03:14:23.355 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-07-27T03:14:23.356 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-07-27T03:14:23.356 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-07-27T03:14:23.357 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-07-27T03:14:23.357 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-07-27T03:14:23.357 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-07-27T03:14:23.358 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-07-27T03:14:23.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:23 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1615393065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:14:23.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:23 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1615393065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:14:24.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:23 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1615393065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:14:24.080 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:24.220 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:24.222 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:24.222 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:24.222 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:24.235 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:24.235 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:24.239 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:24.646 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:24 smithi067 ceph-mon[32814]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:14:24.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:24 smithi175 ceph-mon[37125]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:14:24.983 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:24.983 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:24.983 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:25.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:24 smithi149 ceph-mon[37173]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:14:25.008 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:26.617 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:26 smithi067 ceph-mon[32814]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:14:26.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:26 smithi175 ceph-mon[37125]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:14:27.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:26 smithi149 ceph-mon[37173]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:14:28.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:28 smithi067 ceph-mon[32814]: pgmap v367: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:14:28.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:28 smithi175 ceph-mon[37125]: pgmap v367: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:14:29.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:28 smithi149 ceph-mon[37173]: pgmap v367: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:14:30.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:30 smithi175 ceph-mon[37125]: pgmap v368: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:14:30.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:30 smithi067 ceph-mon[32814]: pgmap v368: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:14:31.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:30 smithi149 ceph-mon[37173]: pgmap v368: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:14:32.799 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:32 smithi067 ceph-mon[32814]: pgmap v369: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:14:32.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:32 smithi175 ceph-mon[37125]: pgmap v369: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:14:33.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:32 smithi149 ceph-mon[37173]: pgmap v369: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:14:33.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.795+0000 7fed289ee640 1 -- 172.21.15.67:0/36690159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed2410d860 msgr2=0x7fed2410fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:33.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.795+0000 7fed289ee640 1 --2- 172.21.15.67:0/36690159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed2410d860 0x7fed2410fd70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009ca0 tx=0x7fed1c02a8c0 comp rx=0 tx=0).stop 2024-07-27T03:14:33.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.796+0000 7fed289ee640 1 -- 172.21.15.67:0/36690159 shutdown_connections 2024-07-27T03:14:33.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.796+0000 7fed289ee640 1 --2- 172.21.15.67:0/36690159 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed2410d860 0x7fed2410fd70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:33.732 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.796+0000 7fed289ee640 1 --2- 172.21.15.67:0/36690159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed24104ce0 0x7fed2410d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.796+0000 7fed289ee640 1 --2- 172.21.15.67:0/36690159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed241042b0 0x7fed241046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.796+0000 7fed289ee640 1 -- 172.21.15.67:0/36690159 >> 172.21.15.67:0/36690159 conn(0x7fed240791a0 msgr2=0x7fed240795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.796+0000 7fed289ee640 1 -- 172.21.15.67:0/36690159 shutdown_connections 2024-07-27T03:14:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.796+0000 7fed289ee640 1 -- 172.21.15.67:0/36690159 wait complete. 2024-07-27T03:14:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.797+0000 7fed289ee640 1 Processor -- start 2024-07-27T03:14:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.797+0000 7fed289ee640 1 -- start start 2024-07-27T03:14:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.798+0000 7fed289ee640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed241042b0 0x7fed24199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.798+0000 7fed289ee640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed24104ce0 0x7fed2419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.798+0000 7fed289ee640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed2410d860 0x7fed2419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.798+0000 7fed289ee640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fed241134e0 con 0x7fed2410d860 2024-07-27T03:14:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.798+0000 7fed289ee640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fed24113360 con 0x7fed24104ce0 2024-07-27T03:14:33.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.798+0000 7fed289ee640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fed24113660 con 0x7fed241042b0 2024-07-27T03:14:33.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.798+0000 7fed22d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed2410d860 0x7fed2419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:33.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.798+0000 7fed22d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed2410d860 0x7fed2419f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58010/0 (socket says 172.21.15.67:58010) 2024-07-27T03:14:33.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.798+0000 7fed22d9d640 1 -- 172.21.15.67:0/2281691485 learned_addr learned my addr 172.21.15.67:0/2281691485 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:14:33.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.798+0000 7fed2259c640 1 --2- 172.21.15.67:0/2281691485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed241042b0 0x7fed24199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:33.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.798+0000 7fed21d9b640 1 --2- 172.21.15.67:0/2281691485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed24104ce0 0x7fed2419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:33.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.798+0000 7fed22d9d640 1 -- 172.21.15.67:0/2281691485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed241042b0 msgr2=0x7fed24199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:33.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.798+0000 7fed22d9d640 1 --2- 172.21.15.67:0/2281691485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed241042b0 0x7fed24199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:33.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.798+0000 7fed22d9d640 1 -- 172.21.15.67:0/2281691485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed24104ce0 msgr2=0x7fed2419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:33.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.798+0000 7fed22d9d640 1 --2- 172.21.15.67:0/2281691485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed24104ce0 0x7fed2419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:33.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.798+0000 7fed22d9d640 1 -- 172.21.15.67:0/2281691485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2419fec0 con 0x7fed2410d860 2024-07-27T03:14:33.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.799+0000 7fed22d9d640 1 --2- 172.21.15.67:0/2281691485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed2410d860 0x7fed2419f790 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fed1c005cc0 tx=0x7fed1c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:33.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.799+0000 7fed137fe640 1 -- 172.21.15.67:0/2281691485 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c003bf0 con 0x7fed2410d860 2024-07-27T03:14:33.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.799+0000 7fed137fe640 1 -- 172.21.15.67:0/2281691485 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed1c003d90 con 0x7fed2410d860 2024-07-27T03:14:33.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.799+0000 7fed289ee640 1 -- 172.21.15.67:0/2281691485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed241a0150 con 0x7fed2410d860 2024-07-27T03:14:33.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.799+0000 7fed137fe640 1 -- 172.21.15.67:0/2281691485 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c02fbc0 con 0x7fed2410d860 2024-07-27T03:14:33.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.799+0000 7fed289ee640 1 -- 172.21.15.67:0/2281691485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed241a6910 con 0x7fed2410d860 2024-07-27T03:14:33.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.800+0000 7fed289ee640 1 -- 172.21.15.67:0/2281691485 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fed24106de0 con 0x7fed2410d860 2024-07-27T03:14:33.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.801+0000 7fed137fe640 1 -- 172.21.15.67:0/2281691485 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7fed1c02fd60 con 0x7fed2410d860 2024-07-27T03:14:33.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.802+0000 7fed137fe640 1 --2- 172.21.15.67:0/2281691485 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fed0c0b3c60 0x7fed0c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:33.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.802+0000 7fed2259c640 1 --2- 172.21.15.67:0/2281691485 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fed0c0b3c60 0x7fed0c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:33.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.802+0000 7fed137fe640 1 -- 172.21.15.67:0/2281691485 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fed1c105840 con 0x7fed2410d860 2024-07-27T03:14:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.803+0000 7fed137fe640 1 --2- 172.21.15.67:0/2281691485 >> [v2:172.21.15.149:6824/2011495212,v1:172.21.15.149:6825/2011495212] conn(0x7fed0c0bf000 0x7fed0c0c14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.803+0000 7fed137fe640 1 -- 172.21.15.67:0/2281691485 --> [v2:172.21.15.149:6824/2011495212,v1:172.21.15.149:6825/2011495212] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed1c0020b0 con 0x7fed0c0bf000 2024-07-27T03:14:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.803+0000 7fed137fe640 1 -- 172.21.15.67:0/2281691485 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fed1c105c90 con 0x7fed2410d860 2024-07-27T03:14:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.803+0000 7fed2259c640 1 --2- 172.21.15.67:0/2281691485 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fed0c0b3c60 0x7fed0c0b6170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fed14005fd0 tx=0x7fed1400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.803+0000 7fed21d9b640 1 --2- 172.21.15.67:0/2281691485 >> [v2:172.21.15.149:6824/2011495212,v1:172.21.15.149:6825/2011495212] conn(0x7fed0c0bf000 0x7fed0c0c14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.803+0000 7fed21d9b640 1 --2- 172.21.15.67:0/2281691485 >> [v2:172.21.15.149:6824/2011495212,v1:172.21.15.149:6825/2011495212] conn(0x7fed0c0bf000 0x7fed0c0c14f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.804+0000 7fed137fe640 1 -- 172.21.15.67:0/2281691485 <== osd.10 v2:172.21.15.149:6824/2011495212 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed1c0020b0 con 0x7fed0c0bf000 2024-07-27T03:14:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.818+0000 7fed289ee640 1 -- 172.21.15.67:0/2281691485 --> [v2:172.21.15.149:6824/2011495212,v1:172.21.15.149:6825/2011495212] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fed2406af70 con 0x7fed0c0bf000 2024-07-27T03:14:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.819+0000 7fed137fe640 1 -- 172.21.15.67:0/2281691485 <== osd.10 v2:172.21.15.149:6824/2011495212 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fed2406af70 con 0x7fed0c0bf000 2024-07-27T03:14:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.819+0000 7fed289ee640 1 -- 172.21.15.67:0/2281691485 >> [v2:172.21.15.149:6824/2011495212,v1:172.21.15.149:6825/2011495212] conn(0x7fed0c0bf000 msgr2=0x7fed0c0c14f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.819+0000 7fed289ee640 1 --2- 172.21.15.67:0/2281691485 >> [v2:172.21.15.149:6824/2011495212,v1:172.21.15.149:6825/2011495212] conn(0x7fed0c0bf000 0x7fed0c0c14f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.819+0000 7fed289ee640 1 -- 172.21.15.67:0/2281691485 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fed0c0b3c60 msgr2=0x7fed0c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.819+0000 7fed289ee640 1 --2- 172.21.15.67:0/2281691485 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fed0c0b3c60 0x7fed0c0b6170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fed14005fd0 tx=0x7fed1400b410 comp rx=0 tx=0).stop 2024-07-27T03:14:33.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.819+0000 7fed289ee640 1 -- 172.21.15.67:0/2281691485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed2410d860 msgr2=0x7fed2419f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:33.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.819+0000 7fed289ee640 1 --2- 172.21.15.67:0/2281691485 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed2410d860 0x7fed2419f790 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fed1c005cc0 tx=0x7fed1c02f380 comp rx=0 tx=0).stop 2024-07-27T03:14:33.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.819+0000 7fed22d9d640 1 -- 172.21.15.67:0/2281691485 reap_dead start 2024-07-27T03:14:33.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.819+0000 7fed289ee640 1 -- 172.21.15.67:0/2281691485 shutdown_connections 2024-07-27T03:14:33.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.819+0000 7fed289ee640 1 -- 172.21.15.67:0/2281691485 >> 172.21.15.67:0/2281691485 conn(0x7fed240791a0 msgr2=0x7fed241063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:33.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.819+0000 7fed289ee640 1 -- 172.21.15.67:0/2281691485 shutdown_connections 2024-07-27T03:14:33.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:32.819+0000 7fed289ee640 1 -- 172.21.15.67:0/2281691485 wait complete. 2024-07-27T03:14:33.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:33 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:14:33.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:33 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:14:34.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:33 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:14:34.647 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:34 smithi067 ceph-mon[32814]: pgmap v370: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-07-27T03:14:34.647 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:14:34.648 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:14:34.648 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:14:34.648 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:34 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:14:34.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:34 smithi175 ceph-mon[37125]: pgmap v370: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-07-27T03:14:34.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:14:34.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:14:34.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:14:34.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:34 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:14:34.901 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:34 smithi149 ceph-mon[37173]: pgmap v370: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-07-27T03:14:34.901 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:14:34.901 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:14:34.901 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:14:34.901 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:34 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:14:36.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.614+0000 7f0b03707640 1 -- 172.21.15.67:0/3614511112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0afc1023b0 msgr2=0x7f0afc112030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:36.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.614+0000 7f0b03707640 1 --2- 172.21.15.67:0/3614511112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0afc1023b0 0x7f0afc112030 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009ca0 tx=0x7f0af402a950 comp rx=0 tx=0).stop 2024-07-27T03:14:36.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.614+0000 7f0b03707640 1 -- 172.21.15.67:0/3614511112 shutdown_connections 2024-07-27T03:14:36.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.614+0000 7f0b03707640 1 --2- 172.21.15.67:0/3614511112 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0afc1023b0 0x7f0afc112030 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:36.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.614+0000 7f0b03707640 1 --2- 172.21.15.67:0/3614511112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0afc101860 0x7f0afc101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:36.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.614+0000 7f0b03707640 1 --2- 172.21.15.67:0/3614511112 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0afc104180 0x7f0afc104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:36.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.614+0000 7f0b03707640 1 -- 172.21.15.67:0/3614511112 >> 172.21.15.67:0/3614511112 conn(0x7f0afc079930 msgr2=0x7f0afc079d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:36.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.614+0000 7f0b03707640 1 -- 172.21.15.67:0/3614511112 shutdown_connections 2024-07-27T03:14:36.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.614+0000 7f0b03707640 1 -- 172.21.15.67:0/3614511112 wait complete. 2024-07-27T03:14:36.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.615+0000 7f0b03707640 1 Processor -- start 2024-07-27T03:14:36.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.615+0000 7f0b03707640 1 -- start start 2024-07-27T03:14:36.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.615+0000 7f0b03707640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0afc101860 0x7f0afc19c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:36.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.615+0000 7f0b03707640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0afc1023b0 0x7f0afc19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.615+0000 7f0b03707640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0afc104180 0x7f0afc1a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.615+0000 7f0b03707640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc115790 con 0x7f0afc101860 2024-07-27T03:14:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.615+0000 7f0b03707640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc115610 con 0x7f0afc1023b0 2024-07-27T03:14:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.615+0000 7f0b03707640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc115910 con 0x7f0afc104180 2024-07-27T03:14:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.615+0000 7f0b014a3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0afc101860 0x7f0afc19c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.615+0000 7f0b014a3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0afc101860 0x7f0afc19c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46540/0 (socket says 172.21.15.67:46540) 2024-07-27T03:14:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.615+0000 7f0b014a3640 1 -- 172.21.15.67:0/3586927557 learned_addr learned my addr 172.21.15.67:0/3586927557 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:14:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.615+0000 7f0b00ca2640 1 --2- 172.21.15.67:0/3586927557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0afc1023b0 0x7f0afc19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.616+0000 7f0b014a3640 1 -- 172.21.15.67:0/3586927557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0afc104180 msgr2=0x7f0afc1a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:36.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.616+0000 7f0b014a3640 1 --2- 172.21.15.67:0/3586927557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0afc104180 0x7f0afc1a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.616+0000 7f0b014a3640 1 -- 172.21.15.67:0/3586927557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0afc1023b0 msgr2=0x7f0afc19c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.616+0000 7f0b014a3640 1 --2- 172.21.15.67:0/3586927557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0afc1023b0 0x7f0afc19c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.616+0000 7f0b014a3640 1 -- 172.21.15.67:0/3586927557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0afc1a2150 con 0x7f0afc101860 2024-07-27T03:14:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.616+0000 7f0b00ca2640 1 --2- 172.21.15.67:0/3586927557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0afc1023b0 0x7f0afc19c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:14:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.616+0000 7f0b014a3640 1 --2- 172.21.15.67:0/3586927557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0afc101860 0x7f0afc19c1e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0af800c890 tx=0x7f0af800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.616+0000 7f0af27fc640 1 -- 172.21.15.67:0/3586927557 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af8004220 con 0x7f0afc101860 2024-07-27T03:14:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.616+0000 7f0af27fc640 1 -- 172.21.15.67:0/3586927557 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0af80043c0 con 0x7f0afc101860 2024-07-27T03:14:36.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.616+0000 7f0b03707640 1 -- 172.21.15.67:0/3586927557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0afc1a2440 con 0x7f0afc101860 2024-07-27T03:14:36.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.616+0000 7f0af27fc640 1 -- 172.21.15.67:0/3586927557 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af8004a00 con 0x7f0afc101860 2024-07-27T03:14:36.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.616+0000 7f0b03707640 1 -- 172.21.15.67:0/3586927557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0afc1a8ba0 con 0x7f0afc101860 2024-07-27T03:14:36.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.617+0000 7f0b03707640 1 -- 172.21.15.67:0/3586927557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0afc10cac0 con 0x7f0afc101860 2024-07-27T03:14:36.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.619+0000 7f0af27fc640 1 -- 172.21.15.67:0/3586927557 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f0af80079b0 con 0x7f0afc101860 2024-07-27T03:14:36.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.619+0000 7f0af27fc640 1 --2- 172.21.15.67:0/3586927557 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0ae80b3ce0 0x7f0ae80b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:36.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.620+0000 7f0b00ca2640 1 --2- 172.21.15.67:0/3586927557 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0ae80b3ce0 0x7f0ae80b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:36.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.620+0000 7f0af27fc640 1 -- 172.21.15.67:0/3586927557 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f0af80e4230 con 0x7f0afc101860 2024-07-27T03:14:36.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.620+0000 7f0af27fc640 1 --2- 172.21.15.67:0/3586927557 >> [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513] conn(0x7f0ae80bf080 0x7f0ae80c1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:36.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.620+0000 7f0af27fc640 1 -- 172.21.15.67:0/3586927557 --> [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0af8012ec0 con 0x7f0ae80bf080 2024-07-27T03:14:36.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.620+0000 7f0af27fc640 1 -- 172.21.15.67:0/3586927557 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f0af80e4680 con 0x7f0afc101860 2024-07-27T03:14:36.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.620+0000 7f0b01ca4640 1 --2- 172.21.15.67:0/3586927557 >> [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513] conn(0x7f0ae80bf080 0x7f0ae80c1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.620+0000 7f0b00ca2640 1 --2- 172.21.15.67:0/3586927557 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0ae80b3ce0 0x7f0ae80b61f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0afc0feaa0 tx=0x7f0aec005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.620+0000 7f0b01ca4640 1 --2- 172.21.15.67:0/3586927557 >> [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513] conn(0x7f0ae80bf080 0x7f0ae80c1570 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.621+0000 7f0af27fc640 1 -- 172.21.15.67:0/3586927557 <== osd.9 v2:172.21.15.67:6826/3411081513 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0af8012ec0 con 0x7f0ae80bf080 2024-07-27T03:14:36.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.636+0000 7f0b03707640 1 -- 172.21.15.67:0/3586927557 --> [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0afc102870 con 0x7f0ae80bf080 2024-07-27T03:14:36.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.636+0000 7f0af27fc640 1 -- 172.21.15.67:0/3586927557 <== osd.9 v2:172.21.15.67:6826/3411081513 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0afc102870 con 0x7f0ae80bf080 2024-07-27T03:14:36.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.636+0000 7f0b03707640 1 -- 172.21.15.67:0/3586927557 >> [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513] conn(0x7f0ae80bf080 msgr2=0x7f0ae80c1570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:36.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.636+0000 7f0b03707640 1 --2- 172.21.15.67:0/3586927557 >> [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513] conn(0x7f0ae80bf080 0x7f0ae80c1570 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:36.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.637+0000 7f0b03707640 1 -- 172.21.15.67:0/3586927557 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0ae80b3ce0 msgr2=0x7f0ae80b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:36.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.637+0000 7f0b03707640 1 --2- 172.21.15.67:0/3586927557 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0ae80b3ce0 0x7f0ae80b61f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0afc0feaa0 tx=0x7f0aec005e50 comp rx=0 tx=0).stop 2024-07-27T03:14:36.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.637+0000 7f0b03707640 1 -- 172.21.15.67:0/3586927557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0afc101860 msgr2=0x7f0afc19c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:36.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.637+0000 7f0b03707640 1 --2- 172.21.15.67:0/3586927557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0afc101860 0x7f0afc19c1e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0af800c890 tx=0x7f0af800cd50 comp rx=0 tx=0).stop 2024-07-27T03:14:36.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.637+0000 7f0b01ca4640 1 -- 172.21.15.67:0/3586927557 reap_dead start 2024-07-27T03:14:36.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.637+0000 7f0b03707640 1 -- 172.21.15.67:0/3586927557 shutdown_connections 2024-07-27T03:14:36.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.637+0000 7f0b03707640 1 -- 172.21.15.67:0/3586927557 >> 172.21.15.67:0/3586927557 conn(0x7f0afc079930 msgr2=0x7f0afc10c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:36.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.637+0000 7f0b03707640 1 -- 172.21.15.67:0/3586927557 shutdown_connections 2024-07-27T03:14:36.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:36.637+0000 7f0b03707640 1 -- 172.21.15.67:0/3586927557 wait complete. 2024-07-27T03:14:36.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:36 smithi175 ceph-mon[37125]: pgmap v371: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:14:36.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:36 smithi067 ceph-mon[32814]: pgmap v371: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:14:37.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:36 smithi149 ceph-mon[37173]: pgmap v371: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:14:37.074 INFO:teuthology.orchestra.run.smithi067.stdout:283467841580 2024-07-27T03:14:37.074 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-07-27T03:14:37.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.416+0000 7fc1dc281640 1 -- 172.21.15.67:0/422024506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1d4108760 msgr2=0x7fc1d410ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:37.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.416+0000 7fc1dc281640 1 --2- 172.21.15.67:0/422024506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1d4108760 0x7fc1d410ef90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009ca0 tx=0x7fc1cc02a8c0 comp rx=0 tx=0).stop 2024-07-27T03:14:37.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.416+0000 7fc1dc281640 1 -- 172.21.15.67:0/422024506 shutdown_connections 2024-07-27T03:14:37.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.416+0000 7fc1dc281640 1 --2- 172.21.15.67:0/422024506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1d4108760 0x7fc1d410ef90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:37.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.416+0000 7fc1dc281640 1 --2- 172.21.15.67:0/422024506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1d4107e10 0x7fc1d4108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:37.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.416+0000 7fc1dc281640 1 --2- 172.21.15.67:0/422024506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1d41024d0 0x7fc1d41028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:37.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.416+0000 7fc1dc281640 1 -- 172.21.15.67:0/422024506 >> 172.21.15.67:0/422024506 conn(0x7fc1d40791a0 msgr2=0x7fc1d40795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:37.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.416+0000 7fc1dc281640 1 -- 172.21.15.67:0/422024506 shutdown_connections 2024-07-27T03:14:37.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.416+0000 7fc1dc281640 1 -- 172.21.15.67:0/422024506 wait complete. 2024-07-27T03:14:37.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.416+0000 7fc1dc281640 1 Processor -- start 2024-07-27T03:14:37.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.417+0000 7fc1dc281640 1 -- start start 2024-07-27T03:14:37.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.417+0000 7fc1dc281640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1d41024d0 0x7fc1d4198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:37.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.417+0000 7fc1dc281640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1d4107e10 0x7fc1d4199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:37.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.417+0000 7fc1dc281640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1d4108760 0x7fc1d419e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:37.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.417+0000 7fc1dc281640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d41123e0 con 0x7fc1d41024d0 2024-07-27T03:14:37.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.417+0000 7fc1dc281640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d4112260 con 0x7fc1d4108760 2024-07-27T03:14:37.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.417+0000 7fc1dc281640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d4112560 con 0x7fc1d4107e10 2024-07-27T03:14:37.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.417+0000 7fc1da01d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1d41024d0 0x7fc1d4198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:37.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.417+0000 7fc1da01d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1d41024d0 0x7fc1d4198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46564/0 (socket says 172.21.15.67:46564) 2024-07-27T03:14:37.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.417+0000 7fc1da01d640 1 -- 172.21.15.67:0/960852770 learned_addr learned my addr 172.21.15.67:0/960852770 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:14:37.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.417+0000 7fc1d981c640 1 --2- 172.21.15.67:0/960852770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1d4107e10 0x7fc1d4199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:37.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.417+0000 7fc1da81e640 1 --2- 172.21.15.67:0/960852770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1d4108760 0x7fc1d419e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:37.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.417+0000 7fc1da01d640 1 -- 172.21.15.67:0/960852770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1d4107e10 msgr2=0x7fc1d4199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:37.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.417+0000 7fc1da01d640 1 --2- 172.21.15.67:0/960852770 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1d4107e10 0x7fc1d4199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:37.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.417+0000 7fc1da01d640 1 -- 172.21.15.67:0/960852770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1d4108760 msgr2=0x7fc1d419e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:37.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.417+0000 7fc1da01d640 1 --2- 172.21.15.67:0/960852770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1d4108760 0x7fc1d419e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:37.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.417+0000 7fc1da01d640 1 -- 172.21.15.67:0/960852770 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d419ed00 con 0x7fc1d41024d0 2024-07-27T03:14:37.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.417+0000 7fc1da81e640 1 --2- 172.21.15.67:0/960852770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1d4108760 0x7fc1d419e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:14:37.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.418+0000 7fc1da01d640 1 --2- 172.21.15.67:0/960852770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1d41024d0 0x7fc1d4198d90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc1d000d770 tx=0x7fc1d000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:37.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.418+0000 7fc1cb7fe640 1 -- 172.21.15.67:0/960852770 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0017070 con 0x7fc1d41024d0 2024-07-27T03:14:37.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.418+0000 7fc1dc281640 1 -- 172.21.15.67:0/960852770 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d419eff0 con 0x7fc1d41024d0 2024-07-27T03:14:37.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.418+0000 7fc1cb7fe640 1 -- 172.21.15.67:0/960852770 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1d00054d0 con 0x7fc1d41024d0 2024-07-27T03:14:37.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.418+0000 7fc1cb7fe640 1 -- 172.21.15.67:0/960852770 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0004c30 con 0x7fc1d41024d0 2024-07-27T03:14:37.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.418+0000 7fc1dc281640 1 -- 172.21.15.67:0/960852770 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d406a8d0 con 0x7fc1d41024d0 2024-07-27T03:14:37.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.419+0000 7fc1dc281640 1 -- 172.21.15.67:0/960852770 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc1d406ae40 con 0x7fc1d41024d0 2024-07-27T03:14:37.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.420+0000 7fc1cb7fe640 1 -- 172.21.15.67:0/960852770 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7fc1d0004dd0 con 0x7fc1d41024d0 2024-07-27T03:14:37.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.421+0000 7fc1cb7fe640 1 --2- 172.21.15.67:0/960852770 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc1c00b3c60 0x7fc1c00b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:37.428 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.421+0000 7fc1d981c640 1 --2- 172.21.15.67:0/960852770 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc1c00b3c60 0x7fc1c00b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:37.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.421+0000 7fc1cb7fe640 1 -- 172.21.15.67:0/960852770 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fc1d00e4030 con 0x7fc1d41024d0 2024-07-27T03:14:37.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.422+0000 7fc1cb7fe640 1 --2- 172.21.15.67:0/960852770 >> [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879] conn(0x7fc1c00bf000 0x7fc1c00c14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:37.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.422+0000 7fc1cb7fe640 1 -- 172.21.15.67:0/960852770 --> [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc1d0001d30 con 0x7fc1c00bf000 2024-07-27T03:14:37.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.422+0000 7fc1cb7fe640 1 -- 172.21.15.67:0/960852770 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fc1d00e4480 con 0x7fc1d41024d0 2024-07-27T03:14:37.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.422+0000 7fc1da81e640 1 --2- 172.21.15.67:0/960852770 >> [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879] conn(0x7fc1c00bf000 0x7fc1c00c14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:37.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.422+0000 7fc1d981c640 1 --2- 172.21.15.67:0/960852770 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc1c00b3c60 0x7fc1c00b6170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4105960 tx=0x7fc1c400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:37.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.422+0000 7fc1da81e640 1 --2- 172.21.15.67:0/960852770 >> [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879] conn(0x7fc1c00bf000 0x7fc1c00c14f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:37.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.423+0000 7fc1cb7fe640 1 -- 172.21.15.67:0/960852770 <== osd.6 v2:172.21.15.67:6818/2660182879 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc1d0001d30 con 0x7fc1c00bf000 2024-07-27T03:14:37.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.439+0000 7fc1dc281640 1 -- 172.21.15.67:0/960852770 --> [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc1d406b2c0 con 0x7fc1c00bf000 2024-07-27T03:14:37.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.439+0000 7fc1cb7fe640 1 -- 172.21.15.67:0/960852770 <== osd.6 v2:172.21.15.67:6818/2660182879 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc1d406b2c0 con 0x7fc1c00bf000 2024-07-27T03:14:37.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.439+0000 7fc1dc281640 1 -- 172.21.15.67:0/960852770 >> [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879] conn(0x7fc1c00bf000 msgr2=0x7fc1c00c14f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:37.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.439+0000 7fc1dc281640 1 --2- 172.21.15.67:0/960852770 >> [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879] conn(0x7fc1c00bf000 0x7fc1c00c14f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:37.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.439+0000 7fc1dc281640 1 -- 172.21.15.67:0/960852770 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc1c00b3c60 msgr2=0x7fc1c00b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:37.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.439+0000 7fc1dc281640 1 --2- 172.21.15.67:0/960852770 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fc1c00b3c60 0x7fc1c00b6170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4105960 tx=0x7fc1c400b390 comp rx=0 tx=0).stop 2024-07-27T03:14:37.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.439+0000 7fc1dc281640 1 -- 172.21.15.67:0/960852770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1d41024d0 msgr2=0x7fc1d4198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:37.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.439+0000 7fc1dc281640 1 --2- 172.21.15.67:0/960852770 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc1d41024d0 0x7fc1d4198d90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc1d000d770 tx=0x7fc1d000dc30 comp rx=0 tx=0).stop 2024-07-27T03:14:37.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.440+0000 7fc1da81e640 1 -- 172.21.15.67:0/960852770 reap_dead start 2024-07-27T03:14:37.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.440+0000 7fc1dc281640 1 -- 172.21.15.67:0/960852770 shutdown_connections 2024-07-27T03:14:37.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.440+0000 7fc1dc281640 1 -- 172.21.15.67:0/960852770 >> 172.21.15.67:0/960852770 conn(0x7fc1d40791a0 msgr2=0x7fc1d4103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:37.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.440+0000 7fc1dc281640 1 -- 172.21.15.67:0/960852770 shutdown_connections 2024-07-27T03:14:37.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:37.440+0000 7fc1dc281640 1 -- 172.21.15.67:0/960852770 wait complete. 2024-07-27T03:14:38.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:38 smithi067 ceph-mon[32814]: pgmap v372: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-07-27T03:14:38.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:38 smithi175 ceph-mon[37125]: pgmap v372: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-07-27T03:14:38.891 INFO:teuthology.orchestra.run.smithi067.stdout:257698037810 2024-07-27T03:14:38.891 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-07-27T03:14:39.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:38 smithi149 ceph-mon[37173]: pgmap v372: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-07-27T03:14:40.282 INFO:teuthology.orchestra.run.smithi067.stdout:184683593792 2024-07-27T03:14:40.283 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-07-27T03:14:40.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:40 smithi067 ceph-mon[32814]: pgmap v373: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:14:40.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:40 smithi175 ceph-mon[37125]: pgmap v373: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:14:41.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:40 smithi149 ceph-mon[37173]: pgmap v373: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:14:41.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.684+0000 7f3a90c8d640 1 -- 172.21.15.67:0/322732932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a8c1024d0 msgr2=0x7f3a8c1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:41.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.684+0000 7f3a90c8d640 1 --2- 172.21.15.67:0/322732932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a8c1024d0 0x7f3a8c1028e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009900 tx=0x7f3a7c02a3f0 comp rx=0 tx=0).stop 2024-07-27T03:14:41.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.684+0000 7f3a90c8d640 1 -- 172.21.15.67:0/322732932 shutdown_connections 2024-07-27T03:14:41.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.684+0000 7f3a90c8d640 1 --2- 172.21.15.67:0/322732932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a8c108760 0x7f3a8c10ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:41.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.684+0000 7f3a90c8d640 1 --2- 172.21.15.67:0/322732932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a8c107e10 0x7f3a8c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:41.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.684+0000 7f3a90c8d640 1 --2- 172.21.15.67:0/322732932 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a8c1024d0 0x7f3a8c1028e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:41.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.684+0000 7f3a90c8d640 1 -- 172.21.15.67:0/322732932 >> 172.21.15.67:0/322732932 conn(0x7f3a8c0791a0 msgr2=0x7f3a8c0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:41.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.684+0000 7f3a90c8d640 1 -- 172.21.15.67:0/322732932 shutdown_connections 2024-07-27T03:14:41.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.684+0000 7f3a90c8d640 1 -- 172.21.15.67:0/322732932 wait complete. 2024-07-27T03:14:41.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.685+0000 7f3a90c8d640 1 Processor -- start 2024-07-27T03:14:41.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.685+0000 7f3a90c8d640 1 -- start start 2024-07-27T03:14:41.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.685+0000 7f3a90c8d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a8c1024d0 0x7f3a8c198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:41.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.685+0000 7f3a90c8d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a8c107e10 0x7f3a8c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:41.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.685+0000 7f3a90c8d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a8c108760 0x7f3a8c19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:41.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.685+0000 7f3a90c8d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1123c0 con 0x7f3a8c1024d0 2024-07-27T03:14:41.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.685+0000 7f3a90c8d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c112240 con 0x7f3a8c108760 2024-07-27T03:14:41.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.685+0000 7f3a90c8d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c112540 con 0x7f3a8c107e10 2024-07-27T03:14:41.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.685+0000 7f3a8a59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a8c1024d0 0x7f3a8c198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:41.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.686+0000 7f3a8a59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a8c1024d0 0x7f3a8c198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46580/0 (socket says 172.21.15.67:46580) 2024-07-27T03:14:41.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.686+0000 7f3a8a59c640 1 -- 172.21.15.67:0/572003427 learned_addr learned my addr 172.21.15.67:0/572003427 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:14:41.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.686+0000 7f3a8ad9d640 1 --2- 172.21.15.67:0/572003427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a8c108760 0x7f3a8c19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:41.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.686+0000 7f3a89d9b640 1 --2- 172.21.15.67:0/572003427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a8c107e10 0x7f3a8c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:41.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.686+0000 7f3a8a59c640 1 -- 172.21.15.67:0/572003427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a8c107e10 msgr2=0x7f3a8c199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:41.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.686+0000 7f3a8a59c640 1 --2- 172.21.15.67:0/572003427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a8c107e10 0x7f3a8c199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:41.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.686+0000 7f3a8a59c640 1 -- 172.21.15.67:0/572003427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a8c108760 msgr2=0x7f3a8c19e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:41.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.686+0000 7f3a8a59c640 1 --2- 172.21.15.67:0/572003427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a8c108760 0x7f3a8c19e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:41.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.686+0000 7f3a8a59c640 1 -- 172.21.15.67:0/572003427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8c19ec80 con 0x7f3a8c1024d0 2024-07-27T03:14:41.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.686+0000 7f3a8a59c640 1 --2- 172.21.15.67:0/572003427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a8c1024d0 0x7f3a8c198dd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c005940 tx=0x7f3a7c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:41.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.686+0000 7f3a7b7fe640 1 -- 172.21.15.67:0/572003427 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c046070 con 0x7f3a8c1024d0 2024-07-27T03:14:41.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.686+0000 7f3a7b7fe640 1 -- 172.21.15.67:0/572003427 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a7c003aa0 con 0x7f3a8c1024d0 2024-07-27T03:14:41.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.686+0000 7f3a90c8d640 1 -- 172.21.15.67:0/572003427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a8c19ef10 con 0x7f3a8c1024d0 2024-07-27T03:14:41.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.686+0000 7f3a7b7fe640 1 -- 172.21.15.67:0/572003427 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c035050 con 0x7f3a8c1024d0 2024-07-27T03:14:41.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.687+0000 7f3a90c8d640 1 -- 172.21.15.67:0/572003427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a8c19f3f0 con 0x7f3a8c1024d0 2024-07-27T03:14:41.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.687+0000 7f3a90c8d640 1 -- 172.21.15.67:0/572003427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3a8c06ac80 con 0x7f3a8c1024d0 2024-07-27T03:14:41.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.689+0000 7f3a7b7fe640 1 -- 172.21.15.67:0/572003427 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f3a7c049070 con 0x7f3a8c1024d0 2024-07-27T03:14:41.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.690+0000 7f3a7b7fe640 1 --2- 172.21.15.67:0/572003427 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3a740b3c90 0x7f3a740b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:41.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.690+0000 7f3a89d9b640 1 --2- 172.21.15.67:0/572003427 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3a740b3c90 0x7f3a740b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:41.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.690+0000 7f3a7b7fe640 1 -- 172.21.15.67:0/572003427 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f3a7c1065e0 con 0x7f3a8c1024d0 2024-07-27T03:14:41.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.690+0000 7f3a7b7fe640 1 --2- 172.21.15.67:0/572003427 >> [v2:172.21.15.149:6800/2561753152,v1:172.21.15.149:6801/2561753152] conn(0x7f3a740bf030 0x7f3a740c1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:41.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.690+0000 7f3a7b7fe640 1 -- 172.21.15.67:0/572003427 --> [v2:172.21.15.149:6800/2561753152,v1:172.21.15.149:6801/2561753152] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3a7c04de50 con 0x7f3a740bf030 2024-07-27T03:14:41.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.690+0000 7f3a7b7fe640 1 -- 172.21.15.67:0/572003427 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f3a7c106a30 con 0x7f3a8c1024d0 2024-07-27T03:14:41.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.690+0000 7f3a89d9b640 1 --2- 172.21.15.67:0/572003427 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3a740b3c90 0x7f3a740b61a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c1005e0 tx=0x7f3a8000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:41.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.691+0000 7f3a8ad9d640 1 --2- 172.21.15.67:0/572003427 >> [v2:172.21.15.149:6800/2561753152,v1:172.21.15.149:6801/2561753152] conn(0x7f3a740bf030 0x7f3a740c1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:41.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.691+0000 7f3a8ad9d640 1 --2- 172.21.15.67:0/572003427 >> [v2:172.21.15.149:6800/2561753152,v1:172.21.15.149:6801/2561753152] conn(0x7f3a740bf030 0x7f3a740c1520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:41.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.692+0000 7f3a7b7fe640 1 -- 172.21.15.67:0/572003427 <== osd.1 v2:172.21.15.149:6800/2561753152 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3a7c04de50 con 0x7f3a740bf030 2024-07-27T03:14:41.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.706+0000 7f3a90c8d640 1 -- 172.21.15.67:0/572003427 --> [v2:172.21.15.149:6800/2561753152,v1:172.21.15.149:6801/2561753152] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3a8c06b080 con 0x7f3a740bf030 2024-07-27T03:14:41.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.707+0000 7f3a7b7fe640 1 -- 172.21.15.67:0/572003427 <== osd.1 v2:172.21.15.149:6800/2561753152 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3a8c06b080 con 0x7f3a740bf030 2024-07-27T03:14:41.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.707+0000 7f3a90c8d640 1 -- 172.21.15.67:0/572003427 >> [v2:172.21.15.149:6800/2561753152,v1:172.21.15.149:6801/2561753152] conn(0x7f3a740bf030 msgr2=0x7f3a740c1520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:41.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.707+0000 7f3a90c8d640 1 --2- 172.21.15.67:0/572003427 >> [v2:172.21.15.149:6800/2561753152,v1:172.21.15.149:6801/2561753152] conn(0x7f3a740bf030 0x7f3a740c1520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:41.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.707+0000 7f3a90c8d640 1 -- 172.21.15.67:0/572003427 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3a740b3c90 msgr2=0x7f3a740b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:41.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.707+0000 7f3a90c8d640 1 --2- 172.21.15.67:0/572003427 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3a740b3c90 0x7f3a740b61a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c1005e0 tx=0x7f3a8000b390 comp rx=0 tx=0).stop 2024-07-27T03:14:41.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.707+0000 7f3a90c8d640 1 -- 172.21.15.67:0/572003427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a8c1024d0 msgr2=0x7f3a8c198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:41.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.707+0000 7f3a90c8d640 1 --2- 172.21.15.67:0/572003427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a8c1024d0 0x7f3a8c198dd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c005940 tx=0x7f3a7c004270 comp rx=0 tx=0).stop 2024-07-27T03:14:41.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.707+0000 7f3a8ad9d640 1 -- 172.21.15.67:0/572003427 reap_dead start 2024-07-27T03:14:41.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.708+0000 7f3a90c8d640 1 -- 172.21.15.67:0/572003427 shutdown_connections 2024-07-27T03:14:41.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.708+0000 7f3a90c8d640 1 -- 172.21.15.67:0/572003427 >> 172.21.15.67:0/572003427 conn(0x7f3a8c0791a0 msgr2=0x7f3a8c100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:41.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.708+0000 7f3a90c8d640 1 -- 172.21.15.67:0/572003427 shutdown_connections 2024-07-27T03:14:41.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:40.708+0000 7f3a90c8d640 1 -- 172.21.15.67:0/572003427 wait complete. 2024-07-27T03:14:42.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.500+0000 7fd3bea2e640 1 -- 172.21.15.67:0/2168642355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3b8109860 msgr2=0x7fd3b8110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:42.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.500+0000 7fd3bea2e640 1 --2- 172.21.15.67:0/2168642355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3b8109860 0x7fd3b8110090 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8009900 tx=0x7fd3a802a410 comp rx=0 tx=0).stop 2024-07-27T03:14:42.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.500+0000 7fd3bea2e640 1 -- 172.21.15.67:0/2168642355 shutdown_connections 2024-07-27T03:14:42.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.500+0000 7fd3bea2e640 1 --2- 172.21.15.67:0/2168642355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3b8109860 0x7fd3b8110090 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:42.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.500+0000 7fd3bea2e640 1 --2- 172.21.15.67:0/2168642355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3b8108f10 0x7fd3b8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:42.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.500+0000 7fd3bea2e640 1 --2- 172.21.15.67:0/2168642355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3b81035d0 0x7fd3b81039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:42.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.500+0000 7fd3bea2e640 1 -- 172.21.15.67:0/2168642355 >> 172.21.15.67:0/2168642355 conn(0x7fd3b807b2c0 msgr2=0x7fd3b807b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:42.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.500+0000 7fd3bea2e640 1 -- 172.21.15.67:0/2168642355 shutdown_connections 2024-07-27T03:14:42.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.500+0000 7fd3bea2e640 1 -- 172.21.15.67:0/2168642355 wait complete. 2024-07-27T03:14:42.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.501+0000 7fd3bea2e640 1 Processor -- start 2024-07-27T03:14:42.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.501+0000 7fd3bea2e640 1 -- start start 2024-07-27T03:14:42.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.501+0000 7fd3bea2e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3b81035d0 0x7fd3b8199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:42.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.501+0000 7fd3bea2e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3b8108f10 0x7fd3b819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:42.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.501+0000 7fd3bea2e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3b8109860 0x7fd3b819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:42.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.501+0000 7fd3bea2e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b8113530 con 0x7fd3b8109860 2024-07-27T03:14:42.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.501+0000 7fd3bea2e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b81133b0 con 0x7fd3b8108f10 2024-07-27T03:14:42.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.501+0000 7fd3bea2e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b81136b0 con 0x7fd3b81035d0 2024-07-27T03:14:42.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.502+0000 7fd3bcfcb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3b8109860 0x7fd3b819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:42.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.502+0000 7fd3bcfcb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3b8109860 0x7fd3b819f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46602/0 (socket says 172.21.15.67:46602) 2024-07-27T03:14:42.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.502+0000 7fd3bcfcb640 1 -- 172.21.15.67:0/185399517 learned_addr learned my addr 172.21.15.67:0/185399517 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:14:42.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.502+0000 7fd3b77fe640 1 --2- 172.21.15.67:0/185399517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3b8108f10 0x7fd3b819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:42.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.502+0000 7fd3b7fff640 1 --2- 172.21.15.67:0/185399517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3b81035d0 0x7fd3b8199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:42.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.502+0000 7fd3bcfcb640 1 -- 172.21.15.67:0/185399517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3b81035d0 msgr2=0x7fd3b8199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:42.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.502+0000 7fd3bcfcb640 1 --2- 172.21.15.67:0/185399517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3b81035d0 0x7fd3b8199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:42.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.502+0000 7fd3bcfcb640 1 -- 172.21.15.67:0/185399517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3b8108f10 msgr2=0x7fd3b819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:42.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.502+0000 7fd3bcfcb640 1 --2- 172.21.15.67:0/185399517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3b8108f10 0x7fd3b819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:42.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.502+0000 7fd3bcfcb640 1 -- 172.21.15.67:0/185399517 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b819fe30 con 0x7fd3b8109860 2024-07-27T03:14:42.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.502+0000 7fd3b7fff640 1 --2- 172.21.15.67:0/185399517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3b81035d0 0x7fd3b8199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:14:42.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.502+0000 7fd3bcfcb640 1 --2- 172.21.15.67:0/185399517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3b8109860 0x7fd3b819f700 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8004770 tx=0x7fd3a8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:42.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.502+0000 7fd3b57fa640 1 -- 172.21.15.67:0/185399517 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a803f030 con 0x7fd3b8109860 2024-07-27T03:14:42.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.502+0000 7fd3bea2e640 1 -- 172.21.15.67:0/185399517 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b81a00c0 con 0x7fd3b8109860 2024-07-27T03:14:42.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.502+0000 7fd3b57fa640 1 -- 172.21.15.67:0/185399517 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3a8003aa0 con 0x7fd3b8109860 2024-07-27T03:14:42.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.502+0000 7fd3b57fa640 1 -- 172.21.15.67:0/185399517 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8035070 con 0x7fd3b8109860 2024-07-27T03:14:42.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.502+0000 7fd3bea2e640 1 -- 172.21.15.67:0/185399517 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b81a6880 con 0x7fd3b8109860 2024-07-27T03:14:42.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.503+0000 7fd3bea2e640 1 -- 172.21.15.67:0/185399517 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd3b80635c0 con 0x7fd3b8109860 2024-07-27T03:14:42.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.505+0000 7fd3b57fa640 1 -- 172.21.15.67:0/185399517 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7fd3a8042030 con 0x7fd3b8109860 2024-07-27T03:14:42.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.506+0000 7fd3b57fa640 1 --2- 172.21.15.67:0/185399517 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd3a00b3c60 0x7fd3a00b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:42.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.506+0000 7fd3b7fff640 1 --2- 172.21.15.67:0/185399517 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd3a00b3c60 0x7fd3a00b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:42.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.506+0000 7fd3b57fa640 1 -- 172.21.15.67:0/185399517 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fd3a8104eb0 con 0x7fd3b8109860 2024-07-27T03:14:42.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.506+0000 7fd3b57fa640 1 --2- 172.21.15.67:0/185399517 >> [v2:172.21.15.67:6802/2648129674,v1:172.21.15.67:6803/2648129674] conn(0x7fd3a00bf000 0x7fd3a00c14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:42.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.506+0000 7fd3b57fa640 1 -- 172.21.15.67:0/185399517 --> [v2:172.21.15.67:6802/2648129674,v1:172.21.15.67:6803/2648129674] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd3a00c1bc0 con 0x7fd3a00bf000 2024-07-27T03:14:42.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.506+0000 7fd3b57fa640 1 -- 172.21.15.67:0/185399517 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fd3a8105300 con 0x7fd3b8109860 2024-07-27T03:14:42.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.506+0000 7fd3b77fe640 1 --2- 172.21.15.67:0/185399517 >> [v2:172.21.15.67:6802/2648129674,v1:172.21.15.67:6803/2648129674] conn(0x7fd3a00bf000 0x7fd3a00c14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:42.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.507+0000 7fd3b7fff640 1 --2- 172.21.15.67:0/185399517 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd3a00b3c60 0x7fd3a00b6170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0005fd0 tx=0x7fd3b0009520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:42.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.507+0000 7fd3b77fe640 1 --2- 172.21.15.67:0/185399517 >> [v2:172.21.15.67:6802/2648129674,v1:172.21.15.67:6803/2648129674] conn(0x7fd3a00bf000 0x7fd3a00c14f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:42.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.508+0000 7fd3b57fa640 1 -- 172.21.15.67:0/185399517 <== osd.0 v2:172.21.15.67:6802/2648129674 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd3a00c1bc0 con 0x7fd3a00bf000 2024-07-27T03:14:42.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.522+0000 7fd3bea2e640 1 -- 172.21.15.67:0/185399517 --> [v2:172.21.15.67:6802/2648129674,v1:172.21.15.67:6803/2648129674] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd3b80639c0 con 0x7fd3a00bf000 2024-07-27T03:14:42.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.522+0000 7fd3b57fa640 1 -- 172.21.15.67:0/185399517 <== osd.0 v2:172.21.15.67:6802/2648129674 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd3b80639c0 con 0x7fd3a00bf000 2024-07-27T03:14:42.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.522+0000 7fd3bea2e640 1 -- 172.21.15.67:0/185399517 >> [v2:172.21.15.67:6802/2648129674,v1:172.21.15.67:6803/2648129674] conn(0x7fd3a00bf000 msgr2=0x7fd3a00c14f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:42.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.522+0000 7fd3bea2e640 1 --2- 172.21.15.67:0/185399517 >> [v2:172.21.15.67:6802/2648129674,v1:172.21.15.67:6803/2648129674] conn(0x7fd3a00bf000 0x7fd3a00c14f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:42.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.523+0000 7fd3bea2e640 1 -- 172.21.15.67:0/185399517 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd3a00b3c60 msgr2=0x7fd3a00b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:42.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.523+0000 7fd3bea2e640 1 --2- 172.21.15.67:0/185399517 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd3a00b3c60 0x7fd3a00b6170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0005fd0 tx=0x7fd3b0009520 comp rx=0 tx=0).stop 2024-07-27T03:14:42.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.523+0000 7fd3bea2e640 1 -- 172.21.15.67:0/185399517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3b8109860 msgr2=0x7fd3b819f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:42.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.523+0000 7fd3bea2e640 1 --2- 172.21.15.67:0/185399517 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3b8109860 0x7fd3b819f700 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8004770 tx=0x7fd3a8004270 comp rx=0 tx=0).stop 2024-07-27T03:14:42.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.523+0000 7fd3bcfcb640 1 -- 172.21.15.67:0/185399517 reap_dead start 2024-07-27T03:14:42.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.523+0000 7fd3bea2e640 1 -- 172.21.15.67:0/185399517 shutdown_connections 2024-07-27T03:14:42.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.523+0000 7fd3bea2e640 1 -- 172.21.15.67:0/185399517 >> 172.21.15.67:0/185399517 conn(0x7fd3b807b2c0 msgr2=0x7fd3b8106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:42.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.523+0000 7fd3bea2e640 1 -- 172.21.15.67:0/185399517 shutdown_connections 2024-07-27T03:14:42.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:42.523+0000 7fd3bea2e640 1 -- 172.21.15.67:0/185399517 wait complete. 2024-07-27T03:14:42.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.948+0000 7f5eb160f640 1 -- 172.21.15.67:0/3785051530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eac107e10 msgr2=0x7f5eac108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:42.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.948+0000 7f5eb160f640 1 --2- 172.21.15.67:0/3785051530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eac107e10 0x7f5eac108220 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009900 tx=0x7f5e9402a410 comp rx=0 tx=0).stop 2024-07-27T03:14:42.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.948+0000 7f5eb160f640 1 -- 172.21.15.67:0/3785051530 shutdown_connections 2024-07-27T03:14:42.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.948+0000 7f5eb160f640 1 --2- 172.21.15.67:0/3785051530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5eac108760 0x7f5eac10ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:42.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.948+0000 7f5eb160f640 1 --2- 172.21.15.67:0/3785051530 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eac107e10 0x7f5eac108220 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:42.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.948+0000 7f5eb160f640 1 --2- 172.21.15.67:0/3785051530 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eac1024d0 0x7f5eac1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:42.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.948+0000 7f5eb160f640 1 -- 172.21.15.67:0/3785051530 >> 172.21.15.67:0/3785051530 conn(0x7f5eac0791a0 msgr2=0x7f5eac0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:42.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.948+0000 7f5eb160f640 1 -- 172.21.15.67:0/3785051530 shutdown_connections 2024-07-27T03:14:42.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.948+0000 7f5eb160f640 1 -- 172.21.15.67:0/3785051530 wait complete. 2024-07-27T03:14:42.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.948+0000 7f5eb160f640 1 Processor -- start 2024-07-27T03:14:42.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.949+0000 7f5eb160f640 1 -- start start 2024-07-27T03:14:42.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.949+0000 7f5eb160f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eac1024d0 0x7f5eac19c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:42.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.949+0000 7f5eb160f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5eac107e10 0x7f5eac19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:42.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.949+0000 7f5eb160f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eac108760 0x7f5eac1a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:42.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.949+0000 7f5eb160f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac112390 con 0x7f5eac1024d0 2024-07-27T03:14:42.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.949+0000 7f5eb160f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac112210 con 0x7f5eac107e10 2024-07-27T03:14:42.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.949+0000 7f5eb160f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac112510 con 0x7f5eac108760 2024-07-27T03:14:42.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.949+0000 7f5eaaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eac1024d0 0x7f5eac19c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:42.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.949+0000 7f5eaaffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eac1024d0 0x7f5eac19c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46590/0 (socket says 172.21.15.67:46590) 2024-07-27T03:14:42.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.949+0000 7f5eaaffd640 1 -- 172.21.15.67:0/584070384 learned_addr learned my addr 172.21.15.67:0/584070384 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:14:42.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.949+0000 7f5ea3fff640 1 --2- 172.21.15.67:0/584070384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5eac107e10 0x7f5eac19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:42.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.949+0000 7f5eab7fe640 1 --2- 172.21.15.67:0/584070384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eac108760 0x7f5eac1a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:42.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.949+0000 7f5eaaffd640 1 -- 172.21.15.67:0/584070384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eac108760 msgr2=0x7f5eac1a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:42.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.949+0000 7f5eaaffd640 1 --2- 172.21.15.67:0/584070384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eac108760 0x7f5eac1a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:42.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.949+0000 7f5eaaffd640 1 -- 172.21.15.67:0/584070384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5eac107e10 msgr2=0x7f5eac19c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:42.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.949+0000 7f5eaaffd640 1 --2- 172.21.15.67:0/584070384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5eac107e10 0x7f5eac19c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:42.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.949+0000 7f5eaaffd640 1 -- 172.21.15.67:0/584070384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eac1a1f40 con 0x7f5eac1024d0 2024-07-27T03:14:42.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.949+0000 7f5eab7fe640 1 --2- 172.21.15.67:0/584070384 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5eac108760 0x7f5eac1a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:14:42.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.950+0000 7f5eaaffd640 1 --2- 172.21.15.67:0/584070384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eac1024d0 0x7f5eac19c090 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00c920 tx=0x7f5e9c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:42.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.950+0000 7f5ea8ff9640 1 -- 172.21.15.67:0/584070384 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c007c70 con 0x7f5eac1024d0 2024-07-27T03:14:42.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.950+0000 7f5ea8ff9640 1 -- 172.21.15.67:0/584070384 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e9c007e10 con 0x7f5eac1024d0 2024-07-27T03:14:42.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.950+0000 7f5eb160f640 1 -- 172.21.15.67:0/584070384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eac1a2230 con 0x7f5eac1024d0 2024-07-27T03:14:42.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.950+0000 7f5ea8ff9640 1 -- 172.21.15.67:0/584070384 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c018820 con 0x7f5eac1024d0 2024-07-27T03:14:42.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.950+0000 7f5eb160f640 1 -- 172.21.15.67:0/584070384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eac1a8a50 con 0x7f5eac1024d0 2024-07-27T03:14:42.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.951+0000 7f5eb160f640 1 -- 172.21.15.67:0/584070384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5eac06b3f0 con 0x7f5eac1024d0 2024-07-27T03:14:42.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.952+0000 7f5ea8ff9640 1 -- 172.21.15.67:0/584070384 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f5e9c00f450 con 0x7f5eac1024d0 2024-07-27T03:14:42.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.953+0000 7f5ea8ff9640 1 --2- 172.21.15.67:0/584070384 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5e980b3d10 0x7f5e980b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:42.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.953+0000 7f5ea3fff640 1 --2- 172.21.15.67:0/584070384 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5e980b3d10 0x7f5e980b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:42.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.953+0000 7f5ea8ff9640 1 -- 172.21.15.67:0/584070384 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f5e9c0e46d0 con 0x7f5eac1024d0 2024-07-27T03:14:42.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.954+0000 7f5ea8ff9640 1 --2- 172.21.15.67:0/584070384 >> [v2:172.21.15.175:6824/2565514213,v1:172.21.15.175:6825/2565514213] conn(0x7f5e980bf0b0 0x7f5e980c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:42.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.954+0000 7f5ea8ff9640 1 -- 172.21.15.67:0/584070384 --> [v2:172.21.15.175:6824/2565514213,v1:172.21.15.175:6825/2565514213] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e9c01ee50 con 0x7f5e980bf0b0 2024-07-27T03:14:42.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.954+0000 7f5ea8ff9640 1 -- 172.21.15.67:0/584070384 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f5e9c0e4b20 con 0x7f5eac1024d0 2024-07-27T03:14:42.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.954+0000 7f5ea3fff640 1 --2- 172.21.15.67:0/584070384 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5e980b3d10 0x7f5e980b6220 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e94000c00 tx=0x7f5e94031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:42.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.954+0000 7f5eab7fe640 1 --2- 172.21.15.67:0/584070384 >> [v2:172.21.15.175:6824/2565514213,v1:172.21.15.175:6825/2565514213] conn(0x7f5e980bf0b0 0x7f5e980c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:42.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.954+0000 7f5eab7fe640 1 --2- 172.21.15.67:0/584070384 >> [v2:172.21.15.175:6824/2565514213,v1:172.21.15.175:6825/2565514213] conn(0x7f5e980bf0b0 0x7f5e980c15a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:42.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.955+0000 7f5ea8ff9640 1 -- 172.21.15.67:0/584070384 <== osd.11 v2:172.21.15.175:6824/2565514213 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5e9c01ee50 con 0x7f5e980bf0b0 2024-07-27T03:14:42.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.969+0000 7f5eb160f640 1 -- 172.21.15.67:0/584070384 --> [v2:172.21.15.175:6824/2565514213,v1:172.21.15.175:6825/2565514213] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5eac103750 con 0x7f5e980bf0b0 2024-07-27T03:14:42.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.970+0000 7f5ea8ff9640 1 -- 172.21.15.67:0/584070384 <== osd.11 v2:172.21.15.175:6824/2565514213 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5eac103750 con 0x7f5e980bf0b0 2024-07-27T03:14:42.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.970+0000 7f5eb160f640 1 -- 172.21.15.67:0/584070384 >> [v2:172.21.15.175:6824/2565514213,v1:172.21.15.175:6825/2565514213] conn(0x7f5e980bf0b0 msgr2=0x7f5e980c15a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:42.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.970+0000 7f5eb160f640 1 --2- 172.21.15.67:0/584070384 >> [v2:172.21.15.175:6824/2565514213,v1:172.21.15.175:6825/2565514213] conn(0x7f5e980bf0b0 0x7f5e980c15a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:42.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.970+0000 7f5eb160f640 1 -- 172.21.15.67:0/584070384 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5e980b3d10 msgr2=0x7f5e980b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:42.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.970+0000 7f5eb160f640 1 --2- 172.21.15.67:0/584070384 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5e980b3d10 0x7f5e980b6220 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e94000c00 tx=0x7f5e94031400 comp rx=0 tx=0).stop 2024-07-27T03:14:42.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.970+0000 7f5eb160f640 1 -- 172.21.15.67:0/584070384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eac1024d0 msgr2=0x7f5eac19c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:42.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.970+0000 7f5eb160f640 1 --2- 172.21.15.67:0/584070384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5eac1024d0 0x7f5eac19c090 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00c920 tx=0x7f5e9c00cde0 comp rx=0 tx=0).stop 2024-07-27T03:14:42.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.970+0000 7f5eab7fe640 1 -- 172.21.15.67:0/584070384 reap_dead start 2024-07-27T03:14:42.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.971+0000 7f5eb160f640 1 -- 172.21.15.67:0/584070384 shutdown_connections 2024-07-27T03:14:42.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.971+0000 7f5eb160f640 1 -- 172.21.15.67:0/584070384 >> 172.21.15.67:0/584070384 conn(0x7f5eac0791a0 msgr2=0x7f5eac06a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:42.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.971+0000 7f5eb160f640 1 -- 172.21.15.67:0/584070384 shutdown_connections 2024-07-27T03:14:42.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:41.971+0000 7f5eb160f640 1 -- 172.21.15.67:0/584070384 wait complete. 2024-07-27T03:14:42.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:42 smithi175 ceph-mon[37125]: pgmap v374: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-07-27T03:14:42.903 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:42 smithi067 ceph-mon[32814]: pgmap v374: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-07-27T03:14:43.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:42 smithi149 ceph-mon[37173]: pgmap v374: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-07-27T03:14:43.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.263+0000 7ff00c59b640 1 -- 172.21.15.67:0/2849316863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0041024d0 msgr2=0x7ff0041028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:43.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.263+0000 7ff00c59b640 1 --2- 172.21.15.67:0/2849316863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0041024d0 0x7ff0041028e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7feff4009ca0 tx=0x7feff402a8c0 comp rx=0 tx=0).stop 2024-07-27T03:14:43.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.263+0000 7ff00c59b640 1 -- 172.21.15.67:0/2849316863 shutdown_connections 2024-07-27T03:14:43.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.263+0000 7ff00c59b640 1 --2- 172.21.15.67:0/2849316863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff004108760 0x7ff00410ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:43.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.263+0000 7ff00c59b640 1 --2- 172.21.15.67:0/2849316863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff004107e10 0x7ff004108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:43.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.263+0000 7ff00c59b640 1 --2- 172.21.15.67:0/2849316863 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0041024d0 0x7ff0041028e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:43.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.263+0000 7ff00c59b640 1 -- 172.21.15.67:0/2849316863 >> 172.21.15.67:0/2849316863 conn(0x7ff0040791a0 msgr2=0x7ff0040795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:43.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.263+0000 7ff00c59b640 1 -- 172.21.15.67:0/2849316863 shutdown_connections 2024-07-27T03:14:43.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.263+0000 7ff00c59b640 1 -- 172.21.15.67:0/2849316863 wait complete. 2024-07-27T03:14:43.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.264+0000 7ff00c59b640 1 Processor -- start 2024-07-27T03:14:43.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.264+0000 7ff00c59b640 1 -- start start 2024-07-27T03:14:43.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.264+0000 7ff00c59b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0041024d0 0x7ff00419acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:43.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.264+0000 7ff00c59b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff004107e10 0x7ff00419b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:43.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.264+0000 7ff00c59b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff004108760 0x7ff00419b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:43.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.264+0000 7ff00c59b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff0041123e0 con 0x7ff004107e10 2024-07-27T03:14:43.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.264+0000 7ff00c59b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff004112260 con 0x7ff004108760 2024-07-27T03:14:43.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.264+0000 7ff00c59b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff004112560 con 0x7ff0041024d0 2024-07-27T03:14:43.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.264+0000 7ff009b36640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff004107e10 0x7ff00419b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:43.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.264+0000 7ff009b36640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff004107e10 0x7ff00419b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46614/0 (socket says 172.21.15.67:46614) 2024-07-27T03:14:43.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.264+0000 7ff009b36640 1 -- 172.21.15.67:0/29246782 learned_addr learned my addr 172.21.15.67:0/29246782 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:14:43.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.265+0000 7ff00ab38640 1 --2- 172.21.15.67:0/29246782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff004108760 0x7ff00419b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:43.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.265+0000 7ff00a337640 1 --2- 172.21.15.67:0/29246782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0041024d0 0x7ff00419acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:43.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.265+0000 7ff009b36640 1 -- 172.21.15.67:0/29246782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0041024d0 msgr2=0x7ff00419acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:43.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.265+0000 7ff009b36640 1 --2- 172.21.15.67:0/29246782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0041024d0 0x7ff00419acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:43.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.265+0000 7ff009b36640 1 -- 172.21.15.67:0/29246782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff004108760 msgr2=0x7ff00419b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:43.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.265+0000 7ff009b36640 1 --2- 172.21.15.67:0/29246782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff004108760 0x7ff00419b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:43.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.265+0000 7ff009b36640 1 -- 172.21.15.67:0/29246782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0041a5ef0 con 0x7ff004107e10 2024-07-27T03:14:43.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.265+0000 7ff009b36640 1 --2- 172.21.15.67:0/29246782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff004107e10 0x7ff00419b230 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7feff800c970 tx=0x7feff800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:43.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.265+0000 7ff0037fe640 1 -- 172.21.15.67:0/29246782 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8007bb0 con 0x7ff004107e10 2024-07-27T03:14:43.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.265+0000 7ff0037fe640 1 -- 172.21.15.67:0/29246782 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff8007d50 con 0x7ff004107e10 2024-07-27T03:14:43.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.265+0000 7ff00c59b640 1 -- 172.21.15.67:0/29246782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0041a61e0 con 0x7ff004107e10 2024-07-27T03:14:43.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.265+0000 7ff0037fe640 1 -- 172.21.15.67:0/29246782 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8018730 con 0x7ff004107e10 2024-07-27T03:14:43.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.265+0000 7ff00c59b640 1 -- 172.21.15.67:0/29246782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0041a66d0 con 0x7ff004107e10 2024-07-27T03:14:43.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.266+0000 7ff00c59b640 1 -- 172.21.15.67:0/29246782 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff0040635c0 con 0x7ff004107e10 2024-07-27T03:14:43.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.268+0000 7ff0037fe640 1 -- 172.21.15.67:0/29246782 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7feff800f990 con 0x7ff004107e10 2024-07-27T03:14:43.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.268+0000 7ff0037fe640 1 --2- 172.21.15.67:0/29246782 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fefec0b3c90 0x7fefec0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:43.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.269+0000 7ff00a337640 1 --2- 172.21.15.67:0/29246782 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fefec0b3c90 0x7fefec0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:43.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.269+0000 7ff0037fe640 1 -- 172.21.15.67:0/29246782 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7feff80e4680 con 0x7ff004107e10 2024-07-27T03:14:43.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.269+0000 7ff0037fe640 1 --2- 172.21.15.67:0/29246782 >> [v2:172.21.15.175:6816/2286032466,v1:172.21.15.175:6817/2286032466] conn(0x7fefec0bf030 0x7fefec0c1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:43.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.269+0000 7ff0037fe640 1 -- 172.21.15.67:0/29246782 --> [v2:172.21.15.175:6816/2286032466,v1:172.21.15.175:6817/2286032466] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feff8022e50 con 0x7fefec0bf030 2024-07-27T03:14:43.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.269+0000 7ff0037fe640 1 -- 172.21.15.67:0/29246782 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7feff80e4ad0 con 0x7ff004107e10 2024-07-27T03:14:43.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.269+0000 7ff00a337640 1 --2- 172.21.15.67:0/29246782 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fefec0b3c90 0x7fefec0b61a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feff40098d0 tx=0x7feff40093e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:43.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.269+0000 7ff00ab38640 1 --2- 172.21.15.67:0/29246782 >> [v2:172.21.15.175:6816/2286032466,v1:172.21.15.175:6817/2286032466] conn(0x7fefec0bf030 0x7fefec0c1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:43.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.270+0000 7ff00ab38640 1 --2- 172.21.15.67:0/29246782 >> [v2:172.21.15.175:6816/2286032466,v1:172.21.15.175:6817/2286032466] conn(0x7fefec0bf030 0x7fefec0c1520 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:43.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.271+0000 7ff0037fe640 1 -- 172.21.15.67:0/29246782 <== osd.8 v2:172.21.15.175:6816/2286032466 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feff8022e50 con 0x7fefec0bf030 2024-07-27T03:14:43.288 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.286+0000 7ff00c59b640 1 -- 172.21.15.67:0/29246782 --> [v2:172.21.15.175:6816/2286032466,v1:172.21.15.175:6817/2286032466] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff004063a40 con 0x7fefec0bf030 2024-07-27T03:14:43.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.286+0000 7ff0037fe640 1 -- 172.21.15.67:0/29246782 <== osd.8 v2:172.21.15.175:6816/2286032466 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff004063a40 con 0x7fefec0bf030 2024-07-27T03:14:43.289 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.287+0000 7ff00c59b640 1 -- 172.21.15.67:0/29246782 >> [v2:172.21.15.175:6816/2286032466,v1:172.21.15.175:6817/2286032466] conn(0x7fefec0bf030 msgr2=0x7fefec0c1520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:43.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.287+0000 7ff00c59b640 1 --2- 172.21.15.67:0/29246782 >> [v2:172.21.15.175:6816/2286032466,v1:172.21.15.175:6817/2286032466] conn(0x7fefec0bf030 0x7fefec0c1520 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:43.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.287+0000 7ff00c59b640 1 -- 172.21.15.67:0/29246782 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fefec0b3c90 msgr2=0x7fefec0b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:43.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.287+0000 7ff00c59b640 1 --2- 172.21.15.67:0/29246782 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fefec0b3c90 0x7fefec0b61a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feff40098d0 tx=0x7feff40093e0 comp rx=0 tx=0).stop 2024-07-27T03:14:43.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.287+0000 7ff00c59b640 1 -- 172.21.15.67:0/29246782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff004107e10 msgr2=0x7ff00419b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:43.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.287+0000 7ff00c59b640 1 --2- 172.21.15.67:0/29246782 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff004107e10 0x7ff00419b230 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7feff800c970 tx=0x7feff800ce30 comp rx=0 tx=0).stop 2024-07-27T03:14:43.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.287+0000 7ff00ab38640 1 -- 172.21.15.67:0/29246782 reap_dead start 2024-07-27T03:14:43.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.287+0000 7ff00c59b640 1 -- 172.21.15.67:0/29246782 shutdown_connections 2024-07-27T03:14:43.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.287+0000 7ff00c59b640 1 -- 172.21.15.67:0/29246782 >> 172.21.15.67:0/29246782 conn(0x7ff0040791a0 msgr2=0x7ff004100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:43.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.287+0000 7ff00c59b640 1 -- 172.21.15.67:0/29246782 shutdown_connections 2024-07-27T03:14:43.290 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.287+0000 7ff00c59b640 1 -- 172.21.15.67:0/29246782 wait complete. 2024-07-27T03:14:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.708+0000 7ff7506eb640 1 -- 172.21.15.67:0/2539908744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff748101d00 msgr2=0x7ff748102110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.708+0000 7ff7506eb640 1 --2- 172.21.15.67:0/2539908744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff748101d00 0x7ff748102110 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff74400aa10 tx=0x7ff74402b720 comp rx=0 tx=0).stop 2024-07-27T03:14:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.709+0000 7ff7506eb640 1 -- 172.21.15.67:0/2539908744 shutdown_connections 2024-07-27T03:14:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.709+0000 7ff7506eb640 1 --2- 172.21.15.67:0/2539908744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff748107f90 0x7ff74810e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.709+0000 7ff7506eb640 1 --2- 172.21.15.67:0/2539908744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff748107640 0x7ff748107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.709+0000 7ff7506eb640 1 --2- 172.21.15.67:0/2539908744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff748101d00 0x7ff748102110 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.709+0000 7ff7506eb640 1 -- 172.21.15.67:0/2539908744 >> 172.21.15.67:0/2539908744 conn(0x7ff748079910 msgr2=0x7ff748079d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.709+0000 7ff7506eb640 1 -- 172.21.15.67:0/2539908744 shutdown_connections 2024-07-27T03:14:43.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.709+0000 7ff7506eb640 1 -- 172.21.15.67:0/2539908744 wait complete. 2024-07-27T03:14:43.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.709+0000 7ff7506eb640 1 Processor -- start 2024-07-27T03:14:43.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.710+0000 7ff7506eb640 1 -- start start 2024-07-27T03:14:43.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.710+0000 7ff7506eb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff748101d00 0x7ff748197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:43.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.710+0000 7ff7506eb640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff748107640 0x7ff7481983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:43.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.710+0000 7ff7506eb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff748107f90 0x7ff74819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:43.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.710+0000 7ff7506eb640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff748111430 con 0x7ff748107f90 2024-07-27T03:14:43.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.710+0000 7ff7506eb640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff7481112b0 con 0x7ff748107640 2024-07-27T03:14:43.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.710+0000 7ff7506eb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff7481115b0 con 0x7ff748101d00 2024-07-27T03:14:43.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.710+0000 7ff74ec88640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff748107f90 0x7ff74819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:43.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.710+0000 7ff74ec88640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff748107f90 0x7ff74819d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46624/0 (socket says 172.21.15.67:46624) 2024-07-27T03:14:43.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.710+0000 7ff74ec88640 1 -- 172.21.15.67:0/3316667436 learned_addr learned my addr 172.21.15.67:0/3316667436 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:14:43.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.710+0000 7ff74e487640 1 --2- 172.21.15.67:0/3316667436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff748101d00 0x7ff748197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:43.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.710+0000 7ff74dc86640 1 --2- 172.21.15.67:0/3316667436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff748107640 0x7ff7481983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:43.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.710+0000 7ff74ec88640 1 -- 172.21.15.67:0/3316667436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff748101d00 msgr2=0x7ff748197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:43.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.710+0000 7ff74ec88640 1 --2- 172.21.15.67:0/3316667436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff748101d00 0x7ff748197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:43.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.710+0000 7ff74ec88640 1 -- 172.21.15.67:0/3316667436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff748107640 msgr2=0x7ff7481983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:43.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.710+0000 7ff74ec88640 1 --2- 172.21.15.67:0/3316667436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff748107640 0x7ff7481983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:43.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.710+0000 7ff74ec88640 1 -- 172.21.15.67:0/3316667436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74819dcb0 con 0x7ff748107f90 2024-07-27T03:14:43.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.710+0000 7ff74e487640 1 --2- 172.21.15.67:0/3316667436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff748101d00 0x7ff748197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:14:43.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.711+0000 7ff74ec88640 1 --2- 172.21.15.67:0/3316667436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff748107f90 0x7ff74819d640 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff74000b970 tx=0x7ff74000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:43.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.711+0000 7ff73f7fe640 1 -- 172.21.15.67:0/3316667436 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740019070 con 0x7ff748107f90 2024-07-27T03:14:43.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.711+0000 7ff73f7fe640 1 -- 172.21.15.67:0/3316667436 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff740014440 con 0x7ff748107f90 2024-07-27T03:14:43.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.711+0000 7ff7506eb640 1 -- 172.21.15.67:0/3316667436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74819dfa0 con 0x7ff748107f90 2024-07-27T03:14:43.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.711+0000 7ff73f7fe640 1 -- 172.21.15.67:0/3316667436 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7400075d0 con 0x7ff748107f90 2024-07-27T03:14:43.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.711+0000 7ff7506eb640 1 -- 172.21.15.67:0/3316667436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7481a47c0 con 0x7ff748107f90 2024-07-27T03:14:43.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.712+0000 7ff7506eb640 1 -- 172.21.15.67:0/3316667436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff7480635c0 con 0x7ff748107f90 2024-07-27T03:14:43.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.713+0000 7ff73f7fe640 1 -- 172.21.15.67:0/3316667436 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7ff74001c070 con 0x7ff748107f90 2024-07-27T03:14:43.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.714+0000 7ff73f7fe640 1 --2- 172.21.15.67:0/3316667436 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff7340b3d10 0x7ff7340b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:43.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.714+0000 7ff74e487640 1 --2- 172.21.15.67:0/3316667436 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff7340b3d10 0x7ff7340b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:43.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.714+0000 7ff73f7fe640 1 -- 172.21.15.67:0/3316667436 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7ff7400e3e30 con 0x7ff748107f90 2024-07-27T03:14:43.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.715+0000 7ff73f7fe640 1 --2- 172.21.15.67:0/3316667436 >> [v2:172.21.15.149:6816/2289624404,v1:172.21.15.149:6817/2289624404] conn(0x7ff7340bf0b0 0x7ff7340c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:43.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.715+0000 7ff73f7fe640 1 -- 172.21.15.67:0/3316667436 --> [v2:172.21.15.149:6816/2289624404,v1:172.21.15.149:6817/2289624404] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff7340c1c70 con 0x7ff7340bf0b0 2024-07-27T03:14:43.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.715+0000 7ff73f7fe640 1 -- 172.21.15.67:0/3316667436 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7ff7400e4280 con 0x7ff748107f90 2024-07-27T03:14:43.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.715+0000 7ff74e487640 1 --2- 172.21.15.67:0/3316667436 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff7340b3d10 0x7ff7340b6220 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff744030550 tx=0x7ff744005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:43.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.715+0000 7ff74dc86640 1 --2- 172.21.15.67:0/3316667436 >> [v2:172.21.15.149:6816/2289624404,v1:172.21.15.149:6817/2289624404] conn(0x7ff7340bf0b0 0x7ff7340c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:43.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.716+0000 7ff74dc86640 1 --2- 172.21.15.67:0/3316667436 >> [v2:172.21.15.149:6816/2289624404,v1:172.21.15.149:6817/2289624404] conn(0x7ff7340bf0b0 0x7ff7340c15a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:43.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.717+0000 7ff73f7fe640 1 -- 172.21.15.67:0/3316667436 <== osd.7 v2:172.21.15.149:6816/2289624404 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff7340c1c70 con 0x7ff7340bf0b0 2024-07-27T03:14:43.733 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.730+0000 7ff7506eb640 1 -- 172.21.15.67:0/3316667436 --> [v2:172.21.15.149:6816/2289624404,v1:172.21.15.149:6817/2289624404] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff748063a40 con 0x7ff7340bf0b0 2024-07-27T03:14:43.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.731+0000 7ff73f7fe640 1 -- 172.21.15.67:0/3316667436 <== osd.7 v2:172.21.15.149:6816/2289624404 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff748063a40 con 0x7ff7340bf0b0 2024-07-27T03:14:43.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.731+0000 7ff7506eb640 1 -- 172.21.15.67:0/3316667436 >> [v2:172.21.15.149:6816/2289624404,v1:172.21.15.149:6817/2289624404] conn(0x7ff7340bf0b0 msgr2=0x7ff7340c15a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:43.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.731+0000 7ff7506eb640 1 --2- 172.21.15.67:0/3316667436 >> [v2:172.21.15.149:6816/2289624404,v1:172.21.15.149:6817/2289624404] conn(0x7ff7340bf0b0 0x7ff7340c15a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:43.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.732+0000 7ff7506eb640 1 -- 172.21.15.67:0/3316667436 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff7340b3d10 msgr2=0x7ff7340b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:43.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.732+0000 7ff7506eb640 1 --2- 172.21.15.67:0/3316667436 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff7340b3d10 0x7ff7340b6220 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff744030550 tx=0x7ff744005d50 comp rx=0 tx=0).stop 2024-07-27T03:14:43.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.732+0000 7ff7506eb640 1 -- 172.21.15.67:0/3316667436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff748107f90 msgr2=0x7ff74819d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:43.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.732+0000 7ff7506eb640 1 --2- 172.21.15.67:0/3316667436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff748107f90 0x7ff74819d640 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff74000b970 tx=0x7ff74000be30 comp rx=0 tx=0).stop 2024-07-27T03:14:43.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.732+0000 7ff74ec88640 1 -- 172.21.15.67:0/3316667436 reap_dead start 2024-07-27T03:14:43.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.732+0000 7ff7506eb640 1 -- 172.21.15.67:0/3316667436 shutdown_connections 2024-07-27T03:14:43.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.732+0000 7ff7506eb640 1 -- 172.21.15.67:0/3316667436 >> 172.21.15.67:0/3316667436 conn(0x7ff748079910 msgr2=0x7ff7481056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:43.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.732+0000 7ff7506eb640 1 -- 172.21.15.67:0/3316667436 shutdown_connections 2024-07-27T03:14:43.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:43.732+0000 7ff7506eb640 1 -- 172.21.15.67:0/3316667436 wait complete. 2024-07-27T03:14:44.163 INFO:teuthology.orchestra.run.smithi067.stdout:55834574936 2024-07-27T03:14:44.164 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-07-27T03:14:44.781 INFO:teuthology.orchestra.run.smithi067.stdout:231928234039 2024-07-27T03:14:44.782 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-07-27T03:14:44.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:44 smithi067 ceph-mon[32814]: pgmap v375: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:14:44.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:44 smithi175 ceph-mon[37125]: pgmap v375: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:14:45.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:44 smithi149 ceph-mon[37173]: pgmap v375: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:14:45.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.366+0000 7f4b49343640 1 -- 172.21.15.67:0/4094699285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b441024f0 msgr2=0x7f4b44102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:45.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.366+0000 7f4b49343640 1 --2- 172.21.15.67:0/4094699285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b441024f0 0x7f4b44102900 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009900 tx=0x7f4b3402a410 comp rx=0 tx=0).stop 2024-07-27T03:14:45.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.366+0000 7f4b49343640 1 -- 172.21.15.67:0/4094699285 shutdown_connections 2024-07-27T03:14:45.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.366+0000 7f4b49343640 1 --2- 172.21.15.67:0/4094699285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b44108780 0x7f4b4410efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:45.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.366+0000 7f4b49343640 1 --2- 172.21.15.67:0/4094699285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b44107e30 0x7f4b44108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:45.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.366+0000 7f4b49343640 1 --2- 172.21.15.67:0/4094699285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b441024f0 0x7f4b44102900 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:45.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.366+0000 7f4b49343640 1 -- 172.21.15.67:0/4094699285 >> 172.21.15.67:0/4094699285 conn(0x7f4b440791b0 msgr2=0x7f4b440795c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:45.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.366+0000 7f4b49343640 1 -- 172.21.15.67:0/4094699285 shutdown_connections 2024-07-27T03:14:45.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.366+0000 7f4b49343640 1 -- 172.21.15.67:0/4094699285 wait complete. 2024-07-27T03:14:45.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.366+0000 7f4b49343640 1 Processor -- start 2024-07-27T03:14:45.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.367+0000 7f4b49343640 1 -- start start 2024-07-27T03:14:45.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.367+0000 7f4b49343640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b441024f0 0x7f4b44199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:45.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.367+0000 7f4b49343640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b44107e30 0x7f4b4419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:45.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.367+0000 7f4b49343640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b44108780 0x7f4b4419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:45.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.367+0000 7f4b49343640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4b44112400 con 0x7f4b44107e30 2024-07-27T03:14:45.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.367+0000 7f4b49343640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4b44112280 con 0x7f4b44108780 2024-07-27T03:14:45.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.367+0000 7f4b49343640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4b44112580 con 0x7f4b441024f0 2024-07-27T03:14:45.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.367+0000 7f4b427fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b44107e30 0x7f4b4419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.367+0000 7f4b427fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b44107e30 0x7f4b4419a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57330/0 (socket says 172.21.15.67:57330) 2024-07-27T03:14:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.367+0000 7f4b427fc640 1 -- 172.21.15.67:0/404537137 learned_addr learned my addr 172.21.15.67:0/404537137 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:14:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.367+0000 7f4b437fe640 1 --2- 172.21.15.67:0/404537137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b44108780 0x7f4b4419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.367+0000 7f4b42ffd640 1 --2- 172.21.15.67:0/404537137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b441024f0 0x7f4b44199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.367+0000 7f4b427fc640 1 -- 172.21.15.67:0/404537137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b441024f0 msgr2=0x7f4b44199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.367+0000 7f4b427fc640 1 --2- 172.21.15.67:0/404537137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b441024f0 0x7f4b44199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.367+0000 7f4b427fc640 1 -- 172.21.15.67:0/404537137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b44108780 msgr2=0x7f4b4419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.367+0000 7f4b427fc640 1 --2- 172.21.15.67:0/404537137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b44108780 0x7f4b4419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.367+0000 7f4b427fc640 1 -- 172.21.15.67:0/404537137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4419fe70 con 0x7f4b44107e30 2024-07-27T03:14:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.367+0000 7f4b42ffd640 1 --2- 172.21.15.67:0/404537137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b441024f0 0x7f4b44199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:14:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.368+0000 7f4b427fc640 1 --2- 172.21.15.67:0/404537137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b44107e30 0x7f4b4419a4a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800d740 tx=0x7f4b3800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.368+0000 7f4b2bfff640 1 -- 172.21.15.67:0/404537137 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38017070 con 0x7f4b44107e30 2024-07-27T03:14:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.368+0000 7f4b2bfff640 1 -- 172.21.15.67:0/404537137 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b3800de50 con 0x7f4b44107e30 2024-07-27T03:14:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.368+0000 7f4b49343640 1 -- 172.21.15.67:0/404537137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b441a0160 con 0x7f4b44107e30 2024-07-27T03:14:45.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.368+0000 7f4b2bfff640 1 -- 172.21.15.67:0/404537137 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38004ce0 con 0x7f4b44107e30 2024-07-27T03:14:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.368+0000 7f4b49343640 1 -- 172.21.15.67:0/404537137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b441a68c0 con 0x7f4b44107e30 2024-07-27T03:14:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.369+0000 7f4b49343640 1 -- 172.21.15.67:0/404537137 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4b44106120 con 0x7f4b44107e30 2024-07-27T03:14:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.370+0000 7f4b2bfff640 1 -- 172.21.15.67:0/404537137 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f4b38004e80 con 0x7f4b44107e30 2024-07-27T03:14:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.371+0000 7f4b2bfff640 1 --2- 172.21.15.67:0/404537137 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4b2c0b3c60 0x7f4b2c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.371+0000 7f4b42ffd640 1 --2- 172.21.15.67:0/404537137 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4b2c0b3c60 0x7f4b2c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.371+0000 7f4b2bfff640 1 -- 172.21.15.67:0/404537137 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f4b380e3fb0 con 0x7f4b44107e30 2024-07-27T03:14:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.371+0000 7f4b2bfff640 1 --2- 172.21.15.67:0/404537137 >> [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693] conn(0x7f4b2c0bf000 0x7f4b2c0c14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.371+0000 7f4b2bfff640 1 -- 172.21.15.67:0/404537137 --> [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4b38016ec0 con 0x7f4b2c0bf000 2024-07-27T03:14:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.371+0000 7f4b2bfff640 1 -- 172.21.15.67:0/404537137 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f4b380e4400 con 0x7f4b44107e30 2024-07-27T03:14:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.372+0000 7f4b437fe640 1 --2- 172.21.15.67:0/404537137 >> [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693] conn(0x7f4b2c0bf000 0x7f4b2c0c14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.372+0000 7f4b42ffd640 1 --2- 172.21.15.67:0/404537137 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4b2c0b3c60 0x7f4b2c0b6170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4b340096c0 tx=0x7f4b34031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.372+0000 7f4b437fe640 1 --2- 172.21.15.67:0/404537137 >> [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693] conn(0x7f4b2c0bf000 0x7f4b2c0c14f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:45.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.373+0000 7f4b2bfff640 1 -- 172.21.15.67:0/404537137 <== osd.3 v2:172.21.15.67:6810/3159144693 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4b38016ec0 con 0x7f4b2c0bf000 2024-07-27T03:14:45.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.387+0000 7f4b49343640 1 -- 172.21.15.67:0/404537137 --> [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4b4406ae00 con 0x7f4b2c0bf000 2024-07-27T03:14:45.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.387+0000 7f4b2bfff640 1 -- 172.21.15.67:0/404537137 <== osd.3 v2:172.21.15.67:6810/3159144693 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4b4406ae00 con 0x7f4b2c0bf000 2024-07-27T03:14:45.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.387+0000 7f4b49343640 1 -- 172.21.15.67:0/404537137 >> [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693] conn(0x7f4b2c0bf000 msgr2=0x7f4b2c0c14f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:45.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.387+0000 7f4b49343640 1 --2- 172.21.15.67:0/404537137 >> [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693] conn(0x7f4b2c0bf000 0x7f4b2c0c14f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:45.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.387+0000 7f4b49343640 1 -- 172.21.15.67:0/404537137 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4b2c0b3c60 msgr2=0x7f4b2c0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:45.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.387+0000 7f4b49343640 1 --2- 172.21.15.67:0/404537137 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4b2c0b3c60 0x7f4b2c0b6170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4b340096c0 tx=0x7f4b34031400 comp rx=0 tx=0).stop 2024-07-27T03:14:45.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.387+0000 7f4b49343640 1 -- 172.21.15.67:0/404537137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b44107e30 msgr2=0x7f4b4419a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:45.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.387+0000 7f4b49343640 1 --2- 172.21.15.67:0/404537137 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4b44107e30 0x7f4b4419a4a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800d740 tx=0x7f4b3800dc00 comp rx=0 tx=0).stop 2024-07-27T03:14:45.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.387+0000 7f4b437fe640 1 -- 172.21.15.67:0/404537137 reap_dead start 2024-07-27T03:14:45.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.387+0000 7f4b49343640 1 -- 172.21.15.67:0/404537137 shutdown_connections 2024-07-27T03:14:45.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.387+0000 7f4b49343640 1 -- 172.21.15.67:0/404537137 >> 172.21.15.67:0/404537137 conn(0x7f4b440791b0 msgr2=0x7f4b44101550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:45.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.388+0000 7f4b49343640 1 -- 172.21.15.67:0/404537137 shutdown_connections 2024-07-27T03:14:45.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:45.388+0000 7f4b49343640 1 -- 172.21.15.67:0/404537137 wait complete. 2024-07-27T03:14:46.485 INFO:teuthology.orchestra.run.smithi067.stdout:206158430268 2024-07-27T03:14:46.486 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-07-27T03:14:46.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.651+0000 7f49e3f21640 1 -- 172.21.15.67:0/121460422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49dc108f20 msgr2=0x7f49dc109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:46.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.651+0000 7f49e3f21640 1 --2- 172.21.15.67:0/121460422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49dc108f20 0x7f49dc109330 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009900 tx=0x7f49d802a410 comp rx=0 tx=0).stop 2024-07-27T03:14:46.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.651+0000 7f49e3f21640 1 -- 172.21.15.67:0/121460422 shutdown_connections 2024-07-27T03:14:46.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.651+0000 7f49e3f21640 1 --2- 172.21.15.67:0/121460422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49dc109870 0x7f49dc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:46.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.651+0000 7f49e3f21640 1 --2- 172.21.15.67:0/121460422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49dc108f20 0x7f49dc109330 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:46.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.651+0000 7f49e3f21640 1 --2- 172.21.15.67:0/121460422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49dc103630 0x7f49dc103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:46.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.651+0000 7f49e3f21640 1 -- 172.21.15.67:0/121460422 >> 172.21.15.67:0/121460422 conn(0x7f49dc07b380 msgr2=0x7f49dc07b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:46.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.651+0000 7f49e3f21640 1 -- 172.21.15.67:0/121460422 shutdown_connections 2024-07-27T03:14:46.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.651+0000 7f49e3f21640 1 -- 172.21.15.67:0/121460422 wait complete. 2024-07-27T03:14:46.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.652+0000 7f49e3f21640 1 Processor -- start 2024-07-27T03:14:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.652+0000 7f49e3f21640 1 -- start start 2024-07-27T03:14:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.652+0000 7f49e3f21640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49dc103630 0x7f49dc199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.652+0000 7f49e3f21640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49dc108f20 0x7f49dc19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.652+0000 7f49e3f21640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49dc109870 0x7f49dc19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.652+0000 7f49e3f21640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc1134e0 con 0x7f49dc108f20 2024-07-27T03:14:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.652+0000 7f49e3f21640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc113360 con 0x7f49dc109870 2024-07-27T03:14:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.652+0000 7f49e3f21640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc113660 con 0x7f49dc103630 2024-07-27T03:14:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.653+0000 7f49e14bc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49dc108f20 0x7f49dc19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.653+0000 7f49e14bc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49dc108f20 0x7f49dc19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57352/0 (socket says 172.21.15.67:57352) 2024-07-27T03:14:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.653+0000 7f49e14bc640 1 -- 172.21.15.67:0/3314961811 learned_addr learned my addr 172.21.15.67:0/3314961811 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:14:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.653+0000 7f49e1cbd640 1 --2- 172.21.15.67:0/3314961811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49dc103630 0x7f49dc199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:46.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.653+0000 7f49e24be640 1 --2- 172.21.15.67:0/3314961811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49dc109870 0x7f49dc19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:46.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.653+0000 7f49e14bc640 1 -- 172.21.15.67:0/3314961811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49dc103630 msgr2=0x7f49dc199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:46.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.653+0000 7f49e14bc640 1 --2- 172.21.15.67:0/3314961811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49dc103630 0x7f49dc199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:46.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.653+0000 7f49e14bc640 1 -- 172.21.15.67:0/3314961811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49dc109870 msgr2=0x7f49dc19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:46.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.653+0000 7f49e14bc640 1 --2- 172.21.15.67:0/3314961811 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f49dc109870 0x7f49dc19f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:46.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.653+0000 7f49e14bc640 1 -- 172.21.15.67:0/3314961811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49dc19fe60 con 0x7f49dc108f20 2024-07-27T03:14:46.659 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.653+0000 7f49e14bc640 1 --2- 172.21.15.67:0/3314961811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49dc108f20 0x7f49dc19a490 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f49d802a930 tx=0x7f49d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:46.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.654+0000 7f49d2ffd640 1 -- 172.21.15.67:0/3314961811 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d803f030 con 0x7f49dc108f20 2024-07-27T03:14:46.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.654+0000 7f49d2ffd640 1 -- 172.21.15.67:0/3314961811 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49d8003aa0 con 0x7f49dc108f20 2024-07-27T03:14:46.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.654+0000 7f49e3f21640 1 -- 172.21.15.67:0/3314961811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49dc1a00f0 con 0x7f49dc108f20 2024-07-27T03:14:46.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.654+0000 7f49d2ffd640 1 -- 172.21.15.67:0/3314961811 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8035070 con 0x7f49dc108f20 2024-07-27T03:14:46.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.654+0000 7f49e3f21640 1 -- 172.21.15.67:0/3314961811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49dc1a68b0 con 0x7f49dc108f20 2024-07-27T03:14:46.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.655+0000 7f49e3f21640 1 -- 172.21.15.67:0/3314961811 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f49dc063580 con 0x7f49dc108f20 2024-07-27T03:14:46.660 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.656+0000 7f49d2ffd640 1 -- 172.21.15.67:0/3314961811 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f49d8042030 con 0x7f49dc108f20 2024-07-27T03:14:46.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.657+0000 7f49d2ffd640 1 --2- 172.21.15.67:0/3314961811 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f49c80b3ce0 0x7f49c80b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:46.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.658+0000 7f49e1cbd640 1 --2- 172.21.15.67:0/3314961811 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f49c80b3ce0 0x7f49c80b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:46.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.658+0000 7f49d2ffd640 1 -- 172.21.15.67:0/3314961811 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f49d8104bb0 con 0x7f49dc108f20 2024-07-27T03:14:46.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.658+0000 7f49d2ffd640 1 --2- 172.21.15.67:0/3314961811 >> [v2:172.21.15.175:6808/2291204784,v1:172.21.15.175:6809/2291204784] conn(0x7f49c80bf080 0x7f49c80c1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:46.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.658+0000 7f49d2ffd640 1 -- 172.21.15.67:0/3314961811 --> [v2:172.21.15.175:6808/2291204784,v1:172.21.15.175:6809/2291204784] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49d8040e70 con 0x7f49c80bf080 2024-07-27T03:14:46.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.658+0000 7f49d2ffd640 1 -- 172.21.15.67:0/3314961811 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f49d8105000 con 0x7f49dc108f20 2024-07-27T03:14:46.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.658+0000 7f49e1cbd640 1 --2- 172.21.15.67:0/3314961811 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f49c80b3ce0 0x7f49c80b61f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00bbe0 tx=0x7f49cc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:46.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.658+0000 7f49e24be640 1 --2- 172.21.15.67:0/3314961811 >> [v2:172.21.15.175:6808/2291204784,v1:172.21.15.175:6809/2291204784] conn(0x7f49c80bf080 0x7f49c80c1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:46.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.659+0000 7f49e24be640 1 --2- 172.21.15.67:0/3314961811 >> [v2:172.21.15.175:6808/2291204784,v1:172.21.15.175:6809/2291204784] conn(0x7f49c80bf080 0x7f49c80c1570 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:46.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.660+0000 7f49d2ffd640 1 -- 172.21.15.67:0/3314961811 <== osd.5 v2:172.21.15.175:6808/2291204784 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f49d8040e70 con 0x7f49c80bf080 2024-07-27T03:14:46.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.687+0000 7f49e3f21640 1 -- 172.21.15.67:0/3314961811 --> [v2:172.21.15.175:6808/2291204784,v1:172.21.15.175:6809/2291204784] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f49dc0639b0 con 0x7f49c80bf080 2024-07-27T03:14:46.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.688+0000 7f49d2ffd640 1 -- 172.21.15.67:0/3314961811 <== osd.5 v2:172.21.15.175:6808/2291204784 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f49dc0639b0 con 0x7f49c80bf080 2024-07-27T03:14:46.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.689+0000 7f49e3f21640 1 -- 172.21.15.67:0/3314961811 >> [v2:172.21.15.175:6808/2291204784,v1:172.21.15.175:6809/2291204784] conn(0x7f49c80bf080 msgr2=0x7f49c80c1570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:46.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.689+0000 7f49e3f21640 1 --2- 172.21.15.67:0/3314961811 >> [v2:172.21.15.175:6808/2291204784,v1:172.21.15.175:6809/2291204784] conn(0x7f49c80bf080 0x7f49c80c1570 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:46.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.689+0000 7f49e3f21640 1 -- 172.21.15.67:0/3314961811 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f49c80b3ce0 msgr2=0x7f49c80b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:46.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.689+0000 7f49e3f21640 1 --2- 172.21.15.67:0/3314961811 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f49c80b3ce0 0x7f49c80b61f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00bbe0 tx=0x7f49cc00b390 comp rx=0 tx=0).stop 2024-07-27T03:14:46.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.689+0000 7f49e3f21640 1 -- 172.21.15.67:0/3314961811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49dc108f20 msgr2=0x7f49dc19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:46.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.689+0000 7f49e3f21640 1 --2- 172.21.15.67:0/3314961811 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f49dc108f20 0x7f49dc19a490 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f49d802a930 tx=0x7f49d8004270 comp rx=0 tx=0).stop 2024-07-27T03:14:46.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.689+0000 7f49e24be640 1 -- 172.21.15.67:0/3314961811 reap_dead start 2024-07-27T03:14:46.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.689+0000 7f49e3f21640 1 -- 172.21.15.67:0/3314961811 shutdown_connections 2024-07-27T03:14:46.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.689+0000 7f49e3f21640 1 -- 172.21.15.67:0/3314961811 >> 172.21.15.67:0/3314961811 conn(0x7f49dc07b380 msgr2=0x7f49dc107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:46.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.689+0000 7f49e3f21640 1 -- 172.21.15.67:0/3314961811 shutdown_connections 2024-07-27T03:14:46.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:46.689+0000 7f49e3f21640 1 -- 172.21.15.67:0/3314961811 wait complete. 2024-07-27T03:14:46.702 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:46 smithi067 ceph-mon[32814]: pgmap v376: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-07-27T03:14:46.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:46 smithi175 ceph-mon[37125]: pgmap v376: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-07-27T03:14:47.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:46 smithi149 ceph-mon[37173]: pgmap v376: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-07-27T03:14:47.216 INFO:teuthology.orchestra.run.smithi067.stdout:34359738461 2024-07-27T03:14:47.216 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-07-27T03:14:47.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.840+0000 7fcf4128f640 1 -- 172.21.15.67:0/2972763795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf3c102510 msgr2=0x7fcf3c102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:47.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.840+0000 7fcf4128f640 1 --2- 172.21.15.67:0/2972763795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf3c102510 0x7fcf3c102920 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009ca0 tx=0x7fcf2c02a8c0 comp rx=0 tx=0).stop 2024-07-27T03:14:47.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.840+0000 7fcf4128f640 1 -- 172.21.15.67:0/2972763795 shutdown_connections 2024-07-27T03:14:47.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.840+0000 7fcf4128f640 1 --2- 172.21.15.67:0/2972763795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf3c1087a0 0x7fcf3c10efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:47.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.840+0000 7fcf4128f640 1 --2- 172.21.15.67:0/2972763795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf3c107e50 0x7fcf3c108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:47.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.840+0000 7fcf4128f640 1 --2- 172.21.15.67:0/2972763795 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf3c102510 0x7fcf3c102920 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:47.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.840+0000 7fcf4128f640 1 -- 172.21.15.67:0/2972763795 >> 172.21.15.67:0/2972763795 conn(0x7fcf3c079250 msgr2=0x7fcf3c079660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:47.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.840+0000 7fcf4128f640 1 -- 172.21.15.67:0/2972763795 shutdown_connections 2024-07-27T03:14:47.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.840+0000 7fcf4128f640 1 -- 172.21.15.67:0/2972763795 wait complete. 2024-07-27T03:14:47.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.841+0000 7fcf4128f640 1 Processor -- start 2024-07-27T03:14:47.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.841+0000 7fcf4128f640 1 -- start start 2024-07-27T03:14:47.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.841+0000 7fcf4128f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf3c102510 0x7fcf3c198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:47.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.841+0000 7fcf4128f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf3c107e50 0x7fcf3c199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:47.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.841+0000 7fcf4128f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf3c1087a0 0x7fcf3c199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:47.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.841+0000 7fcf4128f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c112400 con 0x7fcf3c102510 2024-07-27T03:14:47.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.841+0000 7fcf4128f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c112280 con 0x7fcf3c107e50 2024-07-27T03:14:47.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.841+0000 7fcf4128f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c112580 con 0x7fcf3c1087a0 2024-07-27T03:14:47.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.841+0000 7fcf3affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf3c102510 0x7fcf3c198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:47.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.841+0000 7fcf3affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf3c102510 0x7fcf3c198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57372/0 (socket says 172.21.15.67:57372) 2024-07-27T03:14:47.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.841+0000 7fcf3affd640 1 -- 172.21.15.67:0/3333388965 learned_addr learned my addr 172.21.15.67:0/3333388965 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:14:47.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.842+0000 7fcf3a7fc640 1 --2- 172.21.15.67:0/3333388965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf3c107e50 0x7fcf3c199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:47.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.842+0000 7fcf3b7fe640 1 --2- 172.21.15.67:0/3333388965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf3c1087a0 0x7fcf3c199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:47.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.842+0000 7fcf3affd640 1 -- 172.21.15.67:0/3333388965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf3c1087a0 msgr2=0x7fcf3c199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:47.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.842+0000 7fcf3affd640 1 --2- 172.21.15.67:0/3333388965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf3c1087a0 0x7fcf3c199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:47.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.842+0000 7fcf3affd640 1 -- 172.21.15.67:0/3333388965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf3c107e50 msgr2=0x7fcf3c199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:47.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.842+0000 7fcf3affd640 1 --2- 172.21.15.67:0/3333388965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf3c107e50 0x7fcf3c199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:47.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.842+0000 7fcf3affd640 1 -- 172.21.15.67:0/3333388965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3c1a8240 con 0x7fcf3c102510 2024-07-27T03:14:47.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.842+0000 7fcf3a7fc640 1 --2- 172.21.15.67:0/3333388965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf3c107e50 0x7fcf3c199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:14:47.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.842+0000 7fcf3affd640 1 --2- 172.21.15.67:0/3333388965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf3c102510 0x7fcf3c198bc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c006010 tx=0x7fcf2c0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:47.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.842+0000 7fcf33fff640 1 -- 172.21.15.67:0/3333388965 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c003bf0 con 0x7fcf3c102510 2024-07-27T03:14:47.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.842+0000 7fcf33fff640 1 -- 172.21.15.67:0/3333388965 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf2c003d90 con 0x7fcf3c102510 2024-07-27T03:14:47.852 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.842+0000 7fcf4128f640 1 -- 172.21.15.67:0/3333388965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf3c1a8470 con 0x7fcf3c102510 2024-07-27T03:14:47.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.842+0000 7fcf33fff640 1 -- 172.21.15.67:0/3333388965 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c02fc20 con 0x7fcf3c102510 2024-07-27T03:14:47.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.842+0000 7fcf4128f640 1 -- 172.21.15.67:0/3333388965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf3c1a89b0 con 0x7fcf3c102510 2024-07-27T03:14:47.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.843+0000 7fcf4128f640 1 -- 172.21.15.67:0/3333388965 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcf3c063500 con 0x7fcf3c102510 2024-07-27T03:14:47.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.845+0000 7fcf33fff640 1 -- 172.21.15.67:0/3333388965 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7fcf2c0046a0 con 0x7fcf3c102510 2024-07-27T03:14:47.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.845+0000 7fcf33fff640 1 --2- 172.21.15.67:0/3333388965 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcf280b3c60 0x7fcf280b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:47.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.846+0000 7fcf33fff640 1 -- 172.21.15.67:0/3333388965 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fcf2c105670 con 0x7fcf3c102510 2024-07-27T03:14:47.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.846+0000 7fcf3a7fc640 1 --2- 172.21.15.67:0/3333388965 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcf280b3c60 0x7fcf280b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:47.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.846+0000 7fcf33fff640 1 --2- 172.21.15.67:0/3333388965 >> [v2:172.21.15.149:6808/3393503532,v1:172.21.15.149:6809/3393503532] conn(0x7fcf280bf000 0x7fcf280c14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:47.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.846+0000 7fcf33fff640 1 -- 172.21.15.67:0/3333388965 --> [v2:172.21.15.149:6808/3393503532,v1:172.21.15.149:6809/3393503532] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcf2c036e70 con 0x7fcf280bf000 2024-07-27T03:14:47.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.846+0000 7fcf33fff640 1 -- 172.21.15.67:0/3333388965 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fcf2c10c050 con 0x7fcf3c102510 2024-07-27T03:14:47.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.846+0000 7fcf3b7fe640 1 --2- 172.21.15.67:0/3333388965 >> [v2:172.21.15.149:6808/3393503532,v1:172.21.15.149:6809/3393503532] conn(0x7fcf280bf000 0x7fcf280c14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:47.853 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.846+0000 7fcf3a7fc640 1 --2- 172.21.15.67:0/3333388965 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcf280b3c60 0x7fcf280b6170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcf240058c0 tx=0x7fcf24005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:47.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.847+0000 7fcf3b7fe640 1 --2- 172.21.15.67:0/3333388965 >> [v2:172.21.15.149:6808/3393503532,v1:172.21.15.149:6809/3393503532] conn(0x7fcf280bf000 0x7fcf280c14f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:47.854 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.848+0000 7fcf33fff640 1 -- 172.21.15.67:0/3333388965 <== osd.4 v2:172.21.15.149:6808/3393503532 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcf2c036e70 con 0x7fcf280bf000 2024-07-27T03:14:47.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.862+0000 7fcf4128f640 1 -- 172.21.15.67:0/3333388965 --> [v2:172.21.15.149:6808/3393503532,v1:172.21.15.149:6809/3393503532] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcf3c063900 con 0x7fcf280bf000 2024-07-27T03:14:47.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.862+0000 7fcf33fff640 1 -- 172.21.15.67:0/3333388965 <== osd.4 v2:172.21.15.149:6808/3393503532 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcf3c063900 con 0x7fcf280bf000 2024-07-27T03:14:47.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.863+0000 7fcf4128f640 1 -- 172.21.15.67:0/3333388965 >> [v2:172.21.15.149:6808/3393503532,v1:172.21.15.149:6809/3393503532] conn(0x7fcf280bf000 msgr2=0x7fcf280c14f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:47.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.863+0000 7fcf4128f640 1 --2- 172.21.15.67:0/3333388965 >> [v2:172.21.15.149:6808/3393503532,v1:172.21.15.149:6809/3393503532] conn(0x7fcf280bf000 0x7fcf280c14f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:47.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.863+0000 7fcf4128f640 1 -- 172.21.15.67:0/3333388965 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcf280b3c60 msgr2=0x7fcf280b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:47.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.863+0000 7fcf4128f640 1 --2- 172.21.15.67:0/3333388965 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcf280b3c60 0x7fcf280b6170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcf240058c0 tx=0x7fcf24005850 comp rx=0 tx=0).stop 2024-07-27T03:14:47.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.863+0000 7fcf4128f640 1 -- 172.21.15.67:0/3333388965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf3c102510 msgr2=0x7fcf3c198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:47.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.863+0000 7fcf4128f640 1 --2- 172.21.15.67:0/3333388965 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf3c102510 0x7fcf3c198bc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c006010 tx=0x7fcf2c0319f0 comp rx=0 tx=0).stop 2024-07-27T03:14:47.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.863+0000 7fcf3b7fe640 1 -- 172.21.15.67:0/3333388965 reap_dead start 2024-07-27T03:14:47.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.863+0000 7fcf4128f640 1 -- 172.21.15.67:0/3333388965 shutdown_connections 2024-07-27T03:14:47.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.863+0000 7fcf4128f640 1 -- 172.21.15.67:0/3333388965 >> 172.21.15.67:0/3333388965 conn(0x7fcf3c079250 msgr2=0x7fcf3c100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:47.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.863+0000 7fcf4128f640 1 -- 172.21.15.67:0/3333388965 shutdown_connections 2024-07-27T03:14:47.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:47.863+0000 7fcf4128f640 1 -- 172.21.15.67:0/3333388965 wait complete. 2024-07-27T03:14:48.286 INFO:teuthology.orchestra.run.smithi067.stdout:158913790022 2024-07-27T03:14:48.286 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-07-27T03:14:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.469+0000 7f65df557640 1 -- 172.21.15.67:0/2560790119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d8108760 msgr2=0x7f65d810ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.469+0000 7f65df557640 1 --2- 172.21.15.67:0/2560790119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d8108760 0x7f65d810ef90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f65d0009ca0 tx=0x7f65d002a8c0 comp rx=0 tx=0).stop 2024-07-27T03:14:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.469+0000 7f65df557640 1 -- 172.21.15.67:0/2560790119 shutdown_connections 2024-07-27T03:14:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.469+0000 7f65df557640 1 --2- 172.21.15.67:0/2560790119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d8108760 0x7f65d810ef90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.469+0000 7f65df557640 1 --2- 172.21.15.67:0/2560790119 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65d8107e10 0x7f65d8108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.469+0000 7f65df557640 1 --2- 172.21.15.67:0/2560790119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65d81024d0 0x7f65d81028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.469+0000 7f65df557640 1 -- 172.21.15.67:0/2560790119 >> 172.21.15.67:0/2560790119 conn(0x7f65d80791a0 msgr2=0x7f65d80795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:48.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.469+0000 7f65df557640 1 -- 172.21.15.67:0/2560790119 shutdown_connections 2024-07-27T03:14:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.469+0000 7f65df557640 1 -- 172.21.15.67:0/2560790119 wait complete. 2024-07-27T03:14:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.470+0000 7f65df557640 1 Processor -- start 2024-07-27T03:14:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.470+0000 7f65df557640 1 -- start start 2024-07-27T03:14:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.470+0000 7f65df557640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65d81024d0 0x7f65d819acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.470+0000 7f65df557640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65d8107e10 0x7f65d819b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.470+0000 7f65df557640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d8108760 0x7f65d819b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.470+0000 7f65df557640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f65d8112430 con 0x7f65d8108760 2024-07-27T03:14:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.470+0000 7f65df557640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f65d81122b0 con 0x7f65d81024d0 2024-07-27T03:14:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.470+0000 7f65df557640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65d81125b0 con 0x7f65d8107e10 2024-07-27T03:14:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.471+0000 7f65ddaf4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d8108760 0x7f65d819b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:48.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.471+0000 7f65ddaf4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d8108760 0x7f65d819b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57386/0 (socket says 172.21.15.67:57386) 2024-07-27T03:14:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.471+0000 7f65dd2f3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65d81024d0 0x7f65d819acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.471+0000 7f65ddaf4640 1 -- 172.21.15.67:0/1447482398 learned_addr learned my addr 172.21.15.67:0/1447482398 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:14:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.471+0000 7f65dcaf2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65d8107e10 0x7f65d819b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.471+0000 7f65ddaf4640 1 -- 172.21.15.67:0/1447482398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65d8107e10 msgr2=0x7f65d819b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.471+0000 7f65ddaf4640 1 --2- 172.21.15.67:0/1447482398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65d8107e10 0x7f65d819b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.471+0000 7f65ddaf4640 1 -- 172.21.15.67:0/1447482398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65d81024d0 msgr2=0x7f65d819acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.471+0000 7f65ddaf4640 1 --2- 172.21.15.67:0/1447482398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65d81024d0 0x7f65d819acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.471+0000 7f65ddaf4640 1 -- 172.21.15.67:0/1447482398 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d81a5f50 con 0x7f65d8108760 2024-07-27T03:14:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.471+0000 7f65dcaf2640 1 --2- 172.21.15.67:0/1447482398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65d8107e10 0x7f65d819b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:14:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.471+0000 7f65ddaf4640 1 --2- 172.21.15.67:0/1447482398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d8108760 0x7f65d819b740 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f65d0009430 tx=0x7f65d00317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.472+0000 7f65ca7fc640 1 -- 172.21.15.67:0/1447482398 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d0003b70 con 0x7f65d8108760 2024-07-27T03:14:48.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.472+0000 7f65ca7fc640 1 -- 172.21.15.67:0/1447482398 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65d0003d10 con 0x7f65d8108760 2024-07-27T03:14:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.472+0000 7f65df557640 1 -- 172.21.15.67:0/1447482398 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d81a61e0 con 0x7f65d8108760 2024-07-27T03:14:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.472+0000 7f65ca7fc640 1 -- 172.21.15.67:0/1447482398 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d002fa30 con 0x7f65d8108760 2024-07-27T03:14:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.472+0000 7f65df557640 1 -- 172.21.15.67:0/1447482398 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d81a6720 con 0x7f65d8108760 2024-07-27T03:14:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.473+0000 7f65df557640 1 -- 172.21.15.67:0/1447482398 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f65d80635c0 con 0x7f65d8108760 2024-07-27T03:14:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.475+0000 7f65ca7fc640 1 -- 172.21.15.67:0/1447482398 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f65d002fbd0 con 0x7f65d8108760 2024-07-27T03:14:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.476+0000 7f65ca7fc640 1 --2- 172.21.15.67:0/1447482398 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f65c40b3c90 0x7f65c40b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.476+0000 7f65dd2f3640 1 --2- 172.21.15.67:0/1447482398 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f65c40b3c90 0x7f65c40b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:48.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.476+0000 7f65ca7fc640 1 -- 172.21.15.67:0/1447482398 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f65d01056d0 con 0x7f65d8108760 2024-07-27T03:14:48.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.476+0000 7f65ca7fc640 1 --2- 172.21.15.67:0/1447482398 >> [v2:172.21.15.175:6800/3770940078,v1:172.21.15.175:6801/3770940078] conn(0x7f65c40bf030 0x7f65c40c1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:48.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.476+0000 7f65ca7fc640 1 -- 172.21.15.67:0/1447482398 --> [v2:172.21.15.175:6800/3770940078,v1:172.21.15.175:6801/3770940078] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f65d003fec0 con 0x7f65c40bf030 2024-07-27T03:14:48.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.476+0000 7f65ca7fc640 1 -- 172.21.15.67:0/1447482398 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f65d0105ac0 con 0x7f65d8108760 2024-07-27T03:14:48.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.477+0000 7f65dcaf2640 1 --2- 172.21.15.67:0/1447482398 >> [v2:172.21.15.175:6800/3770940078,v1:172.21.15.175:6801/3770940078] conn(0x7f65c40bf030 0x7f65c40c1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.477+0000 7f65dd2f3640 1 --2- 172.21.15.67:0/1447482398 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f65c40b3c90 0x7f65c40b61a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f65d806b430 tx=0x7f65cc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.477+0000 7f65dcaf2640 1 --2- 172.21.15.67:0/1447482398 >> [v2:172.21.15.175:6800/3770940078,v1:172.21.15.175:6801/3770940078] conn(0x7f65c40bf030 0x7f65c40c1520 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.478+0000 7f65ca7fc640 1 -- 172.21.15.67:0/1447482398 <== osd.2 v2:172.21.15.175:6800/3770940078 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f65d003fec0 con 0x7f65c40bf030 2024-07-27T03:14:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.492+0000 7f65df557640 1 -- 172.21.15.67:0/1447482398 --> [v2:172.21.15.175:6800/3770940078,v1:172.21.15.175:6801/3770940078] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f65d8063a40 con 0x7f65c40bf030 2024-07-27T03:14:48.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.493+0000 7f65ca7fc640 1 -- 172.21.15.67:0/1447482398 <== osd.2 v2:172.21.15.175:6800/3770940078 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f65d8063a40 con 0x7f65c40bf030 2024-07-27T03:14:48.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.493+0000 7f65df557640 1 -- 172.21.15.67:0/1447482398 >> [v2:172.21.15.175:6800/3770940078,v1:172.21.15.175:6801/3770940078] conn(0x7f65c40bf030 msgr2=0x7f65c40c1520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:48.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.493+0000 7f65df557640 1 --2- 172.21.15.67:0/1447482398 >> [v2:172.21.15.175:6800/3770940078,v1:172.21.15.175:6801/3770940078] conn(0x7f65c40bf030 0x7f65c40c1520 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.493+0000 7f65df557640 1 -- 172.21.15.67:0/1447482398 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f65c40b3c90 msgr2=0x7f65c40b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.493+0000 7f65df557640 1 --2- 172.21.15.67:0/1447482398 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f65c40b3c90 0x7f65c40b61a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f65d806b430 tx=0x7f65cc00b390 comp rx=0 tx=0).stop 2024-07-27T03:14:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.494+0000 7f65df557640 1 -- 172.21.15.67:0/1447482398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d8108760 msgr2=0x7f65d819b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.494+0000 7f65df557640 1 --2- 172.21.15.67:0/1447482398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65d8108760 0x7f65d819b740 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f65d0009430 tx=0x7f65d00317c0 comp rx=0 tx=0).stop 2024-07-27T03:14:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.494+0000 7f65ddaf4640 1 -- 172.21.15.67:0/1447482398 reap_dead start 2024-07-27T03:14:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.494+0000 7f65df557640 1 -- 172.21.15.67:0/1447482398 shutdown_connections 2024-07-27T03:14:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.494+0000 7f65df557640 1 -- 172.21.15.67:0/1447482398 >> 172.21.15.67:0/1447482398 conn(0x7f65d80791a0 msgr2=0x7f65d8104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.494+0000 7f65df557640 1 -- 172.21.15.67:0/1447482398 shutdown_connections 2024-07-27T03:14:48.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:48.494+0000 7f65df557640 1 -- 172.21.15.67:0/1447482398 wait complete. 2024-07-27T03:14:48.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:48 smithi175 ceph-mon[37125]: pgmap v377: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:14:48.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:48 smithi067 ceph-mon[32814]: pgmap v377: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:14:49.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:48 smithi149 ceph-mon[37173]: pgmap v377: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:14:49.319 INFO:teuthology.orchestra.run.smithi067.stdout:137438953548 2024-07-27T03:14:49.320 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-07-27T03:14:49.733 INFO:teuthology.orchestra.run.smithi067.stdout:309237645353 2024-07-27T03:14:49.733 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-07-27T03:14:50.190 INFO:teuthology.orchestra.run.smithi067.stdout:77309411413 2024-07-27T03:14:50.190 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-07-27T03:14:50.656 INFO:teuthology.orchestra.run.smithi067.stdout:111669149776 2024-07-27T03:14:50.656 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-07-27T03:14:50.733 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:50.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:50 smithi175 ceph-mon[37125]: pgmap v378: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-07-27T03:14:50.904 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:50 smithi067 ceph-mon[32814]: pgmap v378: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-07-27T03:14:51.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:50 smithi149 ceph-mon[37173]: pgmap v378: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-07-27T03:14:51.042 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:51.042 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:51.048 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:51.050 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:51.051 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:51.054 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:51.055 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:51.059 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:51.474 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:51.474 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:14:52.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:52 smithi175 ceph-mon[37125]: pgmap v379: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-27T03:14:52.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:52 smithi067 ceph-mon[32814]: pgmap v379: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-27T03:14:53.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:52 smithi149 ceph-mon[37173]: pgmap v379: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-07-27T03:14:54.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:54 smithi175 ceph-mon[37125]: pgmap v380: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-07-27T03:14:54.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:54 smithi067 ceph-mon[32814]: pgmap v380: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-07-27T03:14:55.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:54 smithi149 ceph-mon[37173]: pgmap v380: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-07-27T03:14:55.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:55 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3310969943' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-07-27T03:14:55.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:55 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3310969943' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-07-27T03:14:56.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:55 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3310969943' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-07-27T03:14:56.200 INFO:teuthology.orchestra.run.smithi067.stdout:257698037813 2024-07-27T03:14:56.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.376+0000 7f743c5f0640 1 -- 172.21.15.67:0/2245288301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7434108760 msgr2=0x7f743410ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:56.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.376+0000 7f743c5f0640 1 --2- 172.21.15.67:0/2245288301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7434108760 0x7f743410ef90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f742c009ca0 tx=0x7f742c02a8c0 comp rx=0 tx=0).stop 2024-07-27T03:14:56.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.376+0000 7f743c5f0640 1 -- 172.21.15.67:0/2245288301 shutdown_connections 2024-07-27T03:14:56.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.376+0000 7f743c5f0640 1 --2- 172.21.15.67:0/2245288301 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7434108760 0x7f743410ef90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:56.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.376+0000 7f743c5f0640 1 --2- 172.21.15.67:0/2245288301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7434107e10 0x7f7434108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:56.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.376+0000 7f743c5f0640 1 --2- 172.21.15.67:0/2245288301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74341024d0 0x7f74341028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:56.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.376+0000 7f743c5f0640 1 -- 172.21.15.67:0/2245288301 >> 172.21.15.67:0/2245288301 conn(0x7f74340791a0 msgr2=0x7f74340795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:56.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.376+0000 7f743c5f0640 1 -- 172.21.15.67:0/2245288301 shutdown_connections 2024-07-27T03:14:56.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.376+0000 7f743c5f0640 1 -- 172.21.15.67:0/2245288301 wait complete. 2024-07-27T03:14:56.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.377+0000 7f743c5f0640 1 Processor -- start 2024-07-27T03:14:56.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.377+0000 7f743c5f0640 1 -- start start 2024-07-27T03:14:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.378+0000 7f743c5f0640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74341024d0 0x7f74341a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.378+0000 7f743c5f0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7434107e10 0x7f74341a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.378+0000 7f743c5f0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7434108760 0x7f74341a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.378+0000 7f743c5f0640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7434112410 con 0x7f7434107e10 2024-07-27T03:14:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.378+0000 7f743c5f0640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7434112290 con 0x7f74341024d0 2024-07-27T03:14:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.378+0000 7f743c5f0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7434112590 con 0x7f7434108760 2024-07-27T03:14:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.378+0000 7f7439b8b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7434107e10 0x7f74341a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.378+0000 7f7439b8b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7434107e10 0x7f74341a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47600/0 (socket says 172.21.15.67:47600) 2024-07-27T03:14:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.378+0000 7f7439b8b640 1 -- 172.21.15.67:0/3310969943 learned_addr learned my addr 172.21.15.67:0/3310969943 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:14:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.378+0000 7f743a38c640 1 --2- 172.21.15.67:0/3310969943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74341024d0 0x7f74341a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:56.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.378+0000 7f743ab8d640 1 --2- 172.21.15.67:0/3310969943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7434108760 0x7f74341a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:56.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.378+0000 7f7439b8b640 1 -- 172.21.15.67:0/3310969943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7434108760 msgr2=0x7f74341a5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:56.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.378+0000 7f7439b8b640 1 --2- 172.21.15.67:0/3310969943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7434108760 0x7f74341a5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:56.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.378+0000 7f7439b8b640 1 -- 172.21.15.67:0/3310969943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74341024d0 msgr2=0x7f74341a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:56.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.378+0000 7f7439b8b640 1 --2- 172.21.15.67:0/3310969943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74341024d0 0x7f74341a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:56.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.378+0000 7f7439b8b640 1 -- 172.21.15.67:0/3310969943 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74341a6420 con 0x7f7434107e10 2024-07-27T03:14:56.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.378+0000 7f743ab8d640 1 --2- 172.21.15.67:0/3310969943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7434108760 0x7f74341a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:14:56.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.379+0000 7f7439b8b640 1 --2- 172.21.15.67:0/3310969943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7434107e10 0x7f74341a0a60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f742400e9c0 tx=0x7f742400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:56.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.379+0000 7f742b7fe640 1 -- 172.21.15.67:0/3310969943 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742400cd80 con 0x7f7434107e10 2024-07-27T03:14:56.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.379+0000 7f743c5f0640 1 -- 172.21.15.67:0/3310969943 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74341a6710 con 0x7f7434107e10 2024-07-27T03:14:56.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.379+0000 7f742b7fe640 1 -- 172.21.15.67:0/3310969943 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74240054d0 con 0x7f7434107e10 2024-07-27T03:14:56.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.379+0000 7f743c5f0640 1 -- 172.21.15.67:0/3310969943 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74341ace70 con 0x7f7434107e10 2024-07-27T03:14:56.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.379+0000 7f742b7fe640 1 -- 172.21.15.67:0/3310969943 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7424010430 con 0x7f7434107e10 2024-07-27T03:14:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.380+0000 7f743c5f0640 1 -- 172.21.15.67:0/3310969943 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74341028e0 con 0x7f7434107e10 2024-07-27T03:14:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.382+0000 7f742b7fe640 1 -- 172.21.15.67:0/3310969943 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f7424004fb0 con 0x7f7434107e10 2024-07-27T03:14:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.383+0000 7f742b7fe640 1 --2- 172.21.15.67:0/3310969943 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f74200b3ce0 0x7f74200b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.383+0000 7f742b7fe640 1 -- 172.21.15.67:0/3310969943 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f74240e4710 con 0x7f7434107e10 2024-07-27T03:14:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.384+0000 7f743a38c640 1 --2- 172.21.15.67:0/3310969943 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f74200b3ce0 0x7f74200b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.385+0000 7f743a38c640 1 --2- 172.21.15.67:0/3310969943 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f74200b3ce0 0x7f74200b61f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f74300058c0 tx=0x7f743000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.386+0000 7f742b7fe640 1 -- 172.21.15.67:0/3310969943 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f7424055600 con 0x7f7434107e10 2024-07-27T03:14:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.575+0000 7f743c5f0640 1 -- 172.21.15.67:0/3310969943 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f74341ad160 con 0x7f7434107e10 2024-07-27T03:14:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.575+0000 7f742b7fe640 1 -- 172.21.15.67:0/3310969943 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f74240ab850 con 0x7f7434107e10 2024-07-27T03:14:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.577+0000 7f743c5f0640 1 -- 172.21.15.67:0/3310969943 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f74200b3ce0 msgr2=0x7f74200b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.577+0000 7f743c5f0640 1 --2- 172.21.15.67:0/3310969943 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f74200b3ce0 0x7f74200b61f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f74300058c0 tx=0x7f743000a2d0 comp rx=0 tx=0).stop 2024-07-27T03:14:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.577+0000 7f743c5f0640 1 -- 172.21.15.67:0/3310969943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7434107e10 msgr2=0x7f74341a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.577+0000 7f743c5f0640 1 --2- 172.21.15.67:0/3310969943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7434107e10 0x7f74341a0a60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f742400e9c0 tx=0x7f742400ee80 comp rx=0 tx=0).stop 2024-07-27T03:14:56.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.577+0000 7f743c5f0640 1 -- 172.21.15.67:0/3310969943 shutdown_connections 2024-07-27T03:14:56.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.577+0000 7f743c5f0640 1 --2- 172.21.15.67:0/3310969943 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f74200b3ce0 0x7f74200b61f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:56.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.577+0000 7f743c5f0640 1 --2- 172.21.15.67:0/3310969943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7434108760 0x7f74341a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:56.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.577+0000 7f743c5f0640 1 --2- 172.21.15.67:0/3310969943 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7434107e10 0x7f74341a0a60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:56.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.577+0000 7f743c5f0640 1 --2- 172.21.15.67:0/3310969943 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74341024d0 0x7f74341a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:56.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.577+0000 7f743c5f0640 1 -- 172.21.15.67:0/3310969943 >> 172.21.15.67:0/3310969943 conn(0x7f74340791a0 msgr2=0x7f743406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:56.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.577+0000 7f743c5f0640 1 -- 172.21.15.67:0/3310969943 shutdown_connections 2024-07-27T03:14:56.205 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:55.577+0000 7f743c5f0640 1 -- 172.21.15.67:0/3310969943 wait complete. 2024-07-27T03:14:56.830 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:56 smithi067 ceph-mon[32814]: pgmap v381: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:14:56.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:56 smithi175 ceph-mon[37125]: pgmap v381: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:14:57.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:56 smithi149 ceph-mon[37173]: pgmap v381: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:14:57.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.630+0000 7f9e02f5b640 1 -- 172.21.15.67:0/3166985692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dfc108760 msgr2=0x7f9dfc10ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:57.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.630+0000 7f9e02f5b640 1 --2- 172.21.15.67:0/3166985692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dfc108760 0x7f9dfc10ef90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009ca0 tx=0x7f9df402a8c0 comp rx=0 tx=0).stop 2024-07-27T03:14:57.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.630+0000 7f9e02f5b640 1 -- 172.21.15.67:0/3166985692 shutdown_connections 2024-07-27T03:14:57.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.630+0000 7f9e02f5b640 1 --2- 172.21.15.67:0/3166985692 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dfc108760 0x7f9dfc10ef90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:57.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.630+0000 7f9e02f5b640 1 --2- 172.21.15.67:0/3166985692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dfc107e10 0x7f9dfc108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:57.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.630+0000 7f9e02f5b640 1 --2- 172.21.15.67:0/3166985692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9dfc1024d0 0x7f9dfc1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.630+0000 7f9e02f5b640 1 -- 172.21.15.67:0/3166985692 >> 172.21.15.67:0/3166985692 conn(0x7f9dfc0791d0 msgr2=0x7f9dfc0795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.631+0000 7f9e02f5b640 1 -- 172.21.15.67:0/3166985692 shutdown_connections 2024-07-27T03:14:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.631+0000 7f9e02f5b640 1 -- 172.21.15.67:0/3166985692 wait complete. 2024-07-27T03:14:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.631+0000 7f9e02f5b640 1 Processor -- start 2024-07-27T03:14:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.631+0000 7f9e02f5b640 1 -- start start 2024-07-27T03:14:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.632+0000 7f9e02f5b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9dfc1024d0 0x7f9dfc0ff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.632+0000 7f9e02f5b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dfc107e10 0x7f9dfc0ff600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.632+0000 7f9e02f5b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dfc108760 0x7f9dfc0ffb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.632+0000 7f9e02f5b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9dfc112430 con 0x7f9dfc107e10 2024-07-27T03:14:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.632+0000 7f9e02f5b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9dfc1122b0 con 0x7f9dfc108760 2024-07-27T03:14:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.632+0000 7f9e02f5b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9dfc1125b0 con 0x7f9dfc1024d0 2024-07-27T03:14:57.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.632+0000 7f9dfbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dfc107e10 0x7f9dfc0ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:57.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.632+0000 7f9dfbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dfc107e10 0x7f9dfc0ff600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47620/0 (socket says 172.21.15.67:47620) 2024-07-27T03:14:57.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.632+0000 7f9dfbfff640 1 -- 172.21.15.67:0/1314787750 learned_addr learned my addr 172.21.15.67:0/1314787750 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:14:57.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.632+0000 7f9e00cf7640 1 --2- 172.21.15.67:0/1314787750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9dfc1024d0 0x7f9dfc0ff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:57.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.632+0000 7f9e014f8640 1 --2- 172.21.15.67:0/1314787750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dfc108760 0x7f9dfc0ffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:57.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.632+0000 7f9dfbfff640 1 -- 172.21.15.67:0/1314787750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9dfc1024d0 msgr2=0x7f9dfc0ff0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:57.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.632+0000 7f9dfbfff640 1 --2- 172.21.15.67:0/1314787750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9dfc1024d0 0x7f9dfc0ff0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:57.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.632+0000 7f9dfbfff640 1 -- 172.21.15.67:0/1314787750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dfc108760 msgr2=0x7f9dfc0ffb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:57.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.632+0000 7f9dfbfff640 1 --2- 172.21.15.67:0/1314787750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dfc108760 0x7f9dfc0ffb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:57.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.632+0000 7f9dfbfff640 1 -- 172.21.15.67:0/1314787750 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dfc1a8250 con 0x7f9dfc107e10 2024-07-27T03:14:57.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.632+0000 7f9e014f8640 1 --2- 172.21.15.67:0/1314787750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dfc108760 0x7f9dfc0ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:14:57.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.632+0000 7f9dfbfff640 1 --2- 172.21.15.67:0/1314787750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dfc107e10 0x7f9dfc0ff600 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9de800d950 tx=0x7f9de800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:57.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.633+0000 7f9df9ffb640 1 -- 172.21.15.67:0/1314787750 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de8017070 con 0x7f9dfc107e10 2024-07-27T03:14:57.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.633+0000 7f9e02f5b640 1 -- 172.21.15.67:0/1314787750 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dfc1a8540 con 0x7f9dfc107e10 2024-07-27T03:14:57.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.633+0000 7f9df9ffb640 1 -- 172.21.15.67:0/1314787750 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9de8009c30 con 0x7f9dfc107e10 2024-07-27T03:14:57.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.633+0000 7f9df9ffb640 1 -- 172.21.15.67:0/1314787750 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de80047a0 con 0x7f9dfc107e10 2024-07-27T03:14:57.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.633+0000 7f9e02f5b640 1 -- 172.21.15.67:0/1314787750 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dfc1a8a80 con 0x7f9dfc107e10 2024-07-27T03:14:57.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.634+0000 7f9e02f5b640 1 -- 172.21.15.67:0/1314787750 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dfc103750 con 0x7f9dfc107e10 2024-07-27T03:14:57.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.636+0000 7f9df9ffb640 1 -- 172.21.15.67:0/1314787750 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f9de8004020 con 0x7f9dfc107e10 2024-07-27T03:14:57.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.637+0000 7f9df9ffb640 1 --2- 172.21.15.67:0/1314787750 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f9dec0b3c60 0x7f9dec0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:14:57.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.637+0000 7f9e00cf7640 1 --2- 172.21.15.67:0/1314787750 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f9dec0b3c60 0x7f9dec0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:14:57.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.637+0000 7f9df9ffb640 1 -- 172.21.15.67:0/1314787750 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f9de80e4170 con 0x7f9dfc107e10 2024-07-27T03:14:57.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.638+0000 7f9e00cf7640 1 --2- 172.21.15.67:0/1314787750 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f9dec0b3c60 0x7f9dec0b6170 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc100b10 tx=0x7f9df0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:14:57.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.640+0000 7f9df9ffb640 1 -- 172.21.15.67:0/1314787750 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f9de8020020 con 0x7f9dfc107e10 2024-07-27T03:14:57.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.826+0000 7f9e02f5b640 1 -- 172.21.15.67:0/1314787750 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9dfc1003c0 con 0x7f9dfc107e10 2024-07-27T03:14:57.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.827+0000 7f9df9ffb640 1 -- 172.21.15.67:0/1314787750 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9de80ab2b0 con 0x7f9dfc107e10 2024-07-27T03:14:57.830 INFO:teuthology.orchestra.run.smithi067.stdout:158913790024 2024-07-27T03:14:57.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.829+0000 7f9e02f5b640 1 -- 172.21.15.67:0/1314787750 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f9dec0b3c60 msgr2=0x7f9dec0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:57.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.829+0000 7f9e02f5b640 1 --2- 172.21.15.67:0/1314787750 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f9dec0b3c60 0x7f9dec0b6170 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc100b10 tx=0x7f9df0005f90 comp rx=0 tx=0).stop 2024-07-27T03:14:57.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.829+0000 7f9e02f5b640 1 -- 172.21.15.67:0/1314787750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dfc107e10 msgr2=0x7f9dfc0ff600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:14:57.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.829+0000 7f9e02f5b640 1 --2- 172.21.15.67:0/1314787750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dfc107e10 0x7f9dfc0ff600 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9de800d950 tx=0x7f9de800de10 comp rx=0 tx=0).stop 2024-07-27T03:14:57.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.829+0000 7f9e02f5b640 1 -- 172.21.15.67:0/1314787750 shutdown_connections 2024-07-27T03:14:57.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.829+0000 7f9e02f5b640 1 --2- 172.21.15.67:0/1314787750 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f9dec0b3c60 0x7f9dec0b6170 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:57.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.829+0000 7f9e02f5b640 1 --2- 172.21.15.67:0/1314787750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dfc108760 0x7f9dfc0ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:57.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.829+0000 7f9e02f5b640 1 --2- 172.21.15.67:0/1314787750 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9dfc107e10 0x7f9dfc0ff600 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:57.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.829+0000 7f9e02f5b640 1 --2- 172.21.15.67:0/1314787750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9dfc1024d0 0x7f9dfc0ff0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:14:57.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.829+0000 7f9e02f5b640 1 -- 172.21.15.67:0/1314787750 >> 172.21.15.67:0/1314787750 conn(0x7f9dfc0791d0 msgr2=0x7f9dfc06b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:14:57.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.829+0000 7f9e02f5b640 1 -- 172.21.15.67:0/1314787750 shutdown_connections 2024-07-27T03:14:57.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:14:57.829+0000 7f9e02f5b640 1 -- 172.21.15.67:0/1314787750 wait complete. 2024-07-27T03:14:58.655 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:58 smithi067 ceph-mon[32814]: pgmap v382: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:14:58.655 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:14:58 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1314787750' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-07-27T03:14:58.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:58 smithi175 ceph-mon[37125]: pgmap v382: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:14:58.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:14:58 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1314787750' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-07-27T03:14:59.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:58 smithi149 ceph-mon[37173]: pgmap v382: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:14:59.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:14:58 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1314787750' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-07-27T03:15:00.796 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:00 smithi067 ceph-mon[32814]: pgmap v383: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:15:00.831 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790022 got 158913790024 for osd.5 2024-07-27T03:15:00.832 DEBUG:teuthology.parallel:result is None 2024-07-27T03:15:00.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:00 smithi175 ceph-mon[37125]: pgmap v383: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:15:01.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:00 smithi149 ceph-mon[37173]: pgmap v383: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:15:02.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.555+0000 7f7df715d640 1 -- 172.21.15.67:0/1234149257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7df0107e10 msgr2=0x7f7df0108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:02.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.555+0000 7f7df715d640 1 --2- 172.21.15.67:0/1234149257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7df0107e10 0x7f7df0108220 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009900 tx=0x7f7ddc02a410 comp rx=0 tx=0).stop 2024-07-27T03:15:02.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.555+0000 7f7df715d640 1 -- 172.21.15.67:0/1234149257 shutdown_connections 2024-07-27T03:15:02.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.555+0000 7f7df715d640 1 --2- 172.21.15.67:0/1234149257 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7df0108760 0x7f7df010ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:02.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.555+0000 7f7df715d640 1 --2- 172.21.15.67:0/1234149257 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7df0107e10 0x7f7df0108220 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:02.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.555+0000 7f7df715d640 1 --2- 172.21.15.67:0/1234149257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7df01024d0 0x7f7df01028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:02.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.555+0000 7f7df715d640 1 -- 172.21.15.67:0/1234149257 >> 172.21.15.67:0/1234149257 conn(0x7f7df00791a0 msgr2=0x7f7df00795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:02.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.555+0000 7f7df715d640 1 -- 172.21.15.67:0/1234149257 shutdown_connections 2024-07-27T03:15:02.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.555+0000 7f7df715d640 1 -- 172.21.15.67:0/1234149257 wait complete. 2024-07-27T03:15:02.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.556+0000 7f7df715d640 1 Processor -- start 2024-07-27T03:15:02.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.556+0000 7f7df715d640 1 -- start start 2024-07-27T03:15:02.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.556+0000 7f7df715d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7df01024d0 0x7f7df0199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:02.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.556+0000 7f7df715d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7df0107e10 0x7f7df019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:02.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.556+0000 7f7df715d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7df0108760 0x7f7df019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:02.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.556+0000 7f7df715d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7df01123b0 con 0x7f7df0107e10 2024-07-27T03:15:02.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.556+0000 7f7df715d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7df0112230 con 0x7f7df01024d0 2024-07-27T03:15:02.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.556+0000 7f7df715d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7df0112530 con 0x7f7df0108760 2024-07-27T03:15:02.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.556+0000 7f7deffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7df0107e10 0x7f7df019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:02.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.556+0000 7f7deffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7df0107e10 0x7f7df019a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47640/0 (socket says 172.21.15.67:47640) 2024-07-27T03:15:02.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.556+0000 7f7deffff640 1 -- 172.21.15.67:0/1370451134 learned_addr learned my addr 172.21.15.67:0/1370451134 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:02.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.556+0000 7f7df4ef9640 1 --2- 172.21.15.67:0/1370451134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7df01024d0 0x7f7df0199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:02.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.556+0000 7f7df56fa640 1 --2- 172.21.15.67:0/1370451134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7df0108760 0x7f7df019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:02.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.557+0000 7f7deffff640 1 -- 172.21.15.67:0/1370451134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7df0108760 msgr2=0x7f7df019f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:02.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.557+0000 7f7deffff640 1 --2- 172.21.15.67:0/1370451134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7df0108760 0x7f7df019f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:02.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.557+0000 7f7deffff640 1 -- 172.21.15.67:0/1370451134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7df01024d0 msgr2=0x7f7df0199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:02.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.557+0000 7f7deffff640 1 --2- 172.21.15.67:0/1370451134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7df01024d0 0x7f7df0199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:02.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.557+0000 7f7deffff640 1 -- 172.21.15.67:0/1370451134 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df019ff00 con 0x7f7df0107e10 2024-07-27T03:15:02.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.557+0000 7f7df56fa640 1 --2- 172.21.15.67:0/1370451134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7df0108760 0x7f7df019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:15:02.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.557+0000 7f7deffff640 1 --2- 172.21.15.67:0/1370451134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7df0107e10 0x7f7df019a530 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc032fd0 tx=0x7f7ddc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:02.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.557+0000 7f7dedffb640 1 -- 172.21.15.67:0/1370451134 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc03f030 con 0x7f7df0107e10 2024-07-27T03:15:02.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.557+0000 7f7df715d640 1 -- 172.21.15.67:0/1370451134 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df01a0190 con 0x7f7df0107e10 2024-07-27T03:15:02.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.557+0000 7f7dedffb640 1 -- 172.21.15.67:0/1370451134 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ddc004490 con 0x7f7df0107e10 2024-07-27T03:15:02.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.557+0000 7f7dedffb640 1 -- 172.21.15.67:0/1370451134 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc035070 con 0x7f7df0107e10 2024-07-27T03:15:02.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.557+0000 7f7df715d640 1 -- 172.21.15.67:0/1370451134 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df01a6950 con 0x7f7df0107e10 2024-07-27T03:15:02.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.558+0000 7f7df715d640 1 -- 172.21.15.67:0/1370451134 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df01028e0 con 0x7f7df0107e10 2024-07-27T03:15:02.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.560+0000 7f7dedffb640 1 -- 172.21.15.67:0/1370451134 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f7ddc042030 con 0x7f7df0107e10 2024-07-27T03:15:02.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.561+0000 7f7dedffb640 1 --2- 172.21.15.67:0/1370451134 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7de00b3c60 0x7f7de00b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:02.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.561+0000 7f7df4ef9640 1 --2- 172.21.15.67:0/1370451134 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7de00b3c60 0x7f7de00b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:02.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.561+0000 7f7dedffb640 1 -- 172.21.15.67:0/1370451134 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f7ddc104a70 con 0x7f7df0107e10 2024-07-27T03:15:02.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.562+0000 7f7df4ef9640 1 --2- 172.21.15.67:0/1370451134 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7de00b3c60 0x7f7de00b6170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7de4005fd0 tx=0x7f7de400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:02.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.565+0000 7f7dedffb640 1 -- 172.21.15.67:0/1370451134 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f7ddc0cbbb0 con 0x7f7df0107e10 2024-07-27T03:15:02.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.753+0000 7f7df715d640 1 -- 172.21.15.67:0/1370451134 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f7df010b560 con 0x7f7df0107e10 2024-07-27T03:15:02.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.753+0000 7f7dedffb640 1 -- 172.21.15.67:0/1370451134 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7ddc0cb4b0 con 0x7f7df0107e10 2024-07-27T03:15:02.756 INFO:teuthology.orchestra.run.smithi067.stdout:184683593797 2024-07-27T03:15:02.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.756+0000 7f7df715d640 1 -- 172.21.15.67:0/1370451134 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7de00b3c60 msgr2=0x7f7de00b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:02.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.756+0000 7f7df715d640 1 --2- 172.21.15.67:0/1370451134 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7de00b3c60 0x7f7de00b6170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7de4005fd0 tx=0x7f7de400b390 comp rx=0 tx=0).stop 2024-07-27T03:15:02.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.756+0000 7f7df715d640 1 -- 172.21.15.67:0/1370451134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7df0107e10 msgr2=0x7f7df019a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:02.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.756+0000 7f7df715d640 1 --2- 172.21.15.67:0/1370451134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7df0107e10 0x7f7df019a530 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc032fd0 tx=0x7f7ddc004270 comp rx=0 tx=0).stop 2024-07-27T03:15:02.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.756+0000 7f7df715d640 1 -- 172.21.15.67:0/1370451134 shutdown_connections 2024-07-27T03:15:02.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.756+0000 7f7df715d640 1 --2- 172.21.15.67:0/1370451134 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7de00b3c60 0x7f7de00b6170 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:02.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.756+0000 7f7df715d640 1 --2- 172.21.15.67:0/1370451134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7df0108760 0x7f7df019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:02.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.756+0000 7f7df715d640 1 --2- 172.21.15.67:0/1370451134 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7df0107e10 0x7f7df019a530 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:02.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.756+0000 7f7df715d640 1 --2- 172.21.15.67:0/1370451134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7df01024d0 0x7f7df0199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:02.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.756+0000 7f7df715d640 1 -- 172.21.15.67:0/1370451134 >> 172.21.15.67:0/1370451134 conn(0x7f7df00791a0 msgr2=0x7f7df006ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:02.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.756+0000 7f7df715d640 1 -- 172.21.15.67:0/1370451134 shutdown_connections 2024-07-27T03:15:02.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:02.756+0000 7f7df715d640 1 -- 172.21.15.67:0/1370451134 wait complete. 2024-07-27T03:15:02.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:02 smithi175 ceph-mon[37125]: pgmap v384: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-27T03:15:02.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:02 smithi067 ceph-mon[32814]: pgmap v384: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-27T03:15:03.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:02 smithi149 ceph-mon[37173]: pgmap v384: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-07-27T03:15:03.864 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:03 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1370451134' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-07-27T03:15:03.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:03 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1370451134' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-07-27T03:15:04.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:03 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1370451134' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-07-27T03:15:04.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.493+0000 7ff9866e5640 1 -- 172.21.15.67:0/3322452702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff980107e30 msgr2=0x7ff980108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:04.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.493+0000 7ff9866e5640 1 --2- 172.21.15.67:0/3322452702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff980107e30 0x7ff980108240 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff974009900 tx=0x7ff97402a3f0 comp rx=0 tx=0).stop 2024-07-27T03:15:04.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.493+0000 7ff9866e5640 1 -- 172.21.15.67:0/3322452702 shutdown_connections 2024-07-27T03:15:04.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.493+0000 7ff9866e5640 1 --2- 172.21.15.67:0/3322452702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff980108780 0x7ff98010efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:04.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.493+0000 7ff9866e5640 1 --2- 172.21.15.67:0/3322452702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff980107e30 0x7ff980108240 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:04.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.493+0000 7ff9866e5640 1 --2- 172.21.15.67:0/3322452702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9801024f0 0x7ff980102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:04.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.493+0000 7ff9866e5640 1 -- 172.21.15.67:0/3322452702 >> 172.21.15.67:0/3322452702 conn(0x7ff9800791b0 msgr2=0x7ff9800795c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:04.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.493+0000 7ff9866e5640 1 -- 172.21.15.67:0/3322452702 shutdown_connections 2024-07-27T03:15:04.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.493+0000 7ff9866e5640 1 -- 172.21.15.67:0/3322452702 wait complete. 2024-07-27T03:15:04.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.493+0000 7ff9866e5640 1 Processor -- start 2024-07-27T03:15:04.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.493+0000 7ff9866e5640 1 -- start start 2024-07-27T03:15:04.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.494+0000 7ff9866e5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9801024f0 0x7ff980198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:04.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.494+0000 7ff9866e5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff980107e30 0x7ff9801993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:04.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.494+0000 7ff9866e5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff980108780 0x7ff98019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:04.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.494+0000 7ff9866e5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff9801123b0 con 0x7ff980108780 2024-07-27T03:15:04.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.494+0000 7ff9866e5640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff980112230 con 0x7ff9801024f0 2024-07-27T03:15:04.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.494+0000 7ff9866e5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff980112530 con 0x7ff980107e30 2024-07-27T03:15:04.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.494+0000 7ff984c82640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff980108780 0x7ff98019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:04.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.494+0000 7ff984c82640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff980108780 0x7ff98019e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47660/0 (socket says 172.21.15.67:47660) 2024-07-27T03:15:04.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.494+0000 7ff984c82640 1 -- 172.21.15.67:0/2278072508 learned_addr learned my addr 172.21.15.67:0/2278072508 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:04.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.494+0000 7ff97ffff640 1 --2- 172.21.15.67:0/2278072508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9801024f0 0x7ff980198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:04.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.494+0000 7ff97f7fe640 1 --2- 172.21.15.67:0/2278072508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff980107e30 0x7ff9801993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:04.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.494+0000 7ff984c82640 1 -- 172.21.15.67:0/2278072508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff980107e30 msgr2=0x7ff9801993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:04.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.494+0000 7ff984c82640 1 --2- 172.21.15.67:0/2278072508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff980107e30 0x7ff9801993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:04.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.494+0000 7ff984c82640 1 -- 172.21.15.67:0/2278072508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9801024f0 msgr2=0x7ff980198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:04.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.494+0000 7ff984c82640 1 --2- 172.21.15.67:0/2278072508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9801024f0 0x7ff980198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:04.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.494+0000 7ff984c82640 1 -- 172.21.15.67:0/2278072508 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff98019ecd0 con 0x7ff980108780 2024-07-27T03:15:04.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.494+0000 7ff97f7fe640 1 --2- 172.21.15.67:0/2278072508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff980107e30 0x7ff9801993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:15:04.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.495+0000 7ff984c82640 1 --2- 172.21.15.67:0/2278072508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff980108780 0x7ff98019e660 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff97800cce0 tx=0x7ff97800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:04.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.495+0000 7ff97d7fa640 1 -- 172.21.15.67:0/2278072508 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97800f450 con 0x7ff980108780 2024-07-27T03:15:04.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.495+0000 7ff97d7fa640 1 -- 172.21.15.67:0/2278072508 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff97800faa0 con 0x7ff980108780 2024-07-27T03:15:04.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.495+0000 7ff9866e5640 1 -- 172.21.15.67:0/2278072508 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff98019efc0 con 0x7ff980108780 2024-07-27T03:15:04.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.495+0000 7ff97d7fa640 1 -- 172.21.15.67:0/2278072508 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978018b70 con 0x7ff980108780 2024-07-27T03:15:04.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.495+0000 7ff9866e5640 1 -- 172.21.15.67:0/2278072508 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9801a57e0 con 0x7ff980108780 2024-07-27T03:15:04.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.496+0000 7ff9866e5640 1 -- 172.21.15.67:0/2278072508 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff980103770 con 0x7ff980108780 2024-07-27T03:15:04.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.497+0000 7ff97d7fa640 1 -- 172.21.15.67:0/2278072508 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7ff978018d10 con 0x7ff980108780 2024-07-27T03:15:04.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.498+0000 7ff97d7fa640 1 --2- 172.21.15.67:0/2278072508 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff9680b3c90 0x7ff9680b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:04.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.498+0000 7ff97ffff640 1 --2- 172.21.15.67:0/2278072508 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff9680b3c90 0x7ff9680b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:04.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.498+0000 7ff97d7fa640 1 -- 172.21.15.67:0/2278072508 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7ff9780e4730 con 0x7ff980108780 2024-07-27T03:15:04.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.499+0000 7ff97ffff640 1 --2- 172.21.15.67:0/2278072508 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff9680b3c90 0x7ff9680b61a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff970005fd0 tx=0x7ff970005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:04.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.501+0000 7ff97d7fa640 1 -- 172.21.15.67:0/2278072508 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7ff978014030 con 0x7ff980108780 2024-07-27T03:15:04.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.694+0000 7ff9866e5640 1 -- 172.21.15.67:0/2278072508 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff980102900 con 0x7ff980108780 2024-07-27T03:15:04.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.694+0000 7ff97d7fa640 1 -- 172.21.15.67:0/2278072508 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff980102900 con 0x7ff980108780 2024-07-27T03:15:04.697 INFO:teuthology.orchestra.run.smithi067.stdout:34359738465 2024-07-27T03:15:04.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.696+0000 7ff9866e5640 1 -- 172.21.15.67:0/2278072508 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff9680b3c90 msgr2=0x7ff9680b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:04.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.696+0000 7ff9866e5640 1 --2- 172.21.15.67:0/2278072508 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff9680b3c90 0x7ff9680b61a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff970005fd0 tx=0x7ff970005850 comp rx=0 tx=0).stop 2024-07-27T03:15:04.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.696+0000 7ff9866e5640 1 -- 172.21.15.67:0/2278072508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff980108780 msgr2=0x7ff98019e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:04.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.696+0000 7ff9866e5640 1 --2- 172.21.15.67:0/2278072508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff980108780 0x7ff98019e660 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff97800cce0 tx=0x7ff97800a620 comp rx=0 tx=0).stop 2024-07-27T03:15:04.699 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.697+0000 7ff9866e5640 1 -- 172.21.15.67:0/2278072508 shutdown_connections 2024-07-27T03:15:04.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.697+0000 7ff9866e5640 1 --2- 172.21.15.67:0/2278072508 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff9680b3c90 0x7ff9680b61a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:04.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.697+0000 7ff9866e5640 1 --2- 172.21.15.67:0/2278072508 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff980108780 0x7ff98019e660 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:04.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.697+0000 7ff9866e5640 1 --2- 172.21.15.67:0/2278072508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff980107e30 0x7ff9801993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:04.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.697+0000 7ff9866e5640 1 --2- 172.21.15.67:0/2278072508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9801024f0 0x7ff980198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:04.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.697+0000 7ff9866e5640 1 -- 172.21.15.67:0/2278072508 >> 172.21.15.67:0/2278072508 conn(0x7ff9800791b0 msgr2=0x7ff980103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:04.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.697+0000 7ff9866e5640 1 -- 172.21.15.67:0/2278072508 shutdown_connections 2024-07-27T03:15:04.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.697+0000 7ff9866e5640 1 -- 172.21.15.67:0/2278072508 wait complete. 2024-07-27T03:15:04.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.704+0000 7faab0eab640 1 -- 172.21.15.67:0/1117220362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaac107e10 msgr2=0x7faaac108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:04.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.704+0000 7faab0eab640 1 --2- 172.21.15.67:0/1117220362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaac107e10 0x7faaac108220 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faaa0009900 tx=0x7faaa002a410 comp rx=0 tx=0).stop 2024-07-27T03:15:04.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.704+0000 7faab0eab640 1 -- 172.21.15.67:0/1117220362 shutdown_connections 2024-07-27T03:15:04.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.705+0000 7faab0eab640 1 --2- 172.21.15.67:0/1117220362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faaac108760 0x7faaac10ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:04.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.705+0000 7faab0eab640 1 --2- 172.21.15.67:0/1117220362 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaac107e10 0x7faaac108220 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:04.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.705+0000 7faab0eab640 1 --2- 172.21.15.67:0/1117220362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaac1024d0 0x7faaac1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:04.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.705+0000 7faab0eab640 1 -- 172.21.15.67:0/1117220362 >> 172.21.15.67:0/1117220362 conn(0x7faaac0791a0 msgr2=0x7faaac0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:04.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.705+0000 7faab0eab640 1 -- 172.21.15.67:0/1117220362 shutdown_connections 2024-07-27T03:15:04.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.705+0000 7faab0eab640 1 -- 172.21.15.67:0/1117220362 wait complete. 2024-07-27T03:15:04.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.706+0000 7faab0eab640 1 Processor -- start 2024-07-27T03:15:04.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.706+0000 7faab0eab640 1 -- start start 2024-07-27T03:15:04.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.706+0000 7faab0eab640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaac1024d0 0x7faaac199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:04.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.706+0000 7faab0eab640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faaac107e10 0x7faaac19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:04.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.706+0000 7faab0eab640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaac108760 0x7faaac19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:04.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.706+0000 7faab0eab640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faaac06b6b0 con 0x7faaac108760 2024-07-27T03:15:04.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.706+0000 7faab0eab640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faaac06b530 con 0x7faaac107e10 2024-07-27T03:15:04.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.706+0000 7faab0eab640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faaac06b830 con 0x7faaac1024d0 2024-07-27T03:15:04.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.706+0000 7faaaad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaac108760 0x7faaac19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:04.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.706+0000 7faaaad9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaac108760 0x7faaac19f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47672/0 (socket says 172.21.15.67:47672) 2024-07-27T03:15:04.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.706+0000 7faaaad9d640 1 -- 172.21.15.67:0/2797364606 learned_addr learned my addr 172.21.15.67:0/2797364606 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:04.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.706+0000 7faaa9d9b640 1 --2- 172.21.15.67:0/2797364606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faaac107e10 0x7faaac19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:04.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.707+0000 7faaaa59c640 1 --2- 172.21.15.67:0/2797364606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaac1024d0 0x7faaac199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:04.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.707+0000 7faaaad9d640 1 -- 172.21.15.67:0/2797364606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaac1024d0 msgr2=0x7faaac199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:04.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.707+0000 7faaaad9d640 1 --2- 172.21.15.67:0/2797364606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaac1024d0 0x7faaac199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:04.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.707+0000 7faaaad9d640 1 -- 172.21.15.67:0/2797364606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faaac107e10 msgr2=0x7faaac19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:04.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.707+0000 7faaaad9d640 1 --2- 172.21.15.67:0/2797364606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faaac107e10 0x7faaac19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:04.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.707+0000 7faaaad9d640 1 -- 172.21.15.67:0/2797364606 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaac19ff10 con 0x7faaac108760 2024-07-27T03:15:04.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.707+0000 7faaaa59c640 1 --2- 172.21.15.67:0/2797364606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaac1024d0 0x7faaac199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:15:04.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.707+0000 7faaaad9d640 1 --2- 172.21.15.67:0/2797364606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaac108760 0x7faaac19f7e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009fd0 tx=0x7faaa400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:04.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.707+0000 7faa9b7fe640 1 -- 172.21.15.67:0/2797364606 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4017070 con 0x7faaac108760 2024-07-27T03:15:04.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.707+0000 7faab0eab640 1 -- 172.21.15.67:0/2797364606 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaac1a0200 con 0x7faaac108760 2024-07-27T03:15:04.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.707+0000 7faa9b7fe640 1 -- 172.21.15.67:0/2797364606 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaa4003e20 con 0x7faaac108760 2024-07-27T03:15:04.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.707+0000 7faa9b7fe640 1 -- 172.21.15.67:0/2797364606 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa4010460 con 0x7faaac108760 2024-07-27T03:15:04.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.707+0000 7faab0eab640 1 -- 172.21.15.67:0/2797364606 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaac1a6960 con 0x7faaac108760 2024-07-27T03:15:04.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.709+0000 7faab0eab640 1 -- 172.21.15.67:0/2797364606 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaac1028e0 con 0x7faaac108760 2024-07-27T03:15:04.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.710+0000 7faa9b7fe640 1 -- 172.21.15.67:0/2797364606 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7faaa4003970 con 0x7faaac108760 2024-07-27T03:15:04.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.711+0000 7faa9b7fe640 1 --2- 172.21.15.67:0/2797364606 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7faa940b3c60 0x7faa940b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:04.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.711+0000 7faaaa59c640 1 --2- 172.21.15.67:0/2797364606 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7faa940b3c60 0x7faa940b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:04.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.711+0000 7faa9b7fe640 1 -- 172.21.15.67:0/2797364606 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7faaa40e3e10 con 0x7faaac108760 2024-07-27T03:15:04.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.712+0000 7faaaa59c640 1 --2- 172.21.15.67:0/2797364606 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7faa940b3c60 0x7faa940b6170 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faa9c0058c0 tx=0x7faa9c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:04.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.716+0000 7faa9b7fe640 1 -- 172.21.15.67:0/2797364606 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7faaa40aaf50 con 0x7faaac108760 2024-07-27T03:15:04.822 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:04 smithi067 ceph-mon[32814]: pgmap v385: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:04.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:04 smithi175 ceph-mon[37125]: pgmap v385: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:04.900 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:04 smithi149 ceph-mon[37173]: pgmap v385: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:04.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.904+0000 7faab0eab640 1 -- 172.21.15.67:0/2797364606 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7faaac109fc0 con 0x7faaac108760 2024-07-27T03:15:04.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.905+0000 7faa9b7fe640 1 -- 172.21.15.67:0/2797364606 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7faaa40b3220 con 0x7faaac108760 2024-07-27T03:15:04.908 INFO:teuthology.orchestra.run.smithi067.stdout:309237645357 2024-07-27T03:15:04.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.907+0000 7faab0eab640 1 -- 172.21.15.67:0/2797364606 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7faa940b3c60 msgr2=0x7faa940b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:04.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.907+0000 7faab0eab640 1 --2- 172.21.15.67:0/2797364606 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7faa940b3c60 0x7faa940b6170 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faa9c0058c0 tx=0x7faa9c005850 comp rx=0 tx=0).stop 2024-07-27T03:15:04.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.907+0000 7faab0eab640 1 -- 172.21.15.67:0/2797364606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaac108760 msgr2=0x7faaac19f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:04.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.907+0000 7faab0eab640 1 --2- 172.21.15.67:0/2797364606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaac108760 0x7faaac19f7e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7faaa4009fd0 tx=0x7faaa400b260 comp rx=0 tx=0).stop 2024-07-27T03:15:04.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.907+0000 7faab0eab640 1 -- 172.21.15.67:0/2797364606 shutdown_connections 2024-07-27T03:15:04.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.907+0000 7faab0eab640 1 --2- 172.21.15.67:0/2797364606 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7faa940b3c60 0x7faa940b6170 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:04.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.907+0000 7faab0eab640 1 --2- 172.21.15.67:0/2797364606 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaac108760 0x7faaac19f7e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:04.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.907+0000 7faab0eab640 1 --2- 172.21.15.67:0/2797364606 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faaac107e10 0x7faaac19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:04.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.907+0000 7faab0eab640 1 --2- 172.21.15.67:0/2797364606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faaac1024d0 0x7faaac199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:04.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.907+0000 7faab0eab640 1 -- 172.21.15.67:0/2797364606 >> 172.21.15.67:0/2797364606 conn(0x7faaac0791a0 msgr2=0x7faaac103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:04.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.907+0000 7faab0eab640 1 -- 172.21.15.67:0/2797364606 shutdown_connections 2024-07-27T03:15:04.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:04.907+0000 7faab0eab640 1 -- 172.21.15.67:0/2797364606 wait complete. 2024-07-27T03:15:05.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:05 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2278072508' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-07-27T03:15:05.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:05 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2797364606' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-07-27T03:15:05.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:05 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2278072508' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-07-27T03:15:05.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:05 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2797364606' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-07-27T03:15:06.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:05 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2278072508' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-07-27T03:15:06.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:05 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2797364606' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-07-27T03:15:06.282 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037810 got 257698037813 for osd.9 2024-07-27T03:15:06.283 DEBUG:teuthology.parallel:result is None 2024-07-27T03:15:06.743 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593792 got 184683593797 for osd.6 2024-07-27T03:15:06.743 DEBUG:teuthology.parallel:result is None 2024-07-27T03:15:06.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:06 smithi175 ceph-mon[37125]: pgmap v386: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:06.931 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:06 smithi067 ceph-mon[32814]: pgmap v386: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:07.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:06 smithi149 ceph-mon[37173]: pgmap v386: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:07.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.023+0000 7f930ccad640 1 -- 172.21.15.67:0/3036523744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9308107e30 msgr2=0x7f9308108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:07.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.023+0000 7f930ccad640 1 --2- 172.21.15.67:0/3036523744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9308107e30 0x7f9308108240 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009900 tx=0x7f92fc02a3f0 comp rx=0 tx=0).stop 2024-07-27T03:15:07.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.024+0000 7f930ccad640 1 -- 172.21.15.67:0/3036523744 shutdown_connections 2024-07-27T03:15:07.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.024+0000 7f930ccad640 1 --2- 172.21.15.67:0/3036523744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9308108780 0x7f930810efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:07.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.024+0000 7f930ccad640 1 --2- 172.21.15.67:0/3036523744 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9308107e30 0x7f9308108240 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:07.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.024+0000 7f930ccad640 1 --2- 172.21.15.67:0/3036523744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93081024f0 0x7f9308102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:07.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.024+0000 7f930ccad640 1 -- 172.21.15.67:0/3036523744 >> 172.21.15.67:0/3036523744 conn(0x7f93080791b0 msgr2=0x7f93080795c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:07.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.024+0000 7f930ccad640 1 -- 172.21.15.67:0/3036523744 shutdown_connections 2024-07-27T03:15:07.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.024+0000 7f930ccad640 1 -- 172.21.15.67:0/3036523744 wait complete. 2024-07-27T03:15:07.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.024+0000 7f930ccad640 1 Processor -- start 2024-07-27T03:15:07.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.025+0000 7f930ccad640 1 -- start start 2024-07-27T03:15:07.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.025+0000 7f930ccad640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93081024f0 0x7f930810fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:07.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.025+0000 7f930ccad640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9308107e30 0x7f930810ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:07.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.025+0000 7f930ccad640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9308108780 0x7f93081104c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:07.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.025+0000 7f930ccad640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9308113510 con 0x7f9308107e30 2024-07-27T03:15:07.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.025+0000 7f930ccad640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9308113390 con 0x7f9308108780 2024-07-27T03:15:07.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.025+0000 7f930ccad640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9308113690 con 0x7f93081024f0 2024-07-27T03:15:07.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.025+0000 7f9305d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9308107e30 0x7f930810ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:07.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.025+0000 7f9305d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9308107e30 0x7f930810ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46756/0 (socket says 172.21.15.67:46756) 2024-07-27T03:15:07.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.025+0000 7f9305d9b640 1 -- 172.21.15.67:0/1821717772 learned_addr learned my addr 172.21.15.67:0/1821717772 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:07.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.025+0000 7f9306d9d640 1 --2- 172.21.15.67:0/1821717772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9308108780 0x7f93081104c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:07.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.025+0000 7f930659c640 1 --2- 172.21.15.67:0/1821717772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93081024f0 0x7f930810fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:07.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.025+0000 7f9305d9b640 1 -- 172.21.15.67:0/1821717772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93081024f0 msgr2=0x7f930810fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:07.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.025+0000 7f9305d9b640 1 --2- 172.21.15.67:0/1821717772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93081024f0 0x7f930810fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:07.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.025+0000 7f9305d9b640 1 -- 172.21.15.67:0/1821717772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9308108780 msgr2=0x7f93081104c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:07.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.025+0000 7f9305d9b640 1 --2- 172.21.15.67:0/1821717772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9308108780 0x7f93081104c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:07.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.025+0000 7f9305d9b640 1 -- 172.21.15.67:0/1821717772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93081a8150 con 0x7f9308107e30 2024-07-27T03:15:07.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.026+0000 7f9305d9b640 1 --2- 172.21.15.67:0/1821717772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9308107e30 0x7f930810ff80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f92fc032fd0 tx=0x7f92fc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:07.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.026+0000 7f92f77fe640 1 -- 172.21.15.67:0/1821717772 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc03f030 con 0x7f9308107e30 2024-07-27T03:15:07.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.026+0000 7f92f77fe640 1 -- 172.21.15.67:0/1821717772 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92fc004490 con 0x7f9308107e30 2024-07-27T03:15:07.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.026+0000 7f92f77fe640 1 -- 172.21.15.67:0/1821717772 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc035070 con 0x7f9308107e30 2024-07-27T03:15:07.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.026+0000 7f930ccad640 1 -- 172.21.15.67:0/1821717772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93081a83e0 con 0x7f9308107e30 2024-07-27T03:15:07.032 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.026+0000 7f930ccad640 1 -- 172.21.15.67:0/1821717772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93081a8910 con 0x7f9308107e30 2024-07-27T03:15:07.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.027+0000 7f930ccad640 1 -- 172.21.15.67:0/1821717772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9308103770 con 0x7f9308107e30 2024-07-27T03:15:07.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.029+0000 7f92f77fe640 1 -- 172.21.15.67:0/1821717772 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f92fc042030 con 0x7f9308107e30 2024-07-27T03:15:07.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.029+0000 7f92f77fe640 1 --2- 172.21.15.67:0/1821717772 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f92f00b3ce0 0x7f92f00b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:07.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.029+0000 7f930659c640 1 --2- 172.21.15.67:0/1821717772 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f92f00b3ce0 0x7f92f00b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:07.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.029+0000 7f92f77fe640 1 -- 172.21.15.67:0/1821717772 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f92fc104940 con 0x7f9308107e30 2024-07-27T03:15:07.033 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.030+0000 7f930659c640 1 --2- 172.21.15.67:0/1821717772 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f92f00b3ce0 0x7f92f00b61f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9308111490 tx=0x7f92f800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:07.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.032+0000 7f92f77fe640 1 -- 172.21.15.67:0/1821717772 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f92fc0cb430 con 0x7f9308107e30 2024-07-27T03:15:07.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.216+0000 7f930ccad640 1 -- 172.21.15.67:0/1821717772 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9308102900 con 0x7f9308107e30 2024-07-27T03:15:07.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.217+0000 7f92f77fe640 1 -- 172.21.15.67:0/1821717772 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f92fc0cf980 con 0x7f9308107e30 2024-07-27T03:15:07.220 INFO:teuthology.orchestra.run.smithi067.stdout:55834574941 2024-07-27T03:15:07.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.219+0000 7f930ccad640 1 -- 172.21.15.67:0/1821717772 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f92f00b3ce0 msgr2=0x7f92f00b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:07.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.219+0000 7f930ccad640 1 --2- 172.21.15.67:0/1821717772 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f92f00b3ce0 0x7f92f00b61f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9308111490 tx=0x7f92f800b390 comp rx=0 tx=0).stop 2024-07-27T03:15:07.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.219+0000 7f930ccad640 1 -- 172.21.15.67:0/1821717772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9308107e30 msgr2=0x7f930810ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:07.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.219+0000 7f930ccad640 1 --2- 172.21.15.67:0/1821717772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9308107e30 0x7f930810ff80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f92fc032fd0 tx=0x7f92fc004270 comp rx=0 tx=0).stop 2024-07-27T03:15:07.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.219+0000 7f930ccad640 1 -- 172.21.15.67:0/1821717772 shutdown_connections 2024-07-27T03:15:07.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.219+0000 7f930ccad640 1 --2- 172.21.15.67:0/1821717772 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f92f00b3ce0 0x7f92f00b61f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:07.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.219+0000 7f930ccad640 1 --2- 172.21.15.67:0/1821717772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9308108780 0x7f93081104c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:07.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.219+0000 7f930ccad640 1 --2- 172.21.15.67:0/1821717772 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9308107e30 0x7f930810ff80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:07.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.219+0000 7f930ccad640 1 --2- 172.21.15.67:0/1821717772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93081024f0 0x7f930810fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:07.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.219+0000 7f930ccad640 1 -- 172.21.15.67:0/1821717772 >> 172.21.15.67:0/1821717772 conn(0x7f93080791b0 msgr2=0x7f9308103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:07.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.219+0000 7f930ccad640 1 -- 172.21.15.67:0/1821717772 shutdown_connections 2024-07-27T03:15:07.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:07.219+0000 7f930ccad640 1 -- 172.21.15.67:0/1821717772 wait complete. 2024-07-27T03:15:07.430 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645353 got 309237645357 for osd.11 2024-07-27T03:15:07.430 DEBUG:teuthology.parallel:result is None 2024-07-27T03:15:07.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:07 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1821717772' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-07-27T03:15:08.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:07 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1821717772' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-07-27T03:15:08.021 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738461 got 34359738465 for osd.0 2024-07-27T03:15:08.021 DEBUG:teuthology.parallel:result is None 2024-07-27T03:15:08.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:07 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1821717772' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-07-27T03:15:08.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:08 smithi067 ceph-mon[32814]: pgmap v387: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:09.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:08 smithi149 ceph-mon[37173]: pgmap v387: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:09.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:08 smithi175 ceph-mon[37125]: pgmap v387: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:09.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.502+0000 7fd57a1af640 1 -- 172.21.15.67:0/3704233914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd574107e10 msgr2=0x7fd574108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:09.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.502+0000 7fd57a1af640 1 --2- 172.21.15.67:0/3704233914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd574107e10 0x7fd574108220 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd564009900 tx=0x7fd56402a410 comp rx=0 tx=0).stop 2024-07-27T03:15:09.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.502+0000 7fd57a1af640 1 -- 172.21.15.67:0/3704233914 shutdown_connections 2024-07-27T03:15:09.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.502+0000 7fd57a1af640 1 --2- 172.21.15.67:0/3704233914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd574108760 0x7fd57410ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:09.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.502+0000 7fd57a1af640 1 --2- 172.21.15.67:0/3704233914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd574107e10 0x7fd574108220 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:09.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.502+0000 7fd57a1af640 1 --2- 172.21.15.67:0/3704233914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5741024d0 0x7fd5741028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:09.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.502+0000 7fd57a1af640 1 -- 172.21.15.67:0/3704233914 >> 172.21.15.67:0/3704233914 conn(0x7fd5740791a0 msgr2=0x7fd5740795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:09.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.502+0000 7fd57a1af640 1 -- 172.21.15.67:0/3704233914 shutdown_connections 2024-07-27T03:15:09.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.502+0000 7fd57a1af640 1 -- 172.21.15.67:0/3704233914 wait complete. 2024-07-27T03:15:09.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.503+0000 7fd57a1af640 1 Processor -- start 2024-07-27T03:15:09.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.503+0000 7fd57a1af640 1 -- start start 2024-07-27T03:15:09.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.504+0000 7fd57a1af640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5741024d0 0x7fd57419ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:09.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.504+0000 7fd57a1af640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd574107e10 0x7fd57419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:09.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.504+0000 7fd57a1af640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd574108760 0x7fd57419b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:09.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.504+0000 7fd57a1af640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd574112390 con 0x7fd574107e10 2024-07-27T03:15:09.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.504+0000 7fd57a1af640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd574112210 con 0x7fd574108760 2024-07-27T03:15:09.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.504+0000 7fd57a1af640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd574112510 con 0x7fd5741024d0 2024-07-27T03:15:09.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.504+0000 7fd572ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd574107e10 0x7fd57419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:09.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.504+0000 7fd572ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd574107e10 0x7fd57419b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46778/0 (socket says 172.21.15.67:46778) 2024-07-27T03:15:09.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.504+0000 7fd572ffd640 1 -- 172.21.15.67:0/2059818801 learned_addr learned my addr 172.21.15.67:0/2059818801 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:09.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.504+0000 7fd5737fe640 1 --2- 172.21.15.67:0/2059818801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5741024d0 0x7fd57419ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:09.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.504+0000 7fd573fff640 1 --2- 172.21.15.67:0/2059818801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd574108760 0x7fd57419b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:09.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.504+0000 7fd572ffd640 1 -- 172.21.15.67:0/2059818801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5741024d0 msgr2=0x7fd57419ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:09.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.504+0000 7fd572ffd640 1 --2- 172.21.15.67:0/2059818801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5741024d0 0x7fd57419ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:09.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.504+0000 7fd572ffd640 1 -- 172.21.15.67:0/2059818801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd574108760 msgr2=0x7fd57419b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:09.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.505+0000 7fd572ffd640 1 --2- 172.21.15.67:0/2059818801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd574108760 0x7fd57419b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:09.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.505+0000 7fd572ffd640 1 -- 172.21.15.67:0/2059818801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5741a6000 con 0x7fd574107e10 2024-07-27T03:15:09.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.505+0000 7fd573fff640 1 --2- 172.21.15.67:0/2059818801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd574108760 0x7fd57419b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:15:09.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.505+0000 7fd572ffd640 1 --2- 172.21.15.67:0/2059818801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd574107e10 0x7fd57419b280 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd564032fd0 tx=0x7fd564004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:09.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.505+0000 7fd570ff9640 1 -- 172.21.15.67:0/2059818801 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56403f030 con 0x7fd574107e10 2024-07-27T03:15:09.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.505+0000 7fd570ff9640 1 -- 172.21.15.67:0/2059818801 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd564004490 con 0x7fd574107e10 2024-07-27T03:15:09.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.505+0000 7fd57a1af640 1 -- 172.21.15.67:0/2059818801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5741a6290 con 0x7fd574107e10 2024-07-27T03:15:09.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.505+0000 7fd570ff9640 1 -- 172.21.15.67:0/2059818801 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564035070 con 0x7fd574107e10 2024-07-27T03:15:09.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.505+0000 7fd57a1af640 1 -- 172.21.15.67:0/2059818801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5741a67c0 con 0x7fd574107e10 2024-07-27T03:15:09.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.506+0000 7fd57a1af640 1 -- 172.21.15.67:0/2059818801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5741028e0 con 0x7fd574107e10 2024-07-27T03:15:09.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.509+0000 7fd570ff9640 1 -- 172.21.15.67:0/2059818801 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7fd564042030 con 0x7fd574107e10 2024-07-27T03:15:09.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.510+0000 7fd570ff9640 1 --2- 172.21.15.67:0/2059818801 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd5600b3d60 0x7fd5600b6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:09.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.511+0000 7fd570ff9640 1 -- 172.21.15.67:0/2059818801 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fd564104a00 con 0x7fd574107e10 2024-07-27T03:15:09.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.511+0000 7fd5737fe640 1 --2- 172.21.15.67:0/2059818801 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd5600b3d60 0x7fd5600b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:09.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.511+0000 7fd5737fe640 1 --2- 172.21.15.67:0/2059818801 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd5600b3d60 0x7fd5600b6270 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd55c0058c0 tx=0x7fd55c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:09.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.516+0000 7fd570ff9640 1 -- 172.21.15.67:0/2059818801 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fd5640cbae0 con 0x7fd574107e10 2024-07-27T03:15:09.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.718+0000 7fd57a1af640 1 -- 172.21.15.67:0/2059818801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd57402e100 con 0x7fd574107e10 2024-07-27T03:15:09.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.718+0000 7fd570ff9640 1 -- 172.21.15.67:0/2059818801 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd5640cb3e0 con 0x7fd574107e10 2024-07-27T03:15:09.721 INFO:teuthology.orchestra.run.smithi067.stdout:77309411417 2024-07-27T03:15:09.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.720+0000 7fd57a1af640 1 -- 172.21.15.67:0/2059818801 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd5600b3d60 msgr2=0x7fd5600b6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:09.723 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.720+0000 7fd57a1af640 1 --2- 172.21.15.67:0/2059818801 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd5600b3d60 0x7fd5600b6270 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd55c0058c0 tx=0x7fd55c00a2d0 comp rx=0 tx=0).stop 2024-07-27T03:15:09.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.720+0000 7fd57a1af640 1 -- 172.21.15.67:0/2059818801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd574107e10 msgr2=0x7fd57419b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:09.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.720+0000 7fd57a1af640 1 --2- 172.21.15.67:0/2059818801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd574107e10 0x7fd57419b280 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd564032fd0 tx=0x7fd564004270 comp rx=0 tx=0).stop 2024-07-27T03:15:09.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.720+0000 7fd57a1af640 1 -- 172.21.15.67:0/2059818801 shutdown_connections 2024-07-27T03:15:09.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.720+0000 7fd57a1af640 1 --2- 172.21.15.67:0/2059818801 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd5600b3d60 0x7fd5600b6270 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:09.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.720+0000 7fd57a1af640 1 --2- 172.21.15.67:0/2059818801 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd574108760 0x7fd57419b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:09.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.721+0000 7fd57a1af640 1 --2- 172.21.15.67:0/2059818801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd574107e10 0x7fd57419b280 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:09.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.721+0000 7fd57a1af640 1 --2- 172.21.15.67:0/2059818801 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5741024d0 0x7fd57419ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:09.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.721+0000 7fd57a1af640 1 -- 172.21.15.67:0/2059818801 >> 172.21.15.67:0/2059818801 conn(0x7fd5740791a0 msgr2=0x7fd574104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:09.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.721+0000 7fd57a1af640 1 -- 172.21.15.67:0/2059818801 shutdown_connections 2024-07-27T03:15:09.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.721+0000 7fd57a1af640 1 -- 172.21.15.67:0/2059818801 wait complete. 2024-07-27T03:15:09.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.976+0000 7f24fbe57640 1 -- 172.21.15.67:0/2995298901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24f41035e0 msgr2=0x7f24f41039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:09.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.976+0000 7f24fbe57640 1 --2- 172.21.15.67:0/2995298901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24f41035e0 0x7f24f41039f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f24e4009900 tx=0x7f24e402a410 comp rx=0 tx=0).stop 2024-07-27T03:15:09.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.976+0000 7f24fbe57640 1 -- 172.21.15.67:0/2995298901 shutdown_connections 2024-07-27T03:15:09.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.976+0000 7f24fbe57640 1 --2- 172.21.15.67:0/2995298901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24f4109870 0x7f24f41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:09.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.976+0000 7f24fbe57640 1 --2- 172.21.15.67:0/2995298901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24f4108f20 0x7f24f4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:09.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.976+0000 7f24fbe57640 1 --2- 172.21.15.67:0/2995298901 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24f41035e0 0x7f24f41039f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:09.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.976+0000 7f24fbe57640 1 -- 172.21.15.67:0/2995298901 >> 172.21.15.67:0/2995298901 conn(0x7f24f40ff050 msgr2=0x7f24f41014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:09.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.976+0000 7f24fbe57640 1 -- 172.21.15.67:0/2995298901 shutdown_connections 2024-07-27T03:15:09.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.976+0000 7f24fbe57640 1 -- 172.21.15.67:0/2995298901 wait complete. 2024-07-27T03:15:09.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.977+0000 7f24fbe57640 1 Processor -- start 2024-07-27T03:15:09.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.977+0000 7f24fbe57640 1 -- start start 2024-07-27T03:15:09.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.978+0000 7f24fbe57640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24f41035e0 0x7f24f4075180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:09.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.978+0000 7f24fbe57640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24f4108f20 0x7f24f40756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:09.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.978+0000 7f24fbe57640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24f4109870 0x7f24f4075c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:09.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.978+0000 7f24fbe57640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f24f4113520 con 0x7f24f41035e0 2024-07-27T03:15:09.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.978+0000 7f24fbe57640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f24f41133a0 con 0x7f24f4108f20 2024-07-27T03:15:09.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.978+0000 7f24fbe57640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f24f41136a0 con 0x7f24f4109870 2024-07-27T03:15:09.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.978+0000 7f24f9bf3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24f41035e0 0x7f24f4075180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:09.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.978+0000 7f24f9bf3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24f41035e0 0x7f24f4075180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46806/0 (socket says 172.21.15.67:46806) 2024-07-27T03:15:09.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.978+0000 7f24f9bf3640 1 -- 172.21.15.67:0/238746152 learned_addr learned my addr 172.21.15.67:0/238746152 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:09.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.978+0000 7f24fa3f4640 1 --2- 172.21.15.67:0/238746152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24f4109870 0x7f24f4075c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:09.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.978+0000 7f24f93f2640 1 --2- 172.21.15.67:0/238746152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24f4108f20 0x7f24f40756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:09.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.978+0000 7f24f9bf3640 1 -- 172.21.15.67:0/238746152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24f4109870 msgr2=0x7f24f4075c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:09.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.978+0000 7f24f9bf3640 1 --2- 172.21.15.67:0/238746152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24f4109870 0x7f24f4075c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:09.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.978+0000 7f24f9bf3640 1 -- 172.21.15.67:0/238746152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24f4108f20 msgr2=0x7f24f40756c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:09.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.978+0000 7f24f9bf3640 1 --2- 172.21.15.67:0/238746152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24f4108f20 0x7f24f40756c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:09.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.978+0000 7f24f9bf3640 1 -- 172.21.15.67:0/238746152 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f41a8140 con 0x7f24f41035e0 2024-07-27T03:15:09.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.978+0000 7f24fa3f4640 1 --2- 172.21.15.67:0/238746152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24f4109870 0x7f24f4075c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:15:09.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.979+0000 7f24f9bf3640 1 --2- 172.21.15.67:0/238746152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24f41035e0 0x7f24f4075180 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f24e4006010 tx=0x7f24e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:09.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.979+0000 7f24f2ffd640 1 -- 172.21.15.67:0/238746152 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e403f030 con 0x7f24f41035e0 2024-07-27T03:15:09.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.979+0000 7f24fbe57640 1 -- 172.21.15.67:0/238746152 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f41a83d0 con 0x7f24f41035e0 2024-07-27T03:15:09.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.979+0000 7f24f2ffd640 1 -- 172.21.15.67:0/238746152 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24e4003800 con 0x7f24f41035e0 2024-07-27T03:15:09.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.979+0000 7f24f2ffd640 1 -- 172.21.15.67:0/238746152 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4035070 con 0x7f24f41035e0 2024-07-27T03:15:09.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.979+0000 7f24fbe57640 1 -- 172.21.15.67:0/238746152 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f41a8900 con 0x7f24f41035e0 2024-07-27T03:15:09.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.980+0000 7f24fbe57640 1 -- 172.21.15.67:0/238746152 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24f41039f0 con 0x7f24f41035e0 2024-07-27T03:15:09.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.982+0000 7f24f2ffd640 1 -- 172.21.15.67:0/238746152 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f24e4003cb0 con 0x7f24f41035e0 2024-07-27T03:15:09.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.983+0000 7f24f2ffd640 1 --2- 172.21.15.67:0/238746152 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f24dc0b3c60 0x7f24dc0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:09.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.983+0000 7f24f93f2640 1 --2- 172.21.15.67:0/238746152 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f24dc0b3c60 0x7f24dc0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:09.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.983+0000 7f24f2ffd640 1 -- 172.21.15.67:0/238746152 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f24e4104dd0 con 0x7f24f41035e0 2024-07-27T03:15:09.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.984+0000 7f24f93f2640 1 --2- 172.21.15.67:0/238746152 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f24dc0b3c60 0x7f24dc0b6170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f24e8005e10 tx=0x7f24e8005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:09.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:09.987+0000 7f24f2ffd640 1 -- 172.21.15.67:0/238746152 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f24e40cbf10 con 0x7f24f41035e0 2024-07-27T03:15:10.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:10.174+0000 7f24fbe57640 1 -- 172.21.15.67:0/238746152 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f24f410b650 con 0x7f24f41035e0 2024-07-27T03:15:10.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:10.174+0000 7f24f2ffd640 1 -- 172.21.15.67:0/238746152 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f24e40cb810 con 0x7f24f41035e0 2024-07-27T03:15:10.177 INFO:teuthology.orchestra.run.smithi067.stdout:231928234044 2024-07-27T03:15:10.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:10.177+0000 7f24fbe57640 1 -- 172.21.15.67:0/238746152 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f24dc0b3c60 msgr2=0x7f24dc0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:10.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:10.177+0000 7f24fbe57640 1 --2- 172.21.15.67:0/238746152 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f24dc0b3c60 0x7f24dc0b6170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f24e8005e10 tx=0x7f24e8005da0 comp rx=0 tx=0).stop 2024-07-27T03:15:10.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:10.177+0000 7f24fbe57640 1 -- 172.21.15.67:0/238746152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24f41035e0 msgr2=0x7f24f4075180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:10.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:10.177+0000 7f24fbe57640 1 --2- 172.21.15.67:0/238746152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24f41035e0 0x7f24f4075180 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f24e4006010 tx=0x7f24e40042f0 comp rx=0 tx=0).stop 2024-07-27T03:15:10.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:10.177+0000 7f24fbe57640 1 -- 172.21.15.67:0/238746152 shutdown_connections 2024-07-27T03:15:10.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:10.177+0000 7f24fbe57640 1 --2- 172.21.15.67:0/238746152 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f24dc0b3c60 0x7f24dc0b6170 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:10.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:10.177+0000 7f24fbe57640 1 --2- 172.21.15.67:0/238746152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f24f4109870 0x7f24f4075c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:10.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:10.177+0000 7f24fbe57640 1 --2- 172.21.15.67:0/238746152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f24f4108f20 0x7f24f40756c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:10.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:10.177+0000 7f24fbe57640 1 --2- 172.21.15.67:0/238746152 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f24f41035e0 0x7f24f4075180 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:10.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:10.177+0000 7f24fbe57640 1 -- 172.21.15.67:0/238746152 >> 172.21.15.67:0/238746152 conn(0x7f24f40ff050 msgr2=0x7f24f4101470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:10.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:10.177+0000 7f24fbe57640 1 -- 172.21.15.67:0/238746152 shutdown_connections 2024-07-27T03:15:10.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:10.177+0000 7f24fbe57640 1 -- 172.21.15.67:0/238746152 wait complete. 2024-07-27T03:15:10.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:10 smithi067 ceph-mon[32814]: pgmap v388: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:15:10.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:10 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2059818801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-07-27T03:15:10.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:10 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/238746152' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-07-27T03:15:11.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:10 smithi149 ceph-mon[37173]: pgmap v388: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:15:11.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:10 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2059818801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-07-27T03:15:11.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:10 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/238746152' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-07-27T03:15:11.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:10 smithi175 ceph-mon[37125]: pgmap v388: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:15:11.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:10 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2059818801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-07-27T03:15:11.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:10 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/238746152' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-07-27T03:15:11.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.215+0000 7f7a32bb1640 1 -- 172.21.15.67:0/3692238019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a2c1024d0 msgr2=0x7f7a2c1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:11.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.215+0000 7f7a32bb1640 1 --2- 172.21.15.67:0/3692238019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a2c1024d0 0x7f7a2c1028e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7a24009900 tx=0x7f7a2402a410 comp rx=0 tx=0).stop 2024-07-27T03:15:11.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.215+0000 7f7a32bb1640 1 -- 172.21.15.67:0/3692238019 shutdown_connections 2024-07-27T03:15:11.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.215+0000 7f7a32bb1640 1 --2- 172.21.15.67:0/3692238019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a2c108760 0x7f7a2c10ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:11.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.215+0000 7f7a32bb1640 1 --2- 172.21.15.67:0/3692238019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a2c107e10 0x7f7a2c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:11.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.215+0000 7f7a32bb1640 1 --2- 172.21.15.67:0/3692238019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a2c1024d0 0x7f7a2c1028e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:11.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.215+0000 7f7a32bb1640 1 -- 172.21.15.67:0/3692238019 >> 172.21.15.67:0/3692238019 conn(0x7f7a2c0791a0 msgr2=0x7f7a2c0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.215+0000 7f7a32bb1640 1 -- 172.21.15.67:0/3692238019 shutdown_connections 2024-07-27T03:15:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.215+0000 7f7a32bb1640 1 -- 172.21.15.67:0/3692238019 wait complete. 2024-07-27T03:15:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.216+0000 7f7a32bb1640 1 Processor -- start 2024-07-27T03:15:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.216+0000 7f7a32bb1640 1 -- start start 2024-07-27T03:15:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.217+0000 7f7a32bb1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a2c1024d0 0x7f7a2c19acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.217+0000 7f7a32bb1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a2c107e10 0x7f7a2c19b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.217+0000 7f7a32bb1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a2c108760 0x7f7a2c19b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.217+0000 7f7a32bb1640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c1123e0 con 0x7f7a2c108760 2024-07-27T03:15:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.217+0000 7f7a32bb1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c112260 con 0x7f7a2c107e10 2024-07-27T03:15:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.217+0000 7f7a32bb1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c112560 con 0x7f7a2c1024d0 2024-07-27T03:15:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.217+0000 7f7a3114e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a2c108760 0x7f7a2c19b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.217+0000 7f7a3114e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a2c108760 0x7f7a2c19b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46836/0 (socket says 172.21.15.67:46836) 2024-07-27T03:15:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.217+0000 7f7a3114e640 1 -- 172.21.15.67:0/1359714455 learned_addr learned my addr 172.21.15.67:0/1359714455 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.217+0000 7f7a3094d640 1 --2- 172.21.15.67:0/1359714455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a2c1024d0 0x7f7a2c19acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.217+0000 7f7a2bfff640 1 --2- 172.21.15.67:0/1359714455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a2c107e10 0x7f7a2c19b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.217+0000 7f7a3114e640 1 -- 172.21.15.67:0/1359714455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a2c1024d0 msgr2=0x7f7a2c19acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.217+0000 7f7a3114e640 1 --2- 172.21.15.67:0/1359714455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a2c1024d0 0x7f7a2c19acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.217+0000 7f7a3114e640 1 -- 172.21.15.67:0/1359714455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a2c107e10 msgr2=0x7f7a2c19b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.217+0000 7f7a3114e640 1 --2- 172.21.15.67:0/1359714455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a2c107e10 0x7f7a2c19b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:11.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.217+0000 7f7a3114e640 1 -- 172.21.15.67:0/1359714455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2c1a5f60 con 0x7f7a2c108760 2024-07-27T03:15:11.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.217+0000 7f7a3114e640 1 --2- 172.21.15.67:0/1359714455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a2c108760 0x7f7a2c19b750 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00eaa0 tx=0x7f7a1c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:11.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.218+0000 7f7a29ffb640 1 -- 172.21.15.67:0/1359714455 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c00cdb0 con 0x7f7a2c108760 2024-07-27T03:15:11.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.218+0000 7f7a29ffb640 1 -- 172.21.15.67:0/1359714455 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a1c005020 con 0x7f7a2c108760 2024-07-27T03:15:11.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.218+0000 7f7a29ffb640 1 -- 172.21.15.67:0/1359714455 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c0116d0 con 0x7f7a2c108760 2024-07-27T03:15:11.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.218+0000 7f7a32bb1640 1 -- 172.21.15.67:0/1359714455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2c1a6250 con 0x7f7a2c108760 2024-07-27T03:15:11.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.218+0000 7f7a32bb1640 1 -- 172.21.15.67:0/1359714455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a2c1a6790 con 0x7f7a2c108760 2024-07-27T03:15:11.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.219+0000 7f7a32bb1640 1 -- 172.21.15.67:0/1359714455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a2c1028e0 con 0x7f7a2c108760 2024-07-27T03:15:11.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.220+0000 7f7a29ffb640 1 -- 172.21.15.67:0/1359714455 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f7a1c0051d0 con 0x7f7a2c108760 2024-07-27T03:15:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.222+0000 7f7a29ffb640 1 --2- 172.21.15.67:0/1359714455 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7a180b3c90 0x7f7a180b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:11.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.222+0000 7f7a29ffb640 1 -- 172.21.15.67:0/1359714455 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f7a1c0e4260 con 0x7f7a2c108760 2024-07-27T03:15:11.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.223+0000 7f7a3094d640 1 --2- 172.21.15.67:0/1359714455 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7a180b3c90 0x7f7a180b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:11.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.223+0000 7f7a3094d640 1 --2- 172.21.15.67:0/1359714455 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7a180b3c90 0x7f7a180b61a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7a24000c00 tx=0x7f7a24031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:11.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.227+0000 7f7a29ffb640 1 -- 172.21.15.67:0/1359714455 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f7a1c016030 con 0x7f7a2c108760 2024-07-27T03:15:11.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.413+0000 7f7a32bb1640 1 -- 172.21.15.67:0/1359714455 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f7a2c02e100 con 0x7f7a2c108760 2024-07-27T03:15:11.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.413+0000 7f7a29ffb640 1 -- 172.21.15.67:0/1359714455 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7a1c0aacf0 con 0x7f7a2c108760 2024-07-27T03:15:11.416 INFO:teuthology.orchestra.run.smithi067.stdout:137438953552 2024-07-27T03:15:11.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.415+0000 7f7a32bb1640 1 -- 172.21.15.67:0/1359714455 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7a180b3c90 msgr2=0x7f7a180b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:11.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.415+0000 7f7a32bb1640 1 --2- 172.21.15.67:0/1359714455 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7a180b3c90 0x7f7a180b61a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7a24000c00 tx=0x7f7a24031400 comp rx=0 tx=0).stop 2024-07-27T03:15:11.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.415+0000 7f7a32bb1640 1 -- 172.21.15.67:0/1359714455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a2c108760 msgr2=0x7f7a2c19b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:11.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.415+0000 7f7a32bb1640 1 --2- 172.21.15.67:0/1359714455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a2c108760 0x7f7a2c19b750 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00eaa0 tx=0x7f7a1c00ef60 comp rx=0 tx=0).stop 2024-07-27T03:15:11.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.415+0000 7f7a32bb1640 1 -- 172.21.15.67:0/1359714455 shutdown_connections 2024-07-27T03:15:11.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.415+0000 7f7a32bb1640 1 --2- 172.21.15.67:0/1359714455 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7a180b3c90 0x7f7a180b61a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:11.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.415+0000 7f7a32bb1640 1 --2- 172.21.15.67:0/1359714455 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a2c108760 0x7f7a2c19b750 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:11.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.415+0000 7f7a32bb1640 1 --2- 172.21.15.67:0/1359714455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a2c107e10 0x7f7a2c19b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:11.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.415+0000 7f7a32bb1640 1 --2- 172.21.15.67:0/1359714455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a2c1024d0 0x7f7a2c19acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:11.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.415+0000 7f7a32bb1640 1 -- 172.21.15.67:0/1359714455 >> 172.21.15.67:0/1359714455 conn(0x7f7a2c0791a0 msgr2=0x7f7a2c100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:11.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.416+0000 7f7a32bb1640 1 -- 172.21.15.67:0/1359714455 shutdown_connections 2024-07-27T03:15:11.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:11.416+0000 7f7a32bb1640 1 -- 172.21.15.67:0/1359714455 wait complete. 2024-07-27T03:15:11.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:11 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1359714455' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-07-27T03:15:12.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:11 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1359714455' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-07-27T03:15:12.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:11 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1359714455' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-07-27T03:15:12.129 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953548 got 137438953552 for osd.4 2024-07-27T03:15:12.129 DEBUG:teuthology.parallel:result is None 2024-07-27T03:15:12.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.302+0000 7f5282850640 1 -- 172.21.15.67:0/1436976062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c1024f0 msgr2=0x7f527c102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:12.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.302+0000 7f5282850640 1 --2- 172.21.15.67:0/1436976062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c1024f0 0x7f527c102900 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f526c009900 tx=0x7f526c02a410 comp rx=0 tx=0).stop 2024-07-27T03:15:12.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.303+0000 7f5282850640 1 -- 172.21.15.67:0/1436976062 shutdown_connections 2024-07-27T03:15:12.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.303+0000 7f5282850640 1 --2- 172.21.15.67:0/1436976062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f527c108780 0x7f527c10efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:12.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.303+0000 7f5282850640 1 --2- 172.21.15.67:0/1436976062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f527c107e30 0x7f527c108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:12.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.303+0000 7f5282850640 1 --2- 172.21.15.67:0/1436976062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c1024f0 0x7f527c102900 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:12.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.303+0000 7f5282850640 1 -- 172.21.15.67:0/1436976062 >> 172.21.15.67:0/1436976062 conn(0x7f527c0791b0 msgr2=0x7f527c0795c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:12.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.303+0000 7f5282850640 1 -- 172.21.15.67:0/1436976062 shutdown_connections 2024-07-27T03:15:12.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.303+0000 7f5282850640 1 -- 172.21.15.67:0/1436976062 wait complete. 2024-07-27T03:15:12.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.303+0000 7f5282850640 1 Processor -- start 2024-07-27T03:15:12.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.303+0000 7f5282850640 1 -- start start 2024-07-27T03:15:12.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.304+0000 7f5282850640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c1024f0 0x7f527c199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:12.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.304+0000 7f5282850640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f527c107e30 0x7f527c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:12.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.304+0000 7f5282850640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f527c108780 0x7f527c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:12.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.304+0000 7f5282850640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f527c113510 con 0x7f527c1024f0 2024-07-27T03:15:12.308 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.304+0000 7f5282850640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f527c113390 con 0x7f527c108780 2024-07-27T03:15:12.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.304+0000 7f5282850640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f527c113690 con 0x7f527c107e30 2024-07-27T03:15:12.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.304+0000 7f527bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c1024f0 0x7f527c199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:12.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.304+0000 7f527bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c1024f0 0x7f527c199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46866/0 (socket says 172.21.15.67:46866) 2024-07-27T03:15:12.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.304+0000 7f527bfff640 1 -- 172.21.15.67:0/2152640841 learned_addr learned my addr 172.21.15.67:0/2152640841 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:12.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.304+0000 7f527b7fe640 1 --2- 172.21.15.67:0/2152640841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f527c107e30 0x7f527c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:12.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.304+0000 7f5280ded640 1 --2- 172.21.15.67:0/2152640841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f527c108780 0x7f527c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:12.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.304+0000 7f527bfff640 1 -- 172.21.15.67:0/2152640841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f527c107e30 msgr2=0x7f527c19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:12.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.304+0000 7f527bfff640 1 --2- 172.21.15.67:0/2152640841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f527c107e30 0x7f527c19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:12.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.304+0000 7f527bfff640 1 -- 172.21.15.67:0/2152640841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f527c108780 msgr2=0x7f527c19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:12.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.304+0000 7f527bfff640 1 --2- 172.21.15.67:0/2152640841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f527c108780 0x7f527c19f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:12.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.304+0000 7f527bfff640 1 -- 172.21.15.67:0/2152640841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527c19fd70 con 0x7f527c1024f0 2024-07-27T03:15:12.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.305+0000 7f527bfff640 1 --2- 172.21.15.67:0/2152640841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c1024f0 0x7f527c199ec0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f526c031440 tx=0x7f526c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:12.309 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.305+0000 7f52797fa640 1 -- 172.21.15.67:0/2152640841 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c03f030 con 0x7f527c1024f0 2024-07-27T03:15:12.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.305+0000 7f52797fa640 1 -- 172.21.15.67:0/2152640841 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f526c003aa0 con 0x7f527c1024f0 2024-07-27T03:15:12.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.305+0000 7f5282850640 1 -- 172.21.15.67:0/2152640841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f527c1a0000 con 0x7f527c1024f0 2024-07-27T03:15:12.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.305+0000 7f52797fa640 1 -- 172.21.15.67:0/2152640841 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c035070 con 0x7f527c1024f0 2024-07-27T03:15:12.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.305+0000 7f5282850640 1 -- 172.21.15.67:0/2152640841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f527c1a04e0 con 0x7f527c1024f0 2024-07-27T03:15:12.310 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.306+0000 7f5282850640 1 -- 172.21.15.67:0/2152640841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f527c103770 con 0x7f527c1024f0 2024-07-27T03:15:12.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.309+0000 7f52797fa640 1 -- 172.21.15.67:0/2152640841 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f526c042030 con 0x7f527c1024f0 2024-07-27T03:15:12.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.309+0000 7f52797fa640 1 --2- 172.21.15.67:0/2152640841 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f52640b3c90 0x7f52640b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:12.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.309+0000 7f527b7fe640 1 --2- 172.21.15.67:0/2152640841 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f52640b3c90 0x7f52640b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:12.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.310+0000 7f52797fa640 1 -- 172.21.15.67:0/2152640841 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f526c104cb0 con 0x7f527c1024f0 2024-07-27T03:15:12.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.310+0000 7f527b7fe640 1 --2- 172.21.15.67:0/2152640841 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f52640b3c90 0x7f52640b61a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5270005e10 tx=0x7f5270005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:12.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.313+0000 7f52797fa640 1 -- 172.21.15.67:0/2152640841 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f526c049030 con 0x7f527c1024f0 2024-07-27T03:15:12.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.508+0000 7f5282850640 1 -- 172.21.15.67:0/2152640841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f527c109fc0 con 0x7f527c1024f0 2024-07-27T03:15:12.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.509+0000 7f52797fa640 1 -- 172.21.15.67:0/2152640841 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f526c0cbdf0 con 0x7f527c1024f0 2024-07-27T03:15:12.512 INFO:teuthology.orchestra.run.smithi067.stdout:206158430273 2024-07-27T03:15:12.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.511+0000 7f5282850640 1 -- 172.21.15.67:0/2152640841 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f52640b3c90 msgr2=0x7f52640b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:12.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.511+0000 7f5282850640 1 --2- 172.21.15.67:0/2152640841 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f52640b3c90 0x7f52640b61a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5270005e10 tx=0x7f5270005da0 comp rx=0 tx=0).stop 2024-07-27T03:15:12.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.511+0000 7f5282850640 1 -- 172.21.15.67:0/2152640841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c1024f0 msgr2=0x7f527c199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:12.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.511+0000 7f5282850640 1 --2- 172.21.15.67:0/2152640841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c1024f0 0x7f527c199ec0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f526c031440 tx=0x7f526c004270 comp rx=0 tx=0).stop 2024-07-27T03:15:12.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.511+0000 7f5282850640 1 -- 172.21.15.67:0/2152640841 shutdown_connections 2024-07-27T03:15:12.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.511+0000 7f5282850640 1 --2- 172.21.15.67:0/2152640841 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f52640b3c90 0x7f52640b61a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:12.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.511+0000 7f5282850640 1 --2- 172.21.15.67:0/2152640841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f527c108780 0x7f527c19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:12.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.511+0000 7f5282850640 1 --2- 172.21.15.67:0/2152640841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f527c107e30 0x7f527c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:12.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.511+0000 7f5282850640 1 --2- 172.21.15.67:0/2152640841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f527c1024f0 0x7f527c199ec0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:12.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.511+0000 7f5282850640 1 -- 172.21.15.67:0/2152640841 >> 172.21.15.67:0/2152640841 conn(0x7f527c0791b0 msgr2=0x7f527c100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:12.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.511+0000 7f5282850640 1 -- 172.21.15.67:0/2152640841 shutdown_connections 2024-07-27T03:15:12.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.511+0000 7f5282850640 1 -- 172.21.15.67:0/2152640841 wait complete. 2024-07-27T03:15:12.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.548+0000 7f37ed527640 1 -- 172.21.15.67:0/2566334297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37e81035f0 msgr2=0x7f37e8103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:12.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.548+0000 7f37ed527640 1 --2- 172.21.15.67:0/2566334297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37e81035f0 0x7f37e8103a00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009900 tx=0x7f37d802a410 comp rx=0 tx=0).stop 2024-07-27T03:15:12.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.548+0000 7f37ed527640 1 -- 172.21.15.67:0/2566334297 shutdown_connections 2024-07-27T03:15:12.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.548+0000 7f37ed527640 1 --2- 172.21.15.67:0/2566334297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37e8109880 0x7f37e81100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:12.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.548+0000 7f37ed527640 1 --2- 172.21.15.67:0/2566334297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37e8108f30 0x7f37e8109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:12.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.548+0000 7f37ed527640 1 --2- 172.21.15.67:0/2566334297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37e81035f0 0x7f37e8103a00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:12.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.549+0000 7f37ed527640 1 -- 172.21.15.67:0/2566334297 >> 172.21.15.67:0/2566334297 conn(0x7f37e80ff080 msgr2=0x7f37e81014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:12.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.549+0000 7f37ed527640 1 -- 172.21.15.67:0/2566334297 shutdown_connections 2024-07-27T03:15:12.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.549+0000 7f37ed527640 1 -- 172.21.15.67:0/2566334297 wait complete. 2024-07-27T03:15:12.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.550+0000 7f37ed527640 1 Processor -- start 2024-07-27T03:15:12.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.550+0000 7f37ed527640 1 -- start start 2024-07-27T03:15:12.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.551+0000 7f37ed527640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37e81035f0 0x7f37e8075180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:12.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.551+0000 7f37ed527640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37e8108f30 0x7f37e80756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:12.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.551+0000 7f37ed527640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37e8109880 0x7f37e8075c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:12.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.551+0000 7f37ed527640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f37e8113530 con 0x7f37e8108f30 2024-07-27T03:15:12.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.551+0000 7f37ed527640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81133b0 con 0x7f37e8109880 2024-07-27T03:15:12.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.551+0000 7f37ed527640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81136b0 con 0x7f37e81035f0 2024-07-27T03:15:12.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.551+0000 7f37e67fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37e8108f30 0x7f37e80756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:12.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.551+0000 7f37e67fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37e8108f30 0x7f37e80756c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46890/0 (socket says 172.21.15.67:46890) 2024-07-27T03:15:12.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.551+0000 7f37e67fc640 1 -- 172.21.15.67:0/3889559237 learned_addr learned my addr 172.21.15.67:0/3889559237 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:12.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.551+0000 7f37e6ffd640 1 --2- 172.21.15.67:0/3889559237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37e81035f0 0x7f37e8075180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:12.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.551+0000 7f37e77fe640 1 --2- 172.21.15.67:0/3889559237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37e8109880 0x7f37e8075c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:12.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.551+0000 7f37e67fc640 1 -- 172.21.15.67:0/3889559237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37e81035f0 msgr2=0x7f37e8075180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:12.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.551+0000 7f37e67fc640 1 --2- 172.21.15.67:0/3889559237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37e81035f0 0x7f37e8075180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:12.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.551+0000 7f37e67fc640 1 -- 172.21.15.67:0/3889559237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37e8109880 msgr2=0x7f37e8075c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:12.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.551+0000 7f37e67fc640 1 --2- 172.21.15.67:0/3889559237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37e8109880 0x7f37e8075c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:12.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.551+0000 7f37e67fc640 1 -- 172.21.15.67:0/3889559237 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e81a81c0 con 0x7f37e8108f30 2024-07-27T03:15:12.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.552+0000 7f37e67fc640 1 --2- 172.21.15.67:0/3889559237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37e8108f30 0x7f37e80756c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00d950 tx=0x7f37dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:12.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.552+0000 7f37cffff640 1 -- 172.21.15.67:0/3889559237 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc017070 con 0x7f37e8108f30 2024-07-27T03:15:12.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.552+0000 7f37ed527640 1 -- 172.21.15.67:0/3889559237 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e81a84b0 con 0x7f37e8108f30 2024-07-27T03:15:12.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.552+0000 7f37cffff640 1 -- 172.21.15.67:0/3889559237 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37dc009c30 con 0x7f37e8108f30 2024-07-27T03:15:12.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.552+0000 7f37cffff640 1 -- 172.21.15.67:0/3889559237 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0047a0 con 0x7f37e8108f30 2024-07-27T03:15:12.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.552+0000 7f37ed527640 1 -- 172.21.15.67:0/3889559237 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e81a8a40 con 0x7f37e8108f30 2024-07-27T03:15:12.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.553+0000 7f37ed527640 1 -- 172.21.15.67:0/3889559237 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e8104870 con 0x7f37e8108f30 2024-07-27T03:15:12.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.555+0000 7f37cffff640 1 -- 172.21.15.67:0/3889559237 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f37dc004020 con 0x7f37e8108f30 2024-07-27T03:15:12.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.555+0000 7f37cffff640 1 --2- 172.21.15.67:0/3889559237 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f37d00b3c60 0x7f37d00b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:12.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.556+0000 7f37e6ffd640 1 --2- 172.21.15.67:0/3889559237 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f37d00b3c60 0x7f37d00b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:12.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.556+0000 7f37cffff640 1 -- 172.21.15.67:0/3889559237 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f37dc0e3d80 con 0x7f37e8108f30 2024-07-27T03:15:12.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.556+0000 7f37e6ffd640 1 --2- 172.21.15.67:0/3889559237 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f37d00b3c60 0x7f37d00b6170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f37d8000c00 tx=0x7f37d8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:12.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.560+0000 7f37cffff640 1 -- 172.21.15.67:0/3889559237 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f37dc020020 con 0x7f37e8108f30 2024-07-27T03:15:12.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:12 smithi067 ceph-mon[32814]: pgmap v389: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:12.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:12 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:15:12.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:12 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:15:12.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:12 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2152640841' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-07-27T03:15:12.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.754+0000 7f37ed527640 1 -- 172.21.15.67:0/3889559237 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f37e810b070 con 0x7f37e8108f30 2024-07-27T03:15:12.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.754+0000 7f37cffff640 1 -- 172.21.15.67:0/3889559237 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f37dc009de0 con 0x7f37e8108f30 2024-07-27T03:15:12.757 INFO:teuthology.orchestra.run.smithi067.stdout:283467841588 2024-07-27T03:15:12.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.757+0000 7f37ed527640 1 -- 172.21.15.67:0/3889559237 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f37d00b3c60 msgr2=0x7f37d00b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:12.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.757+0000 7f37ed527640 1 --2- 172.21.15.67:0/3889559237 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f37d00b3c60 0x7f37d00b6170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f37d8000c00 tx=0x7f37d8031400 comp rx=0 tx=0).stop 2024-07-27T03:15:12.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.757+0000 7f37ed527640 1 -- 172.21.15.67:0/3889559237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37e8108f30 msgr2=0x7f37e80756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:12.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.757+0000 7f37ed527640 1 --2- 172.21.15.67:0/3889559237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37e8108f30 0x7f37e80756c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00d950 tx=0x7f37dc00de10 comp rx=0 tx=0).stop 2024-07-27T03:15:12.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.757+0000 7f37ed527640 1 -- 172.21.15.67:0/3889559237 shutdown_connections 2024-07-27T03:15:12.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.757+0000 7f37ed527640 1 --2- 172.21.15.67:0/3889559237 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f37d00b3c60 0x7f37d00b6170 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:12.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.757+0000 7f37ed527640 1 --2- 172.21.15.67:0/3889559237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37e8109880 0x7f37e8075c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:12.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.757+0000 7f37ed527640 1 --2- 172.21.15.67:0/3889559237 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f37e8108f30 0x7f37e80756c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:12.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.757+0000 7f37ed527640 1 --2- 172.21.15.67:0/3889559237 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37e81035f0 0x7f37e8075180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:12.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.757+0000 7f37ed527640 1 -- 172.21.15.67:0/3889559237 >> 172.21.15.67:0/3889559237 conn(0x7f37e80ff080 msgr2=0x7f37e81014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:12.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.757+0000 7f37ed527640 1 -- 172.21.15.67:0/3889559237 shutdown_connections 2024-07-27T03:15:12.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:12.757+0000 7f37ed527640 1 -- 172.21.15.67:0/3889559237 wait complete. 2024-07-27T03:15:13.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:12 smithi149 ceph-mon[37173]: pgmap v389: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:13.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:12 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:15:13.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:12 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:15:13.001 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:12 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2152640841' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-07-27T03:15:13.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:12 smithi175 ceph-mon[37125]: pgmap v389: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:13.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:12 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:15:13.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:12 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:15:13.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:12 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2152640841' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-07-27T03:15:13.220 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411413 got 77309411417 for osd.2 2024-07-27T03:15:13.221 DEBUG:teuthology.parallel:result is None 2024-07-27T03:15:13.573 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234039 got 231928234044 for osd.8 2024-07-27T03:15:13.573 DEBUG:teuthology.parallel:result is None 2024-07-27T03:15:13.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:13 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3889559237' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-07-27T03:15:13.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:13 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:15:13.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:13 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:15:13.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:13 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:15:13.982 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:15:14.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:13 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3889559237' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-07-27T03:15:14.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:13 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:15:14.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:13 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:15:14.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:13 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:15:14.011 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574936 got 55834574941 for osd.1 2024-07-27T03:15:14.012 DEBUG:teuthology.parallel:result is None 2024-07-27T03:15:14.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:13 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3889559237' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-07-27T03:15:14.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:13 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:15:14.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:13 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:15:14.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:13 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:15:14.652 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430268 got 206158430273 for osd.7 2024-07-27T03:15:14.653 DEBUG:teuthology.parallel:result is None 2024-07-27T03:15:14.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:14 smithi067 ceph-mon[32814]: pgmap v390: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:15.053 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841580 got 283467841588 for osd.10 2024-07-27T03:15:15.054 DEBUG:teuthology.parallel:result is None 2024-07-27T03:15:15.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:14 smithi175 ceph-mon[37125]: pgmap v390: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:15.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:14 smithi149 ceph-mon[37173]: pgmap v390: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:16.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:15 smithi175 ceph-mon[37125]: pgmap v391: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:16.191 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:15 smithi067 ceph-mon[32814]: pgmap v391: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:16.250 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:15 smithi149 ceph-mon[37173]: pgmap v391: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:16.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.854+0000 7fcf14871640 1 -- 172.21.15.67:0/1715167240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf10108f10 msgr2=0x7fcf10109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:16.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.854+0000 7fcf14871640 1 --2- 172.21.15.67:0/1715167240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf10108f10 0x7fcf10109320 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009900 tx=0x7fcf0402a410 comp rx=0 tx=0).stop 2024-07-27T03:15:16.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.854+0000 7fcf14871640 1 -- 172.21.15.67:0/1715167240 shutdown_connections 2024-07-27T03:15:16.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.854+0000 7fcf14871640 1 --2- 172.21.15.67:0/1715167240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf10109860 0x7fcf10110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:16.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.854+0000 7fcf14871640 1 --2- 172.21.15.67:0/1715167240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf10108f10 0x7fcf10109320 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:16.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.854+0000 7fcf14871640 1 --2- 172.21.15.67:0/1715167240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf101035d0 0x7fcf101039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:16.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.854+0000 7fcf14871640 1 -- 172.21.15.67:0/1715167240 >> 172.21.15.67:0/1715167240 conn(0x7fcf1007b2c0 msgr2=0x7fcf1007b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:16.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.854+0000 7fcf14871640 1 -- 172.21.15.67:0/1715167240 shutdown_connections 2024-07-27T03:15:16.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.854+0000 7fcf14871640 1 -- 172.21.15.67:0/1715167240 wait complete. 2024-07-27T03:15:16.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.855+0000 7fcf14871640 1 Processor -- start 2024-07-27T03:15:16.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.855+0000 7fcf14871640 1 -- start start 2024-07-27T03:15:16.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.855+0000 7fcf14871640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf101035d0 0x7fcf10199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:16.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.855+0000 7fcf14871640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf10108f10 0x7fcf1019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:16.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.855+0000 7fcf14871640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf10109860 0x7fcf1019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:16.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.855+0000 7fcf14871640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcf101134f0 con 0x7fcf10109860 2024-07-27T03:15:16.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.855+0000 7fcf14871640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcf10113370 con 0x7fcf10108f10 2024-07-27T03:15:16.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.855+0000 7fcf14871640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcf10113670 con 0x7fcf101035d0 2024-07-27T03:15:16.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.856+0000 7fcf0ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf10109860 0x7fcf1019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:16.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.856+0000 7fcf0ed9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf10109860 0x7fcf1019f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55436/0 (socket says 172.21.15.67:55436) 2024-07-27T03:15:16.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.856+0000 7fcf0ed9d640 1 -- 172.21.15.67:0/3145899401 learned_addr learned my addr 172.21.15.67:0/3145899401 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:16.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.856+0000 7fcf0dd9b640 1 --2- 172.21.15.67:0/3145899401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf10108f10 0x7fcf1019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:16.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.856+0000 7fcf0e59c640 1 --2- 172.21.15.67:0/3145899401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf101035d0 0x7fcf10199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:16.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.856+0000 7fcf0ed9d640 1 -- 172.21.15.67:0/3145899401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf101035d0 msgr2=0x7fcf10199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:16.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.856+0000 7fcf0ed9d640 1 --2- 172.21.15.67:0/3145899401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf101035d0 0x7fcf10199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:16.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.856+0000 7fcf0ed9d640 1 -- 172.21.15.67:0/3145899401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf10108f10 msgr2=0x7fcf1019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:16.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.856+0000 7fcf0ed9d640 1 --2- 172.21.15.67:0/3145899401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf10108f10 0x7fcf1019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:16.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.856+0000 7fcf0ed9d640 1 -- 172.21.15.67:0/3145899401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1019fef0 con 0x7fcf10109860 2024-07-27T03:15:16.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.856+0000 7fcf0ed9d640 1 --2- 172.21.15.67:0/3145899401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf10109860 0x7fcf1019f7c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800eaa0 tx=0x7fcf0800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:16.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.856+0000 7fceff7fe640 1 -- 172.21.15.67:0/3145899401 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0800cdb0 con 0x7fcf10109860 2024-07-27T03:15:16.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.856+0000 7fcf14871640 1 -- 172.21.15.67:0/3145899401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf101a01e0 con 0x7fcf10109860 2024-07-27T03:15:16.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.856+0000 7fceff7fe640 1 -- 172.21.15.67:0/3145899401 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf08005020 con 0x7fcf10109860 2024-07-27T03:15:16.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.856+0000 7fceff7fe640 1 -- 172.21.15.67:0/3145899401 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf080116d0 con 0x7fcf10109860 2024-07-27T03:15:16.864 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.856+0000 7fcf14871640 1 -- 172.21.15.67:0/3145899401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf101a6940 con 0x7fcf10109860 2024-07-27T03:15:16.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.857+0000 7fcf14871640 1 -- 172.21.15.67:0/3145899401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf101039e0 con 0x7fcf10109860 2024-07-27T03:15:16.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.859+0000 7fceff7fe640 1 -- 172.21.15.67:0/3145899401 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7fcf080051d0 con 0x7fcf10109860 2024-07-27T03:15:16.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.860+0000 7fceff7fe640 1 --2- 172.21.15.67:0/3145899401 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcef80b3ce0 0x7fcef80b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:16.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.860+0000 7fcf0e59c640 1 --2- 172.21.15.67:0/3145899401 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcef80b3ce0 0x7fcef80b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:16.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.860+0000 7fceff7fe640 1 -- 172.21.15.67:0/3145899401 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fcf080e46c0 con 0x7fcf10109860 2024-07-27T03:15:16.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.861+0000 7fcf0e59c640 1 --2- 172.21.15.67:0/3145899401 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcef80b3ce0 0x7fcef80b61f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcf00005fd0 tx=0x7fcf00005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:16.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:16.863+0000 7fceff7fe640 1 -- 172.21.15.67:0/3145899401 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fcf08016030 con 0x7fcf10109860 2024-07-27T03:15:17.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:17.050+0000 7fcf14871640 1 -- 172.21.15.67:0/3145899401 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fcf1010b670 con 0x7fcf10109860 2024-07-27T03:15:17.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:17.050+0000 7fceff7fe640 1 -- 172.21.15.67:0/3145899401 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcf080ab7a0 con 0x7fcf10109860 2024-07-27T03:15:17.053 INFO:teuthology.orchestra.run.smithi067.stdout:111669149782 2024-07-27T03:15:17.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:17.052+0000 7fcf14871640 1 -- 172.21.15.67:0/3145899401 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcef80b3ce0 msgr2=0x7fcef80b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:17.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:17.052+0000 7fcf14871640 1 --2- 172.21.15.67:0/3145899401 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcef80b3ce0 0x7fcef80b61f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcf00005fd0 tx=0x7fcf00005850 comp rx=0 tx=0).stop 2024-07-27T03:15:17.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:17.052+0000 7fcf14871640 1 -- 172.21.15.67:0/3145899401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf10109860 msgr2=0x7fcf1019f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:17.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:17.052+0000 7fcf14871640 1 --2- 172.21.15.67:0/3145899401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf10109860 0x7fcf1019f7c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800eaa0 tx=0x7fcf0800ef60 comp rx=0 tx=0).stop 2024-07-27T03:15:17.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:17.052+0000 7fcf14871640 1 -- 172.21.15.67:0/3145899401 shutdown_connections 2024-07-27T03:15:17.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:17.052+0000 7fcf14871640 1 --2- 172.21.15.67:0/3145899401 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcef80b3ce0 0x7fcef80b61f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:17.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:17.052+0000 7fcf14871640 1 --2- 172.21.15.67:0/3145899401 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcf10109860 0x7fcf1019f7c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:17.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:17.052+0000 7fcf14871640 1 --2- 172.21.15.67:0/3145899401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf10108f10 0x7fcf1019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:17.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:17.052+0000 7fcf14871640 1 --2- 172.21.15.67:0/3145899401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf101035d0 0x7fcf10199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:17.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:17.052+0000 7fcf14871640 1 -- 172.21.15.67:0/3145899401 >> 172.21.15.67:0/3145899401 conn(0x7fcf1007b2c0 msgr2=0x7fcf10104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:17.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:17.053+0000 7fcf14871640 1 -- 172.21.15.67:0/3145899401 shutdown_connections 2024-07-27T03:15:17.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:17.053+0000 7fcf14871640 1 -- 172.21.15.67:0/3145899401 wait complete. 2024-07-27T03:15:17.375 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:17 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3145899401' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-07-27T03:15:17.378 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:17 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3145899401' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-07-27T03:15:17.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:17 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3145899401' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-07-27T03:15:17.687 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149776 got 111669149782 for osd.3 2024-07-27T03:15:17.688 DEBUG:teuthology.parallel:result is None 2024-07-27T03:15:17.688 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-07-27T03:15:17.688 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph pg dump --format=json 2024-07-27T03:15:17.958 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:15:18.395 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:18 smithi067 ceph-mon[32814]: pgmap v392: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:18.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:18 smithi149 ceph-mon[37173]: pgmap v392: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:18.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:18 smithi175 ceph-mon[37125]: pgmap v392: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:19.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.712+0000 7f299494b640 1 -- 172.21.15.67:0/415510590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2990107e10 msgr2=0x7f2990108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:19.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.712+0000 7f299494b640 1 --2- 172.21.15.67:0/415510590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2990107e10 0x7f2990108220 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2984009900 tx=0x7f298402a410 comp rx=0 tx=0).stop 2024-07-27T03:15:19.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.712+0000 7f299494b640 1 -- 172.21.15.67:0/415510590 shutdown_connections 2024-07-27T03:15:19.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.712+0000 7f299494b640 1 --2- 172.21.15.67:0/415510590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2990108760 0x7f299010ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:19.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.712+0000 7f299494b640 1 --2- 172.21.15.67:0/415510590 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2990107e10 0x7f2990108220 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:19.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.712+0000 7f299494b640 1 --2- 172.21.15.67:0/415510590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29901024d0 0x7f29901028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:19.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.712+0000 7f299494b640 1 -- 172.21.15.67:0/415510590 >> 172.21.15.67:0/415510590 conn(0x7f29900791a0 msgr2=0x7f29900795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:19.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.712+0000 7f299494b640 1 -- 172.21.15.67:0/415510590 shutdown_connections 2024-07-27T03:15:19.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.712+0000 7f299494b640 1 -- 172.21.15.67:0/415510590 wait complete. 2024-07-27T03:15:19.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.713+0000 7f299494b640 1 Processor -- start 2024-07-27T03:15:19.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.713+0000 7f299494b640 1 -- start start 2024-07-27T03:15:19.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.713+0000 7f299494b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29901024d0 0x7f2990104b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:19.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.713+0000 7f299494b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2990107e10 0x7f2990105090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:19.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.713+0000 7f299494b640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2990108760 0x7f29901055d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:19.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.713+0000 7f299494b640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2990112390 con 0x7f2990107e10 2024-07-27T03:15:19.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.713+0000 7f299494b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2990112210 con 0x7f29901024d0 2024-07-27T03:15:19.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.713+0000 7f299494b640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2990112510 con 0x7f2990108760 2024-07-27T03:15:19.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.714+0000 7f298dd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2990107e10 0x7f2990105090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:19.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.714+0000 7f298dd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2990107e10 0x7f2990105090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55456/0 (socket says 172.21.15.67:55456) 2024-07-27T03:15:19.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.714+0000 7f298dd9b640 1 -- 172.21.15.67:0/1977273682 learned_addr learned my addr 172.21.15.67:0/1977273682 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:19.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.714+0000 7f298e59c640 1 --2- 172.21.15.67:0/1977273682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29901024d0 0x7f2990104b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:19.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.714+0000 7f298ed9d640 1 --2- 172.21.15.67:0/1977273682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2990108760 0x7f29901055d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:19.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.714+0000 7f298dd9b640 1 -- 172.21.15.67:0/1977273682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2990108760 msgr2=0x7f29901055d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:19.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.714+0000 7f298dd9b640 1 --2- 172.21.15.67:0/1977273682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2990108760 0x7f29901055d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:19.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.714+0000 7f298dd9b640 1 -- 172.21.15.67:0/1977273682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29901024d0 msgr2=0x7f2990104b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:19.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.714+0000 7f298dd9b640 1 --2- 172.21.15.67:0/1977273682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29901024d0 0x7f2990104b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:19.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.714+0000 7f298dd9b640 1 -- 172.21.15.67:0/1977273682 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29901a8100 con 0x7f2990107e10 2024-07-27T03:15:19.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.714+0000 7f298dd9b640 1 --2- 172.21.15.67:0/1977273682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2990107e10 0x7f2990105090 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2984004270 tx=0x7f29840042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:19.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.714+0000 7f297f7fe640 1 -- 172.21.15.67:0/1977273682 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298403f030 con 0x7f2990107e10 2024-07-27T03:15:19.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.714+0000 7f299494b640 1 -- 172.21.15.67:0/1977273682 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29901a8390 con 0x7f2990107e10 2024-07-27T03:15:19.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.714+0000 7f297f7fe640 1 -- 172.21.15.67:0/1977273682 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2984003cb0 con 0x7f2990107e10 2024-07-27T03:15:19.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.715+0000 7f297f7fe640 1 -- 172.21.15.67:0/1977273682 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984035070 con 0x7f2990107e10 2024-07-27T03:15:19.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.715+0000 7f299494b640 1 -- 172.21.15.67:0/1977273682 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29901a88c0 con 0x7f2990107e10 2024-07-27T03:15:19.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.716+0000 7f299494b640 1 -- 172.21.15.67:0/1977273682 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29901028e0 con 0x7f2990107e10 2024-07-27T03:15:19.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.717+0000 7f297f7fe640 1 -- 172.21.15.67:0/1977273682 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f2984042030 con 0x7f2990107e10 2024-07-27T03:15:19.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.718+0000 7f297f7fe640 1 --2- 172.21.15.67:0/1977273682 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f29780b3c60 0x7f29780b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:19.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.718+0000 7f298e59c640 1 --2- 172.21.15.67:0/1977273682 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f29780b3c60 0x7f29780b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:19.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.718+0000 7f297f7fe640 1 -- 172.21.15.67:0/1977273682 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f29841046b0 con 0x7f2990107e10 2024-07-27T03:15:19.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.718+0000 7f298e59c640 1 --2- 172.21.15.67:0/1977273682 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f29780b3c60 0x7f29780b6170 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f29800058c0 tx=0x7f298000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:19.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.722+0000 7f297f7fe640 1 -- 172.21.15.67:0/1977273682 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f2984003e60 con 0x7f2990107e10 2024-07-27T03:15:19.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.905+0000 7f299494b640 1 -- 172.21.15.67:0/1977273682 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f299010b590 con 0x7f29780b3c60 2024-07-27T03:15:19.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.910+0000 7f297f7fe640 1 -- 172.21.15.67:0/1977273682 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340423 (secure 0 0 0) 0x7f299010b590 con 0x7f29780b3c60 2024-07-27T03:15:19.918 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:15:19.920 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-07-27T03:15:19.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.913+0000 7f299494b640 1 -- 172.21.15.67:0/1977273682 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f29780b3c60 msgr2=0x7f29780b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:19.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.913+0000 7f299494b640 1 --2- 172.21.15.67:0/1977273682 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f29780b3c60 0x7f29780b6170 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f29800058c0 tx=0x7f298000a2d0 comp rx=0 tx=0).stop 2024-07-27T03:15:19.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.913+0000 7f299494b640 1 -- 172.21.15.67:0/1977273682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2990107e10 msgr2=0x7f2990105090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:19.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.913+0000 7f299494b640 1 --2- 172.21.15.67:0/1977273682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2990107e10 0x7f2990105090 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2984004270 tx=0x7f29840042a0 comp rx=0 tx=0).stop 2024-07-27T03:15:19.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.914+0000 7f299494b640 1 -- 172.21.15.67:0/1977273682 shutdown_connections 2024-07-27T03:15:19.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.914+0000 7f299494b640 1 --2- 172.21.15.67:0/1977273682 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f29780b3c60 0x7f29780b6170 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:19.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.914+0000 7f299494b640 1 --2- 172.21.15.67:0/1977273682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2990108760 0x7f29901055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:19.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.914+0000 7f299494b640 1 --2- 172.21.15.67:0/1977273682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2990107e10 0x7f2990105090 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:19.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.914+0000 7f299494b640 1 --2- 172.21.15.67:0/1977273682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29901024d0 0x7f2990104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:19.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.914+0000 7f299494b640 1 -- 172.21.15.67:0/1977273682 >> 172.21.15.67:0/1977273682 conn(0x7f29900791a0 msgr2=0x7f299006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:19.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.914+0000 7f299494b640 1 -- 172.21.15.67:0/1977273682 shutdown_connections 2024-07-27T03:15:19.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:19.914+0000 7f299494b640 1 -- 172.21.15.67:0/1977273682 wait complete. 2024-07-27T03:15:20.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:20 smithi067 ceph-mon[32814]: pgmap v393: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:15:20.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:20 smithi149 ceph-mon[37173]: pgmap v393: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:15:20.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:20 smithi175 ceph-mon[37125]: pgmap v393: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:15:20.708 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-07-27T03:15:19.093206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659878,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1074,"num_read_kb":636,"num_write":374,"num_write_kb":2294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951979,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388624,"kb_used_data":14468,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553200,"statfs":{"total":1151940427776,"available":1151542476800,"internally_reserved":0,"allocated":14815232,"data_stored":8254724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003582"},"pg_stats":[{"pgid":"2.1f","version":"89'13","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:14.396927+0000","last_change":"2024-07-27T03:12:29.541107+0000","last_active":"2024-07-27T03:15:14.396927+0000","last_peered":"2024-07-27T03:15:14.396927+0000","last_clean":"2024-07-27T03:15:14.396927+0000","last_became_active":"2024-07-27T03:12:29.540806+0000","last_became_peered":"2024-07-27T03:12:29.540806+0000","last_unstale":"2024-07-27T03:15:14.396927+0000","last_undegraded":"2024-07-27T03:15:14.396927+0000","last_fullsized":"2024-07-27T03:15:14.396927+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:38:24.349354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404651+0000","last_change":"2024-07-27T03:12:32.565515+0000","last_active":"2024-07-27T03:13:57.404651+0000","last_peered":"2024-07-27T03:13:57.404651+0000","last_clean":"2024-07-27T03:13:57.404651+0000","last_became_active":"2024-07-27T03:12:32.565134+0000","last_became_peered":"2024-07-27T03:12:32.565134+0000","last_unstale":"2024-07-27T03:13:57.404651+0000","last_undegraded":"2024-07-27T03:13:57.404651+0000","last_fullsized":"2024-07-27T03:13:57.404651+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:11:56.793139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302752+0000","last_change":"2024-07-27T03:12:42.613432+0000","last_active":"2024-07-27T03:13:57.302752+0000","last_peered":"2024-07-27T03:13:57.302752+0000","last_clean":"2024-07-27T03:13:57.302752+0000","last_became_active":"2024-07-27T03:12:42.612986+0000","last_became_peered":"2024-07-27T03:12:42.612986+0000","last_unstale":"2024-07-27T03:13:57.302752+0000","last_undegraded":"2024-07-27T03:13:57.302752+0000","last_fullsized":"2024-07-27T03:13:57.302752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:25:37.505117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405630+0000","last_change":"2024-07-27T03:12:29.614229+0000","last_active":"2024-07-27T03:13:57.405630+0000","last_peered":"2024-07-27T03:13:57.405630+0000","last_clean":"2024-07-27T03:13:57.405630+0000","last_became_active":"2024-07-27T03:12:29.613844+0000","last_became_peered":"2024-07-27T03:12:29.613844+0000","last_unstale":"2024-07-27T03:13:57.405630+0000","last_undegraded":"2024-07-27T03:13:57.405630+0000","last_fullsized":"2024-07-27T03:13:57.405630+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:10:28.917080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405772+0000","last_change":"2024-07-27T03:12:32.563187+0000","last_active":"2024-07-27T03:13:57.405772+0000","last_peered":"2024-07-27T03:13:57.405772+0000","last_clean":"2024-07-27T03:13:57.405772+0000","last_became_active":"2024-07-27T03:12:32.562674+0000","last_became_peered":"2024-07-27T03:12:32.562674+0000","last_unstale":"2024-07-27T03:13:57.405772+0000","last_undegraded":"2024-07-27T03:13:57.405772+0000","last_fullsized":"2024-07-27T03:13:57.405772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:05:39.211514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303151+0000","last_change":"2024-07-27T03:12:42.611516+0000","last_active":"2024-07-27T03:13:57.303151+0000","last_peered":"2024-07-27T03:13:57.303151+0000","last_clean":"2024-07-27T03:13:57.303151+0000","last_became_active":"2024-07-27T03:12:42.610934+0000","last_became_peered":"2024-07-27T03:12:42.610934+0000","last_unstale":"2024-07-27T03:13:57.303151+0000","last_undegraded":"2024-07-27T03:13:57.303151+0000","last_fullsized":"2024-07-27T03:13:57.303151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:13:17.323063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":58,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305740+0000","last_change":"2024-07-27T03:12:29.539573+0000","last_active":"2024-07-27T03:13:57.305740+0000","last_peered":"2024-07-27T03:13:57.305740+0000","last_clean":"2024-07-27T03:13:57.305740+0000","last_became_active":"2024-07-27T03:12:29.539056+0000","last_became_peered":"2024-07-27T03:12:29.539056+0000","last_unstale":"2024-07-27T03:13:57.305740+0000","last_undegraded":"2024-07-27T03:13:57.305740+0000","last_fullsized":"2024-07-27T03:13:57.305740+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:04:37.513644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405653+0000","last_change":"2024-07-27T03:12:32.567729+0000","last_active":"2024-07-27T03:13:57.405653+0000","last_peered":"2024-07-27T03:13:57.405653+0000","last_clean":"2024-07-27T03:13:57.405653+0000","last_became_active":"2024-07-27T03:12:32.564827+0000","last_became_peered":"2024-07-27T03:12:32.564827+0000","last_unstale":"2024-07-27T03:13:57.405653+0000","last_undegraded":"2024-07-27T03:13:57.405653+0000","last_fullsized":"2024-07-27T03:13:57.405653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:10:55.314988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303339+0000","last_change":"2024-07-27T03:12:42.612776+0000","last_active":"2024-07-27T03:13:57.303339+0000","last_peered":"2024-07-27T03:13:57.303339+0000","last_clean":"2024-07-27T03:13:57.303339+0000","last_became_active":"2024-07-27T03:12:42.612258+0000","last_became_peered":"2024-07-27T03:12:42.612258+0000","last_unstale":"2024-07-27T03:13:57.303339+0000","last_undegraded":"2024-07-27T03:13:57.303339+0000","last_fullsized":"2024-07-27T03:13:57.303339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:48:06.516002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305959+0000","last_change":"2024-07-27T03:12:29.540070+0000","last_active":"2024-07-27T03:13:57.305959+0000","last_peered":"2024-07-27T03:13:57.305959+0000","last_clean":"2024-07-27T03:13:57.305959+0000","last_became_active":"2024-07-27T03:12:29.539657+0000","last_became_peered":"2024-07-27T03:12:29.539657+0000","last_unstale":"2024-07-27T03:13:57.305959+0000","last_undegraded":"2024-07-27T03:13:57.305959+0000","last_fullsized":"2024-07-27T03:13:57.305959+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:42:48.222613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405070+0000","last_change":"2024-07-27T03:12:32.574418+0000","last_active":"2024-07-27T03:13:57.405070+0000","last_peered":"2024-07-27T03:13:57.405070+0000","last_clean":"2024-07-27T03:13:57.405070+0000","last_became_active":"2024-07-27T03:12:32.573974+0000","last_became_peered":"2024-07-27T03:12:32.573974+0000","last_unstale":"2024-07-27T03:13:57.405070+0000","last_undegraded":"2024-07-27T03:13:57.405070+0000","last_fullsized":"2024-07-27T03:13:57.405070+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:34:30.442833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303642+0000","last_change":"2024-07-27T03:12:42.605155+0000","last_active":"2024-07-27T03:13:57.303642+0000","last_peered":"2024-07-27T03:13:57.303642+0000","last_clean":"2024-07-27T03:13:57.303642+0000","last_became_active":"2024-07-27T03:12:42.604685+0000","last_became_peered":"2024-07-27T03:12:42.604685+0000","last_unstale":"2024-07-27T03:13:57.303642+0000","last_undegraded":"2024-07-27T03:13:57.303642+0000","last_fullsized":"2024-07-27T03:13:57.303642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:28:58.931408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305862+0000","last_change":"2024-07-27T03:12:32.574711+0000","last_active":"2024-07-27T03:13:57.305862+0000","last_peered":"2024-07-27T03:13:57.305862+0000","last_clean":"2024-07-27T03:13:57.305862+0000","last_became_active":"2024-07-27T03:12:32.574365+0000","last_became_peered":"2024-07-27T03:12:32.574365+0000","last_unstale":"2024-07-27T03:13:57.305862+0000","last_undegraded":"2024-07-27T03:13:57.305862+0000","last_fullsized":"2024-07-27T03:13:57.305862+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:50:56.621371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305326+0000","last_change":"2024-07-27T03:12:29.540835+0000","last_active":"2024-07-27T03:13:57.305326+0000","last_peered":"2024-07-27T03:13:57.305326+0000","last_clean":"2024-07-27T03:13:57.305326+0000","last_became_active":"2024-07-27T03:12:29.540461+0000","last_became_peered":"2024-07-27T03:12:29.540461+0000","last_unstale":"2024-07-27T03:13:57.305326+0000","last_undegraded":"2024-07-27T03:13:57.305326+0000","last_fullsized":"2024-07-27T03:13:57.305326+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:19:20.817038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303344+0000","last_change":"2024-07-27T03:12:42.718584+0000","last_active":"2024-07-27T03:13:57.303344+0000","last_peered":"2024-07-27T03:13:57.303344+0000","last_clean":"2024-07-27T03:13:57.303344+0000","last_became_active":"2024-07-27T03:12:42.717752+0000","last_became_peered":"2024-07-27T03:12:42.717752+0000","last_unstale":"2024-07-27T03:13:57.303344+0000","last_undegraded":"2024-07-27T03:13:57.303344+0000","last_fullsized":"2024-07-27T03:13:57.303344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:23:52.629794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404408+0000","last_change":"2024-07-27T03:12:32.560672+0000","last_active":"2024-07-27T03:13:57.404408+0000","last_peered":"2024-07-27T03:13:57.404408+0000","last_clean":"2024-07-27T03:13:57.404408+0000","last_became_active":"2024-07-27T03:12:32.559172+0000","last_became_peered":"2024-07-27T03:12:32.559172+0000","last_unstale":"2024-07-27T03:13:57.404408+0000","last_undegraded":"2024-07-27T03:13:57.404408+0000","last_fullsized":"2024-07-27T03:13:57.404408+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:02:49.346983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405206+0000","last_change":"2024-07-27T03:12:57.434569+0000","last_active":"2024-07-27T03:13:57.405206+0000","last_peered":"2024-07-27T03:13:57.405206+0000","last_clean":"2024-07-27T03:13:57.405206+0000","last_became_active":"2024-07-27T03:12:57.434038+0000","last_became_peered":"2024-07-27T03:12:57.434038+0000","last_unstale":"2024-07-27T03:13:57.405206+0000","last_undegraded":"2024-07-27T03:13:57.405206+0000","last_fullsized":"2024-07-27T03:13:57.405206+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:43:24.462838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.412383+0000","last_change":"2024-07-27T03:13:57.411931+0000","last_active":"2024-07-27T03:13:57.412383+0000","last_peered":"2024-07-27T03:13:57.412383+0000","last_clean":"2024-07-27T03:13:57.412383+0000","last_became_active":"2024-07-27T03:13:57.407362+0000","last_became_peered":"2024-07-27T03:13:57.407362+0000","last_unstale":"2024-07-27T03:13:57.412383+0000","last_undegraded":"2024-07-27T03:13:57.412383+0000","last_fullsized":"2024-07-27T03:13:57.412383+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:52:08.444268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405164+0000","last_change":"2024-07-27T03:12:32.572129+0000","last_active":"2024-07-27T03:13:57.405164+0000","last_peered":"2024-07-27T03:13:57.405164+0000","last_clean":"2024-07-27T03:13:57.405164+0000","last_became_active":"2024-07-27T03:12:32.571774+0000","last_became_peered":"2024-07-27T03:12:32.571774+0000","last_unstale":"2024-07-27T03:13:57.405164+0000","last_undegraded":"2024-07-27T03:13:57.405164+0000","last_fullsized":"2024-07-27T03:13:57.405164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:52:23.316957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404446+0000","last_change":"2024-07-27T03:12:29.539768+0000","last_active":"2024-07-27T03:13:57.404446+0000","last_peered":"2024-07-27T03:13:57.404446+0000","last_clean":"2024-07-27T03:13:57.404446+0000","last_became_active":"2024-07-27T03:12:29.539389+0000","last_became_peered":"2024-07-27T03:12:29.539389+0000","last_unstale":"2024-07-27T03:13:57.404446+0000","last_undegraded":"2024-07-27T03:13:57.404446+0000","last_fullsized":"2024-07-27T03:13:57.404446+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:09:35.961328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302623+0000","last_change":"2024-07-27T03:12:42.617569+0000","last_active":"2024-07-27T03:13:57.302623+0000","last_peered":"2024-07-27T03:13:57.302623+0000","last_clean":"2024-07-27T03:13:57.302623+0000","last_became_active":"2024-07-27T03:12:42.616357+0000","last_became_peered":"2024-07-27T03:12:42.616357+0000","last_unstale":"2024-07-27T03:13:57.302623+0000","last_undegraded":"2024-07-27T03:13:57.302623+0000","last_fullsized":"2024-07-27T03:13:57.302623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:35:11.297631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304839+0000","last_change":"2024-07-27T03:12:32.566272+0000","last_active":"2024-07-27T03:13:57.304839+0000","last_peered":"2024-07-27T03:13:57.304839+0000","last_clean":"2024-07-27T03:13:57.304839+0000","last_became_active":"2024-07-27T03:12:32.565865+0000","last_became_peered":"2024-07-27T03:12:32.565865+0000","last_unstale":"2024-07-27T03:13:57.304839+0000","last_undegraded":"2024-07-27T03:13:57.304839+0000","last_fullsized":"2024-07-27T03:13:57.304839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:42:09.539586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405359+0000","last_change":"2024-07-27T03:12:29.613733+0000","last_active":"2024-07-27T03:13:57.405359+0000","last_peered":"2024-07-27T03:13:57.405359+0000","last_clean":"2024-07-27T03:13:57.405359+0000","last_became_active":"2024-07-27T03:12:29.613395+0000","last_became_peered":"2024-07-27T03:12:29.613395+0000","last_unstale":"2024-07-27T03:13:57.405359+0000","last_undegraded":"2024-07-27T03:13:57.405359+0000","last_fullsized":"2024-07-27T03:13:57.405359+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:21:34.431900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.408340+0000","last_change":"2024-07-27T03:13:57.407798+0000","last_active":"2024-07-27T03:13:57.408340+0000","last_peered":"2024-07-27T03:13:57.408340+0000","last_clean":"2024-07-27T03:13:57.408340+0000","last_became_active":"2024-07-27T03:13:57.407275+0000","last_became_peered":"2024-07-27T03:13:57.407275+0000","last_unstale":"2024-07-27T03:13:57.408340+0000","last_undegraded":"2024-07-27T03:13:57.408340+0000","last_fullsized":"2024-07-27T03:13:57.408340+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:47:27.817092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,8],"acting":[4,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405521+0000","last_change":"2024-07-27T03:12:32.567006+0000","last_active":"2024-07-27T03:13:57.405521+0000","last_peered":"2024-07-27T03:13:57.405521+0000","last_clean":"2024-07-27T03:13:57.405521+0000","last_became_active":"2024-07-27T03:12:32.566205+0000","last_became_peered":"2024-07-27T03:12:32.566205+0000","last_unstale":"2024-07-27T03:13:57.405521+0000","last_undegraded":"2024-07-27T03:13:57.405521+0000","last_fullsized":"2024-07-27T03:13:57.405521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:11:23.171341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303695+0000","last_change":"2024-07-27T03:12:29.533909+0000","last_active":"2024-07-27T03:13:57.303695+0000","last_peered":"2024-07-27T03:13:57.303695+0000","last_clean":"2024-07-27T03:13:57.303695+0000","last_became_active":"2024-07-27T03:12:29.533327+0000","last_became_peered":"2024-07-27T03:12:29.533327+0000","last_unstale":"2024-07-27T03:13:57.303695+0000","last_undegraded":"2024-07-27T03:13:57.303695+0000","last_fullsized":"2024-07-27T03:13:57.303695+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:35:24.705096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302923+0000","last_change":"2024-07-27T03:12:42.609066+0000","last_active":"2024-07-27T03:13:57.302923+0000","last_peered":"2024-07-27T03:13:57.302923+0000","last_clean":"2024-07-27T03:13:57.302923+0000","last_became_active":"2024-07-27T03:12:42.608561+0000","last_became_peered":"2024-07-27T03:12:42.608561+0000","last_unstale":"2024-07-27T03:13:57.302923+0000","last_undegraded":"2024-07-27T03:13:57.302923+0000","last_fullsized":"2024-07-27T03:13:57.302923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:27:53.657498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405653+0000","last_change":"2024-07-27T03:12:32.560931+0000","last_active":"2024-07-27T03:13:57.405653+0000","last_peered":"2024-07-27T03:13:57.405653+0000","last_clean":"2024-07-27T03:13:57.405653+0000","last_became_active":"2024-07-27T03:12:32.560549+0000","last_became_peered":"2024-07-27T03:12:32.560549+0000","last_unstale":"2024-07-27T03:13:57.405653+0000","last_undegraded":"2024-07-27T03:13:57.405653+0000","last_fullsized":"2024-07-27T03:13:57.405653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T15:11:25.136035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'15","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:02.583204+0000","last_change":"2024-07-27T03:12:29.535365+0000","last_active":"2024-07-27T03:14:02.583204+0000","last_peered":"2024-07-27T03:14:02.583204+0000","last_clean":"2024-07-27T03:14:02.583204+0000","last_became_active":"2024-07-27T03:12:29.534952+0000","last_became_peered":"2024-07-27T03:12:29.534952+0000","last_unstale":"2024-07-27T03:14:02.583204+0000","last_undegraded":"2024-07-27T03:14:02.583204+0000","last_fullsized":"2024-07-27T03:14:02.583204+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:34:59.889488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404064+0000","last_change":"2024-07-27T03:12:42.718416+0000","last_active":"2024-07-27T03:13:57.404064+0000","last_peered":"2024-07-27T03:13:57.404064+0000","last_clean":"2024-07-27T03:13:57.404064+0000","last_became_active":"2024-07-27T03:12:42.717949+0000","last_became_peered":"2024-07-27T03:12:42.717949+0000","last_unstale":"2024-07-27T03:13:57.404064+0000","last_undegraded":"2024-07-27T03:13:57.404064+0000","last_fullsized":"2024-07-27T03:13:57.404064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:05:05.762577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348872+0000","last_change":"2024-07-27T03:12:32.559824+0000","last_active":"2024-07-27T03:13:57.348872+0000","last_peered":"2024-07-27T03:13:57.348872+0000","last_clean":"2024-07-27T03:13:57.348872+0000","last_became_active":"2024-07-27T03:12:32.559435+0000","last_became_peered":"2024-07-27T03:12:32.559435+0000","last_unstale":"2024-07-27T03:13:57.348872+0000","last_undegraded":"2024-07-27T03:13:57.348872+0000","last_fullsized":"2024-07-27T03:13:57.348872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:16:44.514272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305857+0000","last_change":"2024-07-27T03:12:29.614296+0000","last_active":"2024-07-27T03:13:57.305857+0000","last_peered":"2024-07-27T03:13:57.305857+0000","last_clean":"2024-07-27T03:13:57.305857+0000","last_became_active":"2024-07-27T03:12:29.612987+0000","last_became_peered":"2024-07-27T03:12:29.612987+0000","last_unstale":"2024-07-27T03:13:57.305857+0000","last_undegraded":"2024-07-27T03:13:57.305857+0000","last_fullsized":"2024-07-27T03:13:57.305857+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:23:09.226523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405200+0000","last_change":"2024-07-27T03:12:42.616575+0000","last_active":"2024-07-27T03:13:57.405200+0000","last_peered":"2024-07-27T03:13:57.405200+0000","last_clean":"2024-07-27T03:13:57.405200+0000","last_became_active":"2024-07-27T03:12:42.615980+0000","last_became_peered":"2024-07-27T03:12:42.615980+0000","last_unstale":"2024-07-27T03:13:57.405200+0000","last_undegraded":"2024-07-27T03:13:57.405200+0000","last_fullsized":"2024-07-27T03:13:57.405200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:00:59.041910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406044+0000","last_change":"2024-07-27T03:12:32.556109+0000","last_active":"2024-07-27T03:13:57.406044+0000","last_peered":"2024-07-27T03:13:57.406044+0000","last_clean":"2024-07-27T03:13:57.406044+0000","last_became_active":"2024-07-27T03:12:32.555747+0000","last_became_peered":"2024-07-27T03:12:32.555747+0000","last_unstale":"2024-07-27T03:13:57.406044+0000","last_undegraded":"2024-07-27T03:13:57.406044+0000","last_fullsized":"2024-07-27T03:13:57.406044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:27:58.286605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302565+0000","last_change":"2024-07-27T03:12:29.613523+0000","last_active":"2024-07-27T03:13:57.302565+0000","last_peered":"2024-07-27T03:13:57.302565+0000","last_clean":"2024-07-27T03:13:57.302565+0000","last_became_active":"2024-07-27T03:12:29.613172+0000","last_became_peered":"2024-07-27T03:12:29.613172+0000","last_unstale":"2024-07-27T03:13:57.302565+0000","last_undegraded":"2024-07-27T03:13:57.302565+0000","last_fullsized":"2024-07-27T03:13:57.302565+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:49:45.225221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304930+0000","last_change":"2024-07-27T03:12:42.718536+0000","last_active":"2024-07-27T03:13:57.304930+0000","last_peered":"2024-07-27T03:13:57.304930+0000","last_clean":"2024-07-27T03:13:57.304930+0000","last_became_active":"2024-07-27T03:12:42.718068+0000","last_became_peered":"2024-07-27T03:12:42.718068+0000","last_unstale":"2024-07-27T03:13:57.304930+0000","last_undegraded":"2024-07-27T03:13:57.304930+0000","last_fullsized":"2024-07-27T03:13:57.304930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:18:28.478700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406800+0000","last_change":"2024-07-27T03:12:32.558060+0000","last_active":"2024-07-27T03:13:57.406800+0000","last_peered":"2024-07-27T03:13:57.406800+0000","last_clean":"2024-07-27T03:13:57.406800+0000","last_became_active":"2024-07-27T03:12:32.557260+0000","last_became_peered":"2024-07-27T03:12:32.557260+0000","last_unstale":"2024-07-27T03:13:57.406800+0000","last_undegraded":"2024-07-27T03:13:57.406800+0000","last_fullsized":"2024-07-27T03:13:57.406800+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:36:54.945761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":148,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:15.399209+0000","last_change":"2024-07-27T03:12:29.541080+0000","last_active":"2024-07-27T03:15:15.399209+0000","last_peered":"2024-07-27T03:15:15.399209+0000","last_clean":"2024-07-27T03:15:15.399209+0000","last_became_active":"2024-07-27T03:12:29.540628+0000","last_became_peered":"2024-07-27T03:12:29.540628+0000","last_unstale":"2024-07-27T03:15:15.399209+0000","last_undegraded":"2024-07-27T03:15:15.399209+0000","last_fullsized":"2024-07-27T03:15:15.399209+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:45:17.045124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348865+0000","last_change":"2024-07-27T03:12:42.611561+0000","last_active":"2024-07-27T03:13:57.348865+0000","last_peered":"2024-07-27T03:13:57.348865+0000","last_clean":"2024-07-27T03:13:57.348865+0000","last_became_active":"2024-07-27T03:12:42.610507+0000","last_became_peered":"2024-07-27T03:12:42.610507+0000","last_unstale":"2024-07-27T03:13:57.348865+0000","last_undegraded":"2024-07-27T03:13:57.348865+0000","last_fullsized":"2024-07-27T03:13:57.348865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:15:08.972807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404352+0000","last_change":"2024-07-27T03:12:32.562316+0000","last_active":"2024-07-27T03:13:57.404352+0000","last_peered":"2024-07-27T03:13:57.404352+0000","last_clean":"2024-07-27T03:13:57.404352+0000","last_became_active":"2024-07-27T03:12:32.561729+0000","last_became_peered":"2024-07-27T03:12:32.561729+0000","last_unstale":"2024-07-27T03:13:57.404352+0000","last_undegraded":"2024-07-27T03:13:57.404352+0000","last_fullsized":"2024-07-27T03:13:57.404352+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:01:19.960269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":146,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:12.394543+0000","last_change":"2024-07-27T03:12:29.613459+0000","last_active":"2024-07-27T03:15:12.394543+0000","last_peered":"2024-07-27T03:15:12.394543+0000","last_clean":"2024-07-27T03:15:12.394543+0000","last_became_active":"2024-07-27T03:12:29.613009+0000","last_became_peered":"2024-07-27T03:12:29.613009+0000","last_unstale":"2024-07-27T03:15:12.394543+0000","last_undegraded":"2024-07-27T03:15:12.394543+0000","last_fullsized":"2024-07-27T03:15:12.394543+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:16:37.230471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348695+0000","last_change":"2024-07-27T03:12:42.718516+0000","last_active":"2024-07-27T03:13:57.348695+0000","last_peered":"2024-07-27T03:13:57.348695+0000","last_clean":"2024-07-27T03:13:57.348695+0000","last_became_active":"2024-07-27T03:12:42.718041+0000","last_became_peered":"2024-07-27T03:12:42.718041+0000","last_unstale":"2024-07-27T03:13:57.348695+0000","last_undegraded":"2024-07-27T03:13:57.348695+0000","last_fullsized":"2024-07-27T03:13:57.348695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:35:39.633977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348259+0000","last_change":"2024-07-27T03:12:32.560752+0000","last_active":"2024-07-27T03:13:57.348259+0000","last_peered":"2024-07-27T03:13:57.348259+0000","last_clean":"2024-07-27T03:13:57.348259+0000","last_became_active":"2024-07-27T03:12:32.560248+0000","last_became_peered":"2024-07-27T03:12:32.560248+0000","last_unstale":"2024-07-27T03:13:57.348259+0000","last_undegraded":"2024-07-27T03:13:57.348259+0000","last_fullsized":"2024-07-27T03:13:57.348259+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:59:48.056867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305452+0000","last_change":"2024-07-27T03:12:29.539591+0000","last_active":"2024-07-27T03:13:57.305452+0000","last_peered":"2024-07-27T03:13:57.305452+0000","last_clean":"2024-07-27T03:13:57.305452+0000","last_became_active":"2024-07-27T03:12:29.539165+0000","last_became_peered":"2024-07-27T03:12:29.539165+0000","last_unstale":"2024-07-27T03:13:57.305452+0000","last_undegraded":"2024-07-27T03:13:57.305452+0000","last_fullsized":"2024-07-27T03:13:57.305452+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:04:21.940511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.408418+0000","last_change":"2024-07-27T03:13:57.407921+0000","last_active":"2024-07-27T03:13:57.408418+0000","last_peered":"2024-07-27T03:13:57.408418+0000","last_clean":"2024-07-27T03:13:57.408418+0000","last_became_active":"2024-07-27T03:13:57.407382+0000","last_became_peered":"2024-07-27T03:13:57.407382+0000","last_unstale":"2024-07-27T03:13:57.408418+0000","last_undegraded":"2024-07-27T03:13:57.408418+0000","last_fullsized":"2024-07-27T03:13:57.408418+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:48:27.362759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404373+0000","last_change":"2024-07-27T03:12:32.570653+0000","last_active":"2024-07-27T03:13:57.404373+0000","last_peered":"2024-07-27T03:13:57.404373+0000","last_clean":"2024-07-27T03:13:57.404373+0000","last_became_active":"2024-07-27T03:12:32.569088+0000","last_became_peered":"2024-07-27T03:12:32.569088+0000","last_unstale":"2024-07-27T03:13:57.404373+0000","last_undegraded":"2024-07-27T03:13:57.404373+0000","last_fullsized":"2024-07-27T03:13:57.404373+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:50:24.669431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303915+0000","last_change":"2024-07-27T03:12:29.541654+0000","last_active":"2024-07-27T03:13:57.303915+0000","last_peered":"2024-07-27T03:13:57.303915+0000","last_clean":"2024-07-27T03:13:57.303915+0000","last_became_active":"2024-07-27T03:12:29.541264+0000","last_became_peered":"2024-07-27T03:12:29.541264+0000","last_unstale":"2024-07-27T03:13:57.303915+0000","last_undegraded":"2024-07-27T03:13:57.303915+0000","last_fullsized":"2024-07-27T03:13:57.303915+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:48:05.179016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":122,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306449+0000","last_change":"2024-07-27T03:11:11.343199+0000","last_active":"2024-07-27T03:13:57.306449+0000","last_peered":"2024-07-27T03:13:57.306449+0000","last_clean":"2024-07-27T03:13:57.306449+0000","last_became_active":"2024-07-27T03:11:11.337635+0000","last_became_peered":"2024-07-27T03:11:11.337635+0000","last_unstale":"2024-07-27T03:13:57.306449+0000","last_undegraded":"2024-07-27T03:13:57.306449+0000","last_fullsized":"2024-07-27T03:13:57.306449+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:08:01.694830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:08:01.694830+0000","last_clean_scrub_stamp":"2024-07-27T03:08:01.694830+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:48:49.979610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305836+0000","last_change":"2024-07-27T03:12:42.614346+0000","last_active":"2024-07-27T03:13:57.305836+0000","last_peered":"2024-07-27T03:13:57.305836+0000","last_clean":"2024-07-27T03:13:57.305836+0000","last_became_active":"2024-07-27T03:12:42.613881+0000","last_became_peered":"2024-07-27T03:12:42.613881+0000","last_unstale":"2024-07-27T03:13:57.305836+0000","last_undegraded":"2024-07-27T03:13:57.305836+0000","last_fullsized":"2024-07-27T03:13:57.305836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:12:38.299348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306115+0000","last_change":"2024-07-27T03:12:32.574330+0000","last_active":"2024-07-27T03:13:57.306115+0000","last_peered":"2024-07-27T03:13:57.306115+0000","last_clean":"2024-07-27T03:13:57.306115+0000","last_became_active":"2024-07-27T03:12:32.573977+0000","last_became_peered":"2024-07-27T03:12:32.573977+0000","last_unstale":"2024-07-27T03:13:57.306115+0000","last_undegraded":"2024-07-27T03:13:57.306115+0000","last_fullsized":"2024-07-27T03:13:57.306115+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:39:09.925794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":148,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:13.396770+0000","last_change":"2024-07-27T03:12:29.613848+0000","last_active":"2024-07-27T03:15:13.396770+0000","last_peered":"2024-07-27T03:15:13.396770+0000","last_clean":"2024-07-27T03:15:13.396770+0000","last_became_active":"2024-07-27T03:12:29.613473+0000","last_became_peered":"2024-07-27T03:12:29.613473+0000","last_unstale":"2024-07-27T03:15:13.396770+0000","last_undegraded":"2024-07-27T03:15:13.396770+0000","last_fullsized":"2024-07-27T03:15:13.396770+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:52:56.894962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":97,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405373+0000","last_change":"2024-07-27T03:12:42.611295+0000","last_active":"2024-07-27T03:13:57.405373+0000","last_peered":"2024-07-27T03:13:57.405373+0000","last_clean":"2024-07-27T03:13:57.405373+0000","last_became_active":"2024-07-27T03:12:42.610782+0000","last_became_peered":"2024-07-27T03:12:42.610782+0000","last_unstale":"2024-07-27T03:13:57.405373+0000","last_undegraded":"2024-07-27T03:13:57.405373+0000","last_fullsized":"2024-07-27T03:13:57.405373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:24:02.849882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305956+0000","last_change":"2024-07-27T03:12:56.725924+0000","last_active":"2024-07-27T03:13:57.305956+0000","last_peered":"2024-07-27T03:13:57.305956+0000","last_clean":"2024-07-27T03:13:57.305956+0000","last_became_active":"2024-07-27T03:12:56.725567+0000","last_became_peered":"2024-07-27T03:12:56.725567+0000","last_unstale":"2024-07-27T03:13:57.305956+0000","last_undegraded":"2024-07-27T03:13:57.305956+0000","last_fullsized":"2024-07-27T03:13:57.305956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:32:23.771483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405225+0000","last_change":"2024-07-27T03:12:29.537863+0000","last_active":"2024-07-27T03:13:57.405225+0000","last_peered":"2024-07-27T03:13:57.405225+0000","last_clean":"2024-07-27T03:13:57.405225+0000","last_became_active":"2024-07-27T03:12:29.537444+0000","last_became_peered":"2024-07-27T03:12:29.537444+0000","last_unstale":"2024-07-27T03:13:57.405225+0000","last_undegraded":"2024-07-27T03:13:57.405225+0000","last_fullsized":"2024-07-27T03:13:57.405225+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:52:05.239809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405265+0000","last_change":"2024-07-27T03:12:42.622599+0000","last_active":"2024-07-27T03:13:57.405265+0000","last_peered":"2024-07-27T03:13:57.405265+0000","last_clean":"2024-07-27T03:13:57.405265+0000","last_became_active":"2024-07-27T03:12:42.622151+0000","last_became_peered":"2024-07-27T03:12:42.622151+0000","last_unstale":"2024-07-27T03:13:57.405265+0000","last_undegraded":"2024-07-27T03:13:57.405265+0000","last_fullsized":"2024-07-27T03:13:57.405265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:34:31.622281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306372+0000","last_change":"2024-07-27T03:12:32.562300+0000","last_active":"2024-07-27T03:13:57.306372+0000","last_peered":"2024-07-27T03:13:57.306372+0000","last_clean":"2024-07-27T03:13:57.306372+0000","last_became_active":"2024-07-27T03:12:32.561343+0000","last_became_peered":"2024-07-27T03:12:32.561343+0000","last_unstale":"2024-07-27T03:13:57.306372+0000","last_undegraded":"2024-07-27T03:13:57.306372+0000","last_fullsized":"2024-07-27T03:13:57.306372+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:01:28.626943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348835+0000","last_change":"2024-07-27T03:12:29.535177+0000","last_active":"2024-07-27T03:13:57.348835+0000","last_peered":"2024-07-27T03:13:57.348835+0000","last_clean":"2024-07-27T03:13:57.348835+0000","last_became_active":"2024-07-27T03:12:29.534662+0000","last_became_peered":"2024-07-27T03:12:29.534662+0000","last_unstale":"2024-07-27T03:13:57.348835+0000","last_undegraded":"2024-07-27T03:13:57.348835+0000","last_fullsized":"2024-07-27T03:13:57.348835+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:50:49.661992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404084+0000","last_change":"2024-07-27T03:12:42.609925+0000","last_active":"2024-07-27T03:13:57.404084+0000","last_peered":"2024-07-27T03:13:57.404084+0000","last_clean":"2024-07-27T03:13:57.404084+0000","last_became_active":"2024-07-27T03:12:42.609456+0000","last_became_peered":"2024-07-27T03:12:42.609456+0000","last_unstale":"2024-07-27T03:13:57.404084+0000","last_undegraded":"2024-07-27T03:13:57.404084+0000","last_fullsized":"2024-07-27T03:13:57.404084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:07:52.128381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406018+0000","last_change":"2024-07-27T03:12:56.726170+0000","last_active":"2024-07-27T03:13:57.406018+0000","last_peered":"2024-07-27T03:13:57.406018+0000","last_clean":"2024-07-27T03:13:57.406018+0000","last_became_active":"2024-07-27T03:12:56.725685+0000","last_became_peered":"2024-07-27T03:12:56.725685+0000","last_unstale":"2024-07-27T03:13:57.406018+0000","last_undegraded":"2024-07-27T03:13:57.406018+0000","last_fullsized":"2024-07-27T03:13:57.406018+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:53:41.143333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405969+0000","last_change":"2024-07-27T03:12:56.726059+0000","last_active":"2024-07-27T03:13:57.405969+0000","last_peered":"2024-07-27T03:13:57.405969+0000","last_clean":"2024-07-27T03:13:57.405969+0000","last_became_active":"2024-07-27T03:12:56.722294+0000","last_became_peered":"2024-07-27T03:12:56.722294+0000","last_unstale":"2024-07-27T03:13:57.405969+0000","last_undegraded":"2024-07-27T03:13:57.405969+0000","last_fullsized":"2024-07-27T03:13:57.405969+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:18:42.627768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404320+0000","last_change":"2024-07-27T03:12:42.611966+0000","last_active":"2024-07-27T03:13:57.404320+0000","last_peered":"2024-07-27T03:13:57.404320+0000","last_clean":"2024-07-27T03:13:57.404320+0000","last_became_active":"2024-07-27T03:12:42.611522+0000","last_became_peered":"2024-07-27T03:12:42.611522+0000","last_unstale":"2024-07-27T03:13:57.404320+0000","last_undegraded":"2024-07-27T03:13:57.404320+0000","last_fullsized":"2024-07-27T03:13:57.404320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:09:54.034597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.301967+0000","last_change":"2024-07-27T03:12:56.719638+0000","last_active":"2024-07-27T03:13:57.301967+0000","last_peered":"2024-07-27T03:13:57.301967+0000","last_clean":"2024-07-27T03:13:57.301967+0000","last_became_active":"2024-07-27T03:12:56.719227+0000","last_became_peered":"2024-07-27T03:12:56.719227+0000","last_unstale":"2024-07-27T03:13:57.301967+0000","last_undegraded":"2024-07-27T03:13:57.301967+0000","last_fullsized":"2024-07-27T03:13:57.301967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:54:16.891518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304967+0000","last_change":"2024-07-27T03:12:29.614346+0000","last_active":"2024-07-27T03:13:57.304967+0000","last_peered":"2024-07-27T03:13:57.304967+0000","last_clean":"2024-07-27T03:13:57.304967+0000","last_became_active":"2024-07-27T03:12:29.613672+0000","last_became_peered":"2024-07-27T03:12:29.613672+0000","last_unstale":"2024-07-27T03:13:57.304967+0000","last_undegraded":"2024-07-27T03:13:57.304967+0000","last_fullsized":"2024-07-27T03:13:57.304967+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:46:35.505735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348712+0000","last_change":"2024-07-27T03:12:42.618364+0000","last_active":"2024-07-27T03:13:57.348712+0000","last_peered":"2024-07-27T03:13:57.348712+0000","last_clean":"2024-07-27T03:13:57.348712+0000","last_became_active":"2024-07-27T03:12:42.617988+0000","last_became_peered":"2024-07-27T03:12:42.617988+0000","last_unstale":"2024-07-27T03:13:57.348712+0000","last_undegraded":"2024-07-27T03:13:57.348712+0000","last_fullsized":"2024-07-27T03:13:57.348712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:40:21.725877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305356+0000","last_change":"2024-07-27T03:12:32.563809+0000","last_active":"2024-07-27T03:13:57.305356+0000","last_peered":"2024-07-27T03:13:57.305356+0000","last_clean":"2024-07-27T03:13:57.305356+0000","last_became_active":"2024-07-27T03:12:32.563479+0000","last_became_peered":"2024-07-27T03:12:32.563479+0000","last_unstale":"2024-07-27T03:13:57.305356+0000","last_undegraded":"2024-07-27T03:13:57.305356+0000","last_fullsized":"2024-07-27T03:13:57.305356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:45:47.719241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:14.398423+0000","last_change":"2024-07-27T03:12:29.543786+0000","last_active":"2024-07-27T03:15:14.398423+0000","last_peered":"2024-07-27T03:15:14.398423+0000","last_clean":"2024-07-27T03:15:14.398423+0000","last_became_active":"2024-07-27T03:12:29.543203+0000","last_became_peered":"2024-07-27T03:12:29.543203+0000","last_unstale":"2024-07-27T03:15:14.398423+0000","last_undegraded":"2024-07-27T03:15:14.398423+0000","last_fullsized":"2024-07-27T03:15:14.398423+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:47:06.742171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302718+0000","last_change":"2024-07-27T03:12:42.622142+0000","last_active":"2024-07-27T03:13:57.302718+0000","last_peered":"2024-07-27T03:13:57.302718+0000","last_clean":"2024-07-27T03:13:57.302718+0000","last_became_active":"2024-07-27T03:12:42.621681+0000","last_became_peered":"2024-07-27T03:12:42.621681+0000","last_unstale":"2024-07-27T03:13:57.302718+0000","last_undegraded":"2024-07-27T03:13:57.302718+0000","last_fullsized":"2024-07-27T03:13:57.302718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:05:08.168424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306186+0000","last_change":"2024-07-27T03:12:32.567119+0000","last_active":"2024-07-27T03:13:57.306186+0000","last_peered":"2024-07-27T03:13:57.306186+0000","last_clean":"2024-07-27T03:13:57.306186+0000","last_became_active":"2024-07-27T03:12:32.566757+0000","last_became_peered":"2024-07-27T03:12:32.566757+0000","last_unstale":"2024-07-27T03:13:57.306186+0000","last_undegraded":"2024-07-27T03:13:57.306186+0000","last_fullsized":"2024-07-27T03:13:57.306186+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:31:12.972804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404725+0000","last_change":"2024-07-27T03:12:29.540453+0000","last_active":"2024-07-27T03:13:57.404725+0000","last_peered":"2024-07-27T03:13:57.404725+0000","last_clean":"2024-07-27T03:13:57.404725+0000","last_became_active":"2024-07-27T03:12:29.539195+0000","last_became_peered":"2024-07-27T03:12:29.539195+0000","last_unstale":"2024-07-27T03:13:57.404725+0000","last_undegraded":"2024-07-27T03:13:57.404725+0000","last_fullsized":"2024-07-27T03:13:57.404725+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:44:31.144845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405363+0000","last_change":"2024-07-27T03:12:42.616671+0000","last_active":"2024-07-27T03:13:57.405363+0000","last_peered":"2024-07-27T03:13:57.405363+0000","last_clean":"2024-07-27T03:13:57.405363+0000","last_became_active":"2024-07-27T03:12:42.616198+0000","last_became_peered":"2024-07-27T03:12:42.616198+0000","last_unstale":"2024-07-27T03:13:57.405363+0000","last_undegraded":"2024-07-27T03:13:57.405363+0000","last_fullsized":"2024-07-27T03:13:57.405363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:01:31.584686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348516+0000","last_change":"2024-07-27T03:12:32.562350+0000","last_active":"2024-07-27T03:13:57.348516+0000","last_peered":"2024-07-27T03:13:57.348516+0000","last_clean":"2024-07-27T03:13:57.348516+0000","last_became_active":"2024-07-27T03:12:32.562016+0000","last_became_peered":"2024-07-27T03:12:32.562016+0000","last_unstale":"2024-07-27T03:13:57.348516+0000","last_undegraded":"2024-07-27T03:13:57.348516+0000","last_fullsized":"2024-07-27T03:13:57.348516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:44:35.027329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303241+0000","last_change":"2024-07-27T03:12:29.541387+0000","last_active":"2024-07-27T03:13:57.303241+0000","last_peered":"2024-07-27T03:13:57.303241+0000","last_clean":"2024-07-27T03:13:57.303241+0000","last_became_active":"2024-07-27T03:12:29.541005+0000","last_became_peered":"2024-07-27T03:12:29.541005+0000","last_unstale":"2024-07-27T03:13:57.303241+0000","last_undegraded":"2024-07-27T03:13:57.303241+0000","last_fullsized":"2024-07-27T03:13:57.303241+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:22:16.268348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404849+0000","last_change":"2024-07-27T03:12:42.610029+0000","last_active":"2024-07-27T03:13:57.404849+0000","last_peered":"2024-07-27T03:13:57.404849+0000","last_clean":"2024-07-27T03:13:57.404849+0000","last_became_active":"2024-07-27T03:12:42.609592+0000","last_became_peered":"2024-07-27T03:12:42.609592+0000","last_unstale":"2024-07-27T03:13:57.404849+0000","last_undegraded":"2024-07-27T03:13:57.404849+0000","last_fullsized":"2024-07-27T03:13:57.404849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:47:15.254770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305612+0000","last_change":"2024-07-27T03:12:32.568941+0000","last_active":"2024-07-27T03:13:57.305612+0000","last_peered":"2024-07-27T03:13:57.305612+0000","last_clean":"2024-07-27T03:13:57.305612+0000","last_became_active":"2024-07-27T03:12:32.568531+0000","last_became_peered":"2024-07-27T03:12:32.568531+0000","last_unstale":"2024-07-27T03:13:57.305612+0000","last_undegraded":"2024-07-27T03:13:57.305612+0000","last_fullsized":"2024-07-27T03:13:57.305612+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:06:22.020505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":279,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:14.397793+0000","last_change":"2024-07-27T03:12:29.545131+0000","last_active":"2024-07-27T03:15:14.397793+0000","last_peered":"2024-07-27T03:15:14.397793+0000","last_clean":"2024-07-27T03:15:14.397793+0000","last_became_active":"2024-07-27T03:12:29.544768+0000","last_became_peered":"2024-07-27T03:12:29.544768+0000","last_unstale":"2024-07-27T03:15:14.397793+0000","last_undegraded":"2024-07-27T03:15:14.397793+0000","last_fullsized":"2024-07-27T03:15:14.397793+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:20:57.556128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62072,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":199,"num_read_kb":160,"num_write":37,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302087+0000","last_change":"2024-07-27T03:12:42.617483+0000","last_active":"2024-07-27T03:13:57.302087+0000","last_peered":"2024-07-27T03:13:57.302087+0000","last_clean":"2024-07-27T03:13:57.302087+0000","last_became_active":"2024-07-27T03:12:42.617065+0000","last_became_peered":"2024-07-27T03:12:42.617065+0000","last_unstale":"2024-07-27T03:13:57.302087+0000","last_undegraded":"2024-07-27T03:13:57.302087+0000","last_fullsized":"2024-07-27T03:13:57.302087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:52:17.448919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405888+0000","last_change":"2024-07-27T03:12:56.722257+0000","last_active":"2024-07-27T03:13:57.405888+0000","last_peered":"2024-07-27T03:13:57.405888+0000","last_clean":"2024-07-27T03:13:57.405888+0000","last_became_active":"2024-07-27T03:12:56.721773+0000","last_became_peered":"2024-07-27T03:12:56.721773+0000","last_unstale":"2024-07-27T03:13:57.405888+0000","last_undegraded":"2024-07-27T03:13:57.405888+0000","last_fullsized":"2024-07-27T03:13:57.405888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:30:44.310705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405205+0000","last_change":"2024-07-27T03:12:29.614171+0000","last_active":"2024-07-27T03:13:57.405205+0000","last_peered":"2024-07-27T03:13:57.405205+0000","last_clean":"2024-07-27T03:13:57.405205+0000","last_became_active":"2024-07-27T03:12:29.613270+0000","last_became_peered":"2024-07-27T03:12:29.613270+0000","last_unstale":"2024-07-27T03:13:57.405205+0000","last_undegraded":"2024-07-27T03:13:57.405205+0000","last_fullsized":"2024-07-27T03:13:57.405205+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:39:32.283778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303808+0000","last_change":"2024-07-27T03:12:42.610375+0000","last_active":"2024-07-27T03:13:57.303808+0000","last_peered":"2024-07-27T03:13:57.303808+0000","last_clean":"2024-07-27T03:13:57.303808+0000","last_became_active":"2024-07-27T03:12:42.609957+0000","last_became_peered":"2024-07-27T03:12:42.609957+0000","last_unstale":"2024-07-27T03:13:57.303808+0000","last_undegraded":"2024-07-27T03:13:57.303808+0000","last_fullsized":"2024-07-27T03:13:57.303808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:11:02.707179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405928+0000","last_change":"2024-07-27T03:12:56.722332+0000","last_active":"2024-07-27T03:13:57.405928+0000","last_peered":"2024-07-27T03:13:57.405928+0000","last_clean":"2024-07-27T03:13:57.405928+0000","last_became_active":"2024-07-27T03:12:56.721972+0000","last_became_peered":"2024-07-27T03:12:56.721972+0000","last_unstale":"2024-07-27T03:13:57.405928+0000","last_undegraded":"2024-07-27T03:13:57.405928+0000","last_fullsized":"2024-07-27T03:13:57.405928+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:54:31.880137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306094+0000","last_change":"2024-07-27T03:12:29.541051+0000","last_active":"2024-07-27T03:13:57.306094+0000","last_peered":"2024-07-27T03:13:57.306094+0000","last_clean":"2024-07-27T03:13:57.306094+0000","last_became_active":"2024-07-27T03:12:29.540619+0000","last_became_peered":"2024-07-27T03:12:29.540619+0000","last_unstale":"2024-07-27T03:13:57.306094+0000","last_undegraded":"2024-07-27T03:13:57.306094+0000","last_fullsized":"2024-07-27T03:13:57.306094+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:23:27.587419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304742+0000","last_change":"2024-07-27T03:12:56.720496+0000","last_active":"2024-07-27T03:13:57.304742+0000","last_peered":"2024-07-27T03:13:57.304742+0000","last_clean":"2024-07-27T03:13:57.304742+0000","last_became_active":"2024-07-27T03:12:56.719784+0000","last_became_peered":"2024-07-27T03:12:56.719784+0000","last_unstale":"2024-07-27T03:13:57.304742+0000","last_undegraded":"2024-07-27T03:13:57.304742+0000","last_fullsized":"2024-07-27T03:13:57.304742+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:07:20.518139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406680+0000","last_change":"2024-07-27T03:12:32.568409+0000","last_active":"2024-07-27T03:13:57.406680+0000","last_peered":"2024-07-27T03:13:57.406680+0000","last_clean":"2024-07-27T03:13:57.406680+0000","last_became_active":"2024-07-27T03:12:32.568029+0000","last_became_peered":"2024-07-27T03:12:32.568029+0000","last_unstale":"2024-07-27T03:13:57.406680+0000","last_undegraded":"2024-07-27T03:13:57.406680+0000","last_fullsized":"2024-07-27T03:13:57.406680+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:44:53.857986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":239,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:14.397656+0000","last_change":"2024-07-27T03:12:29.544286+0000","last_active":"2024-07-27T03:15:14.397656+0000","last_peered":"2024-07-27T03:15:14.397656+0000","last_clean":"2024-07-27T03:15:14.397656+0000","last_became_active":"2024-07-27T03:12:29.543879+0000","last_became_peered":"2024-07-27T03:12:29.543879+0000","last_unstale":"2024-07-27T03:15:14.397656+0000","last_undegraded":"2024-07-27T03:15:14.397656+0000","last_fullsized":"2024-07-27T03:15:14.397656+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:20:36.997298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404174+0000","last_change":"2024-07-27T03:12:42.609824+0000","last_active":"2024-07-27T03:13:57.404174+0000","last_peered":"2024-07-27T03:13:57.404174+0000","last_clean":"2024-07-27T03:13:57.404174+0000","last_became_active":"2024-07-27T03:12:42.608502+0000","last_became_peered":"2024-07-27T03:12:42.608502+0000","last_unstale":"2024-07-27T03:13:57.404174+0000","last_undegraded":"2024-07-27T03:13:57.404174+0000","last_fullsized":"2024-07-27T03:13:57.404174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:10:23.253921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405172+0000","last_change":"2024-07-27T03:12:42.610744+0000","last_active":"2024-07-27T03:13:57.405172+0000","last_peered":"2024-07-27T03:13:57.405172+0000","last_clean":"2024-07-27T03:13:57.405172+0000","last_became_active":"2024-07-27T03:12:42.610322+0000","last_became_peered":"2024-07-27T03:12:42.610322+0000","last_unstale":"2024-07-27T03:13:57.405172+0000","last_undegraded":"2024-07-27T03:13:57.405172+0000","last_fullsized":"2024-07-27T03:13:57.405172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:01:47.618013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302086+0000","last_change":"2024-07-27T03:12:32.558372+0000","last_active":"2024-07-27T03:13:57.302086+0000","last_peered":"2024-07-27T03:13:57.302086+0000","last_clean":"2024-07-27T03:13:57.302086+0000","last_became_active":"2024-07-27T03:12:32.558003+0000","last_became_peered":"2024-07-27T03:12:32.558003+0000","last_unstale":"2024-07-27T03:13:57.302086+0000","last_undegraded":"2024-07-27T03:13:57.302086+0000","last_fullsized":"2024-07-27T03:13:57.302086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:41:43.673807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404495+0000","last_change":"2024-07-27T03:12:29.543182+0000","last_active":"2024-07-27T03:13:57.404495+0000","last_peered":"2024-07-27T03:13:57.404495+0000","last_clean":"2024-07-27T03:13:57.404495+0000","last_became_active":"2024-07-27T03:12:29.542744+0000","last_became_peered":"2024-07-27T03:12:29.542744+0000","last_unstale":"2024-07-27T03:13:57.404495+0000","last_undegraded":"2024-07-27T03:13:57.404495+0000","last_fullsized":"2024-07-27T03:13:57.404495+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:01:17.276549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405941+0000","last_change":"2024-07-27T03:12:42.622297+0000","last_active":"2024-07-27T03:13:57.405941+0000","last_peered":"2024-07-27T03:13:57.405941+0000","last_clean":"2024-07-27T03:13:57.405941+0000","last_became_active":"2024-07-27T03:12:42.621947+0000","last_became_peered":"2024-07-27T03:12:42.621947+0000","last_unstale":"2024-07-27T03:13:57.405941+0000","last_undegraded":"2024-07-27T03:13:57.405941+0000","last_fullsized":"2024-07-27T03:13:57.405941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:15:59.480216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404239+0000","last_change":"2024-07-27T03:12:32.565987+0000","last_active":"2024-07-27T03:13:57.404239+0000","last_peered":"2024-07-27T03:13:57.404239+0000","last_clean":"2024-07-27T03:13:57.404239+0000","last_became_active":"2024-07-27T03:12:32.565603+0000","last_became_peered":"2024-07-27T03:12:32.565603+0000","last_unstale":"2024-07-27T03:13:57.404239+0000","last_undegraded":"2024-07-27T03:13:57.404239+0000","last_fullsized":"2024-07-27T03:13:57.404239+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:30:48.954683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348273+0000","last_change":"2024-07-27T03:12:29.532149+0000","last_active":"2024-07-27T03:13:57.348273+0000","last_peered":"2024-07-27T03:13:57.348273+0000","last_clean":"2024-07-27T03:13:57.348273+0000","last_became_active":"2024-07-27T03:12:29.531761+0000","last_became_peered":"2024-07-27T03:12:29.531761+0000","last_unstale":"2024-07-27T03:13:57.348273+0000","last_undegraded":"2024-07-27T03:13:57.348273+0000","last_fullsized":"2024-07-27T03:13:57.348273+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:44:30.902538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406091+0000","last_change":"2024-07-27T03:12:29.543148+0000","last_active":"2024-07-27T03:13:57.406091+0000","last_peered":"2024-07-27T03:13:57.406091+0000","last_clean":"2024-07-27T03:13:57.406091+0000","last_became_active":"2024-07-27T03:12:29.542774+0000","last_became_peered":"2024-07-27T03:12:29.542774+0000","last_unstale":"2024-07-27T03:13:57.406091+0000","last_undegraded":"2024-07-27T03:13:57.406091+0000","last_fullsized":"2024-07-27T03:13:57.406091+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:20:06.786121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404598+0000","last_change":"2024-07-27T03:12:32.561516+0000","last_active":"2024-07-27T03:13:57.404598+0000","last_peered":"2024-07-27T03:13:57.404598+0000","last_clean":"2024-07-27T03:13:57.404598+0000","last_became_active":"2024-07-27T03:12:32.561145+0000","last_became_peered":"2024-07-27T03:12:32.561145+0000","last_unstale":"2024-07-27T03:13:57.404598+0000","last_undegraded":"2024-07-27T03:13:57.404598+0000","last_fullsized":"2024-07-27T03:13:57.404598+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:49:26.317206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302982+0000","last_change":"2024-07-27T03:12:42.616735+0000","last_active":"2024-07-27T03:13:57.302982+0000","last_peered":"2024-07-27T03:13:57.302982+0000","last_clean":"2024-07-27T03:13:57.302982+0000","last_became_active":"2024-07-27T03:12:42.616171+0000","last_became_peered":"2024-07-27T03:12:42.616171+0000","last_unstale":"2024-07-27T03:13:57.302982+0000","last_undegraded":"2024-07-27T03:13:57.302982+0000","last_fullsized":"2024-07-27T03:13:57.302982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:45:46.236613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":152,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:17.403421+0000","last_change":"2024-07-27T03:12:29.539775+0000","last_active":"2024-07-27T03:15:17.403421+0000","last_peered":"2024-07-27T03:15:17.403421+0000","last_clean":"2024-07-27T03:15:17.403421+0000","last_became_active":"2024-07-27T03:12:29.539276+0000","last_became_peered":"2024-07-27T03:12:29.539276+0000","last_unstale":"2024-07-27T03:15:17.403421+0000","last_undegraded":"2024-07-27T03:15:17.403421+0000","last_fullsized":"2024-07-27T03:15:17.403421+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:09:19.364679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305126+0000","last_change":"2024-07-27T03:12:32.562673+0000","last_active":"2024-07-27T03:13:57.305126+0000","last_peered":"2024-07-27T03:13:57.305126+0000","last_clean":"2024-07-27T03:13:57.305126+0000","last_became_active":"2024-07-27T03:12:32.561591+0000","last_became_peered":"2024-07-27T03:12:32.561591+0000","last_unstale":"2024-07-27T03:13:57.305126+0000","last_undegraded":"2024-07-27T03:13:57.305126+0000","last_fullsized":"2024-07-27T03:13:57.305126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:49:38.095004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303410+0000","last_change":"2024-07-27T03:12:42.611756+0000","last_active":"2024-07-27T03:13:57.303410+0000","last_peered":"2024-07-27T03:13:57.303410+0000","last_clean":"2024-07-27T03:13:57.303410+0000","last_became_active":"2024-07-27T03:12:42.611246+0000","last_became_peered":"2024-07-27T03:12:42.611246+0000","last_unstale":"2024-07-27T03:13:57.303410+0000","last_undegraded":"2024-07-27T03:13:57.303410+0000","last_fullsized":"2024-07-27T03:13:57.303410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:13:06.027887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405688+0000","last_change":"2024-07-27T03:12:32.571339+0000","last_active":"2024-07-27T03:13:57.405688+0000","last_peered":"2024-07-27T03:13:57.405688+0000","last_clean":"2024-07-27T03:13:57.405688+0000","last_became_active":"2024-07-27T03:12:32.570973+0000","last_became_peered":"2024-07-27T03:12:32.570973+0000","last_unstale":"2024-07-27T03:13:57.405688+0000","last_undegraded":"2024-07-27T03:13:57.405688+0000","last_fullsized":"2024-07-27T03:13:57.405688+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:24:25.357833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303364+0000","last_change":"2024-07-27T03:12:32.561243+0000","last_active":"2024-07-27T03:13:57.303364+0000","last_peered":"2024-07-27T03:13:57.303364+0000","last_clean":"2024-07-27T03:13:57.303364+0000","last_became_active":"2024-07-27T03:12:32.560748+0000","last_became_peered":"2024-07-27T03:12:32.560748+0000","last_unstale":"2024-07-27T03:13:57.303364+0000","last_undegraded":"2024-07-27T03:13:57.303364+0000","last_fullsized":"2024-07-27T03:13:57.303364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:47:40.430283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302948+0000","last_change":"2024-07-27T03:12:32.562545+0000","last_active":"2024-07-27T03:13:57.302948+0000","last_peered":"2024-07-27T03:13:57.302948+0000","last_clean":"2024-07-27T03:13:57.302948+0000","last_became_active":"2024-07-27T03:12:32.562202+0000","last_became_peered":"2024-07-27T03:12:32.562202+0000","last_unstale":"2024-07-27T03:13:57.302948+0000","last_undegraded":"2024-07-27T03:13:57.302948+0000","last_fullsized":"2024-07-27T03:13:57.302948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:05:20.229232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405456+0000","last_change":"2024-07-27T03:12:32.565162+0000","last_active":"2024-07-27T03:13:57.405456+0000","last_peered":"2024-07-27T03:13:57.405456+0000","last_clean":"2024-07-27T03:13:57.405456+0000","last_became_active":"2024-07-27T03:12:32.564798+0000","last_became_peered":"2024-07-27T03:12:32.564798+0000","last_unstale":"2024-07-27T03:13:57.405456+0000","last_undegraded":"2024-07-27T03:13:57.405456+0000","last_fullsized":"2024-07-27T03:13:57.405456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:54:29.742624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348739+0000","last_change":"2024-07-27T03:12:32.556874+0000","last_active":"2024-07-27T03:13:57.348739+0000","last_peered":"2024-07-27T03:13:57.348739+0000","last_clean":"2024-07-27T03:13:57.348739+0000","last_became_active":"2024-07-27T03:12:32.556467+0000","last_became_peered":"2024-07-27T03:12:32.556467+0000","last_unstale":"2024-07-27T03:13:57.348739+0000","last_undegraded":"2024-07-27T03:13:57.348739+0000","last_fullsized":"2024-07-27T03:13:57.348739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:51:20.806785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406013+0000","last_change":"2024-07-27T03:12:32.569136+0000","last_active":"2024-07-27T03:13:57.406013+0000","last_peered":"2024-07-27T03:13:57.406013+0000","last_clean":"2024-07-27T03:13:57.406013+0000","last_became_active":"2024-07-27T03:12:32.568778+0000","last_became_peered":"2024-07-27T03:12:32.568778+0000","last_unstale":"2024-07-27T03:13:57.406013+0000","last_undegraded":"2024-07-27T03:13:57.406013+0000","last_fullsized":"2024-07-27T03:13:57.406013+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:26:22.654974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303529+0000","last_change":"2024-07-27T03:12:32.569507+0000","last_active":"2024-07-27T03:13:57.303529+0000","last_peered":"2024-07-27T03:13:57.303529+0000","last_clean":"2024-07-27T03:13:57.303529+0000","last_became_active":"2024-07-27T03:12:32.569109+0000","last_became_peered":"2024-07-27T03:12:32.569109+0000","last_unstale":"2024-07-27T03:13:57.303529+0000","last_undegraded":"2024-07-27T03:13:57.303529+0000","last_fullsized":"2024-07-27T03:13:57.303529+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:35:12.944933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405595+0000","last_change":"2024-07-27T03:12:32.566479+0000","last_active":"2024-07-27T03:13:57.405595+0000","last_peered":"2024-07-27T03:13:57.405595+0000","last_clean":"2024-07-27T03:13:57.405595+0000","last_became_active":"2024-07-27T03:12:32.566114+0000","last_became_peered":"2024-07-27T03:12:32.566114+0000","last_unstale":"2024-07-27T03:13:57.405595+0000","last_undegraded":"2024-07-27T03:13:57.405595+0000","last_fullsized":"2024-07-27T03:13:57.405595+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:24:19.944031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404254+0000","last_change":"2024-07-27T03:12:56.726118+0000","last_active":"2024-07-27T03:13:57.404254+0000","last_peered":"2024-07-27T03:13:57.404254+0000","last_clean":"2024-07-27T03:13:57.404254+0000","last_became_active":"2024-07-27T03:12:56.725637+0000","last_became_peered":"2024-07-27T03:12:56.725637+0000","last_unstale":"2024-07-27T03:13:57.404254+0000","last_undegraded":"2024-07-27T03:13:57.404254+0000","last_fullsized":"2024-07-27T03:13:57.404254+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:36:41.725150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404405+0000","last_change":"2024-07-27T03:12:32.568232+0000","last_active":"2024-07-27T03:13:57.404405+0000","last_peered":"2024-07-27T03:13:57.404405+0000","last_clean":"2024-07-27T03:13:57.404405+0000","last_became_active":"2024-07-27T03:12:32.567834+0000","last_became_peered":"2024-07-27T03:12:32.567834+0000","last_unstale":"2024-07-27T03:13:57.404405+0000","last_undegraded":"2024-07-27T03:13:57.404405+0000","last_fullsized":"2024-07-27T03:13:57.404405+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:40:28.234539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405616+0000","last_change":"2024-07-27T03:12:32.573451+0000","last_active":"2024-07-27T03:13:57.405616+0000","last_peered":"2024-07-27T03:13:57.405616+0000","last_clean":"2024-07-27T03:13:57.405616+0000","last_became_active":"2024-07-27T03:12:32.573066+0000","last_became_peered":"2024-07-27T03:12:32.573066+0000","last_unstale":"2024-07-27T03:13:57.405616+0000","last_undegraded":"2024-07-27T03:13:57.405616+0000","last_fullsized":"2024-07-27T03:13:57.405616+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:13:29.104055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348685+0000","last_change":"2024-07-27T03:12:32.561067+0000","last_active":"2024-07-27T03:13:57.348685+0000","last_peered":"2024-07-27T03:13:57.348685+0000","last_clean":"2024-07-27T03:13:57.348685+0000","last_became_active":"2024-07-27T03:12:32.560358+0000","last_became_peered":"2024-07-27T03:12:32.560358+0000","last_unstale":"2024-07-27T03:13:57.348685+0000","last_undegraded":"2024-07-27T03:13:57.348685+0000","last_fullsized":"2024-07-27T03:13:57.348685+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:41:41.026804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405528+0000","last_change":"2024-07-27T03:12:32.572290+0000","last_active":"2024-07-27T03:13:57.405528+0000","last_peered":"2024-07-27T03:13:57.405528+0000","last_clean":"2024-07-27T03:13:57.405528+0000","last_became_active":"2024-07-27T03:12:32.571877+0000","last_became_peered":"2024-07-27T03:12:32.571877+0000","last_unstale":"2024-07-27T03:13:57.405528+0000","last_undegraded":"2024-07-27T03:13:57.405528+0000","last_fullsized":"2024-07-27T03:13:57.405528+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:34:40.559787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303465+0000","last_change":"2024-07-27T03:12:32.565495+0000","last_active":"2024-07-27T03:13:57.303465+0000","last_peered":"2024-07-27T03:13:57.303465+0000","last_clean":"2024-07-27T03:13:57.303465+0000","last_became_active":"2024-07-27T03:12:32.563136+0000","last_became_peered":"2024-07-27T03:12:32.563136+0000","last_unstale":"2024-07-27T03:13:57.303465+0000","last_undegraded":"2024-07-27T03:13:57.303465+0000","last_fullsized":"2024-07-27T03:13:57.303465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:31:02.032776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302646+0000","last_change":"2024-07-27T03:12:32.569477+0000","last_active":"2024-07-27T03:13:57.302646+0000","last_peered":"2024-07-27T03:13:57.302646+0000","last_clean":"2024-07-27T03:13:57.302646+0000","last_became_active":"2024-07-27T03:12:32.569098+0000","last_became_peered":"2024-07-27T03:12:32.569098+0000","last_unstale":"2024-07-27T03:13:57.302646+0000","last_undegraded":"2024-07-27T03:13:57.302646+0000","last_fullsized":"2024-07-27T03:13:57.302646+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:44:26.082539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406172+0000","last_change":"2024-07-27T03:12:32.574704+0000","last_active":"2024-07-27T03:13:57.406172+0000","last_peered":"2024-07-27T03:13:57.406172+0000","last_clean":"2024-07-27T03:13:57.406172+0000","last_became_active":"2024-07-27T03:12:32.574261+0000","last_became_peered":"2024-07-27T03:12:32.574261+0000","last_unstale":"2024-07-27T03:13:57.406172+0000","last_undegraded":"2024-07-27T03:13:57.406172+0000","last_fullsized":"2024-07-27T03:13:57.406172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:53:06.445267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405155+0000","last_change":"2024-07-27T03:12:32.573835+0000","last_active":"2024-07-27T03:13:57.405155+0000","last_peered":"2024-07-27T03:13:57.405155+0000","last_clean":"2024-07-27T03:13:57.405155+0000","last_became_active":"2024-07-27T03:12:32.573450+0000","last_became_peered":"2024-07-27T03:12:32.573450+0000","last_unstale":"2024-07-27T03:13:57.405155+0000","last_undegraded":"2024-07-27T03:13:57.405155+0000","last_fullsized":"2024-07-27T03:13:57.405155+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:40:45.264000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404972+0000","last_change":"2024-07-27T03:12:32.568542+0000","last_active":"2024-07-27T03:13:57.404972+0000","last_peered":"2024-07-27T03:13:57.404972+0000","last_clean":"2024-07-27T03:13:57.404972+0000","last_became_active":"2024-07-27T03:12:32.568179+0000","last_became_peered":"2024-07-27T03:12:32.568179+0000","last_unstale":"2024-07-27T03:13:57.404972+0000","last_undegraded":"2024-07-27T03:13:57.404972+0000","last_fullsized":"2024-07-27T03:13:57.404972+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:32:10.005601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404937+0000","last_change":"2024-07-27T03:12:32.564063+0000","last_active":"2024-07-27T03:13:57.404937+0000","last_peered":"2024-07-27T03:13:57.404937+0000","last_clean":"2024-07-27T03:13:57.404937+0000","last_became_active":"2024-07-27T03:12:32.563705+0000","last_became_peered":"2024-07-27T03:12:32.563705+0000","last_unstale":"2024-07-27T03:13:57.404937+0000","last_undegraded":"2024-07-27T03:13:57.404937+0000","last_fullsized":"2024-07-27T03:13:57.404937+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:40:19.234194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348625+0000","last_change":"2024-07-27T03:12:57.178963+0000","last_active":"2024-07-27T03:13:57.348625+0000","last_peered":"2024-07-27T03:13:57.348625+0000","last_clean":"2024-07-27T03:13:57.348625+0000","last_became_active":"2024-07-27T03:12:57.178550+0000","last_became_peered":"2024-07-27T03:12:57.178550+0000","last_unstale":"2024-07-27T03:13:57.348625+0000","last_undegraded":"2024-07-27T03:13:57.348625+0000","last_fullsized":"2024-07-27T03:13:57.348625+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:20:06.586957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304244+0000","last_change":"2024-07-27T03:12:32.569863+0000","last_active":"2024-07-27T03:13:57.304244+0000","last_peered":"2024-07-27T03:13:57.304244+0000","last_clean":"2024-07-27T03:13:57.304244+0000","last_became_active":"2024-07-27T03:12:32.566457+0000","last_became_peered":"2024-07-27T03:12:32.566457+0000","last_unstale":"2024-07-27T03:13:57.304244+0000","last_undegraded":"2024-07-27T03:13:57.304244+0000","last_fullsized":"2024-07-27T03:13:57.304244+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:28:13.141002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306014+0000","last_change":"2024-07-27T03:12:32.574490+0000","last_active":"2024-07-27T03:13:57.306014+0000","last_peered":"2024-07-27T03:13:57.306014+0000","last_clean":"2024-07-27T03:13:57.306014+0000","last_became_active":"2024-07-27T03:12:32.574130+0000","last_became_peered":"2024-07-27T03:12:32.574130+0000","last_unstale":"2024-07-27T03:13:57.306014+0000","last_undegraded":"2024-07-27T03:13:57.306014+0000","last_fullsized":"2024-07-27T03:13:57.306014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:44:22.696669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406538+0000","last_change":"2024-07-27T03:12:32.560106+0000","last_active":"2024-07-27T03:13:57.406538+0000","last_peered":"2024-07-27T03:13:57.406538+0000","last_clean":"2024-07-27T03:13:57.406538+0000","last_became_active":"2024-07-27T03:12:32.559751+0000","last_became_peered":"2024-07-27T03:12:32.559751+0000","last_unstale":"2024-07-27T03:13:57.406538+0000","last_undegraded":"2024-07-27T03:13:57.406538+0000","last_fullsized":"2024-07-27T03:13:57.406538+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:16:37.647462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406784+0000","last_change":"2024-07-27T03:12:32.569352+0000","last_active":"2024-07-27T03:13:57.406784+0000","last_peered":"2024-07-27T03:13:57.406784+0000","last_clean":"2024-07-27T03:13:57.406784+0000","last_became_active":"2024-07-27T03:12:32.569004+0000","last_became_peered":"2024-07-27T03:12:32.569004+0000","last_unstale":"2024-07-27T03:13:57.406784+0000","last_undegraded":"2024-07-27T03:13:57.406784+0000","last_fullsized":"2024-07-27T03:13:57.406784+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:21:25.999214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404116+0000","last_change":"2024-07-27T03:12:32.562775+0000","last_active":"2024-07-27T03:13:57.404116+0000","last_peered":"2024-07-27T03:13:57.404116+0000","last_clean":"2024-07-27T03:13:57.404116+0000","last_became_active":"2024-07-27T03:12:32.562403+0000","last_became_peered":"2024-07-27T03:12:32.562403+0000","last_unstale":"2024-07-27T03:13:57.404116+0000","last_undegraded":"2024-07-27T03:13:57.404116+0000","last_fullsized":"2024-07-27T03:13:57.404116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:16:09.474505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305583+0000","last_change":"2024-07-27T03:12:32.563271+0000","last_active":"2024-07-27T03:13:57.305583+0000","last_peered":"2024-07-27T03:13:57.305583+0000","last_clean":"2024-07-27T03:13:57.305583+0000","last_became_active":"2024-07-27T03:12:32.562850+0000","last_became_peered":"2024-07-27T03:12:32.562850+0000","last_unstale":"2024-07-27T03:13:57.305583+0000","last_undegraded":"2024-07-27T03:13:57.305583+0000","last_fullsized":"2024-07-27T03:13:57.305583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:40:27.688205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405997+0000","last_change":"2024-07-27T03:12:32.557580+0000","last_active":"2024-07-27T03:13:57.405997+0000","last_peered":"2024-07-27T03:13:57.405997+0000","last_clean":"2024-07-27T03:13:57.405997+0000","last_became_active":"2024-07-27T03:12:32.557190+0000","last_became_peered":"2024-07-27T03:12:32.557190+0000","last_unstale":"2024-07-27T03:13:57.405997+0000","last_undegraded":"2024-07-27T03:13:57.405997+0000","last_fullsized":"2024-07-27T03:13:57.405997+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:58:17.609035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305207+0000","last_change":"2024-07-27T03:12:32.563669+0000","last_active":"2024-07-27T03:13:57.305207+0000","last_peered":"2024-07-27T03:13:57.305207+0000","last_clean":"2024-07-27T03:13:57.305207+0000","last_became_active":"2024-07-27T03:12:32.563200+0000","last_became_peered":"2024-07-27T03:12:32.563200+0000","last_unstale":"2024-07-27T03:13:57.305207+0000","last_undegraded":"2024-07-27T03:13:57.305207+0000","last_fullsized":"2024-07-27T03:13:57.305207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:14:05.624118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304165+0000","last_change":"2024-07-27T03:12:32.561793+0000","last_active":"2024-07-27T03:13:57.304165+0000","last_peered":"2024-07-27T03:13:57.304165+0000","last_clean":"2024-07-27T03:13:57.304165+0000","last_became_active":"2024-07-27T03:12:32.561452+0000","last_became_peered":"2024-07-27T03:12:32.561452+0000","last_unstale":"2024-07-27T03:13:57.304165+0000","last_undegraded":"2024-07-27T03:13:57.304165+0000","last_fullsized":"2024-07-27T03:13:57.304165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:22:07.204824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405301+0000","last_change":"2024-07-27T03:12:32.569811+0000","last_active":"2024-07-27T03:13:57.405301+0000","last_peered":"2024-07-27T03:13:57.405301+0000","last_clean":"2024-07-27T03:13:57.405301+0000","last_became_active":"2024-07-27T03:12:32.569474+0000","last_became_peered":"2024-07-27T03:12:32.569474+0000","last_unstale":"2024-07-27T03:13:57.405301+0000","last_undegraded":"2024-07-27T03:13:57.405301+0000","last_fullsized":"2024-07-27T03:13:57.405301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:38:39.317362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405344+0000","last_change":"2024-07-27T03:12:32.565159+0000","last_active":"2024-07-27T03:13:57.405344+0000","last_peered":"2024-07-27T03:13:57.405344+0000","last_clean":"2024-07-27T03:13:57.405344+0000","last_became_active":"2024-07-27T03:12:32.564750+0000","last_became_peered":"2024-07-27T03:12:32.564750+0000","last_unstale":"2024-07-27T03:13:57.405344+0000","last_undegraded":"2024-07-27T03:13:57.405344+0000","last_fullsized":"2024-07-27T03:13:57.405344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:51:26.186510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406142+0000","last_change":"2024-07-27T03:12:32.574429+0000","last_active":"2024-07-27T03:13:57.406142+0000","last_peered":"2024-07-27T03:13:57.406142+0000","last_clean":"2024-07-27T03:13:57.406142+0000","last_became_active":"2024-07-27T03:12:32.573986+0000","last_became_peered":"2024-07-27T03:12:32.573986+0000","last_unstale":"2024-07-27T03:13:57.406142+0000","last_undegraded":"2024-07-27T03:13:57.406142+0000","last_fullsized":"2024-07-27T03:13:57.406142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:35:00.345561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69276,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":998,"num_read_kb":572,"num_write":249,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":207828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645360,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":525562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841589,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37044,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708108,"statfs":{"total":95995035648,"available":95957102592,"internally_reserved":0,"allocated":1748992,"data_stored":1178915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037818,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1077248,"data_stored":525470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234046,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":525826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430275,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1671168,"data_stored":1122736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593800,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":520974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790029,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1679360,"data_stored":1117927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953554,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1060864,"data_stored":525439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149783,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":520682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1138688,"data_stored":584687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1118208,"data_stored":584076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1077248,"data_stored":522430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-27T03:15:20.714 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph pg dump --format=json 2024-07-27T03:15:20.983 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:15:21.359 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:21 smithi067 ceph-mon[32814]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:15:21.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:21 smithi149 ceph-mon[37173]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:15:21.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:21 smithi175 ceph-mon[37125]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:15:22.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:22 smithi067 ceph-mon[32814]: pgmap v394: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:22.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:22 smithi149 ceph-mon[37173]: pgmap v394: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:22.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:22 smithi175 ceph-mon[37125]: pgmap v394: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:22.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.731+0000 7f683c734640 1 -- 172.21.15.67:0/3733355645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6834104d40 msgr2=0x7f683410d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:22.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.731+0000 7f683c734640 1 --2- 172.21.15.67:0/3733355645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6834104d40 0x7f683410d380 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6824009900 tx=0x7f682402a410 comp rx=0 tx=0).stop 2024-07-27T03:15:22.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.731+0000 7f683c734640 1 -- 172.21.15.67:0/3733355645 shutdown_connections 2024-07-27T03:15:22.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.731+0000 7f683c734640 1 --2- 172.21.15.67:0/3733355645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f683410d8c0 0x7f683410fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:22.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.731+0000 7f683c734640 1 --2- 172.21.15.67:0/3733355645 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6834104d40 0x7f683410d380 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:22.734 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.731+0000 7f683c734640 1 --2- 172.21.15.67:0/3733355645 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6834104310 0x7f6834104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:22.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.731+0000 7f683c734640 1 -- 172.21.15.67:0/3733355645 >> 172.21.15.67:0/3733355645 conn(0x7f68340791b0 msgr2=0x7f68340795c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:22.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.731+0000 7f683c734640 1 -- 172.21.15.67:0/3733355645 shutdown_connections 2024-07-27T03:15:22.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.731+0000 7f683c734640 1 -- 172.21.15.67:0/3733355645 wait complete. 2024-07-27T03:15:22.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.732+0000 7f683c734640 1 Processor -- start 2024-07-27T03:15:22.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.732+0000 7f683c734640 1 -- start start 2024-07-27T03:15:22.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.732+0000 7f683c734640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6834104310 0x7f6834199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:22.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.732+0000 7f683c734640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6834104d40 0x7f683419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:22.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.732+0000 7f683c734640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f683410d8c0 0x7f683419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:22.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.732+0000 7f683c734640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f68341134e0 con 0x7f683410d8c0 2024-07-27T03:15:22.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.732+0000 7f683c734640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6834113360 con 0x7f6834104310 2024-07-27T03:15:22.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.732+0000 7f683c734640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6834113660 con 0x7f6834104d40 2024-07-27T03:15:22.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.732+0000 7f683acd1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f683410d8c0 0x7f683419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:22.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.732+0000 7f683acd1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f683410d8c0 0x7f683419f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55470/0 (socket says 172.21.15.67:55470) 2024-07-27T03:15:22.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.732+0000 7f683acd1640 1 -- 172.21.15.67:0/1902813424 learned_addr learned my addr 172.21.15.67:0/1902813424 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:22.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.732+0000 7f683a4d0640 1 --2- 172.21.15.67:0/1902813424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6834104310 0x7f6834199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:22.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.733+0000 7f6839ccf640 1 --2- 172.21.15.67:0/1902813424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6834104d40 0x7f683419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:22.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.733+0000 7f683acd1640 1 -- 172.21.15.67:0/1902813424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6834104d40 msgr2=0x7f683419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:22.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.733+0000 7f683acd1640 1 --2- 172.21.15.67:0/1902813424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6834104d40 0x7f683419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:22.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.733+0000 7f683acd1640 1 -- 172.21.15.67:0/1902813424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6834104310 msgr2=0x7f6834199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:22.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.733+0000 7f683acd1640 1 --2- 172.21.15.67:0/1902813424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6834104310 0x7f6834199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:22.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.733+0000 7f683acd1640 1 -- 172.21.15.67:0/1902813424 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683419fe40 con 0x7f683410d8c0 2024-07-27T03:15:22.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.733+0000 7f683a4d0640 1 --2- 172.21.15.67:0/1902813424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6834104310 0x7f6834199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:15:22.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.733+0000 7f683acd1640 1 --2- 172.21.15.67:0/1902813424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f683410d8c0 0x7f683419f710 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f682c00eb10 tx=0x7f682c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:22.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.733+0000 7f682b7fe640 1 -- 172.21.15.67:0/1902813424 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c004b20 con 0x7f683410d8c0 2024-07-27T03:15:22.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.733+0000 7f682b7fe640 1 -- 172.21.15.67:0/1902813424 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f682c004cc0 con 0x7f683410d8c0 2024-07-27T03:15:22.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.733+0000 7f683c734640 1 -- 172.21.15.67:0/1902813424 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68341a0130 con 0x7f683410d8c0 2024-07-27T03:15:22.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.733+0000 7f682b7fe640 1 -- 172.21.15.67:0/1902813424 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c010670 con 0x7f683410d8c0 2024-07-27T03:15:22.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.733+0000 7f683c734640 1 -- 172.21.15.67:0/1902813424 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68341a6890 con 0x7f683410d8c0 2024-07-27T03:15:22.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.734+0000 7f683c734640 1 -- 172.21.15.67:0/1902813424 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f683406bc10 con 0x7f683410d8c0 2024-07-27T03:15:22.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.736+0000 7f682b7fe640 1 -- 172.21.15.67:0/1902813424 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f682c005000 con 0x7f683410d8c0 2024-07-27T03:15:22.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.737+0000 7f682b7fe640 1 --2- 172.21.15.67:0/1902813424 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f68200b3ce0 0x7f68200b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:22.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.737+0000 7f683a4d0640 1 --2- 172.21.15.67:0/1902813424 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f68200b3ce0 0x7f68200b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:22.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.737+0000 7f682b7fe640 1 -- 172.21.15.67:0/1902813424 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f682c0e4850 con 0x7f683410d8c0 2024-07-27T03:15:22.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.737+0000 7f683a4d0640 1 --2- 172.21.15.67:0/1902813424 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f68200b3ce0 0x7f68200b61f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f683406c310 tx=0x7f6830005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:22.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.740+0000 7f682b7fe640 1 -- 172.21.15.67:0/1902813424 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f682c055600 con 0x7f683410d8c0 2024-07-27T03:15:22.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.921+0000 7f683c734640 1 -- 172.21.15.67:0/1902813424 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6834063270 con 0x7f68200b3ce0 2024-07-27T03:15:22.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.927+0000 7f682b7fe640 1 -- 172.21.15.67:0/1902813424 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340427 (secure 0 0 0) 0x7f6834063270 con 0x7f68200b3ce0 2024-07-27T03:15:22.933 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:15:22.935 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-07-27T03:15:22.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.930+0000 7f683c734640 1 -- 172.21.15.67:0/1902813424 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f68200b3ce0 msgr2=0x7f68200b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:22.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.930+0000 7f683c734640 1 --2- 172.21.15.67:0/1902813424 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f68200b3ce0 0x7f68200b61f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f683406c310 tx=0x7f6830005f90 comp rx=0 tx=0).stop 2024-07-27T03:15:22.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.930+0000 7f683c734640 1 -- 172.21.15.67:0/1902813424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f683410d8c0 msgr2=0x7f683419f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:22.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.930+0000 7f683c734640 1 --2- 172.21.15.67:0/1902813424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f683410d8c0 0x7f683419f710 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f682c00eb10 tx=0x7f682c00c2d0 comp rx=0 tx=0).stop 2024-07-27T03:15:22.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.930+0000 7f683c734640 1 -- 172.21.15.67:0/1902813424 shutdown_connections 2024-07-27T03:15:22.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.930+0000 7f683c734640 1 --2- 172.21.15.67:0/1902813424 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f68200b3ce0 0x7f68200b61f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:22.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.930+0000 7f683c734640 1 --2- 172.21.15.67:0/1902813424 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f683410d8c0 0x7f683419f710 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:22.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.930+0000 7f683c734640 1 --2- 172.21.15.67:0/1902813424 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6834104d40 0x7f683419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:22.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.930+0000 7f683c734640 1 --2- 172.21.15.67:0/1902813424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6834104310 0x7f6834199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:22.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.930+0000 7f683c734640 1 -- 172.21.15.67:0/1902813424 >> 172.21.15.67:0/1902813424 conn(0x7f68340791b0 msgr2=0x7f6834100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:22.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.931+0000 7f683c734640 1 -- 172.21.15.67:0/1902813424 shutdown_connections 2024-07-27T03:15:22.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:22.931+0000 7f683c734640 1 -- 172.21.15.67:0/1902813424 wait complete. 2024-07-27T03:15:23.769 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-07-27T03:15:21.093846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659878,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1109,"num_read_kb":656,"num_write":374,"num_write_kb":2294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951979,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388624,"kb_used_data":14468,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553200,"statfs":{"total":1151940427776,"available":1151542476800,"internally_reserved":0,"allocated":14815232,"data_stored":8254724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003700"},"pg_stats":[{"pgid":"2.1f","version":"89'13","reported_seq":155,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:14.396927+0000","last_change":"2024-07-27T03:12:29.541107+0000","last_active":"2024-07-27T03:15:14.396927+0000","last_peered":"2024-07-27T03:15:14.396927+0000","last_clean":"2024-07-27T03:15:14.396927+0000","last_became_active":"2024-07-27T03:12:29.540806+0000","last_became_peered":"2024-07-27T03:12:29.540806+0000","last_unstale":"2024-07-27T03:15:14.396927+0000","last_undegraded":"2024-07-27T03:15:14.396927+0000","last_fullsized":"2024-07-27T03:15:14.396927+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:38:24.349354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404651+0000","last_change":"2024-07-27T03:12:32.565515+0000","last_active":"2024-07-27T03:13:57.404651+0000","last_peered":"2024-07-27T03:13:57.404651+0000","last_clean":"2024-07-27T03:13:57.404651+0000","last_became_active":"2024-07-27T03:12:32.565134+0000","last_became_peered":"2024-07-27T03:12:32.565134+0000","last_unstale":"2024-07-27T03:13:57.404651+0000","last_undegraded":"2024-07-27T03:13:57.404651+0000","last_fullsized":"2024-07-27T03:13:57.404651+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:11:56.793139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302752+0000","last_change":"2024-07-27T03:12:42.613432+0000","last_active":"2024-07-27T03:13:57.302752+0000","last_peered":"2024-07-27T03:13:57.302752+0000","last_clean":"2024-07-27T03:13:57.302752+0000","last_became_active":"2024-07-27T03:12:42.612986+0000","last_became_peered":"2024-07-27T03:12:42.612986+0000","last_unstale":"2024-07-27T03:13:57.302752+0000","last_undegraded":"2024-07-27T03:13:57.302752+0000","last_fullsized":"2024-07-27T03:13:57.302752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:25:37.505117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405630+0000","last_change":"2024-07-27T03:12:29.614229+0000","last_active":"2024-07-27T03:13:57.405630+0000","last_peered":"2024-07-27T03:13:57.405630+0000","last_clean":"2024-07-27T03:13:57.405630+0000","last_became_active":"2024-07-27T03:12:29.613844+0000","last_became_peered":"2024-07-27T03:12:29.613844+0000","last_unstale":"2024-07-27T03:13:57.405630+0000","last_undegraded":"2024-07-27T03:13:57.405630+0000","last_fullsized":"2024-07-27T03:13:57.405630+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:10:28.917080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405772+0000","last_change":"2024-07-27T03:12:32.563187+0000","last_active":"2024-07-27T03:13:57.405772+0000","last_peered":"2024-07-27T03:13:57.405772+0000","last_clean":"2024-07-27T03:13:57.405772+0000","last_became_active":"2024-07-27T03:12:32.562674+0000","last_became_peered":"2024-07-27T03:12:32.562674+0000","last_unstale":"2024-07-27T03:13:57.405772+0000","last_undegraded":"2024-07-27T03:13:57.405772+0000","last_fullsized":"2024-07-27T03:13:57.405772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:05:39.211514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303151+0000","last_change":"2024-07-27T03:12:42.611516+0000","last_active":"2024-07-27T03:13:57.303151+0000","last_peered":"2024-07-27T03:13:57.303151+0000","last_clean":"2024-07-27T03:13:57.303151+0000","last_became_active":"2024-07-27T03:12:42.610934+0000","last_became_peered":"2024-07-27T03:12:42.610934+0000","last_unstale":"2024-07-27T03:13:57.303151+0000","last_undegraded":"2024-07-27T03:13:57.303151+0000","last_fullsized":"2024-07-27T03:13:57.303151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:13:17.323063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":58,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305740+0000","last_change":"2024-07-27T03:12:29.539573+0000","last_active":"2024-07-27T03:13:57.305740+0000","last_peered":"2024-07-27T03:13:57.305740+0000","last_clean":"2024-07-27T03:13:57.305740+0000","last_became_active":"2024-07-27T03:12:29.539056+0000","last_became_peered":"2024-07-27T03:12:29.539056+0000","last_unstale":"2024-07-27T03:13:57.305740+0000","last_undegraded":"2024-07-27T03:13:57.305740+0000","last_fullsized":"2024-07-27T03:13:57.305740+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:04:37.513644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405653+0000","last_change":"2024-07-27T03:12:32.567729+0000","last_active":"2024-07-27T03:13:57.405653+0000","last_peered":"2024-07-27T03:13:57.405653+0000","last_clean":"2024-07-27T03:13:57.405653+0000","last_became_active":"2024-07-27T03:12:32.564827+0000","last_became_peered":"2024-07-27T03:12:32.564827+0000","last_unstale":"2024-07-27T03:13:57.405653+0000","last_undegraded":"2024-07-27T03:13:57.405653+0000","last_fullsized":"2024-07-27T03:13:57.405653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:10:55.314988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303339+0000","last_change":"2024-07-27T03:12:42.612776+0000","last_active":"2024-07-27T03:13:57.303339+0000","last_peered":"2024-07-27T03:13:57.303339+0000","last_clean":"2024-07-27T03:13:57.303339+0000","last_became_active":"2024-07-27T03:12:42.612258+0000","last_became_peered":"2024-07-27T03:12:42.612258+0000","last_unstale":"2024-07-27T03:13:57.303339+0000","last_undegraded":"2024-07-27T03:13:57.303339+0000","last_fullsized":"2024-07-27T03:13:57.303339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:48:06.516002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305959+0000","last_change":"2024-07-27T03:12:29.540070+0000","last_active":"2024-07-27T03:13:57.305959+0000","last_peered":"2024-07-27T03:13:57.305959+0000","last_clean":"2024-07-27T03:13:57.305959+0000","last_became_active":"2024-07-27T03:12:29.539657+0000","last_became_peered":"2024-07-27T03:12:29.539657+0000","last_unstale":"2024-07-27T03:13:57.305959+0000","last_undegraded":"2024-07-27T03:13:57.305959+0000","last_fullsized":"2024-07-27T03:13:57.305959+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:42:48.222613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405070+0000","last_change":"2024-07-27T03:12:32.574418+0000","last_active":"2024-07-27T03:13:57.405070+0000","last_peered":"2024-07-27T03:13:57.405070+0000","last_clean":"2024-07-27T03:13:57.405070+0000","last_became_active":"2024-07-27T03:12:32.573974+0000","last_became_peered":"2024-07-27T03:12:32.573974+0000","last_unstale":"2024-07-27T03:13:57.405070+0000","last_undegraded":"2024-07-27T03:13:57.405070+0000","last_fullsized":"2024-07-27T03:13:57.405070+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:34:30.442833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303642+0000","last_change":"2024-07-27T03:12:42.605155+0000","last_active":"2024-07-27T03:13:57.303642+0000","last_peered":"2024-07-27T03:13:57.303642+0000","last_clean":"2024-07-27T03:13:57.303642+0000","last_became_active":"2024-07-27T03:12:42.604685+0000","last_became_peered":"2024-07-27T03:12:42.604685+0000","last_unstale":"2024-07-27T03:13:57.303642+0000","last_undegraded":"2024-07-27T03:13:57.303642+0000","last_fullsized":"2024-07-27T03:13:57.303642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:28:58.931408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305862+0000","last_change":"2024-07-27T03:12:32.574711+0000","last_active":"2024-07-27T03:13:57.305862+0000","last_peered":"2024-07-27T03:13:57.305862+0000","last_clean":"2024-07-27T03:13:57.305862+0000","last_became_active":"2024-07-27T03:12:32.574365+0000","last_became_peered":"2024-07-27T03:12:32.574365+0000","last_unstale":"2024-07-27T03:13:57.305862+0000","last_undegraded":"2024-07-27T03:13:57.305862+0000","last_fullsized":"2024-07-27T03:13:57.305862+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:50:56.621371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305326+0000","last_change":"2024-07-27T03:12:29.540835+0000","last_active":"2024-07-27T03:13:57.305326+0000","last_peered":"2024-07-27T03:13:57.305326+0000","last_clean":"2024-07-27T03:13:57.305326+0000","last_became_active":"2024-07-27T03:12:29.540461+0000","last_became_peered":"2024-07-27T03:12:29.540461+0000","last_unstale":"2024-07-27T03:13:57.305326+0000","last_undegraded":"2024-07-27T03:13:57.305326+0000","last_fullsized":"2024-07-27T03:13:57.305326+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:19:20.817038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303344+0000","last_change":"2024-07-27T03:12:42.718584+0000","last_active":"2024-07-27T03:13:57.303344+0000","last_peered":"2024-07-27T03:13:57.303344+0000","last_clean":"2024-07-27T03:13:57.303344+0000","last_became_active":"2024-07-27T03:12:42.717752+0000","last_became_peered":"2024-07-27T03:12:42.717752+0000","last_unstale":"2024-07-27T03:13:57.303344+0000","last_undegraded":"2024-07-27T03:13:57.303344+0000","last_fullsized":"2024-07-27T03:13:57.303344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:23:52.629794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404408+0000","last_change":"2024-07-27T03:12:32.560672+0000","last_active":"2024-07-27T03:13:57.404408+0000","last_peered":"2024-07-27T03:13:57.404408+0000","last_clean":"2024-07-27T03:13:57.404408+0000","last_became_active":"2024-07-27T03:12:32.559172+0000","last_became_peered":"2024-07-27T03:12:32.559172+0000","last_unstale":"2024-07-27T03:13:57.404408+0000","last_undegraded":"2024-07-27T03:13:57.404408+0000","last_fullsized":"2024-07-27T03:13:57.404408+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:02:49.346983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405206+0000","last_change":"2024-07-27T03:12:57.434569+0000","last_active":"2024-07-27T03:13:57.405206+0000","last_peered":"2024-07-27T03:13:57.405206+0000","last_clean":"2024-07-27T03:13:57.405206+0000","last_became_active":"2024-07-27T03:12:57.434038+0000","last_became_peered":"2024-07-27T03:12:57.434038+0000","last_unstale":"2024-07-27T03:13:57.405206+0000","last_undegraded":"2024-07-27T03:13:57.405206+0000","last_fullsized":"2024-07-27T03:13:57.405206+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:43:24.462838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.412383+0000","last_change":"2024-07-27T03:13:57.411931+0000","last_active":"2024-07-27T03:13:57.412383+0000","last_peered":"2024-07-27T03:13:57.412383+0000","last_clean":"2024-07-27T03:13:57.412383+0000","last_became_active":"2024-07-27T03:13:57.407362+0000","last_became_peered":"2024-07-27T03:13:57.407362+0000","last_unstale":"2024-07-27T03:13:57.412383+0000","last_undegraded":"2024-07-27T03:13:57.412383+0000","last_fullsized":"2024-07-27T03:13:57.412383+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:52:08.444268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405164+0000","last_change":"2024-07-27T03:12:32.572129+0000","last_active":"2024-07-27T03:13:57.405164+0000","last_peered":"2024-07-27T03:13:57.405164+0000","last_clean":"2024-07-27T03:13:57.405164+0000","last_became_active":"2024-07-27T03:12:32.571774+0000","last_became_peered":"2024-07-27T03:12:32.571774+0000","last_unstale":"2024-07-27T03:13:57.405164+0000","last_undegraded":"2024-07-27T03:13:57.405164+0000","last_fullsized":"2024-07-27T03:13:57.405164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:52:23.316957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404446+0000","last_change":"2024-07-27T03:12:29.539768+0000","last_active":"2024-07-27T03:13:57.404446+0000","last_peered":"2024-07-27T03:13:57.404446+0000","last_clean":"2024-07-27T03:13:57.404446+0000","last_became_active":"2024-07-27T03:12:29.539389+0000","last_became_peered":"2024-07-27T03:12:29.539389+0000","last_unstale":"2024-07-27T03:13:57.404446+0000","last_undegraded":"2024-07-27T03:13:57.404446+0000","last_fullsized":"2024-07-27T03:13:57.404446+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:09:35.961328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302623+0000","last_change":"2024-07-27T03:12:42.617569+0000","last_active":"2024-07-27T03:13:57.302623+0000","last_peered":"2024-07-27T03:13:57.302623+0000","last_clean":"2024-07-27T03:13:57.302623+0000","last_became_active":"2024-07-27T03:12:42.616357+0000","last_became_peered":"2024-07-27T03:12:42.616357+0000","last_unstale":"2024-07-27T03:13:57.302623+0000","last_undegraded":"2024-07-27T03:13:57.302623+0000","last_fullsized":"2024-07-27T03:13:57.302623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:35:11.297631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304839+0000","last_change":"2024-07-27T03:12:32.566272+0000","last_active":"2024-07-27T03:13:57.304839+0000","last_peered":"2024-07-27T03:13:57.304839+0000","last_clean":"2024-07-27T03:13:57.304839+0000","last_became_active":"2024-07-27T03:12:32.565865+0000","last_became_peered":"2024-07-27T03:12:32.565865+0000","last_unstale":"2024-07-27T03:13:57.304839+0000","last_undegraded":"2024-07-27T03:13:57.304839+0000","last_fullsized":"2024-07-27T03:13:57.304839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:42:09.539586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405359+0000","last_change":"2024-07-27T03:12:29.613733+0000","last_active":"2024-07-27T03:13:57.405359+0000","last_peered":"2024-07-27T03:13:57.405359+0000","last_clean":"2024-07-27T03:13:57.405359+0000","last_became_active":"2024-07-27T03:12:29.613395+0000","last_became_peered":"2024-07-27T03:12:29.613395+0000","last_unstale":"2024-07-27T03:13:57.405359+0000","last_undegraded":"2024-07-27T03:13:57.405359+0000","last_fullsized":"2024-07-27T03:13:57.405359+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:21:34.431900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.408340+0000","last_change":"2024-07-27T03:13:57.407798+0000","last_active":"2024-07-27T03:13:57.408340+0000","last_peered":"2024-07-27T03:13:57.408340+0000","last_clean":"2024-07-27T03:13:57.408340+0000","last_became_active":"2024-07-27T03:13:57.407275+0000","last_became_peered":"2024-07-27T03:13:57.407275+0000","last_unstale":"2024-07-27T03:13:57.408340+0000","last_undegraded":"2024-07-27T03:13:57.408340+0000","last_fullsized":"2024-07-27T03:13:57.408340+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:47:27.817092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,8],"acting":[4,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405521+0000","last_change":"2024-07-27T03:12:32.567006+0000","last_active":"2024-07-27T03:13:57.405521+0000","last_peered":"2024-07-27T03:13:57.405521+0000","last_clean":"2024-07-27T03:13:57.405521+0000","last_became_active":"2024-07-27T03:12:32.566205+0000","last_became_peered":"2024-07-27T03:12:32.566205+0000","last_unstale":"2024-07-27T03:13:57.405521+0000","last_undegraded":"2024-07-27T03:13:57.405521+0000","last_fullsized":"2024-07-27T03:13:57.405521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:11:23.171341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303695+0000","last_change":"2024-07-27T03:12:29.533909+0000","last_active":"2024-07-27T03:13:57.303695+0000","last_peered":"2024-07-27T03:13:57.303695+0000","last_clean":"2024-07-27T03:13:57.303695+0000","last_became_active":"2024-07-27T03:12:29.533327+0000","last_became_peered":"2024-07-27T03:12:29.533327+0000","last_unstale":"2024-07-27T03:13:57.303695+0000","last_undegraded":"2024-07-27T03:13:57.303695+0000","last_fullsized":"2024-07-27T03:13:57.303695+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:35:24.705096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302923+0000","last_change":"2024-07-27T03:12:42.609066+0000","last_active":"2024-07-27T03:13:57.302923+0000","last_peered":"2024-07-27T03:13:57.302923+0000","last_clean":"2024-07-27T03:13:57.302923+0000","last_became_active":"2024-07-27T03:12:42.608561+0000","last_became_peered":"2024-07-27T03:12:42.608561+0000","last_unstale":"2024-07-27T03:13:57.302923+0000","last_undegraded":"2024-07-27T03:13:57.302923+0000","last_fullsized":"2024-07-27T03:13:57.302923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:27:53.657498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405653+0000","last_change":"2024-07-27T03:12:32.560931+0000","last_active":"2024-07-27T03:13:57.405653+0000","last_peered":"2024-07-27T03:13:57.405653+0000","last_clean":"2024-07-27T03:13:57.405653+0000","last_became_active":"2024-07-27T03:12:32.560549+0000","last_became_peered":"2024-07-27T03:12:32.560549+0000","last_unstale":"2024-07-27T03:13:57.405653+0000","last_undegraded":"2024-07-27T03:13:57.405653+0000","last_fullsized":"2024-07-27T03:13:57.405653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T15:11:25.136035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'15","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:02.583204+0000","last_change":"2024-07-27T03:12:29.535365+0000","last_active":"2024-07-27T03:14:02.583204+0000","last_peered":"2024-07-27T03:14:02.583204+0000","last_clean":"2024-07-27T03:14:02.583204+0000","last_became_active":"2024-07-27T03:12:29.534952+0000","last_became_peered":"2024-07-27T03:12:29.534952+0000","last_unstale":"2024-07-27T03:14:02.583204+0000","last_undegraded":"2024-07-27T03:14:02.583204+0000","last_fullsized":"2024-07-27T03:14:02.583204+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:34:59.889488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404064+0000","last_change":"2024-07-27T03:12:42.718416+0000","last_active":"2024-07-27T03:13:57.404064+0000","last_peered":"2024-07-27T03:13:57.404064+0000","last_clean":"2024-07-27T03:13:57.404064+0000","last_became_active":"2024-07-27T03:12:42.717949+0000","last_became_peered":"2024-07-27T03:12:42.717949+0000","last_unstale":"2024-07-27T03:13:57.404064+0000","last_undegraded":"2024-07-27T03:13:57.404064+0000","last_fullsized":"2024-07-27T03:13:57.404064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:05:05.762577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348872+0000","last_change":"2024-07-27T03:12:32.559824+0000","last_active":"2024-07-27T03:13:57.348872+0000","last_peered":"2024-07-27T03:13:57.348872+0000","last_clean":"2024-07-27T03:13:57.348872+0000","last_became_active":"2024-07-27T03:12:32.559435+0000","last_became_peered":"2024-07-27T03:12:32.559435+0000","last_unstale":"2024-07-27T03:13:57.348872+0000","last_undegraded":"2024-07-27T03:13:57.348872+0000","last_fullsized":"2024-07-27T03:13:57.348872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:16:44.514272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305857+0000","last_change":"2024-07-27T03:12:29.614296+0000","last_active":"2024-07-27T03:13:57.305857+0000","last_peered":"2024-07-27T03:13:57.305857+0000","last_clean":"2024-07-27T03:13:57.305857+0000","last_became_active":"2024-07-27T03:12:29.612987+0000","last_became_peered":"2024-07-27T03:12:29.612987+0000","last_unstale":"2024-07-27T03:13:57.305857+0000","last_undegraded":"2024-07-27T03:13:57.305857+0000","last_fullsized":"2024-07-27T03:13:57.305857+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:23:09.226523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405200+0000","last_change":"2024-07-27T03:12:42.616575+0000","last_active":"2024-07-27T03:13:57.405200+0000","last_peered":"2024-07-27T03:13:57.405200+0000","last_clean":"2024-07-27T03:13:57.405200+0000","last_became_active":"2024-07-27T03:12:42.615980+0000","last_became_peered":"2024-07-27T03:12:42.615980+0000","last_unstale":"2024-07-27T03:13:57.405200+0000","last_undegraded":"2024-07-27T03:13:57.405200+0000","last_fullsized":"2024-07-27T03:13:57.405200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:00:59.041910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406044+0000","last_change":"2024-07-27T03:12:32.556109+0000","last_active":"2024-07-27T03:13:57.406044+0000","last_peered":"2024-07-27T03:13:57.406044+0000","last_clean":"2024-07-27T03:13:57.406044+0000","last_became_active":"2024-07-27T03:12:32.555747+0000","last_became_peered":"2024-07-27T03:12:32.555747+0000","last_unstale":"2024-07-27T03:13:57.406044+0000","last_undegraded":"2024-07-27T03:13:57.406044+0000","last_fullsized":"2024-07-27T03:13:57.406044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:27:58.286605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302565+0000","last_change":"2024-07-27T03:12:29.613523+0000","last_active":"2024-07-27T03:13:57.302565+0000","last_peered":"2024-07-27T03:13:57.302565+0000","last_clean":"2024-07-27T03:13:57.302565+0000","last_became_active":"2024-07-27T03:12:29.613172+0000","last_became_peered":"2024-07-27T03:12:29.613172+0000","last_unstale":"2024-07-27T03:13:57.302565+0000","last_undegraded":"2024-07-27T03:13:57.302565+0000","last_fullsized":"2024-07-27T03:13:57.302565+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:49:45.225221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304930+0000","last_change":"2024-07-27T03:12:42.718536+0000","last_active":"2024-07-27T03:13:57.304930+0000","last_peered":"2024-07-27T03:13:57.304930+0000","last_clean":"2024-07-27T03:13:57.304930+0000","last_became_active":"2024-07-27T03:12:42.718068+0000","last_became_peered":"2024-07-27T03:12:42.718068+0000","last_unstale":"2024-07-27T03:13:57.304930+0000","last_undegraded":"2024-07-27T03:13:57.304930+0000","last_fullsized":"2024-07-27T03:13:57.304930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:18:28.478700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406800+0000","last_change":"2024-07-27T03:12:32.558060+0000","last_active":"2024-07-27T03:13:57.406800+0000","last_peered":"2024-07-27T03:13:57.406800+0000","last_clean":"2024-07-27T03:13:57.406800+0000","last_became_active":"2024-07-27T03:12:32.557260+0000","last_became_peered":"2024-07-27T03:12:32.557260+0000","last_unstale":"2024-07-27T03:13:57.406800+0000","last_undegraded":"2024-07-27T03:13:57.406800+0000","last_fullsized":"2024-07-27T03:13:57.406800+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:36:54.945761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":148,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:15.399209+0000","last_change":"2024-07-27T03:12:29.541080+0000","last_active":"2024-07-27T03:15:15.399209+0000","last_peered":"2024-07-27T03:15:15.399209+0000","last_clean":"2024-07-27T03:15:15.399209+0000","last_became_active":"2024-07-27T03:12:29.540628+0000","last_became_peered":"2024-07-27T03:12:29.540628+0000","last_unstale":"2024-07-27T03:15:15.399209+0000","last_undegraded":"2024-07-27T03:15:15.399209+0000","last_fullsized":"2024-07-27T03:15:15.399209+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:45:17.045124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348865+0000","last_change":"2024-07-27T03:12:42.611561+0000","last_active":"2024-07-27T03:13:57.348865+0000","last_peered":"2024-07-27T03:13:57.348865+0000","last_clean":"2024-07-27T03:13:57.348865+0000","last_became_active":"2024-07-27T03:12:42.610507+0000","last_became_peered":"2024-07-27T03:12:42.610507+0000","last_unstale":"2024-07-27T03:13:57.348865+0000","last_undegraded":"2024-07-27T03:13:57.348865+0000","last_fullsized":"2024-07-27T03:13:57.348865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:15:08.972807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404352+0000","last_change":"2024-07-27T03:12:32.562316+0000","last_active":"2024-07-27T03:13:57.404352+0000","last_peered":"2024-07-27T03:13:57.404352+0000","last_clean":"2024-07-27T03:13:57.404352+0000","last_became_active":"2024-07-27T03:12:32.561729+0000","last_became_peered":"2024-07-27T03:12:32.561729+0000","last_unstale":"2024-07-27T03:13:57.404352+0000","last_undegraded":"2024-07-27T03:13:57.404352+0000","last_fullsized":"2024-07-27T03:13:57.404352+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:01:19.960269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:17.402096+0000","last_change":"2024-07-27T03:12:29.613459+0000","last_active":"2024-07-27T03:15:17.402096+0000","last_peered":"2024-07-27T03:15:17.402096+0000","last_clean":"2024-07-27T03:15:17.402096+0000","last_became_active":"2024-07-27T03:12:29.613009+0000","last_became_peered":"2024-07-27T03:12:29.613009+0000","last_unstale":"2024-07-27T03:15:17.402096+0000","last_undegraded":"2024-07-27T03:15:17.402096+0000","last_fullsized":"2024-07-27T03:15:17.402096+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:16:37.230471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348695+0000","last_change":"2024-07-27T03:12:42.718516+0000","last_active":"2024-07-27T03:13:57.348695+0000","last_peered":"2024-07-27T03:13:57.348695+0000","last_clean":"2024-07-27T03:13:57.348695+0000","last_became_active":"2024-07-27T03:12:42.718041+0000","last_became_peered":"2024-07-27T03:12:42.718041+0000","last_unstale":"2024-07-27T03:13:57.348695+0000","last_undegraded":"2024-07-27T03:13:57.348695+0000","last_fullsized":"2024-07-27T03:13:57.348695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:35:39.633977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348259+0000","last_change":"2024-07-27T03:12:32.560752+0000","last_active":"2024-07-27T03:13:57.348259+0000","last_peered":"2024-07-27T03:13:57.348259+0000","last_clean":"2024-07-27T03:13:57.348259+0000","last_became_active":"2024-07-27T03:12:32.560248+0000","last_became_peered":"2024-07-27T03:12:32.560248+0000","last_unstale":"2024-07-27T03:13:57.348259+0000","last_undegraded":"2024-07-27T03:13:57.348259+0000","last_fullsized":"2024-07-27T03:13:57.348259+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:59:48.056867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305452+0000","last_change":"2024-07-27T03:12:29.539591+0000","last_active":"2024-07-27T03:13:57.305452+0000","last_peered":"2024-07-27T03:13:57.305452+0000","last_clean":"2024-07-27T03:13:57.305452+0000","last_became_active":"2024-07-27T03:12:29.539165+0000","last_became_peered":"2024-07-27T03:12:29.539165+0000","last_unstale":"2024-07-27T03:13:57.305452+0000","last_undegraded":"2024-07-27T03:13:57.305452+0000","last_fullsized":"2024-07-27T03:13:57.305452+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:04:21.940511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.408418+0000","last_change":"2024-07-27T03:13:57.407921+0000","last_active":"2024-07-27T03:13:57.408418+0000","last_peered":"2024-07-27T03:13:57.408418+0000","last_clean":"2024-07-27T03:13:57.408418+0000","last_became_active":"2024-07-27T03:13:57.407382+0000","last_became_peered":"2024-07-27T03:13:57.407382+0000","last_unstale":"2024-07-27T03:13:57.408418+0000","last_undegraded":"2024-07-27T03:13:57.408418+0000","last_fullsized":"2024-07-27T03:13:57.408418+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:48:27.362759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404373+0000","last_change":"2024-07-27T03:12:32.570653+0000","last_active":"2024-07-27T03:13:57.404373+0000","last_peered":"2024-07-27T03:13:57.404373+0000","last_clean":"2024-07-27T03:13:57.404373+0000","last_became_active":"2024-07-27T03:12:32.569088+0000","last_became_peered":"2024-07-27T03:12:32.569088+0000","last_unstale":"2024-07-27T03:13:57.404373+0000","last_undegraded":"2024-07-27T03:13:57.404373+0000","last_fullsized":"2024-07-27T03:13:57.404373+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:50:24.669431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303915+0000","last_change":"2024-07-27T03:12:29.541654+0000","last_active":"2024-07-27T03:13:57.303915+0000","last_peered":"2024-07-27T03:13:57.303915+0000","last_clean":"2024-07-27T03:13:57.303915+0000","last_became_active":"2024-07-27T03:12:29.541264+0000","last_became_peered":"2024-07-27T03:12:29.541264+0000","last_unstale":"2024-07-27T03:13:57.303915+0000","last_undegraded":"2024-07-27T03:13:57.303915+0000","last_fullsized":"2024-07-27T03:13:57.303915+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:48:05.179016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":122,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306449+0000","last_change":"2024-07-27T03:11:11.343199+0000","last_active":"2024-07-27T03:13:57.306449+0000","last_peered":"2024-07-27T03:13:57.306449+0000","last_clean":"2024-07-27T03:13:57.306449+0000","last_became_active":"2024-07-27T03:11:11.337635+0000","last_became_peered":"2024-07-27T03:11:11.337635+0000","last_unstale":"2024-07-27T03:13:57.306449+0000","last_undegraded":"2024-07-27T03:13:57.306449+0000","last_fullsized":"2024-07-27T03:13:57.306449+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:08:01.694830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:08:01.694830+0000","last_clean_scrub_stamp":"2024-07-27T03:08:01.694830+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:48:49.979610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305836+0000","last_change":"2024-07-27T03:12:42.614346+0000","last_active":"2024-07-27T03:13:57.305836+0000","last_peered":"2024-07-27T03:13:57.305836+0000","last_clean":"2024-07-27T03:13:57.305836+0000","last_became_active":"2024-07-27T03:12:42.613881+0000","last_became_peered":"2024-07-27T03:12:42.613881+0000","last_unstale":"2024-07-27T03:13:57.305836+0000","last_undegraded":"2024-07-27T03:13:57.305836+0000","last_fullsized":"2024-07-27T03:13:57.305836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:12:38.299348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306115+0000","last_change":"2024-07-27T03:12:32.574330+0000","last_active":"2024-07-27T03:13:57.306115+0000","last_peered":"2024-07-27T03:13:57.306115+0000","last_clean":"2024-07-27T03:13:57.306115+0000","last_became_active":"2024-07-27T03:12:32.573977+0000","last_became_peered":"2024-07-27T03:12:32.573977+0000","last_unstale":"2024-07-27T03:13:57.306115+0000","last_undegraded":"2024-07-27T03:13:57.306115+0000","last_fullsized":"2024-07-27T03:13:57.306115+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:39:09.925794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":153,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:18.404752+0000","last_change":"2024-07-27T03:12:29.613848+0000","last_active":"2024-07-27T03:15:18.404752+0000","last_peered":"2024-07-27T03:15:18.404752+0000","last_clean":"2024-07-27T03:15:18.404752+0000","last_became_active":"2024-07-27T03:12:29.613473+0000","last_became_peered":"2024-07-27T03:12:29.613473+0000","last_unstale":"2024-07-27T03:15:18.404752+0000","last_undegraded":"2024-07-27T03:15:18.404752+0000","last_fullsized":"2024-07-27T03:15:18.404752+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:52:56.894962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405373+0000","last_change":"2024-07-27T03:12:42.611295+0000","last_active":"2024-07-27T03:13:57.405373+0000","last_peered":"2024-07-27T03:13:57.405373+0000","last_clean":"2024-07-27T03:13:57.405373+0000","last_became_active":"2024-07-27T03:12:42.610782+0000","last_became_peered":"2024-07-27T03:12:42.610782+0000","last_unstale":"2024-07-27T03:13:57.405373+0000","last_undegraded":"2024-07-27T03:13:57.405373+0000","last_fullsized":"2024-07-27T03:13:57.405373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:24:02.849882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305956+0000","last_change":"2024-07-27T03:12:56.725924+0000","last_active":"2024-07-27T03:13:57.305956+0000","last_peered":"2024-07-27T03:13:57.305956+0000","last_clean":"2024-07-27T03:13:57.305956+0000","last_became_active":"2024-07-27T03:12:56.725567+0000","last_became_peered":"2024-07-27T03:12:56.725567+0000","last_unstale":"2024-07-27T03:13:57.305956+0000","last_undegraded":"2024-07-27T03:13:57.305956+0000","last_fullsized":"2024-07-27T03:13:57.305956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:32:23.771483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405225+0000","last_change":"2024-07-27T03:12:29.537863+0000","last_active":"2024-07-27T03:13:57.405225+0000","last_peered":"2024-07-27T03:13:57.405225+0000","last_clean":"2024-07-27T03:13:57.405225+0000","last_became_active":"2024-07-27T03:12:29.537444+0000","last_became_peered":"2024-07-27T03:12:29.537444+0000","last_unstale":"2024-07-27T03:13:57.405225+0000","last_undegraded":"2024-07-27T03:13:57.405225+0000","last_fullsized":"2024-07-27T03:13:57.405225+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:52:05.239809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405265+0000","last_change":"2024-07-27T03:12:42.622599+0000","last_active":"2024-07-27T03:13:57.405265+0000","last_peered":"2024-07-27T03:13:57.405265+0000","last_clean":"2024-07-27T03:13:57.405265+0000","last_became_active":"2024-07-27T03:12:42.622151+0000","last_became_peered":"2024-07-27T03:12:42.622151+0000","last_unstale":"2024-07-27T03:13:57.405265+0000","last_undegraded":"2024-07-27T03:13:57.405265+0000","last_fullsized":"2024-07-27T03:13:57.405265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:34:31.622281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306372+0000","last_change":"2024-07-27T03:12:32.562300+0000","last_active":"2024-07-27T03:13:57.306372+0000","last_peered":"2024-07-27T03:13:57.306372+0000","last_clean":"2024-07-27T03:13:57.306372+0000","last_became_active":"2024-07-27T03:12:32.561343+0000","last_became_peered":"2024-07-27T03:12:32.561343+0000","last_unstale":"2024-07-27T03:13:57.306372+0000","last_undegraded":"2024-07-27T03:13:57.306372+0000","last_fullsized":"2024-07-27T03:13:57.306372+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:01:28.626943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348835+0000","last_change":"2024-07-27T03:12:29.535177+0000","last_active":"2024-07-27T03:13:57.348835+0000","last_peered":"2024-07-27T03:13:57.348835+0000","last_clean":"2024-07-27T03:13:57.348835+0000","last_became_active":"2024-07-27T03:12:29.534662+0000","last_became_peered":"2024-07-27T03:12:29.534662+0000","last_unstale":"2024-07-27T03:13:57.348835+0000","last_undegraded":"2024-07-27T03:13:57.348835+0000","last_fullsized":"2024-07-27T03:13:57.348835+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:50:49.661992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404084+0000","last_change":"2024-07-27T03:12:42.609925+0000","last_active":"2024-07-27T03:13:57.404084+0000","last_peered":"2024-07-27T03:13:57.404084+0000","last_clean":"2024-07-27T03:13:57.404084+0000","last_became_active":"2024-07-27T03:12:42.609456+0000","last_became_peered":"2024-07-27T03:12:42.609456+0000","last_unstale":"2024-07-27T03:13:57.404084+0000","last_undegraded":"2024-07-27T03:13:57.404084+0000","last_fullsized":"2024-07-27T03:13:57.404084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:07:52.128381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406018+0000","last_change":"2024-07-27T03:12:56.726170+0000","last_active":"2024-07-27T03:13:57.406018+0000","last_peered":"2024-07-27T03:13:57.406018+0000","last_clean":"2024-07-27T03:13:57.406018+0000","last_became_active":"2024-07-27T03:12:56.725685+0000","last_became_peered":"2024-07-27T03:12:56.725685+0000","last_unstale":"2024-07-27T03:13:57.406018+0000","last_undegraded":"2024-07-27T03:13:57.406018+0000","last_fullsized":"2024-07-27T03:13:57.406018+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:53:41.143333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405969+0000","last_change":"2024-07-27T03:12:56.726059+0000","last_active":"2024-07-27T03:13:57.405969+0000","last_peered":"2024-07-27T03:13:57.405969+0000","last_clean":"2024-07-27T03:13:57.405969+0000","last_became_active":"2024-07-27T03:12:56.722294+0000","last_became_peered":"2024-07-27T03:12:56.722294+0000","last_unstale":"2024-07-27T03:13:57.405969+0000","last_undegraded":"2024-07-27T03:13:57.405969+0000","last_fullsized":"2024-07-27T03:13:57.405969+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:18:42.627768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404320+0000","last_change":"2024-07-27T03:12:42.611966+0000","last_active":"2024-07-27T03:13:57.404320+0000","last_peered":"2024-07-27T03:13:57.404320+0000","last_clean":"2024-07-27T03:13:57.404320+0000","last_became_active":"2024-07-27T03:12:42.611522+0000","last_became_peered":"2024-07-27T03:12:42.611522+0000","last_unstale":"2024-07-27T03:13:57.404320+0000","last_undegraded":"2024-07-27T03:13:57.404320+0000","last_fullsized":"2024-07-27T03:13:57.404320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:09:54.034597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.301967+0000","last_change":"2024-07-27T03:12:56.719638+0000","last_active":"2024-07-27T03:13:57.301967+0000","last_peered":"2024-07-27T03:13:57.301967+0000","last_clean":"2024-07-27T03:13:57.301967+0000","last_became_active":"2024-07-27T03:12:56.719227+0000","last_became_peered":"2024-07-27T03:12:56.719227+0000","last_unstale":"2024-07-27T03:13:57.301967+0000","last_undegraded":"2024-07-27T03:13:57.301967+0000","last_fullsized":"2024-07-27T03:13:57.301967+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:54:16.891518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304967+0000","last_change":"2024-07-27T03:12:29.614346+0000","last_active":"2024-07-27T03:13:57.304967+0000","last_peered":"2024-07-27T03:13:57.304967+0000","last_clean":"2024-07-27T03:13:57.304967+0000","last_became_active":"2024-07-27T03:12:29.613672+0000","last_became_peered":"2024-07-27T03:12:29.613672+0000","last_unstale":"2024-07-27T03:13:57.304967+0000","last_undegraded":"2024-07-27T03:13:57.304967+0000","last_fullsized":"2024-07-27T03:13:57.304967+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:46:35.505735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348712+0000","last_change":"2024-07-27T03:12:42.618364+0000","last_active":"2024-07-27T03:13:57.348712+0000","last_peered":"2024-07-27T03:13:57.348712+0000","last_clean":"2024-07-27T03:13:57.348712+0000","last_became_active":"2024-07-27T03:12:42.617988+0000","last_became_peered":"2024-07-27T03:12:42.617988+0000","last_unstale":"2024-07-27T03:13:57.348712+0000","last_undegraded":"2024-07-27T03:13:57.348712+0000","last_fullsized":"2024-07-27T03:13:57.348712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:40:21.725877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305356+0000","last_change":"2024-07-27T03:12:32.563809+0000","last_active":"2024-07-27T03:13:57.305356+0000","last_peered":"2024-07-27T03:13:57.305356+0000","last_clean":"2024-07-27T03:13:57.305356+0000","last_became_active":"2024-07-27T03:12:32.563479+0000","last_became_peered":"2024-07-27T03:12:32.563479+0000","last_unstale":"2024-07-27T03:13:57.305356+0000","last_undegraded":"2024-07-27T03:13:57.305356+0000","last_fullsized":"2024-07-27T03:13:57.305356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:45:47.719241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":156,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:19.405773+0000","last_change":"2024-07-27T03:12:29.543786+0000","last_active":"2024-07-27T03:15:19.405773+0000","last_peered":"2024-07-27T03:15:19.405773+0000","last_clean":"2024-07-27T03:15:19.405773+0000","last_became_active":"2024-07-27T03:12:29.543203+0000","last_became_peered":"2024-07-27T03:12:29.543203+0000","last_unstale":"2024-07-27T03:15:19.405773+0000","last_undegraded":"2024-07-27T03:15:19.405773+0000","last_fullsized":"2024-07-27T03:15:19.405773+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:47:06.742171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302718+0000","last_change":"2024-07-27T03:12:42.622142+0000","last_active":"2024-07-27T03:13:57.302718+0000","last_peered":"2024-07-27T03:13:57.302718+0000","last_clean":"2024-07-27T03:13:57.302718+0000","last_became_active":"2024-07-27T03:12:42.621681+0000","last_became_peered":"2024-07-27T03:12:42.621681+0000","last_unstale":"2024-07-27T03:13:57.302718+0000","last_undegraded":"2024-07-27T03:13:57.302718+0000","last_fullsized":"2024-07-27T03:13:57.302718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:05:08.168424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306186+0000","last_change":"2024-07-27T03:12:32.567119+0000","last_active":"2024-07-27T03:13:57.306186+0000","last_peered":"2024-07-27T03:13:57.306186+0000","last_clean":"2024-07-27T03:13:57.306186+0000","last_became_active":"2024-07-27T03:12:32.566757+0000","last_became_peered":"2024-07-27T03:12:32.566757+0000","last_unstale":"2024-07-27T03:13:57.306186+0000","last_undegraded":"2024-07-27T03:13:57.306186+0000","last_fullsized":"2024-07-27T03:13:57.306186+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:31:12.972804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404725+0000","last_change":"2024-07-27T03:12:29.540453+0000","last_active":"2024-07-27T03:13:57.404725+0000","last_peered":"2024-07-27T03:13:57.404725+0000","last_clean":"2024-07-27T03:13:57.404725+0000","last_became_active":"2024-07-27T03:12:29.539195+0000","last_became_peered":"2024-07-27T03:12:29.539195+0000","last_unstale":"2024-07-27T03:13:57.404725+0000","last_undegraded":"2024-07-27T03:13:57.404725+0000","last_fullsized":"2024-07-27T03:13:57.404725+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:44:31.144845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405363+0000","last_change":"2024-07-27T03:12:42.616671+0000","last_active":"2024-07-27T03:13:57.405363+0000","last_peered":"2024-07-27T03:13:57.405363+0000","last_clean":"2024-07-27T03:13:57.405363+0000","last_became_active":"2024-07-27T03:12:42.616198+0000","last_became_peered":"2024-07-27T03:12:42.616198+0000","last_unstale":"2024-07-27T03:13:57.405363+0000","last_undegraded":"2024-07-27T03:13:57.405363+0000","last_fullsized":"2024-07-27T03:13:57.405363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:01:31.584686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348516+0000","last_change":"2024-07-27T03:12:32.562350+0000","last_active":"2024-07-27T03:13:57.348516+0000","last_peered":"2024-07-27T03:13:57.348516+0000","last_clean":"2024-07-27T03:13:57.348516+0000","last_became_active":"2024-07-27T03:12:32.562016+0000","last_became_peered":"2024-07-27T03:12:32.562016+0000","last_unstale":"2024-07-27T03:13:57.348516+0000","last_undegraded":"2024-07-27T03:13:57.348516+0000","last_fullsized":"2024-07-27T03:13:57.348516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:44:35.027329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303241+0000","last_change":"2024-07-27T03:12:29.541387+0000","last_active":"2024-07-27T03:13:57.303241+0000","last_peered":"2024-07-27T03:13:57.303241+0000","last_clean":"2024-07-27T03:13:57.303241+0000","last_became_active":"2024-07-27T03:12:29.541005+0000","last_became_peered":"2024-07-27T03:12:29.541005+0000","last_unstale":"2024-07-27T03:13:57.303241+0000","last_undegraded":"2024-07-27T03:13:57.303241+0000","last_fullsized":"2024-07-27T03:13:57.303241+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:22:16.268348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404849+0000","last_change":"2024-07-27T03:12:42.610029+0000","last_active":"2024-07-27T03:13:57.404849+0000","last_peered":"2024-07-27T03:13:57.404849+0000","last_clean":"2024-07-27T03:13:57.404849+0000","last_became_active":"2024-07-27T03:12:42.609592+0000","last_became_peered":"2024-07-27T03:12:42.609592+0000","last_unstale":"2024-07-27T03:13:57.404849+0000","last_undegraded":"2024-07-27T03:13:57.404849+0000","last_fullsized":"2024-07-27T03:13:57.404849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:47:15.254770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305612+0000","last_change":"2024-07-27T03:12:32.568941+0000","last_active":"2024-07-27T03:13:57.305612+0000","last_peered":"2024-07-27T03:13:57.305612+0000","last_clean":"2024-07-27T03:13:57.305612+0000","last_became_active":"2024-07-27T03:12:32.568531+0000","last_became_peered":"2024-07-27T03:12:32.568531+0000","last_unstale":"2024-07-27T03:13:57.305612+0000","last_undegraded":"2024-07-27T03:13:57.305612+0000","last_fullsized":"2024-07-27T03:13:57.305612+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:06:22.020505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'35","reported_seq":289,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:19.405732+0000","last_change":"2024-07-27T03:12:29.545131+0000","last_active":"2024-07-27T03:15:19.405732+0000","last_peered":"2024-07-27T03:15:19.405732+0000","last_clean":"2024-07-27T03:15:19.405732+0000","last_became_active":"2024-07-27T03:12:29.544768+0000","last_became_peered":"2024-07-27T03:12:29.544768+0000","last_unstale":"2024-07-27T03:15:19.405732+0000","last_undegraded":"2024-07-27T03:15:19.405732+0000","last_fullsized":"2024-07-27T03:15:19.405732+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:20:57.556128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62072,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":209,"num_read_kb":165,"num_write":37,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302087+0000","last_change":"2024-07-27T03:12:42.617483+0000","last_active":"2024-07-27T03:13:57.302087+0000","last_peered":"2024-07-27T03:13:57.302087+0000","last_clean":"2024-07-27T03:13:57.302087+0000","last_became_active":"2024-07-27T03:12:42.617065+0000","last_became_peered":"2024-07-27T03:12:42.617065+0000","last_unstale":"2024-07-27T03:13:57.302087+0000","last_undegraded":"2024-07-27T03:13:57.302087+0000","last_fullsized":"2024-07-27T03:13:57.302087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:52:17.448919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405888+0000","last_change":"2024-07-27T03:12:56.722257+0000","last_active":"2024-07-27T03:13:57.405888+0000","last_peered":"2024-07-27T03:13:57.405888+0000","last_clean":"2024-07-27T03:13:57.405888+0000","last_became_active":"2024-07-27T03:12:56.721773+0000","last_became_peered":"2024-07-27T03:12:56.721773+0000","last_unstale":"2024-07-27T03:13:57.405888+0000","last_undegraded":"2024-07-27T03:13:57.405888+0000","last_fullsized":"2024-07-27T03:13:57.405888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:30:44.310705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405205+0000","last_change":"2024-07-27T03:12:29.614171+0000","last_active":"2024-07-27T03:13:57.405205+0000","last_peered":"2024-07-27T03:13:57.405205+0000","last_clean":"2024-07-27T03:13:57.405205+0000","last_became_active":"2024-07-27T03:12:29.613270+0000","last_became_peered":"2024-07-27T03:12:29.613270+0000","last_unstale":"2024-07-27T03:13:57.405205+0000","last_undegraded":"2024-07-27T03:13:57.405205+0000","last_fullsized":"2024-07-27T03:13:57.405205+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:39:32.283778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303808+0000","last_change":"2024-07-27T03:12:42.610375+0000","last_active":"2024-07-27T03:13:57.303808+0000","last_peered":"2024-07-27T03:13:57.303808+0000","last_clean":"2024-07-27T03:13:57.303808+0000","last_became_active":"2024-07-27T03:12:42.609957+0000","last_became_peered":"2024-07-27T03:12:42.609957+0000","last_unstale":"2024-07-27T03:13:57.303808+0000","last_undegraded":"2024-07-27T03:13:57.303808+0000","last_fullsized":"2024-07-27T03:13:57.303808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:11:02.707179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405928+0000","last_change":"2024-07-27T03:12:56.722332+0000","last_active":"2024-07-27T03:13:57.405928+0000","last_peered":"2024-07-27T03:13:57.405928+0000","last_clean":"2024-07-27T03:13:57.405928+0000","last_became_active":"2024-07-27T03:12:56.721972+0000","last_became_peered":"2024-07-27T03:12:56.721972+0000","last_unstale":"2024-07-27T03:13:57.405928+0000","last_undegraded":"2024-07-27T03:13:57.405928+0000","last_fullsized":"2024-07-27T03:13:57.405928+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:54:31.880137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306094+0000","last_change":"2024-07-27T03:12:29.541051+0000","last_active":"2024-07-27T03:13:57.306094+0000","last_peered":"2024-07-27T03:13:57.306094+0000","last_clean":"2024-07-27T03:13:57.306094+0000","last_became_active":"2024-07-27T03:12:29.540619+0000","last_became_peered":"2024-07-27T03:12:29.540619+0000","last_unstale":"2024-07-27T03:13:57.306094+0000","last_undegraded":"2024-07-27T03:13:57.306094+0000","last_fullsized":"2024-07-27T03:13:57.306094+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:23:27.587419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304810+0000","last_change":"2024-07-27T03:12:56.720496+0000","last_active":"2024-07-27T03:13:57.304810+0000","last_peered":"2024-07-27T03:13:57.304810+0000","last_clean":"2024-07-27T03:13:57.304810+0000","last_became_active":"2024-07-27T03:12:56.719784+0000","last_became_peered":"2024-07-27T03:12:56.719784+0000","last_unstale":"2024-07-27T03:13:57.304810+0000","last_undegraded":"2024-07-27T03:13:57.304810+0000","last_fullsized":"2024-07-27T03:13:57.304810+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:07:20.518139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406680+0000","last_change":"2024-07-27T03:12:32.568409+0000","last_active":"2024-07-27T03:13:57.406680+0000","last_peered":"2024-07-27T03:13:57.406680+0000","last_clean":"2024-07-27T03:13:57.406680+0000","last_became_active":"2024-07-27T03:12:32.568029+0000","last_became_peered":"2024-07-27T03:12:32.568029+0000","last_unstale":"2024-07-27T03:13:57.406680+0000","last_undegraded":"2024-07-27T03:13:57.406680+0000","last_fullsized":"2024-07-27T03:13:57.406680+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:44:53.857986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":249,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:19.405743+0000","last_change":"2024-07-27T03:12:29.544286+0000","last_active":"2024-07-27T03:15:19.405743+0000","last_peered":"2024-07-27T03:15:19.405743+0000","last_clean":"2024-07-27T03:15:19.405743+0000","last_became_active":"2024-07-27T03:12:29.543879+0000","last_became_peered":"2024-07-27T03:12:29.543879+0000","last_unstale":"2024-07-27T03:15:19.405743+0000","last_undegraded":"2024-07-27T03:15:19.405743+0000","last_fullsized":"2024-07-27T03:15:19.405743+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:20:36.997298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":200,"num_read_kb":200,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404174+0000","last_change":"2024-07-27T03:12:42.609824+0000","last_active":"2024-07-27T03:13:57.404174+0000","last_peered":"2024-07-27T03:13:57.404174+0000","last_clean":"2024-07-27T03:13:57.404174+0000","last_became_active":"2024-07-27T03:12:42.608502+0000","last_became_peered":"2024-07-27T03:12:42.608502+0000","last_unstale":"2024-07-27T03:13:57.404174+0000","last_undegraded":"2024-07-27T03:13:57.404174+0000","last_fullsized":"2024-07-27T03:13:57.404174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:10:23.253921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405172+0000","last_change":"2024-07-27T03:12:42.610744+0000","last_active":"2024-07-27T03:13:57.405172+0000","last_peered":"2024-07-27T03:13:57.405172+0000","last_clean":"2024-07-27T03:13:57.405172+0000","last_became_active":"2024-07-27T03:12:42.610322+0000","last_became_peered":"2024-07-27T03:12:42.610322+0000","last_unstale":"2024-07-27T03:13:57.405172+0000","last_undegraded":"2024-07-27T03:13:57.405172+0000","last_fullsized":"2024-07-27T03:13:57.405172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:01:47.618013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302086+0000","last_change":"2024-07-27T03:12:32.558372+0000","last_active":"2024-07-27T03:13:57.302086+0000","last_peered":"2024-07-27T03:13:57.302086+0000","last_clean":"2024-07-27T03:13:57.302086+0000","last_became_active":"2024-07-27T03:12:32.558003+0000","last_became_peered":"2024-07-27T03:12:32.558003+0000","last_unstale":"2024-07-27T03:13:57.302086+0000","last_undegraded":"2024-07-27T03:13:57.302086+0000","last_fullsized":"2024-07-27T03:13:57.302086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:41:43.673807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404495+0000","last_change":"2024-07-27T03:12:29.543182+0000","last_active":"2024-07-27T03:13:57.404495+0000","last_peered":"2024-07-27T03:13:57.404495+0000","last_clean":"2024-07-27T03:13:57.404495+0000","last_became_active":"2024-07-27T03:12:29.542744+0000","last_became_peered":"2024-07-27T03:12:29.542744+0000","last_unstale":"2024-07-27T03:13:57.404495+0000","last_undegraded":"2024-07-27T03:13:57.404495+0000","last_fullsized":"2024-07-27T03:13:57.404495+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:01:17.276549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405941+0000","last_change":"2024-07-27T03:12:42.622297+0000","last_active":"2024-07-27T03:13:57.405941+0000","last_peered":"2024-07-27T03:13:57.405941+0000","last_clean":"2024-07-27T03:13:57.405941+0000","last_became_active":"2024-07-27T03:12:42.621947+0000","last_became_peered":"2024-07-27T03:12:42.621947+0000","last_unstale":"2024-07-27T03:13:57.405941+0000","last_undegraded":"2024-07-27T03:13:57.405941+0000","last_fullsized":"2024-07-27T03:13:57.405941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:15:59.480216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404239+0000","last_change":"2024-07-27T03:12:32.565987+0000","last_active":"2024-07-27T03:13:57.404239+0000","last_peered":"2024-07-27T03:13:57.404239+0000","last_clean":"2024-07-27T03:13:57.404239+0000","last_became_active":"2024-07-27T03:12:32.565603+0000","last_became_peered":"2024-07-27T03:12:32.565603+0000","last_unstale":"2024-07-27T03:13:57.404239+0000","last_undegraded":"2024-07-27T03:13:57.404239+0000","last_fullsized":"2024-07-27T03:13:57.404239+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:30:48.954683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348273+0000","last_change":"2024-07-27T03:12:29.532149+0000","last_active":"2024-07-27T03:13:57.348273+0000","last_peered":"2024-07-27T03:13:57.348273+0000","last_clean":"2024-07-27T03:13:57.348273+0000","last_became_active":"2024-07-27T03:12:29.531761+0000","last_became_peered":"2024-07-27T03:12:29.531761+0000","last_unstale":"2024-07-27T03:13:57.348273+0000","last_undegraded":"2024-07-27T03:13:57.348273+0000","last_fullsized":"2024-07-27T03:13:57.348273+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:44:30.902538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406091+0000","last_change":"2024-07-27T03:12:29.543148+0000","last_active":"2024-07-27T03:13:57.406091+0000","last_peered":"2024-07-27T03:13:57.406091+0000","last_clean":"2024-07-27T03:13:57.406091+0000","last_became_active":"2024-07-27T03:12:29.542774+0000","last_became_peered":"2024-07-27T03:12:29.542774+0000","last_unstale":"2024-07-27T03:13:57.406091+0000","last_undegraded":"2024-07-27T03:13:57.406091+0000","last_fullsized":"2024-07-27T03:13:57.406091+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:20:06.786121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404598+0000","last_change":"2024-07-27T03:12:32.561516+0000","last_active":"2024-07-27T03:13:57.404598+0000","last_peered":"2024-07-27T03:13:57.404598+0000","last_clean":"2024-07-27T03:13:57.404598+0000","last_became_active":"2024-07-27T03:12:32.561145+0000","last_became_peered":"2024-07-27T03:12:32.561145+0000","last_unstale":"2024-07-27T03:13:57.404598+0000","last_undegraded":"2024-07-27T03:13:57.404598+0000","last_fullsized":"2024-07-27T03:13:57.404598+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:49:26.317206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302982+0000","last_change":"2024-07-27T03:12:42.616735+0000","last_active":"2024-07-27T03:13:57.302982+0000","last_peered":"2024-07-27T03:13:57.302982+0000","last_clean":"2024-07-27T03:13:57.302982+0000","last_became_active":"2024-07-27T03:12:42.616171+0000","last_became_peered":"2024-07-27T03:12:42.616171+0000","last_unstale":"2024-07-27T03:13:57.302982+0000","last_undegraded":"2024-07-27T03:13:57.302982+0000","last_fullsized":"2024-07-27T03:13:57.302982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:45:46.236613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":152,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:17.403421+0000","last_change":"2024-07-27T03:12:29.539775+0000","last_active":"2024-07-27T03:15:17.403421+0000","last_peered":"2024-07-27T03:15:17.403421+0000","last_clean":"2024-07-27T03:15:17.403421+0000","last_became_active":"2024-07-27T03:12:29.539276+0000","last_became_peered":"2024-07-27T03:12:29.539276+0000","last_unstale":"2024-07-27T03:15:17.403421+0000","last_undegraded":"2024-07-27T03:15:17.403421+0000","last_fullsized":"2024-07-27T03:15:17.403421+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:09:19.364679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305126+0000","last_change":"2024-07-27T03:12:32.562673+0000","last_active":"2024-07-27T03:13:57.305126+0000","last_peered":"2024-07-27T03:13:57.305126+0000","last_clean":"2024-07-27T03:13:57.305126+0000","last_became_active":"2024-07-27T03:12:32.561591+0000","last_became_peered":"2024-07-27T03:12:32.561591+0000","last_unstale":"2024-07-27T03:13:57.305126+0000","last_undegraded":"2024-07-27T03:13:57.305126+0000","last_fullsized":"2024-07-27T03:13:57.305126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:49:38.095004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303410+0000","last_change":"2024-07-27T03:12:42.611756+0000","last_active":"2024-07-27T03:13:57.303410+0000","last_peered":"2024-07-27T03:13:57.303410+0000","last_clean":"2024-07-27T03:13:57.303410+0000","last_became_active":"2024-07-27T03:12:42.611246+0000","last_became_peered":"2024-07-27T03:12:42.611246+0000","last_unstale":"2024-07-27T03:13:57.303410+0000","last_undegraded":"2024-07-27T03:13:57.303410+0000","last_fullsized":"2024-07-27T03:13:57.303410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:13:06.027887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405688+0000","last_change":"2024-07-27T03:12:32.571339+0000","last_active":"2024-07-27T03:13:57.405688+0000","last_peered":"2024-07-27T03:13:57.405688+0000","last_clean":"2024-07-27T03:13:57.405688+0000","last_became_active":"2024-07-27T03:12:32.570973+0000","last_became_peered":"2024-07-27T03:12:32.570973+0000","last_unstale":"2024-07-27T03:13:57.405688+0000","last_undegraded":"2024-07-27T03:13:57.405688+0000","last_fullsized":"2024-07-27T03:13:57.405688+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:24:25.357833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303364+0000","last_change":"2024-07-27T03:12:32.561243+0000","last_active":"2024-07-27T03:13:57.303364+0000","last_peered":"2024-07-27T03:13:57.303364+0000","last_clean":"2024-07-27T03:13:57.303364+0000","last_became_active":"2024-07-27T03:12:32.560748+0000","last_became_peered":"2024-07-27T03:12:32.560748+0000","last_unstale":"2024-07-27T03:13:57.303364+0000","last_undegraded":"2024-07-27T03:13:57.303364+0000","last_fullsized":"2024-07-27T03:13:57.303364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:47:40.430283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302948+0000","last_change":"2024-07-27T03:12:32.562545+0000","last_active":"2024-07-27T03:13:57.302948+0000","last_peered":"2024-07-27T03:13:57.302948+0000","last_clean":"2024-07-27T03:13:57.302948+0000","last_became_active":"2024-07-27T03:12:32.562202+0000","last_became_peered":"2024-07-27T03:12:32.562202+0000","last_unstale":"2024-07-27T03:13:57.302948+0000","last_undegraded":"2024-07-27T03:13:57.302948+0000","last_fullsized":"2024-07-27T03:13:57.302948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:05:20.229232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405456+0000","last_change":"2024-07-27T03:12:32.565162+0000","last_active":"2024-07-27T03:13:57.405456+0000","last_peered":"2024-07-27T03:13:57.405456+0000","last_clean":"2024-07-27T03:13:57.405456+0000","last_became_active":"2024-07-27T03:12:32.564798+0000","last_became_peered":"2024-07-27T03:12:32.564798+0000","last_unstale":"2024-07-27T03:13:57.405456+0000","last_undegraded":"2024-07-27T03:13:57.405456+0000","last_fullsized":"2024-07-27T03:13:57.405456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:54:29.742624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348739+0000","last_change":"2024-07-27T03:12:32.556874+0000","last_active":"2024-07-27T03:13:57.348739+0000","last_peered":"2024-07-27T03:13:57.348739+0000","last_clean":"2024-07-27T03:13:57.348739+0000","last_became_active":"2024-07-27T03:12:32.556467+0000","last_became_peered":"2024-07-27T03:12:32.556467+0000","last_unstale":"2024-07-27T03:13:57.348739+0000","last_undegraded":"2024-07-27T03:13:57.348739+0000","last_fullsized":"2024-07-27T03:13:57.348739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:51:20.806785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406013+0000","last_change":"2024-07-27T03:12:32.569136+0000","last_active":"2024-07-27T03:13:57.406013+0000","last_peered":"2024-07-27T03:13:57.406013+0000","last_clean":"2024-07-27T03:13:57.406013+0000","last_became_active":"2024-07-27T03:12:32.568778+0000","last_became_peered":"2024-07-27T03:12:32.568778+0000","last_unstale":"2024-07-27T03:13:57.406013+0000","last_undegraded":"2024-07-27T03:13:57.406013+0000","last_fullsized":"2024-07-27T03:13:57.406013+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:26:22.654974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303529+0000","last_change":"2024-07-27T03:12:32.569507+0000","last_active":"2024-07-27T03:13:57.303529+0000","last_peered":"2024-07-27T03:13:57.303529+0000","last_clean":"2024-07-27T03:13:57.303529+0000","last_became_active":"2024-07-27T03:12:32.569109+0000","last_became_peered":"2024-07-27T03:12:32.569109+0000","last_unstale":"2024-07-27T03:13:57.303529+0000","last_undegraded":"2024-07-27T03:13:57.303529+0000","last_fullsized":"2024-07-27T03:13:57.303529+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:35:12.944933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405595+0000","last_change":"2024-07-27T03:12:32.566479+0000","last_active":"2024-07-27T03:13:57.405595+0000","last_peered":"2024-07-27T03:13:57.405595+0000","last_clean":"2024-07-27T03:13:57.405595+0000","last_became_active":"2024-07-27T03:12:32.566114+0000","last_became_peered":"2024-07-27T03:12:32.566114+0000","last_unstale":"2024-07-27T03:13:57.405595+0000","last_undegraded":"2024-07-27T03:13:57.405595+0000","last_fullsized":"2024-07-27T03:13:57.405595+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:24:19.944031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404254+0000","last_change":"2024-07-27T03:12:56.726118+0000","last_active":"2024-07-27T03:13:57.404254+0000","last_peered":"2024-07-27T03:13:57.404254+0000","last_clean":"2024-07-27T03:13:57.404254+0000","last_became_active":"2024-07-27T03:12:56.725637+0000","last_became_peered":"2024-07-27T03:12:56.725637+0000","last_unstale":"2024-07-27T03:13:57.404254+0000","last_undegraded":"2024-07-27T03:13:57.404254+0000","last_fullsized":"2024-07-27T03:13:57.404254+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:36:41.725150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404405+0000","last_change":"2024-07-27T03:12:32.568232+0000","last_active":"2024-07-27T03:13:57.404405+0000","last_peered":"2024-07-27T03:13:57.404405+0000","last_clean":"2024-07-27T03:13:57.404405+0000","last_became_active":"2024-07-27T03:12:32.567834+0000","last_became_peered":"2024-07-27T03:12:32.567834+0000","last_unstale":"2024-07-27T03:13:57.404405+0000","last_undegraded":"2024-07-27T03:13:57.404405+0000","last_fullsized":"2024-07-27T03:13:57.404405+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:40:28.234539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405616+0000","last_change":"2024-07-27T03:12:32.573451+0000","last_active":"2024-07-27T03:13:57.405616+0000","last_peered":"2024-07-27T03:13:57.405616+0000","last_clean":"2024-07-27T03:13:57.405616+0000","last_became_active":"2024-07-27T03:12:32.573066+0000","last_became_peered":"2024-07-27T03:12:32.573066+0000","last_unstale":"2024-07-27T03:13:57.405616+0000","last_undegraded":"2024-07-27T03:13:57.405616+0000","last_fullsized":"2024-07-27T03:13:57.405616+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:13:29.104055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348685+0000","last_change":"2024-07-27T03:12:32.561067+0000","last_active":"2024-07-27T03:13:57.348685+0000","last_peered":"2024-07-27T03:13:57.348685+0000","last_clean":"2024-07-27T03:13:57.348685+0000","last_became_active":"2024-07-27T03:12:32.560358+0000","last_became_peered":"2024-07-27T03:12:32.560358+0000","last_unstale":"2024-07-27T03:13:57.348685+0000","last_undegraded":"2024-07-27T03:13:57.348685+0000","last_fullsized":"2024-07-27T03:13:57.348685+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:41:41.026804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405528+0000","last_change":"2024-07-27T03:12:32.572290+0000","last_active":"2024-07-27T03:13:57.405528+0000","last_peered":"2024-07-27T03:13:57.405528+0000","last_clean":"2024-07-27T03:13:57.405528+0000","last_became_active":"2024-07-27T03:12:32.571877+0000","last_became_peered":"2024-07-27T03:12:32.571877+0000","last_unstale":"2024-07-27T03:13:57.405528+0000","last_undegraded":"2024-07-27T03:13:57.405528+0000","last_fullsized":"2024-07-27T03:13:57.405528+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:34:40.559787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303465+0000","last_change":"2024-07-27T03:12:32.565495+0000","last_active":"2024-07-27T03:13:57.303465+0000","last_peered":"2024-07-27T03:13:57.303465+0000","last_clean":"2024-07-27T03:13:57.303465+0000","last_became_active":"2024-07-27T03:12:32.563136+0000","last_became_peered":"2024-07-27T03:12:32.563136+0000","last_unstale":"2024-07-27T03:13:57.303465+0000","last_undegraded":"2024-07-27T03:13:57.303465+0000","last_fullsized":"2024-07-27T03:13:57.303465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:31:02.032776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302646+0000","last_change":"2024-07-27T03:12:32.569477+0000","last_active":"2024-07-27T03:13:57.302646+0000","last_peered":"2024-07-27T03:13:57.302646+0000","last_clean":"2024-07-27T03:13:57.302646+0000","last_became_active":"2024-07-27T03:12:32.569098+0000","last_became_peered":"2024-07-27T03:12:32.569098+0000","last_unstale":"2024-07-27T03:13:57.302646+0000","last_undegraded":"2024-07-27T03:13:57.302646+0000","last_fullsized":"2024-07-27T03:13:57.302646+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:44:26.082539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406172+0000","last_change":"2024-07-27T03:12:32.574704+0000","last_active":"2024-07-27T03:13:57.406172+0000","last_peered":"2024-07-27T03:13:57.406172+0000","last_clean":"2024-07-27T03:13:57.406172+0000","last_became_active":"2024-07-27T03:12:32.574261+0000","last_became_peered":"2024-07-27T03:12:32.574261+0000","last_unstale":"2024-07-27T03:13:57.406172+0000","last_undegraded":"2024-07-27T03:13:57.406172+0000","last_fullsized":"2024-07-27T03:13:57.406172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:53:06.445267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405155+0000","last_change":"2024-07-27T03:12:32.573835+0000","last_active":"2024-07-27T03:13:57.405155+0000","last_peered":"2024-07-27T03:13:57.405155+0000","last_clean":"2024-07-27T03:13:57.405155+0000","last_became_active":"2024-07-27T03:12:32.573450+0000","last_became_peered":"2024-07-27T03:12:32.573450+0000","last_unstale":"2024-07-27T03:13:57.405155+0000","last_undegraded":"2024-07-27T03:13:57.405155+0000","last_fullsized":"2024-07-27T03:13:57.405155+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:40:45.264000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404972+0000","last_change":"2024-07-27T03:12:32.568542+0000","last_active":"2024-07-27T03:13:57.404972+0000","last_peered":"2024-07-27T03:13:57.404972+0000","last_clean":"2024-07-27T03:13:57.404972+0000","last_became_active":"2024-07-27T03:12:32.568179+0000","last_became_peered":"2024-07-27T03:12:32.568179+0000","last_unstale":"2024-07-27T03:13:57.404972+0000","last_undegraded":"2024-07-27T03:13:57.404972+0000","last_fullsized":"2024-07-27T03:13:57.404972+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:32:10.005601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404937+0000","last_change":"2024-07-27T03:12:32.564063+0000","last_active":"2024-07-27T03:13:57.404937+0000","last_peered":"2024-07-27T03:13:57.404937+0000","last_clean":"2024-07-27T03:13:57.404937+0000","last_became_active":"2024-07-27T03:12:32.563705+0000","last_became_peered":"2024-07-27T03:12:32.563705+0000","last_unstale":"2024-07-27T03:13:57.404937+0000","last_undegraded":"2024-07-27T03:13:57.404937+0000","last_fullsized":"2024-07-27T03:13:57.404937+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:40:19.234194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348625+0000","last_change":"2024-07-27T03:12:57.178963+0000","last_active":"2024-07-27T03:13:57.348625+0000","last_peered":"2024-07-27T03:13:57.348625+0000","last_clean":"2024-07-27T03:13:57.348625+0000","last_became_active":"2024-07-27T03:12:57.178550+0000","last_became_peered":"2024-07-27T03:12:57.178550+0000","last_unstale":"2024-07-27T03:13:57.348625+0000","last_undegraded":"2024-07-27T03:13:57.348625+0000","last_fullsized":"2024-07-27T03:13:57.348625+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:20:06.586957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304244+0000","last_change":"2024-07-27T03:12:32.569863+0000","last_active":"2024-07-27T03:13:57.304244+0000","last_peered":"2024-07-27T03:13:57.304244+0000","last_clean":"2024-07-27T03:13:57.304244+0000","last_became_active":"2024-07-27T03:12:32.566457+0000","last_became_peered":"2024-07-27T03:12:32.566457+0000","last_unstale":"2024-07-27T03:13:57.304244+0000","last_undegraded":"2024-07-27T03:13:57.304244+0000","last_fullsized":"2024-07-27T03:13:57.304244+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:28:13.141002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306014+0000","last_change":"2024-07-27T03:12:32.574490+0000","last_active":"2024-07-27T03:13:57.306014+0000","last_peered":"2024-07-27T03:13:57.306014+0000","last_clean":"2024-07-27T03:13:57.306014+0000","last_became_active":"2024-07-27T03:12:32.574130+0000","last_became_peered":"2024-07-27T03:12:32.574130+0000","last_unstale":"2024-07-27T03:13:57.306014+0000","last_undegraded":"2024-07-27T03:13:57.306014+0000","last_fullsized":"2024-07-27T03:13:57.306014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:44:22.696669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406538+0000","last_change":"2024-07-27T03:12:32.560106+0000","last_active":"2024-07-27T03:13:57.406538+0000","last_peered":"2024-07-27T03:13:57.406538+0000","last_clean":"2024-07-27T03:13:57.406538+0000","last_became_active":"2024-07-27T03:12:32.559751+0000","last_became_peered":"2024-07-27T03:12:32.559751+0000","last_unstale":"2024-07-27T03:13:57.406538+0000","last_undegraded":"2024-07-27T03:13:57.406538+0000","last_fullsized":"2024-07-27T03:13:57.406538+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:16:37.647462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406784+0000","last_change":"2024-07-27T03:12:32.569352+0000","last_active":"2024-07-27T03:13:57.406784+0000","last_peered":"2024-07-27T03:13:57.406784+0000","last_clean":"2024-07-27T03:13:57.406784+0000","last_became_active":"2024-07-27T03:12:32.569004+0000","last_became_peered":"2024-07-27T03:12:32.569004+0000","last_unstale":"2024-07-27T03:13:57.406784+0000","last_undegraded":"2024-07-27T03:13:57.406784+0000","last_fullsized":"2024-07-27T03:13:57.406784+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:21:25.999214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404116+0000","last_change":"2024-07-27T03:12:32.562775+0000","last_active":"2024-07-27T03:13:57.404116+0000","last_peered":"2024-07-27T03:13:57.404116+0000","last_clean":"2024-07-27T03:13:57.404116+0000","last_became_active":"2024-07-27T03:12:32.562403+0000","last_became_peered":"2024-07-27T03:12:32.562403+0000","last_unstale":"2024-07-27T03:13:57.404116+0000","last_undegraded":"2024-07-27T03:13:57.404116+0000","last_fullsized":"2024-07-27T03:13:57.404116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:16:09.474505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305583+0000","last_change":"2024-07-27T03:12:32.563271+0000","last_active":"2024-07-27T03:13:57.305583+0000","last_peered":"2024-07-27T03:13:57.305583+0000","last_clean":"2024-07-27T03:13:57.305583+0000","last_became_active":"2024-07-27T03:12:32.562850+0000","last_became_peered":"2024-07-27T03:12:32.562850+0000","last_unstale":"2024-07-27T03:13:57.305583+0000","last_undegraded":"2024-07-27T03:13:57.305583+0000","last_fullsized":"2024-07-27T03:13:57.305583+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:40:27.688205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405997+0000","last_change":"2024-07-27T03:12:32.557580+0000","last_active":"2024-07-27T03:13:57.405997+0000","last_peered":"2024-07-27T03:13:57.405997+0000","last_clean":"2024-07-27T03:13:57.405997+0000","last_became_active":"2024-07-27T03:12:32.557190+0000","last_became_peered":"2024-07-27T03:12:32.557190+0000","last_unstale":"2024-07-27T03:13:57.405997+0000","last_undegraded":"2024-07-27T03:13:57.405997+0000","last_fullsized":"2024-07-27T03:13:57.405997+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:58:17.609035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305207+0000","last_change":"2024-07-27T03:12:32.563669+0000","last_active":"2024-07-27T03:13:57.305207+0000","last_peered":"2024-07-27T03:13:57.305207+0000","last_clean":"2024-07-27T03:13:57.305207+0000","last_became_active":"2024-07-27T03:12:32.563200+0000","last_became_peered":"2024-07-27T03:12:32.563200+0000","last_unstale":"2024-07-27T03:13:57.305207+0000","last_undegraded":"2024-07-27T03:13:57.305207+0000","last_fullsized":"2024-07-27T03:13:57.305207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:14:05.624118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304165+0000","last_change":"2024-07-27T03:12:32.561793+0000","last_active":"2024-07-27T03:13:57.304165+0000","last_peered":"2024-07-27T03:13:57.304165+0000","last_clean":"2024-07-27T03:13:57.304165+0000","last_became_active":"2024-07-27T03:12:32.561452+0000","last_became_peered":"2024-07-27T03:12:32.561452+0000","last_unstale":"2024-07-27T03:13:57.304165+0000","last_undegraded":"2024-07-27T03:13:57.304165+0000","last_fullsized":"2024-07-27T03:13:57.304165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:22:07.204824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405301+0000","last_change":"2024-07-27T03:12:32.569811+0000","last_active":"2024-07-27T03:13:57.405301+0000","last_peered":"2024-07-27T03:13:57.405301+0000","last_clean":"2024-07-27T03:13:57.405301+0000","last_became_active":"2024-07-27T03:12:32.569474+0000","last_became_peered":"2024-07-27T03:12:32.569474+0000","last_unstale":"2024-07-27T03:13:57.405301+0000","last_undegraded":"2024-07-27T03:13:57.405301+0000","last_fullsized":"2024-07-27T03:13:57.405301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:38:39.317362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405344+0000","last_change":"2024-07-27T03:12:32.565159+0000","last_active":"2024-07-27T03:13:57.405344+0000","last_peered":"2024-07-27T03:13:57.405344+0000","last_clean":"2024-07-27T03:13:57.405344+0000","last_became_active":"2024-07-27T03:12:32.564750+0000","last_became_peered":"2024-07-27T03:12:32.564750+0000","last_unstale":"2024-07-27T03:13:57.405344+0000","last_undegraded":"2024-07-27T03:13:57.405344+0000","last_fullsized":"2024-07-27T03:13:57.405344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:51:26.186510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406142+0000","last_change":"2024-07-27T03:12:32.574429+0000","last_active":"2024-07-27T03:13:57.406142+0000","last_peered":"2024-07-27T03:13:57.406142+0000","last_clean":"2024-07-27T03:13:57.406142+0000","last_became_active":"2024-07-27T03:12:32.573986+0000","last_became_peered":"2024-07-27T03:12:32.573986+0000","last_unstale":"2024-07-27T03:13:57.406142+0000","last_undegraded":"2024-07-27T03:13:57.406142+0000","last_fullsized":"2024-07-27T03:13:57.406142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:35:00.345561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69276,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1033,"num_read_kb":592,"num_write":249,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":207828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645361,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":525562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841590,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37044,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708108,"statfs":{"total":95995035648,"available":95957102592,"internally_reserved":0,"allocated":1748992,"data_stored":1178915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037818,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1077248,"data_stored":525470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234047,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":525826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430275,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1671168,"data_stored":1122736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593801,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":520974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790029,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1679360,"data_stored":1117927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953555,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1060864,"data_stored":525439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149783,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":520682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1138688,"data_stored":584687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1118208,"data_stored":584076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1077248,"data_stored":522430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-27T03:15:23.775 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-07-27T03:15:23.775 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-07-27T03:15:23.775 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-07-27T03:15:23.775 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph health --format=json 2024-07-27T03:15:24.044 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:15:24.443 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:24 smithi067 ceph-mon[32814]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:15:24.443 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:24 smithi067 ceph-mon[32814]: pgmap v395: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:24.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:24 smithi149 ceph-mon[37173]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:15:24.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:24 smithi149 ceph-mon[37173]: pgmap v395: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:24.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:24 smithi175 ceph-mon[37125]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:15:24.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:24 smithi175 ceph-mon[37125]: pgmap v395: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:25.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.698+0000 7f007d692640 1 -- 172.21.15.67:0/357252002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0078108f10 msgr2=0x7f0078109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:25.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.698+0000 7f007d692640 1 --2- 172.21.15.67:0/357252002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0078108f10 0x7f0078109320 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f006c009900 tx=0x7f006c02a3f0 comp rx=0 tx=0).stop 2024-07-27T03:15:25.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.698+0000 7f007d692640 1 -- 172.21.15.67:0/357252002 shutdown_connections 2024-07-27T03:15:25.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.698+0000 7f007d692640 1 --2- 172.21.15.67:0/357252002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0078109860 0x7f0078110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:25.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.698+0000 7f007d692640 1 --2- 172.21.15.67:0/357252002 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0078108f10 0x7f0078109320 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:25.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.698+0000 7f007d692640 1 --2- 172.21.15.67:0/357252002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00781035d0 0x7f00781039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:25.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.698+0000 7f007d692640 1 -- 172.21.15.67:0/357252002 >> 172.21.15.67:0/357252002 conn(0x7f007807b2c0 msgr2=0x7f007807b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:25.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.698+0000 7f007d692640 1 -- 172.21.15.67:0/357252002 shutdown_connections 2024-07-27T03:15:25.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.698+0000 7f007d692640 1 -- 172.21.15.67:0/357252002 wait complete. 2024-07-27T03:15:25.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.698+0000 7f007d692640 1 Processor -- start 2024-07-27T03:15:25.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.699+0000 7f007d692640 1 -- start start 2024-07-27T03:15:25.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.699+0000 7f007d692640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00781035d0 0x7f0078199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:25.718 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.699+0000 7f007d692640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0078108f10 0x7f007819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:25.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.699+0000 7f007d692640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0078109860 0x7f007819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:25.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.699+0000 7f007d692640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f00781134d0 con 0x7f0078109860 2024-07-27T03:15:25.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.699+0000 7f007d692640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0078113350 con 0x7f00781035d0 2024-07-27T03:15:25.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.699+0000 7f007d692640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0078113650 con 0x7f0078108f10 2024-07-27T03:15:25.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.699+0000 7f00777fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0078109860 0x7f007819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:25.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.699+0000 7f00777fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0078109860 0x7f007819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50984/0 (socket says 172.21.15.67:50984) 2024-07-27T03:15:25.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.699+0000 7f00777fe640 1 -- 172.21.15.67:0/3885258961 learned_addr learned my addr 172.21.15.67:0/3885258961 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:25.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.699+0000 7f00767fc640 1 --2- 172.21.15.67:0/3885258961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0078108f10 0x7f007819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:25.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.699+0000 7f0076ffd640 1 --2- 172.21.15.67:0/3885258961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00781035d0 0x7f0078199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:25.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.699+0000 7f00777fe640 1 -- 172.21.15.67:0/3885258961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0078108f10 msgr2=0x7f007819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:25.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.699+0000 7f00777fe640 1 --2- 172.21.15.67:0/3885258961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0078108f10 0x7f007819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:25.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.699+0000 7f00777fe640 1 -- 172.21.15.67:0/3885258961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00781035d0 msgr2=0x7f0078199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:25.719 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.699+0000 7f00777fe640 1 --2- 172.21.15.67:0/3885258961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00781035d0 0x7f0078199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:25.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.699+0000 7f00777fe640 1 -- 172.21.15.67:0/3885258961 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007819fd90 con 0x7f0078109860 2024-07-27T03:15:25.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.700+0000 7f00777fe640 1 --2- 172.21.15.67:0/3885258961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0078109860 0x7f007819f720 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f007000cce0 tx=0x7f007000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:25.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.700+0000 7f005ffff640 1 -- 172.21.15.67:0/3885258961 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f007000f450 con 0x7f0078109860 2024-07-27T03:15:25.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.700+0000 7f007d692640 1 -- 172.21.15.67:0/3885258961 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00781a0080 con 0x7f0078109860 2024-07-27T03:15:25.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.700+0000 7f005ffff640 1 -- 172.21.15.67:0/3885258961 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f007000faa0 con 0x7f0078109860 2024-07-27T03:15:25.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.700+0000 7f005ffff640 1 -- 172.21.15.67:0/3885258961 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0070018b70 con 0x7f0078109860 2024-07-27T03:15:25.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.700+0000 7f007d692640 1 -- 172.21.15.67:0/3885258961 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00781a0540 con 0x7f0078109860 2024-07-27T03:15:25.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.701+0000 7f007d692640 1 -- 172.21.15.67:0/3885258961 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00781039e0 con 0x7f0078109860 2024-07-27T03:15:25.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.702+0000 7f005ffff640 1 -- 172.21.15.67:0/3885258961 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 17) ==== 151110+0+0 (secure 0 0 0) 0x7f0070018d10 con 0x7f0078109860 2024-07-27T03:15:25.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.703+0000 7f005ffff640 1 --2- 172.21.15.67:0/3885258961 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f00600b3c90 0x7f00600b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:25.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.703+0000 7f0076ffd640 1 --2- 172.21.15.67:0/3885258961 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f00600b3c90 0x7f00600b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:25.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.703+0000 7f005ffff640 1 -- 172.21.15.67:0/3885258961 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f00700e4650 con 0x7f0078109860 2024-07-27T03:15:25.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.704+0000 7f0076ffd640 1 --2- 172.21.15.67:0/3885258961 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f00600b3c90 0x7f00600b61a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0068005e10 tx=0x7f0068005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:25.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.706+0000 7f005ffff640 1 -- 172.21.15.67:0/3885258961 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f0070014030 con 0x7f0078109860 2024-07-27T03:15:25.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.927+0000 7f007d692640 1 -- 172.21.15.67:0/3885258961 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f007819b010 con 0x7f0078109860 2024-07-27T03:15:25.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.927+0000 7f005ffff640 1 -- 172.21.15.67:0/3885258961 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f007819b010 con 0x7f0078109860 2024-07-27T03:15:25.930 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:15:25.931 INFO:teuthology.orchestra.run.smithi067.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-07-27T03:15:25.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.929+0000 7f007d692640 1 -- 172.21.15.67:0/3885258961 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f00600b3c90 msgr2=0x7f00600b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:25.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.930+0000 7f007d692640 1 --2- 172.21.15.67:0/3885258961 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f00600b3c90 0x7f00600b61a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0068005e10 tx=0x7f0068005da0 comp rx=0 tx=0).stop 2024-07-27T03:15:25.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.930+0000 7f007d692640 1 -- 172.21.15.67:0/3885258961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0078109860 msgr2=0x7f007819f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:25.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.930+0000 7f007d692640 1 --2- 172.21.15.67:0/3885258961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0078109860 0x7f007819f720 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f007000cce0 tx=0x7f007000a620 comp rx=0 tx=0).stop 2024-07-27T03:15:25.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.930+0000 7f007d692640 1 -- 172.21.15.67:0/3885258961 shutdown_connections 2024-07-27T03:15:25.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.930+0000 7f007d692640 1 --2- 172.21.15.67:0/3885258961 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f00600b3c90 0x7f00600b61a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:25.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.930+0000 7f007d692640 1 --2- 172.21.15.67:0/3885258961 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0078109860 0x7f007819f720 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:25.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.930+0000 7f007d692640 1 --2- 172.21.15.67:0/3885258961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0078108f10 0x7f007819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:25.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.930+0000 7f007d692640 1 --2- 172.21.15.67:0/3885258961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00781035d0 0x7f0078199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:25.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.930+0000 7f007d692640 1 -- 172.21.15.67:0/3885258961 >> 172.21.15.67:0/3885258961 conn(0x7f007807b2c0 msgr2=0x7f0078106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:25.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.930+0000 7f007d692640 1 -- 172.21.15.67:0/3885258961 shutdown_connections 2024-07-27T03:15:25.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:25.930+0000 7f007d692640 1 -- 172.21.15.67:0/3885258961 wait complete. 2024-07-27T03:15:26.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:26 smithi067 ceph-mon[32814]: pgmap v396: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:26.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:26 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3885258961' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-07-27T03:15:26.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:26 smithi149 ceph-mon[37173]: pgmap v396: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:26.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:26 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3885258961' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-07-27T03:15:26.601 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-07-27T03:15:26.602 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:15:26.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:26 smithi175 ceph-mon[37125]: pgmap v396: 129 pgs: 129 active+clean; 644 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:26.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:26 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3885258961' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-07-27T03:15:26.867 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:15:27.243 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:27 smithi067 ceph-mon[32814]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-07-27T03:15:27.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:27 smithi149 ceph-mon[37173]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-07-27T03:15:27.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:27 smithi175 ceph-mon[37125]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-07-27T03:15:28.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:28 smithi067 ceph-mon[32814]: pgmap v397: 129 pgs: 129 active+clean; 645 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-27T03:15:28.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:28 smithi149 ceph-mon[37173]: pgmap v397: 129 pgs: 129 active+clean; 645 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-27T03:15:28.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.570+0000 7f94d8a49640 1 -- 172.21.15.67:0/3197026453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94d4075b10 msgr2=0x7f94d410f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:28.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.570+0000 7f94d8a49640 1 --2- 172.21.15.67:0/3197026453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94d4075b10 0x7f94d410f680 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f94c8009900 tx=0x7f94c802a410 comp rx=0 tx=0).stop 2024-07-27T03:15:28.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.570+0000 7f94d8a49640 1 -- 172.21.15.67:0/3197026453 shutdown_connections 2024-07-27T03:15:28.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.570+0000 7f94d8a49640 1 --2- 172.21.15.67:0/3197026453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94d410fbc0 0x7f94d4111fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:28.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.570+0000 7f94d8a49640 1 --2- 172.21.15.67:0/3197026453 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94d4075b10 0x7f94d410f680 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:28.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.570+0000 7f94d8a49640 1 --2- 172.21.15.67:0/3197026453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94d40751c0 0x7f94d40755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:28.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.570+0000 7f94d8a49640 1 -- 172.21.15.67:0/3197026453 >> 172.21.15.67:0/3197026453 conn(0x7f94d40ff050 msgr2=0x7f94d41014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:28.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.570+0000 7f94d8a49640 1 -- 172.21.15.67:0/3197026453 shutdown_connections 2024-07-27T03:15:28.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.570+0000 7f94d8a49640 1 -- 172.21.15.67:0/3197026453 wait complete. 2024-07-27T03:15:28.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.571+0000 7f94d8a49640 1 Processor -- start 2024-07-27T03:15:28.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.571+0000 7f94d8a49640 1 -- start start 2024-07-27T03:15:28.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.571+0000 7f94d8a49640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94d40751c0 0x7f94d419c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:28.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.571+0000 7f94d8a49640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94d4075b10 0x7f94d419c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:28.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.572+0000 7f94d8a49640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94d410fbc0 0x7f94d41a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:28.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.572+0000 7f94d8a49640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f94d41156e0 con 0x7f94d4075b10 2024-07-27T03:15:28.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.572+0000 7f94d8a49640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f94d4115560 con 0x7f94d40751c0 2024-07-27T03:15:28.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.572+0000 7f94d8a49640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f94d4115860 con 0x7f94d410fbc0 2024-07-27T03:15:28.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.572+0000 7f94d1d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94d4075b10 0x7f94d419c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:28.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.572+0000 7f94d259c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94d40751c0 0x7f94d419c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:28.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.572+0000 7f94d1d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94d4075b10 0x7f94d419c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50994/0 (socket says 172.21.15.67:50994) 2024-07-27T03:15:28.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.572+0000 7f94d1d9b640 1 -- 172.21.15.67:0/1232437877 learned_addr learned my addr 172.21.15.67:0/1232437877 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:28.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.572+0000 7f94d2d9d640 1 --2- 172.21.15.67:0/1232437877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94d410fbc0 0x7f94d41a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:28.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.572+0000 7f94d1d9b640 1 -- 172.21.15.67:0/1232437877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94d410fbc0 msgr2=0x7f94d41a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:28.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.572+0000 7f94d1d9b640 1 --2- 172.21.15.67:0/1232437877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94d410fbc0 0x7f94d41a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:28.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.572+0000 7f94d1d9b640 1 -- 172.21.15.67:0/1232437877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94d40751c0 msgr2=0x7f94d419c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:28.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.572+0000 7f94d1d9b640 1 --2- 172.21.15.67:0/1232437877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94d40751c0 0x7f94d419c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:28.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.572+0000 7f94d1d9b640 1 -- 172.21.15.67:0/1232437877 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d41a2080 con 0x7f94d4075b10 2024-07-27T03:15:28.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.572+0000 7f94d259c640 1 --2- 172.21.15.67:0/1232437877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94d40751c0 0x7f94d419c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:15:28.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.572+0000 7f94d1d9b640 1 --2- 172.21.15.67:0/1232437877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94d4075b10 0x7f94d419c6b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f94c802a930 tx=0x7f94c8031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:28.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.573+0000 7f94c37fe640 1 -- 172.21.15.67:0/1232437877 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c803f030 con 0x7f94d4075b10 2024-07-27T03:15:28.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.573+0000 7f94c37fe640 1 -- 172.21.15.67:0/1232437877 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94c8004490 con 0x7f94d4075b10 2024-07-27T03:15:28.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.573+0000 7f94d8a49640 1 -- 172.21.15.67:0/1232437877 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d41a2310 con 0x7f94d4075b10 2024-07-27T03:15:28.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.573+0000 7f94c37fe640 1 -- 172.21.15.67:0/1232437877 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c8035070 con 0x7f94d4075b10 2024-07-27T03:15:28.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.573+0000 7f94d8a49640 1 -- 172.21.15.67:0/1232437877 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d41a8ad0 con 0x7f94d4075b10 2024-07-27T03:15:28.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.574+0000 7f94d8a49640 1 -- 172.21.15.67:0/1232437877 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94d40766d0 con 0x7f94d4075b10 2024-07-27T03:15:28.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.576+0000 7f94c37fe640 1 -- 172.21.15.67:0/1232437877 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f94c8042030 con 0x7f94d4075b10 2024-07-27T03:15:28.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.576+0000 7f94c37fe640 1 --2- 172.21.15.67:0/1232437877 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f94bc0b3bd0 0x7f94bc0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:28.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.576+0000 7f94d259c640 1 --2- 172.21.15.67:0/1232437877 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f94bc0b3bd0 0x7f94bc0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:28.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.576+0000 7f94c37fe640 1 -- 172.21.15.67:0/1232437877 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f94c8104660 con 0x7f94d4075b10 2024-07-27T03:15:28.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.577+0000 7f94d259c640 1 --2- 172.21.15.67:0/1232437877 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f94bc0b3bd0 0x7f94bc0b60e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009fc0 tx=0x7f94c4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:28.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.580+0000 7f94c37fe640 1 -- 172.21.15.67:0/1232437877 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f94c80cb2b0 con 0x7f94d4075b10 2024-07-27T03:15:28.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:28 smithi175 ceph-mon[37125]: pgmap v397: 129 pgs: 129 active+clean; 645 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-27T03:15:28.804 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.801+0000 7f94d8a49640 1 -- 172.21.15.67:0/1232437877 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94d41a8ca0 con 0x7f94d4075b10 2024-07-27T03:15:28.805 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.802+0000 7f94c37fe640 1 -- 172.21.15.67:0/1232437877 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11983 (secure 0 0 0) 0x7f94c80cf800 con 0x7f94d4075b10 2024-07-27T03:15:28.805 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:15:28.806 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":29,"btime":"2024-07-27T03:13:42:247804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14679,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/252686369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":252686369},{"type":"v1","addr":"172.21.15.67:6835","nonce":252686369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/3358832762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3358832762},{"type":"v1","addr":"172.21.15.67:6837","nonce":3358832762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-27T03:12:34.320208+0000","modified":"2024-07-27T03:13:42.062021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24620,"mds_1":24587,"mds_2":24583,"mds_3":24581,"mds_4":24563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14736":{"gid":14736,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/2162708317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2162708317},{"type":"v1","addr":"172.21.15.67:6839","nonce":2162708317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14748":{"gid":14748,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/795704244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":795704244},{"type":"v1","addr":"172.21.15.67:6841","nonce":795704244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/3868325215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3868325215},{"type":"v1","addr":"172.21.15.149:6833","nonce":3868325215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2462749027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2462749027},{"type":"v1","addr":"172.21.15.175:6835","nonce":2462749027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.149:6835/3730964806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3730964806},{"type":"v1","addr":"172.21.15.149:6835","nonce":3730964806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.175:6837/1304803502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1304803502},{"type":"v1","addr":"172.21.15.175:6837","nonce":1304803502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.175:6839/1670568853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1670568853},{"type":"v1","addr":"172.21.15.175:6839","nonce":1670568853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24587":{"gid":24587,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.149:6837/2932951845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2932951845},{"type":"v1","addr":"172.21.15.149:6837","nonce":2932951845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.149:6839/281746498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":281746498},{"type":"v1","addr":"172.21.15.149:6839","nonce":281746498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24620,"qdb_cluster":[24563,24581,24620,24583,24587]},"id":1}]} 2024-07-27T03:15:28.806 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-07-27T03:15:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.804+0000 7f94d8a49640 1 -- 172.21.15.67:0/1232437877 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f94bc0b3bd0 msgr2=0x7f94bc0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.804+0000 7f94d8a49640 1 --2- 172.21.15.67:0/1232437877 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f94bc0b3bd0 0x7f94bc0b60e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009fc0 tx=0x7f94c4005f90 comp rx=0 tx=0).stop 2024-07-27T03:15:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.804+0000 7f94d8a49640 1 -- 172.21.15.67:0/1232437877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94d4075b10 msgr2=0x7f94d419c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.804+0000 7f94d8a49640 1 --2- 172.21.15.67:0/1232437877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94d4075b10 0x7f94d419c6b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f94c802a930 tx=0x7f94c8031a20 comp rx=0 tx=0).stop 2024-07-27T03:15:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.804+0000 7f94d8a49640 1 -- 172.21.15.67:0/1232437877 shutdown_connections 2024-07-27T03:15:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.804+0000 7f94d8a49640 1 --2- 172.21.15.67:0/1232437877 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f94bc0b3bd0 0x7f94bc0b60e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.804+0000 7f94d8a49640 1 --2- 172.21.15.67:0/1232437877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94d410fbc0 0x7f94d41a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.804+0000 7f94d8a49640 1 --2- 172.21.15.67:0/1232437877 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94d4075b10 0x7f94d419c6b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.804+0000 7f94d8a49640 1 --2- 172.21.15.67:0/1232437877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94d40751c0 0x7f94d419c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.804+0000 7f94d8a49640 1 -- 172.21.15.67:0/1232437877 >> 172.21.15.67:0/1232437877 conn(0x7f94d40ff050 msgr2=0x7f94d4109760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:28.807 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.804+0000 7f94d8a49640 1 -- 172.21.15.67:0/1232437877 shutdown_connections 2024-07-27T03:15:28.808 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:28.804+0000 7f94d8a49640 1 -- 172.21.15.67:0/1232437877 wait complete. 2024-07-27T03:15:29.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:29 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1232437877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:15:29.684 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:29 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1232437877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:15:29.685 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:29 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1232437877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:15:29.686 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:15:29.955 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:15:30.336 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:30 smithi067 ceph-mon[32814]: pgmap v398: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-07-27T03:15:30.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:30 smithi149 ceph-mon[37173]: pgmap v398: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-07-27T03:15:30.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:30 smithi175 ceph-mon[37125]: pgmap v398: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-07-27T03:15:31.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.717+0000 7f78bc94d640 1 -- 172.21.15.67:0/993020705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78b8107e30 msgr2=0x7f78b8108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:31.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.717+0000 7f78bc94d640 1 --2- 172.21.15.67:0/993020705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78b8107e30 0x7f78b8108240 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009900 tx=0x7f78ac02a410 comp rx=0 tx=0).stop 2024-07-27T03:15:31.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.717+0000 7f78bc94d640 1 -- 172.21.15.67:0/993020705 shutdown_connections 2024-07-27T03:15:31.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.717+0000 7f78bc94d640 1 --2- 172.21.15.67:0/993020705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78b8108780 0x7f78b810efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:31.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.717+0000 7f78bc94d640 1 --2- 172.21.15.67:0/993020705 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78b8107e30 0x7f78b8108240 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:31.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.717+0000 7f78bc94d640 1 --2- 172.21.15.67:0/993020705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78b81024f0 0x7f78b8102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:31.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.717+0000 7f78bc94d640 1 -- 172.21.15.67:0/993020705 >> 172.21.15.67:0/993020705 conn(0x7f78b80791b0 msgr2=0x7f78b80795c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:31.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.717+0000 7f78bc94d640 1 -- 172.21.15.67:0/993020705 shutdown_connections 2024-07-27T03:15:31.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.717+0000 7f78bc94d640 1 -- 172.21.15.67:0/993020705 wait complete. 2024-07-27T03:15:31.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.718+0000 7f78bc94d640 1 Processor -- start 2024-07-27T03:15:31.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.718+0000 7f78bc94d640 1 -- start start 2024-07-27T03:15:31.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.718+0000 7f78bc94d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78b81024f0 0x7f78b8199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:31.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.718+0000 7f78bc94d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78b8107e30 0x7f78b819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:31.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.718+0000 7f78bc94d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78b8108780 0x7f78b819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:31.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.718+0000 7f78bc94d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f78b8112430 con 0x7f78b8108780 2024-07-27T03:15:31.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.718+0000 7f78bc94d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f78b81122b0 con 0x7f78b8107e30 2024-07-27T03:15:31.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.718+0000 7f78bc94d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f78b81125b0 con 0x7f78b81024f0 2024-07-27T03:15:31.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.718+0000 7f78b6d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78b8108780 0x7f78b819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:31.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.718+0000 7f78b6d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78b8108780 0x7f78b819f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51016/0 (socket says 172.21.15.67:51016) 2024-07-27T03:15:31.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.718+0000 7f78b6d9d640 1 -- 172.21.15.67:0/2130859390 learned_addr learned my addr 172.21.15.67:0/2130859390 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:31.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.718+0000 7f78b659c640 1 --2- 172.21.15.67:0/2130859390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78b81024f0 0x7f78b8199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:31.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.718+0000 7f78b5d9b640 1 --2- 172.21.15.67:0/2130859390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78b8107e30 0x7f78b819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:31.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.719+0000 7f78b6d9d640 1 -- 172.21.15.67:0/2130859390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78b81024f0 msgr2=0x7f78b8199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:31.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.719+0000 7f78b6d9d640 1 --2- 172.21.15.67:0/2130859390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78b81024f0 0x7f78b8199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:31.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.719+0000 7f78b6d9d640 1 -- 172.21.15.67:0/2130859390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78b8107e30 msgr2=0x7f78b819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:31.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.719+0000 7f78b6d9d640 1 --2- 172.21.15.67:0/2130859390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78b8107e30 0x7f78b819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:31.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.719+0000 7f78b6d9d640 1 -- 172.21.15.67:0/2130859390 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b819fee0 con 0x7f78b8108780 2024-07-27T03:15:31.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.719+0000 7f78b6d9d640 1 --2- 172.21.15.67:0/2130859390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78b8108780 0x7f78b819f7b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f78b000eaa0 tx=0x7f78b000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:31.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.719+0000 7f78a77fe640 1 -- 172.21.15.67:0/2130859390 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0004d20 con 0x7f78b8108780 2024-07-27T03:15:31.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.719+0000 7f78a77fe640 1 -- 172.21.15.67:0/2130859390 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78b0005020 con 0x7f78b8108780 2024-07-27T03:15:31.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.719+0000 7f78a77fe640 1 -- 172.21.15.67:0/2130859390 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b00116b0 con 0x7f78b8108780 2024-07-27T03:15:31.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.719+0000 7f78bc94d640 1 -- 172.21.15.67:0/2130859390 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b81a01d0 con 0x7f78b8108780 2024-07-27T03:15:31.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.719+0000 7f78bc94d640 1 -- 172.21.15.67:0/2130859390 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b81a6930 con 0x7f78b8108780 2024-07-27T03:15:31.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.720+0000 7f78bc94d640 1 -- 172.21.15.67:0/2130859390 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78b8103770 con 0x7f78b8108780 2024-07-27T03:15:31.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.722+0000 7f78a77fe640 1 -- 172.21.15.67:0/2130859390 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f78b00051d0 con 0x7f78b8108780 2024-07-27T03:15:31.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.722+0000 7f78a77fe640 1 --2- 172.21.15.67:0/2130859390 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f78a00b3bd0 0x7f78a00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:31.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.723+0000 7f78b659c640 1 --2- 172.21.15.67:0/2130859390 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f78a00b3bd0 0x7f78a00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:31.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.723+0000 7f78a77fe640 1 -- 172.21.15.67:0/2130859390 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f78b00e45a0 con 0x7f78b8108780 2024-07-27T03:15:31.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.723+0000 7f78b659c640 1 --2- 172.21.15.67:0/2130859390 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f78a00b3bd0 0x7f78a00b60e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009990 tx=0x7f78a8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:31.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.725+0000 7f78a77fe640 1 -- 172.21.15.67:0/2130859390 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f78b0016030 con 0x7f78b8108780 2024-07-27T03:15:31.950 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.947+0000 7f78bc94d640 1 -- 172.21.15.67:0/2130859390 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78b8102900 con 0x7f78b8108780 2024-07-27T03:15:31.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.947+0000 7f78a77fe640 1 -- 172.21.15.67:0/2130859390 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11983 (secure 0 0 0) 0x7f78b8102900 con 0x7f78b8108780 2024-07-27T03:15:31.951 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:15:31.951 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":29,"btime":"2024-07-27T03:13:42:247804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14679,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/252686369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":252686369},{"type":"v1","addr":"172.21.15.67:6835","nonce":252686369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/3358832762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3358832762},{"type":"v1","addr":"172.21.15.67:6837","nonce":3358832762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-27T03:12:34.320208+0000","modified":"2024-07-27T03:13:42.062021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24620,"mds_1":24587,"mds_2":24583,"mds_3":24581,"mds_4":24563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14736":{"gid":14736,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/2162708317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2162708317},{"type":"v1","addr":"172.21.15.67:6839","nonce":2162708317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14748":{"gid":14748,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/795704244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":795704244},{"type":"v1","addr":"172.21.15.67:6841","nonce":795704244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/3868325215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3868325215},{"type":"v1","addr":"172.21.15.149:6833","nonce":3868325215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2462749027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2462749027},{"type":"v1","addr":"172.21.15.175:6835","nonce":2462749027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.149:6835/3730964806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3730964806},{"type":"v1","addr":"172.21.15.149:6835","nonce":3730964806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.175:6837/1304803502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1304803502},{"type":"v1","addr":"172.21.15.175:6837","nonce":1304803502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.175:6839/1670568853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1670568853},{"type":"v1","addr":"172.21.15.175:6839","nonce":1670568853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24587":{"gid":24587,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.149:6837/2932951845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2932951845},{"type":"v1","addr":"172.21.15.149:6837","nonce":2932951845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.149:6839/281746498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":281746498},{"type":"v1","addr":"172.21.15.149:6839","nonce":281746498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24620,"qdb_cluster":[24563,24581,24620,24583,24587]},"id":1}]} 2024-07-27T03:15:31.952 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-07-27T03:15:31.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.950+0000 7f78bc94d640 1 -- 172.21.15.67:0/2130859390 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f78a00b3bd0 msgr2=0x7f78a00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:31.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.950+0000 7f78bc94d640 1 --2- 172.21.15.67:0/2130859390 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f78a00b3bd0 0x7f78a00b60e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009990 tx=0x7f78a8009110 comp rx=0 tx=0).stop 2024-07-27T03:15:31.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.950+0000 7f78bc94d640 1 -- 172.21.15.67:0/2130859390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78b8108780 msgr2=0x7f78b819f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:31.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.950+0000 7f78bc94d640 1 --2- 172.21.15.67:0/2130859390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78b8108780 0x7f78b819f7b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f78b000eaa0 tx=0x7f78b000eea0 comp rx=0 tx=0).stop 2024-07-27T03:15:31.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.950+0000 7f78bc94d640 1 -- 172.21.15.67:0/2130859390 shutdown_connections 2024-07-27T03:15:31.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.950+0000 7f78bc94d640 1 --2- 172.21.15.67:0/2130859390 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f78a00b3bd0 0x7f78a00b60e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:31.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.950+0000 7f78bc94d640 1 --2- 172.21.15.67:0/2130859390 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78b8108780 0x7f78b819f7b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:31.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.950+0000 7f78bc94d640 1 --2- 172.21.15.67:0/2130859390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78b8107e30 0x7f78b819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:31.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.950+0000 7f78bc94d640 1 --2- 172.21.15.67:0/2130859390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78b81024f0 0x7f78b8199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:31.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.950+0000 7f78bc94d640 1 -- 172.21.15.67:0/2130859390 >> 172.21.15.67:0/2130859390 conn(0x7f78b80791b0 msgr2=0x7f78b806b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:31.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.951+0000 7f78bc94d640 1 -- 172.21.15.67:0/2130859390 shutdown_connections 2024-07-27T03:15:31.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:31.951+0000 7f78bc94d640 1 -- 172.21.15.67:0/2130859390 wait complete. 2024-07-27T03:15:32.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:32 smithi067 ceph-mon[32814]: pgmap v399: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-07-27T03:15:32.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:32 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2130859390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:15:32.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:32 smithi149 ceph-mon[37173]: pgmap v399: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-07-27T03:15:32.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:32 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2130859390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:15:32.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:32 smithi175 ceph-mon[37125]: pgmap v399: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-07-27T03:15:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:32 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2130859390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:15:32.679 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd dump --format=json 2024-07-27T03:15:32.946 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:15:34.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:34 smithi067 ceph-mon[32814]: pgmap v400: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-07-27T03:15:34.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:34 smithi149 ceph-mon[37173]: pgmap v400: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-07-27T03:15:34.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:34 smithi175 ceph-mon[37125]: pgmap v400: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-07-27T03:15:34.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.738+0000 7f13d6fbc640 1 -- 172.21.15.67:0/2051168720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13d0104320 msgr2=0x7f13d0104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:34.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.738+0000 7f13d6fbc640 1 --2- 172.21.15.67:0/2051168720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13d0104320 0x7f13d0104730 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009ca0 tx=0x7f13c002a8c0 comp rx=0 tx=0).stop 2024-07-27T03:15:34.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.738+0000 7f13d6fbc640 1 -- 172.21.15.67:0/2051168720 shutdown_connections 2024-07-27T03:15:34.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.738+0000 7f13d6fbc640 1 --2- 172.21.15.67:0/2051168720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13d010d8d0 0x7f13d010fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:34.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.738+0000 7f13d6fbc640 1 --2- 172.21.15.67:0/2051168720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13d0104d50 0x7f13d010d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:34.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.738+0000 7f13d6fbc640 1 --2- 172.21.15.67:0/2051168720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13d0104320 0x7f13d0104730 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:34.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.738+0000 7f13d6fbc640 1 -- 172.21.15.67:0/2051168720 >> 172.21.15.67:0/2051168720 conn(0x7f13d00791a0 msgr2=0x7f13d00795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:34.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.738+0000 7f13d6fbc640 1 -- 172.21.15.67:0/2051168720 shutdown_connections 2024-07-27T03:15:34.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.738+0000 7f13d6fbc640 1 -- 172.21.15.67:0/2051168720 wait complete. 2024-07-27T03:15:34.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.739+0000 7f13d6fbc640 1 Processor -- start 2024-07-27T03:15:34.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.739+0000 7f13d6fbc640 1 -- start start 2024-07-27T03:15:34.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.739+0000 7f13d6fbc640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13d0104320 0x7f13d0199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:34.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.739+0000 7f13d6fbc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13d0104d50 0x7f13d019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:34.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.739+0000 7f13d6fbc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13d010d8d0 0x7f13d019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:34.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.739+0000 7f13d6fbc640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f13d0113520 con 0x7f13d010d8d0 2024-07-27T03:15:34.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.739+0000 7f13d6fbc640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f13d01133a0 con 0x7f13d0104d50 2024-07-27T03:15:34.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.739+0000 7f13d6fbc640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f13d01136a0 con 0x7f13d0104320 2024-07-27T03:15:34.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.740+0000 7f13d5559640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13d010d8d0 0x7f13d019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:34.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.740+0000 7f13d5559640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13d010d8d0 0x7f13d019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51036/0 (socket says 172.21.15.67:51036) 2024-07-27T03:15:34.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.740+0000 7f13d5559640 1 -- 172.21.15.67:0/87538420 learned_addr learned my addr 172.21.15.67:0/87538420 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:34.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.740+0000 7f13cffff640 1 --2- 172.21.15.67:0/87538420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13d0104d50 0x7f13d019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:34.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.740+0000 7f13d4d58640 1 --2- 172.21.15.67:0/87538420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13d0104320 0x7f13d0199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:34.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.740+0000 7f13d5559640 1 -- 172.21.15.67:0/87538420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13d0104320 msgr2=0x7f13d0199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:34.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.740+0000 7f13d5559640 1 --2- 172.21.15.67:0/87538420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13d0104320 0x7f13d0199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:34.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.740+0000 7f13d5559640 1 -- 172.21.15.67:0/87538420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13d0104d50 msgr2=0x7f13d019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:34.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.740+0000 7f13d5559640 1 --2- 172.21.15.67:0/87538420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13d0104d50 0x7f13d019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:34.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.740+0000 7f13d5559640 1 -- 172.21.15.67:0/87538420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d019fe80 con 0x7f13d010d8d0 2024-07-27T03:15:34.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.740+0000 7f13d4d58640 1 --2- 172.21.15.67:0/87538420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13d0104320 0x7f13d0199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:15:34.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.740+0000 7f13d5559640 1 --2- 172.21.15.67:0/87538420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13d010d8d0 0x7f13d019f750 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f13c800c970 tx=0x7f13c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:34.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.740+0000 7f13cdffb640 1 -- 172.21.15.67:0/87538420 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8007bb0 con 0x7f13d010d8d0 2024-07-27T03:15:34.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.740+0000 7f13cdffb640 1 -- 172.21.15.67:0/87538420 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13c8007d50 con 0x7f13d010d8d0 2024-07-27T03:15:34.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.740+0000 7f13d6fbc640 1 -- 172.21.15.67:0/87538420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d01a0170 con 0x7f13d010d8d0 2024-07-27T03:15:34.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.741+0000 7f13cdffb640 1 -- 172.21.15.67:0/87538420 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8004e60 con 0x7f13d010d8d0 2024-07-27T03:15:34.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.741+0000 7f13d6fbc640 1 -- 172.21.15.67:0/87538420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d01a68d0 con 0x7f13d010d8d0 2024-07-27T03:15:34.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.742+0000 7f13d6fbc640 1 -- 172.21.15.67:0/87538420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13d006bc10 con 0x7f13d010d8d0 2024-07-27T03:15:34.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.743+0000 7f13cdffb640 1 -- 172.21.15.67:0/87538420 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f13c8005020 con 0x7f13d010d8d0 2024-07-27T03:15:34.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.744+0000 7f13cdffb640 1 --2- 172.21.15.67:0/87538420 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f13bc0b3c50 0x7f13bc0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:34.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.744+0000 7f13d4d58640 1 --2- 172.21.15.67:0/87538420 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f13bc0b3c50 0x7f13bc0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:34.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.744+0000 7f13cdffb640 1 -- 172.21.15.67:0/87538420 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f13c80e3bb0 con 0x7f13d010d8d0 2024-07-27T03:15:34.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.745+0000 7f13d4d58640 1 --2- 172.21.15.67:0/87538420 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f13bc0b3c50 0x7f13bc0b6160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f13c00094e0 tx=0x7f13c00093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:34.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.748+0000 7f13cdffb640 1 -- 172.21.15.67:0/87538420 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f13c8014030 con 0x7f13d010d8d0 2024-07-27T03:15:34.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.933+0000 7f13d6fbc640 1 -- 172.21.15.67:0/87538420 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13d0108470 con 0x7f13d010d8d0 2024-07-27T03:15:34.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.935+0000 7f13cdffb640 1 -- 172.21.15.67:0/87538420 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24829 (secure 0 0 0) 0x7f13c80aae50 con 0x7f13d010d8d0 2024-07-27T03:15:34.939 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:15:34.939 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":91,"fsid":"c0782a00-4bc4-11ef-bcc7-c7b262605968","created":"2024-07-27T03:04:08.096824+0000","modified":"2024-07-27T03:13:57.294556+0000","last_up_change":"2024-07-27T03:11:33.314800+0000","last_in_change":"2024-07-27T03:11:14.028095+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-27T03:08:01.042479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-27T03:12:27.565948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-27T03:12:31.353726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-27T03:12:40.599486+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9f614272-18e8-4562-812c-4f43e40efd2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6803","nonce":2648129674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6805","nonce":2648129674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6809","nonce":2648129674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6807","nonce":2648129674}]},"public_addr":"172.21.15.67:6803/2648129674","cluster_addr":"172.21.15.67:6805/2648129674","heartbeat_back_addr":"172.21.15.67:6809/2648129674","heartbeat_front_addr":"172.21.15.67:6807/2648129674","state":["exists","up"]},{"osd":1,"uuid":"f59f5879-163a-4269-bbe7-1200eea65341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6801","nonce":2561753152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6803","nonce":2561753152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6807","nonce":2561753152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6805","nonce":2561753152}]},"public_addr":"172.21.15.149:6801/2561753152","cluster_addr":"172.21.15.149:6803/2561753152","heartbeat_back_addr":"172.21.15.149:6807/2561753152","heartbeat_front_addr":"172.21.15.149:6805/2561753152","state":["exists","up"]},{"osd":2,"uuid":"444e0ebe-7ca3-4be9-8bd5-db859f824445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6801","nonce":3770940078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6803","nonce":3770940078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6807","nonce":3770940078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6805","nonce":3770940078}]},"public_addr":"172.21.15.175:6801/3770940078","cluster_addr":"172.21.15.175:6803/3770940078","heartbeat_back_addr":"172.21.15.175:6807/3770940078","heartbeat_front_addr":"172.21.15.175:6805/3770940078","state":["exists","up"]},{"osd":3,"uuid":"d554db3d-9604-427c-8736-b1bb379f8d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6811","nonce":3159144693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6813","nonce":3159144693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6817","nonce":3159144693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6815","nonce":3159144693}]},"public_addr":"172.21.15.67:6811/3159144693","cluster_addr":"172.21.15.67:6813/3159144693","heartbeat_back_addr":"172.21.15.67:6817/3159144693","heartbeat_front_addr":"172.21.15.67:6815/3159144693","state":["exists","up"]},{"osd":4,"uuid":"c317dbe5-3b49-47a6-8408-7042a3dac345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3393503532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3393503532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3393503532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3393503532}]},"public_addr":"172.21.15.149:6809/3393503532","cluster_addr":"172.21.15.149:6811/3393503532","heartbeat_back_addr":"172.21.15.149:6815/3393503532","heartbeat_front_addr":"172.21.15.149:6813/3393503532","state":["exists","up"]},{"osd":5,"uuid":"5c85abd6-2b42-4957-a9b7-59a9ee36b841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6809","nonce":2291204784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6811","nonce":2291204784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6815","nonce":2291204784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6813","nonce":2291204784}]},"public_addr":"172.21.15.175:6809/2291204784","cluster_addr":"172.21.15.175:6811/2291204784","heartbeat_back_addr":"172.21.15.175:6815/2291204784","heartbeat_front_addr":"172.21.15.175:6813/2291204784","state":["exists","up"]},{"osd":6,"uuid":"bf6c277d-ec24-4470-98de-cf6f28876ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6819","nonce":2660182879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6821","nonce":2660182879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6825","nonce":2660182879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6823","nonce":2660182879}]},"public_addr":"172.21.15.67:6819/2660182879","cluster_addr":"172.21.15.67:6821/2660182879","heartbeat_back_addr":"172.21.15.67:6825/2660182879","heartbeat_front_addr":"172.21.15.67:6823/2660182879","state":["exists","up"]},{"osd":7,"uuid":"fea2c7cf-d5cb-45ae-9b29-f6e56c903891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6817","nonce":2289624404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6819","nonce":2289624404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6823","nonce":2289624404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6821","nonce":2289624404}]},"public_addr":"172.21.15.149:6817/2289624404","cluster_addr":"172.21.15.149:6819/2289624404","heartbeat_back_addr":"172.21.15.149:6823/2289624404","heartbeat_front_addr":"172.21.15.149:6821/2289624404","state":["exists","up"]},{"osd":8,"uuid":"14d04d96-579a-4dc9-a4d4-50e4850b48e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6817","nonce":2286032466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6819","nonce":2286032466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6823","nonce":2286032466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6821","nonce":2286032466}]},"public_addr":"172.21.15.175:6817/2286032466","cluster_addr":"172.21.15.175:6819/2286032466","heartbeat_back_addr":"172.21.15.175:6823/2286032466","heartbeat_front_addr":"172.21.15.175:6821/2286032466","state":["exists","up"]},{"osd":9,"uuid":"b65dfb6b-1efe-4314-a8be-ce6a295e99fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6827","nonce":3411081513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6829","nonce":3411081513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6833","nonce":3411081513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6831","nonce":3411081513}]},"public_addr":"172.21.15.67:6827/3411081513","cluster_addr":"172.21.15.67:6829/3411081513","heartbeat_back_addr":"172.21.15.67:6833/3411081513","heartbeat_front_addr":"172.21.15.67:6831/3411081513","state":["exists","up"]},{"osd":10,"uuid":"e3dabae0-0d64-4222-984c-fc1cce883456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6825","nonce":2011495212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6827","nonce":2011495212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6831","nonce":2011495212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6829","nonce":2011495212}]},"public_addr":"172.21.15.149:6825/2011495212","cluster_addr":"172.21.15.149:6827/2011495212","heartbeat_back_addr":"172.21.15.149:6831/2011495212","heartbeat_front_addr":"172.21.15.149:6829/2011495212","state":["exists","up"]},{"osd":11,"uuid":"406c082c-9b5f-4781-8cbf-3c04fa49e410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6825","nonce":2565514213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6827","nonce":2565514213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6831","nonce":2565514213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6829","nonce":2565514213}]},"public_addr":"172.21.15.175:6825/2565514213","cluster_addr":"172.21.15.175:6827/2565514213","heartbeat_back_addr":"172.21.15.175:6831/2565514213","heartbeat_front_addr":"172.21.15.175:6829/2565514213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:11.399380+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:33.728518+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:56.411181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:19.871731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:42.994576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:06.712815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:31.321967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:55.070898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:18.686108+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:43.187795+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:07.318538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:31.374843+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/3477601235":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/4130679770":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6800/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/1029332005":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6801/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6801/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/284078926":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6800/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/2559121542":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/925406021":"2024-07-28T03:04:30.600273+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-27T03:15:34.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.937+0000 7f13d6fbc640 1 -- 172.21.15.67:0/87538420 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f13bc0b3c50 msgr2=0x7f13bc0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:34.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.937+0000 7f13d6fbc640 1 --2- 172.21.15.67:0/87538420 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f13bc0b3c50 0x7f13bc0b6160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f13c00094e0 tx=0x7f13c00093f0 comp rx=0 tx=0).stop 2024-07-27T03:15:34.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.937+0000 7f13d6fbc640 1 -- 172.21.15.67:0/87538420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13d010d8d0 msgr2=0x7f13d019f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:34.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.937+0000 7f13d6fbc640 1 --2- 172.21.15.67:0/87538420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13d010d8d0 0x7f13d019f750 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f13c800c970 tx=0x7f13c800ce30 comp rx=0 tx=0).stop 2024-07-27T03:15:34.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.937+0000 7f13d6fbc640 1 -- 172.21.15.67:0/87538420 shutdown_connections 2024-07-27T03:15:34.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.937+0000 7f13d6fbc640 1 --2- 172.21.15.67:0/87538420 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f13bc0b3c50 0x7f13bc0b6160 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:34.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.937+0000 7f13d6fbc640 1 --2- 172.21.15.67:0/87538420 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13d010d8d0 0x7f13d019f750 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:34.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.937+0000 7f13d6fbc640 1 --2- 172.21.15.67:0/87538420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13d0104d50 0x7f13d019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:34.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.937+0000 7f13d6fbc640 1 --2- 172.21.15.67:0/87538420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13d0104320 0x7f13d0199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:34.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.937+0000 7f13d6fbc640 1 -- 172.21.15.67:0/87538420 >> 172.21.15.67:0/87538420 conn(0x7f13d00791a0 msgr2=0x7f13d01002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:34.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.937+0000 7f13d6fbc640 1 -- 172.21.15.67:0/87538420 shutdown_connections 2024-07-27T03:15:34.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:34.937+0000 7f13d6fbc640 1 -- 172.21.15.67:0/87538420 wait complete. 2024-07-27T03:15:35.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:35 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/87538420' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:15:35.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:35 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/87538420' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:15:35.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:35 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/87538420' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:15:35.752 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:15:36.021 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:15:36.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:36 smithi067 ceph-mon[32814]: pgmap v401: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-07-27T03:15:36.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:36 smithi149 ceph-mon[37173]: pgmap v401: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-07-27T03:15:36.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:36 smithi175 ceph-mon[37125]: pgmap v401: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-07-27T03:15:37.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.749+0000 7f23871d6640 1 -- 172.21.15.67:0/3230651620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2380107ef0 msgr2=0x7f238010e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:37.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.749+0000 7f23871d6640 1 --2- 172.21.15.67:0/3230651620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2380107ef0 0x7f238010e720 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2378009ca0 tx=0x7f237802a8c0 comp rx=0 tx=0).stop 2024-07-27T03:15:37.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.749+0000 7f23871d6640 1 -- 172.21.15.67:0/3230651620 shutdown_connections 2024-07-27T03:15:37.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.749+0000 7f23871d6640 1 --2- 172.21.15.67:0/3230651620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2380107ef0 0x7f238010e720 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:37.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.749+0000 7f23871d6640 1 --2- 172.21.15.67:0/3230651620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23801075a0 0x7f23801079b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:37.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.749+0000 7f23871d6640 1 --2- 172.21.15.67:0/3230651620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2380101cb0 0x7f23801020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:37.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.749+0000 7f23871d6640 1 -- 172.21.15.67:0/3230651620 >> 172.21.15.67:0/3230651620 conn(0x7f2380077630 msgr2=0x7f2380077a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:37.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.749+0000 7f23871d6640 1 -- 172.21.15.67:0/3230651620 shutdown_connections 2024-07-27T03:15:37.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.749+0000 7f23871d6640 1 -- 172.21.15.67:0/3230651620 wait complete. 2024-07-27T03:15:37.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.750+0000 7f23871d6640 1 Processor -- start 2024-07-27T03:15:37.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.750+0000 7f23871d6640 1 -- start start 2024-07-27T03:15:37.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.750+0000 7f23871d6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2380101cb0 0x7f2380199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:37.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.750+0000 7f23871d6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23801075a0 0x7f238019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:37.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.750+0000 7f23871d6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2380107ef0 0x7f238019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:37.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.750+0000 7f23871d6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2380111330 con 0x7f2380101cb0 2024-07-27T03:15:37.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.750+0000 7f23871d6640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f23801111b0 con 0x7f2380107ef0 2024-07-27T03:15:37.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.750+0000 7f23871d6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23801114b0 con 0x7f23801075a0 2024-07-27T03:15:37.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.750+0000 7f2384f72640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2380101cb0 0x7f2380199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:37.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.750+0000 7f2384f72640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2380101cb0 0x7f2380199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52222/0 (socket says 172.21.15.67:52222) 2024-07-27T03:15:37.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.750+0000 7f2384f72640 1 -- 172.21.15.67:0/4119619916 learned_addr learned my addr 172.21.15.67:0/4119619916 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:37.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.750+0000 7f237ffff640 1 --2- 172.21.15.67:0/4119619916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23801075a0 0x7f238019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:37.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.750+0000 7f2385773640 1 --2- 172.21.15.67:0/4119619916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2380107ef0 0x7f238019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:37.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.750+0000 7f2384f72640 1 -- 172.21.15.67:0/4119619916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23801075a0 msgr2=0x7f238019a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:37.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.750+0000 7f2384f72640 1 --2- 172.21.15.67:0/4119619916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23801075a0 0x7f238019a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:37.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.750+0000 7f2384f72640 1 -- 172.21.15.67:0/4119619916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2380107ef0 msgr2=0x7f238019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:37.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.750+0000 7f2384f72640 1 --2- 172.21.15.67:0/4119619916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2380107ef0 0x7f238019f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:37.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.750+0000 7f2384f72640 1 -- 172.21.15.67:0/4119619916 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238019fed0 con 0x7f2380101cb0 2024-07-27T03:15:37.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.751+0000 7f237ffff640 1 --2- 172.21.15.67:0/4119619916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23801075a0 0x7f238019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:15:37.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.751+0000 7f2385773640 1 --2- 172.21.15.67:0/4119619916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2380107ef0 0x7f238019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:15:37.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.751+0000 7f2384f72640 1 --2- 172.21.15.67:0/4119619916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2380101cb0 0x7f2380199f60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f237000ddc0 tx=0x7f237000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:37.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.751+0000 7f237dffb640 1 -- 172.21.15.67:0/4119619916 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237001b070 con 0x7f2380101cb0 2024-07-27T03:15:37.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.751+0000 7f237dffb640 1 -- 172.21.15.67:0/4119619916 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f237000be40 con 0x7f2380101cb0 2024-07-27T03:15:37.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.751+0000 7f23871d6640 1 -- 172.21.15.67:0/4119619916 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23801a01c0 con 0x7f2380101cb0 2024-07-27T03:15:37.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.751+0000 7f237dffb640 1 -- 172.21.15.67:0/4119619916 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2370016a80 con 0x7f2380101cb0 2024-07-27T03:15:37.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.751+0000 7f23871d6640 1 -- 172.21.15.67:0/4119619916 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23801a6920 con 0x7f2380101cb0 2024-07-27T03:15:37.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.752+0000 7f23871d6640 1 -- 172.21.15.67:0/4119619916 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23801020c0 con 0x7f2380101cb0 2024-07-27T03:15:37.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.754+0000 7f237dffb640 1 -- 172.21.15.67:0/4119619916 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f237001e070 con 0x7f2380101cb0 2024-07-27T03:15:37.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.755+0000 7f237dffb640 1 --2- 172.21.15.67:0/4119619916 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f23680b3bd0 0x7f23680b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:37.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.755+0000 7f237ffff640 1 --2- 172.21.15.67:0/4119619916 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f23680b3bd0 0x7f23680b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:37.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.755+0000 7f237dffb640 1 -- 172.21.15.67:0/4119619916 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f23700e41d0 con 0x7f2380101cb0 2024-07-27T03:15:37.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.756+0000 7f237ffff640 1 --2- 172.21.15.67:0/4119619916 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f23680b3bd0 0x7f23680b60e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2380104620 tx=0x7f2374005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:37.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.758+0000 7f237dffb640 1 -- 172.21.15.67:0/4119619916 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f23700ab410 con 0x7f2380101cb0 2024-07-27T03:15:37.983 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.981+0000 7f23871d6640 1 -- 172.21.15.67:0/4119619916 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f238019b090 con 0x7f2380101cb0 2024-07-27T03:15:37.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.981+0000 7f237dffb640 1 -- 172.21.15.67:0/4119619916 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11983 (secure 0 0 0) 0x7f23700aad10 con 0x7f2380101cb0 2024-07-27T03:15:37.985 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:15:37.985 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":29,"btime":"2024-07-27T03:13:42:247804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14679,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/252686369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":252686369},{"type":"v1","addr":"172.21.15.67:6835","nonce":252686369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/3358832762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3358832762},{"type":"v1","addr":"172.21.15.67:6837","nonce":3358832762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-27T03:12:34.320208+0000","modified":"2024-07-27T03:13:42.062021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24620,"mds_1":24587,"mds_2":24583,"mds_3":24581,"mds_4":24563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14736":{"gid":14736,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/2162708317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2162708317},{"type":"v1","addr":"172.21.15.67:6839","nonce":2162708317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14748":{"gid":14748,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/795704244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":795704244},{"type":"v1","addr":"172.21.15.67:6841","nonce":795704244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/3868325215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3868325215},{"type":"v1","addr":"172.21.15.149:6833","nonce":3868325215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2462749027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2462749027},{"type":"v1","addr":"172.21.15.175:6835","nonce":2462749027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.149:6835/3730964806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3730964806},{"type":"v1","addr":"172.21.15.149:6835","nonce":3730964806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.175:6837/1304803502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1304803502},{"type":"v1","addr":"172.21.15.175:6837","nonce":1304803502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.175:6839/1670568853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1670568853},{"type":"v1","addr":"172.21.15.175:6839","nonce":1670568853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24587":{"gid":24587,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.149:6837/2932951845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2932951845},{"type":"v1","addr":"172.21.15.149:6837","nonce":2932951845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.149:6839/281746498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":281746498},{"type":"v1","addr":"172.21.15.149:6839","nonce":281746498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24620,"qdb_cluster":[24563,24581,24620,24583,24587]},"id":1}]} 2024-07-27T03:15:37.985 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-07-27T03:15:37.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.983+0000 7f23871d6640 1 -- 172.21.15.67:0/4119619916 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f23680b3bd0 msgr2=0x7f23680b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:37.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.983+0000 7f23871d6640 1 --2- 172.21.15.67:0/4119619916 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f23680b3bd0 0x7f23680b60e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2380104620 tx=0x7f2374005f90 comp rx=0 tx=0).stop 2024-07-27T03:15:37.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.983+0000 7f23871d6640 1 -- 172.21.15.67:0/4119619916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2380101cb0 msgr2=0x7f2380199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:37.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.983+0000 7f23871d6640 1 --2- 172.21.15.67:0/4119619916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2380101cb0 0x7f2380199f60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f237000ddc0 tx=0x7f237000a620 comp rx=0 tx=0).stop 2024-07-27T03:15:37.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.984+0000 7f23871d6640 1 -- 172.21.15.67:0/4119619916 shutdown_connections 2024-07-27T03:15:37.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.984+0000 7f23871d6640 1 --2- 172.21.15.67:0/4119619916 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f23680b3bd0 0x7f23680b60e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:37.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.984+0000 7f23871d6640 1 --2- 172.21.15.67:0/4119619916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2380107ef0 0x7f238019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:37.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.984+0000 7f23871d6640 1 --2- 172.21.15.67:0/4119619916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23801075a0 0x7f238019a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:37.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.984+0000 7f23871d6640 1 --2- 172.21.15.67:0/4119619916 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2380101cb0 0x7f2380199f60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:37.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.984+0000 7f23871d6640 1 -- 172.21.15.67:0/4119619916 >> 172.21.15.67:0/4119619916 conn(0x7f2380077630 msgr2=0x7f2380105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:37.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.984+0000 7f23871d6640 1 -- 172.21.15.67:0/4119619916 shutdown_connections 2024-07-27T03:15:37.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:37.984+0000 7f23871d6640 1 -- 172.21.15.67:0/4119619916 wait complete. 2024-07-27T03:15:38.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:38 smithi067 ceph-mon[32814]: pgmap v402: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-07-27T03:15:38.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:38 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/4119619916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:15:38.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:38 smithi149 ceph-mon[37173]: pgmap v402: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-07-27T03:15:38.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:38 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/4119619916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:15:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:38 smithi175 ceph-mon[37125]: pgmap v402: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-07-27T03:15:38.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:38 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/4119619916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:15:38.767 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-07-27T03:12:34.320208+0000', 'modified': '2024-07-27T03:13:42.062021+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24620, 'mds_1': 24587, 'mds_2': 24583, 'mds_3': 24581, 'mds_4': 24563}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14736': {'gid': 14736, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.67:6839/2162708317', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6838', 'nonce': 2162708317}, {'type': 'v1', 'addr': '172.21.15.67:6839', 'nonce': 2162708317}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14748': {'gid': 14748, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.67:6841/795704244', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6840', 'nonce': 795704244}, {'type': 'v1', 'addr': '172.21.15.67:6841', 'nonce': 795704244}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24526': {'gid': 24526, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6833/3804638701', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6832', 'nonce': 3804638701}, {'type': 'v1', 'addr': '172.21.15.175:6833', 'nonce': 3804638701}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24533': {'gid': 24533, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6833/3868325215', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6832', 'nonce': 3868325215}, {'type': 'v1', 'addr': '172.21.15.149:6833', 'nonce': 3868325215}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24562': {'gid': 24562, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6835/2462749027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 2462749027}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 2462749027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24563': {'gid': 24563, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.149:6835/3730964806', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6834', 'nonce': 3730964806}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 3730964806}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24581': {'gid': 24581, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.175:6837/1304803502', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 1304803502}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 1304803502}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24583': {'gid': 24583, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.175:6839/1670568853', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 1670568853}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 1670568853}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24587': {'gid': 24587, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.149:6837/2932951845', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 2932951845}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 2932951845}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24620': {'gid': 24620, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.149:6839/281746498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 281746498}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 281746498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24620, 'qdb_cluster': [24563, 24581, 24620, 24583, 24587]} 2024-07-27T03:15:38.767 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-07-27T03:15:38.768 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.e status 2024-07-27T03:15:39.033 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:15:40.461 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:40 smithi067 ceph-mon[32814]: pgmap v403: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-07-27T03:15:40.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:40 smithi149 ceph-mon[37173]: pgmap v403: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-07-27T03:15:40.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:40 smithi175 ceph-mon[37125]: pgmap v403: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 511 B/s wr, 9 op/s 2024-07-27T03:15:40.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.868+0000 7f3943d36640 1 -- 172.21.15.67:0/3322946947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f393c108760 msgr2=0x7f393c10ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:40.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.868+0000 7f3943d36640 1 --2- 172.21.15.67:0/3322946947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f393c108760 0x7f393c10ef90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3934009ca0 tx=0x7f393402a8c0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.868+0000 7f3943d36640 1 -- 172.21.15.67:0/3322946947 shutdown_connections 2024-07-27T03:15:40.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.868+0000 7f3943d36640 1 --2- 172.21.15.67:0/3322946947 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f393c108760 0x7f393c10ef90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.868+0000 7f3943d36640 1 --2- 172.21.15.67:0/3322946947 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f393c107e10 0x7f393c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.868+0000 7f3943d36640 1 --2- 172.21.15.67:0/3322946947 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f393c1024d0 0x7f393c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.868+0000 7f3943d36640 1 -- 172.21.15.67:0/3322946947 >> 172.21.15.67:0/3322946947 conn(0x7f393c0791a0 msgr2=0x7f393c0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:40.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.868+0000 7f3943d36640 1 -- 172.21.15.67:0/3322946947 shutdown_connections 2024-07-27T03:15:40.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.868+0000 7f3943d36640 1 -- 172.21.15.67:0/3322946947 wait complete. 2024-07-27T03:15:40.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.869+0000 7f3943d36640 1 Processor -- start 2024-07-27T03:15:40.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.869+0000 7f3943d36640 1 -- start start 2024-07-27T03:15:40.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.869+0000 7f3943d36640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f393c1024d0 0x7f393c198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:40.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.869+0000 7f3943d36640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f393c107e10 0x7f393c199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:40.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.869+0000 7f3943d36640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f393c108760 0x7f393c19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:40.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.869+0000 7f3943d36640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f393c1123c0 con 0x7f393c107e10 2024-07-27T03:15:40.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.869+0000 7f3943d36640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f393c112240 con 0x7f393c1024d0 2024-07-27T03:15:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.869+0000 7f3943d36640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f393c112540 con 0x7f393c108760 2024-07-27T03:15:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.869+0000 7f39412d1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f393c107e10 0x7f393c199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.869+0000 7f39412d1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f393c107e10 0x7f393c199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52236/0 (socket says 172.21.15.67:52236) 2024-07-27T03:15:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.869+0000 7f39412d1640 1 -- 172.21.15.67:0/1079144287 learned_addr learned my addr 172.21.15.67:0/1079144287 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.869+0000 7f3941ad2640 1 --2- 172.21.15.67:0/1079144287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f393c1024d0 0x7f393c198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.870+0000 7f39422d3640 1 --2- 172.21.15.67:0/1079144287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f393c108760 0x7f393c19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.870+0000 7f39412d1640 1 -- 172.21.15.67:0/1079144287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f393c108760 msgr2=0x7f393c19e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.870+0000 7f39412d1640 1 --2- 172.21.15.67:0/1079144287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f393c108760 0x7f393c19e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.870+0000 7f39412d1640 1 -- 172.21.15.67:0/1079144287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f393c1024d0 msgr2=0x7f393c198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.870+0000 7f39412d1640 1 --2- 172.21.15.67:0/1079144287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f393c1024d0 0x7f393c198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.870+0000 7f39412d1640 1 -- 172.21.15.67:0/1079144287 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393c19ede0 con 0x7f393c107e10 2024-07-27T03:15:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.870+0000 7f39412d1640 1 --2- 172.21.15.67:0/1079144287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f393c107e10 0x7f393c199410 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f392c009860 tx=0x7f392c009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:40.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.870+0000 7f3932ffd640 1 -- 172.21.15.67:0/1079144287 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c01d070 con 0x7f393c107e10 2024-07-27T03:15:40.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.870+0000 7f3943d36640 1 -- 172.21.15.67:0/1079144287 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f393c19f0d0 con 0x7f393c107e10 2024-07-27T03:15:40.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.870+0000 7f3932ffd640 1 -- 172.21.15.67:0/1079144287 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f392c007c50 con 0x7f393c107e10 2024-07-27T03:15:40.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.870+0000 7f3932ffd640 1 -- 172.21.15.67:0/1079144287 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c018d30 con 0x7f393c107e10 2024-07-27T03:15:40.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.870+0000 7f3943d36640 1 -- 172.21.15.67:0/1079144287 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f393c06a8d0 con 0x7f393c107e10 2024-07-27T03:15:40.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.873+0000 7f3932ffd640 1 -- 172.21.15.67:0/1079144287 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f392c020070 con 0x7f393c107e10 2024-07-27T03:15:40.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.873+0000 7f394a2bb740 1 Processor -- start 2024-07-27T03:15:40.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.873+0000 7f394a2bb740 1 -- start start 2024-07-27T03:15:40.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.873+0000 7f394a2bb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb45dd3f60 0x55fb46179220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:40.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.873+0000 7f394a2bb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45dab0a0 0x55fb46179760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:40.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.873+0000 7f394a2bb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb45db3280 0x55fb46179ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:40.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.873+0000 7f394a2bb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fb460612d0 con 0x55fb45dab0a0 2024-07-27T03:15:40.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.873+0000 7f394a2bb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fb460ef8b0 con 0x55fb45dd3f60 2024-07-27T03:15:40.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.873+0000 7f394a2bb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fb460dedb0 con 0x55fb45db3280 2024-07-27T03:15:40.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.874+0000 7f3941ad2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45dab0a0 0x55fb46179760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:40.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.874+0000 7f3941ad2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45dab0a0 0x55fb46179760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52244/0 (socket says 172.21.15.67:52244) 2024-07-27T03:15:40.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.874+0000 7f3941ad2640 1 -- 172.21.15.67:0/212987074 learned_addr learned my addr 172.21.15.67:0/212987074 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:40.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.874+0000 7f3932ffd640 1 --2- 172.21.15.67:0/1079144287 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f39280b3bd0 0x7f39280b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:40.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.874+0000 7f39422d3640 1 --2- 172.21.15.67:0/212987074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb45dd3f60 0x55fb46179220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:40.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.874+0000 7f3941ad2640 1 -- 172.21.15.67:0/212987074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb45db3280 msgr2=0x55fb46179ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:40.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.874+0000 7f3941ad2640 1 --2- 172.21.15.67:0/212987074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb45db3280 0x55fb46179ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.874+0000 7f3941ad2640 1 -- 172.21.15.67:0/212987074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb45dd3f60 msgr2=0x55fb46179220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:40.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.874+0000 7f3941ad2640 1 --2- 172.21.15.67:0/212987074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb45dd3f60 0x55fb46179220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.874+0000 7f3932ffd640 1 -- 172.21.15.67:0/1079144287 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f392c0e51f0 con 0x7f393c107e10 2024-07-27T03:15:40.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.874+0000 7f3941ad2640 1 -- 172.21.15.67:0/212987074 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb46118b10 con 0x55fb45dab0a0 2024-07-27T03:15:40.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.874+0000 7f39422d3640 1 --2- 172.21.15.67:0/212987074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb45dd3f60 0x55fb46179220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:15:40.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.874+0000 7f39422d3640 1 --2- 172.21.15.67:0/1079144287 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f39280b3bd0 0x7f39280b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:40.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.874+0000 7f3941ad2640 1 --2- 172.21.15.67:0/212987074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45dab0a0 0x55fb46179760 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f393800dcb0 tx=0x7f393800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=541a8c4a657809e7 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:40.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.875+0000 7f3927fff640 1 -- 172.21.15.67:0/212987074 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938011440 con 0x55fb45dab0a0 2024-07-27T03:15:40.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.875+0000 7f3927fff640 1 -- 172.21.15.67:0/212987074 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3938011a90 con 0x55fb45dab0a0 2024-07-27T03:15:40.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.875+0000 7f3927fff640 1 -- 172.21.15.67:0/212987074 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393801ba00 con 0x55fb45dab0a0 2024-07-27T03:15:40.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.876+0000 7f394a2bb740 1 -- 172.21.15.67:0/212987074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45dab0a0 msgr2=0x55fb46179760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:40.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.876+0000 7f39422d3640 1 --2- 172.21.15.67:0/1079144287 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f39280b3bd0 0x7f39280b60e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f393402f500 tx=0x7f3934005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:40.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.876+0000 7f394a2bb740 1 --2- 172.21.15.67:0/212987074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45dab0a0 0x55fb46179760 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f393800dcb0 tx=0x7f393800f730 comp rx=0 tx=0).stop 2024-07-27T03:15:40.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.876+0000 7f394a2bb740 1 -- 172.21.15.67:0/212987074 shutdown_connections 2024-07-27T03:15:40.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.876+0000 7f394a2bb740 1 --2- 172.21.15.67:0/212987074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb45db3280 0x55fb46179ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.876+0000 7f394a2bb740 1 --2- 172.21.15.67:0/212987074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45dab0a0 0x55fb46179760 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.876+0000 7f394a2bb740 1 --2- 172.21.15.67:0/212987074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb45dd3f60 0x55fb46179220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.876+0000 7f394a2bb740 1 -- 172.21.15.67:0/212987074 >> 172.21.15.67:0/212987074 conn(0x55fb45dc8350 msgr2=0x55fb46177110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:40.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.876+0000 7f394a2bb740 1 -- 172.21.15.67:0/212987074 shutdown_connections 2024-07-27T03:15:40.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.876+0000 7f394a2bb740 1 -- 172.21.15.67:0/212987074 wait complete. 2024-07-27T03:15:40.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.876+0000 7f394a2bb740 1 -- start start 2024-07-27T03:15:40.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.876+0000 7f394a2bb740 1 Processor -- start 2024-07-27T03:15:40.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.877+0000 7f394a2bb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb45dab0a0 0x55fb46184e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:40.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.877+0000 7f394a2bb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45db3280 0x55fb46185400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:40.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.877+0000 7f394a2bb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb45dd3f60 0x55fb46190b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:40.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.877+0000 7f394a2bb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fb460817d0 con 0x55fb45db3280 2024-07-27T03:15:40.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.877+0000 7f394a2bb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fb460e2df0 con 0x55fb45dab0a0 2024-07-27T03:15:40.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.877+0000 7f394a2bb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fb45f425f0 con 0x55fb45dd3f60 2024-07-27T03:15:40.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.877+0000 7f394a2bb740 10 client.0 _ll_register_callbacks cb 0x55fb46089800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:15:40.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.877+0000 7f39412d1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45db3280 0x55fb46185400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:40.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.877+0000 7f39412d1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45db3280 0x55fb46185400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52252/0 (socket says 172.21.15.67:52252) 2024-07-27T03:15:40.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.877+0000 7f39412d1640 1 -- 172.21.15.67:0/2348288764 learned_addr learned my addr 172.21.15.67:0/2348288764 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:40.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.877+0000 7f39422d3640 1 --2- 172.21.15.67:0/2348288764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb45dd3f60 0x55fb46190b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:40.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.877+0000 7f3941ad2640 1 --2- 172.21.15.67:0/2348288764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb45dab0a0 0x55fb46184e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:40.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.877+0000 7f39412d1640 1 -- 172.21.15.67:0/2348288764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb45dd3f60 msgr2=0x55fb46190b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:40.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.877+0000 7f39412d1640 1 --2- 172.21.15.67:0/2348288764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb45dd3f60 0x55fb46190b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.877+0000 7f39412d1640 1 -- 172.21.15.67:0/2348288764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb45dab0a0 msgr2=0x55fb46184e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.877+0000 7f39412d1640 1 --2- 172.21.15.67:0/2348288764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb45dab0a0 0x55fb46184e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.877+0000 7f39412d1640 1 -- 172.21.15.67:0/2348288764 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb46074640 con 0x55fb45db3280 2024-07-27T03:15:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.878+0000 7f39412d1640 1 --2- 172.21.15.67:0/2348288764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45db3280 0x55fb46185400 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f392c007590 tx=0x7f392c0ab5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45cefc4718ee8fba server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.878+0000 7f39257fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:15:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.878+0000 7f39257fa640 1 -- 172.21.15.67:0/2348288764 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c007e00 con 0x55fb45db3280 2024-07-27T03:15:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.878+0000 7f39257fa640 1 -- 172.21.15.67:0/2348288764 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f392c0a8380 con 0x55fb45db3280 2024-07-27T03:15:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.878+0000 7f394a2bb740 1 -- 172.21.15.67:0/2348288764 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3938011a90 con 0x55fb45db3280 2024-07-27T03:15:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.878+0000 7f39257fa640 1 -- 172.21.15.67:0/2348288764 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f392c0abd90 con 0x55fb45db3280 2024-07-27T03:15:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.878+0000 7f394a2bb740 10 client.15204 fetch_fsmap learned FSMap version 29 2024-07-27T03:15:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.878+0000 7f394a2bb740 1 -- 172.21.15.67:0/2348288764 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fb46085ac0 con 0x55fb45db3280 2024-07-27T03:15:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.878+0000 7f39257fa640 1 -- 172.21.15.67:0/2348288764 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3938011a90 con 0x55fb45db3280 2024-07-27T03:15:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.878+0000 7f39257fa640 1 -- 172.21.15.67:0/2348288764 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f392c0077d0 con 0x55fb45db3280 2024-07-27T03:15:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.878+0000 7f394a2bb740 10 client.15204 fetch_fsmap finished waiting for FSMap version 29 2024-07-27T03:15:40.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.879+0000 7f394a2bb740 10 client.15204 resolve_mds: resolved name 'e' to daemon mds.e 2024-07-27T03:15:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.879+0000 7f394a2bb740 20 client.15204 populate_metadata read hostname 'smithi067' 2024-07-27T03:15:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.879+0000 7f394a2bb740 1 --2- 172.21.15.67:0/2348288764 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55fb45d928e0 0x55fb4619b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.879+0000 7f394a2bb740 4 client.15204 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:15:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.879+0000 7f394a2bb740 1 -- 172.21.15.67:0/2348288764 --> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb460f1800 con 0x55fb45d928e0 2024-07-27T03:15:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.879+0000 7f3941ad2640 1 --2- 172.21.15.67:0/2348288764 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55fb45d928e0 0x55fb4619b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.879+0000 7f3941ad2640 1 --2- 172.21.15.67:0/2348288764 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55fb45d928e0 0x55fb4619b4e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a6c1073cfa8283a server_cookie=8c14fd923bd55d6f in_seq=0 out_seq=0 2024-07-27T03:15:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.879+0000 7f39257fa640 10 client.15204 ms_handle_connect on v2:172.21.15.149:6834/3730964806 2024-07-27T03:15:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.880+0000 7f39257fa640 1 -- 172.21.15.67:0/2348288764 <== mds.4 v2:172.21.15.149:6834/3730964806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb460f1800 con 0x55fb45d928e0 2024-07-27T03:15:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.880+0000 7f39257fa640 10 client.15204 handle_command_reply: tid=0 2024-07-27T03:15:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.880+0000 7f394a2bb740 1 client.15204 shutdown 2024-07-27T03:15:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.881+0000 7f394a2bb740 1 -- 172.21.15.67:0/2348288764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45db3280 msgr2=0x55fb46185400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.881+0000 7f394a2bb740 1 --2- 172.21.15.67:0/2348288764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45db3280 0x55fb46185400 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f392c007590 tx=0x7f392c0ab5c0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.881+0000 7f394a2bb740 1 -- 172.21.15.67:0/2348288764 shutdown_connections 2024-07-27T03:15:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.881+0000 7f394a2bb740 1 --2- 172.21.15.67:0/2348288764 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55fb45d928e0 0x55fb4619b4e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.881+0000 7f394a2bb740 1 --2- 172.21.15.67:0/2348288764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb45dd3f60 0x55fb46190b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.881+0000 7f394a2bb740 1 --2- 172.21.15.67:0/2348288764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45db3280 0x55fb46185400 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.881+0000 7f394a2bb740 1 --2- 172.21.15.67:0/2348288764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb45dab0a0 0x55fb46184e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.881+0000 7f39257fa640 0 client.15204 ms_handle_reset on v2:172.21.15.149:6834/3730964806 2024-07-27T03:15:40.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.881+0000 7f394a2bb740 1 -- 172.21.15.67:0/2348288764 >> 172.21.15.67:0/2348288764 conn(0x55fb45dc8350 msgr2=0x55fb4617c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:40.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.881+0000 7f394a2bb740 1 -- 172.21.15.67:0/2348288764 shutdown_connections 2024-07-27T03:15:40.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.881+0000 7f394a2bb740 1 -- 172.21.15.67:0/2348288764 wait complete. 2024-07-27T03:15:40.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.881+0000 7f394a2bb740 20 client.15204 trim_cache size 0 max 16384 2024-07-27T03:15:40.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.899+0000 7f394a2bb740 1 Processor -- start 2024-07-27T03:15:40.903 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.899+0000 7f394a2bb740 1 -- start start 2024-07-27T03:15:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.899+0000 7f394a2bb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb45dab0a0 0x55fb4617e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.899+0000 7f394a2bb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45db3280 0x55fb4617e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.899+0000 7f394a2bb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb45dd3f60 0x55fb4617fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.899+0000 7f394a2bb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fb46199820 con 0x55fb45db3280 2024-07-27T03:15:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.899+0000 7f394a2bb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fb46199ef0 con 0x55fb45dab0a0 2024-07-27T03:15:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.899+0000 7f394a2bb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fb4619a5c0 con 0x55fb45dd3f60 2024-07-27T03:15:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.899+0000 7f39412d1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45db3280 0x55fb4617e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.899+0000 7f39412d1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45db3280 0x55fb4617e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52264/0 (socket says 172.21.15.67:52264) 2024-07-27T03:15:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.899+0000 7f39412d1640 1 -- 172.21.15.67:0/1851947773 learned_addr learned my addr 172.21.15.67:0/1851947773 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.899+0000 7f3941ad2640 1 --2- 172.21.15.67:0/1851947773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb45dab0a0 0x55fb4617e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.900+0000 7f39422d3640 1 --2- 172.21.15.67:0/1851947773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb45dd3f60 0x55fb4617fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.900+0000 7f39412d1640 1 -- 172.21.15.67:0/1851947773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb45dd3f60 msgr2=0x55fb4617fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:40.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.900+0000 7f39412d1640 1 --2- 172.21.15.67:0/1851947773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb45dd3f60 0x55fb4617fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.900+0000 7f39412d1640 1 -- 172.21.15.67:0/1851947773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb45dab0a0 msgr2=0x55fb4617e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:40.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.900+0000 7f39412d1640 1 --2- 172.21.15.67:0/1851947773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb45dab0a0 0x55fb4617e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.900+0000 7f39412d1640 1 -- 172.21.15.67:0/1851947773 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb45e3f540 con 0x55fb45db3280 2024-07-27T03:15:40.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.900+0000 7f3941ad2640 1 --2- 172.21.15.67:0/1851947773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb45dab0a0 0x55fb4617e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:15:40.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.900+0000 7f39422d3640 1 --2- 172.21.15.67:0/1851947773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb45dd3f60 0x55fb4617fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:15:40.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.900+0000 7f39412d1640 1 --2- 172.21.15.67:0/1851947773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45db3280 0x55fb4617e9e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f392c0abf50 tx=0x7f392c0ab630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aceddcacb4b0cdef server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:40.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.900+0000 7f39277fe640 1 -- 172.21.15.67:0/1851947773 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c0b0390 con 0x55fb45db3280 2024-07-27T03:15:40.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.900+0000 7f39277fe640 1 -- 172.21.15.67:0/1851947773 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f392c0b4e40 con 0x55fb45db3280 2024-07-27T03:15:40.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.900+0000 7f39277fe640 1 -- 172.21.15.67:0/1851947773 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c007920 con 0x55fb45db3280 2024-07-27T03:15:40.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.901+0000 7f394a2bb740 1 -- 172.21.15.67:0/1851947773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45db3280 msgr2=0x55fb4617e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:40.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.901+0000 7f394a2bb740 1 --2- 172.21.15.67:0/1851947773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45db3280 0x55fb4617e9e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f392c0abf50 tx=0x7f392c0ab630 comp rx=0 tx=0).stop 2024-07-27T03:15:40.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.901+0000 7f394a2bb740 1 -- 172.21.15.67:0/1851947773 shutdown_connections 2024-07-27T03:15:40.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.901+0000 7f394a2bb740 1 --2- 172.21.15.67:0/1851947773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb45dd3f60 0x55fb4617fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.901+0000 7f394a2bb740 1 --2- 172.21.15.67:0/1851947773 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45db3280 0x55fb4617e9e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.901+0000 7f394a2bb740 1 --2- 172.21.15.67:0/1851947773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb45dab0a0 0x55fb4617e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.901+0000 7f394a2bb740 1 -- 172.21.15.67:0/1851947773 >> 172.21.15.67:0/1851947773 conn(0x55fb45dc8350 msgr2=0x55fb46172060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:40.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.901+0000 7f394a2bb740 1 -- 172.21.15.67:0/1851947773 shutdown_connections 2024-07-27T03:15:40.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.901+0000 7f394a2bb740 1 -- 172.21.15.67:0/1851947773 wait complete. 2024-07-27T03:15:40.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.901+0000 7f394a2bb740 1 -- start start 2024-07-27T03:15:40.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.901+0000 7f394a2bb740 1 Processor -- start 2024-07-27T03:15:40.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.902+0000 7f394a2bb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45dab0a0 0x55fb46190ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:40.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.902+0000 7f394a2bb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb45db3280 0x55fb4618fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:40.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.902+0000 7f394a2bb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb45dd3f60 0x55fb46190010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:40.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.902+0000 7f394a2bb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fb4619b360 con 0x55fb45dab0a0 2024-07-27T03:15:40.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.902+0000 7f394a2bb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fb4619ac90 con 0x55fb45db3280 2024-07-27T03:15:40.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.902+0000 7f394a2bb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fb4619ba30 con 0x55fb45dd3f60 2024-07-27T03:15:40.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.902+0000 7f394a2bb740 10 client.0 _ll_register_callbacks cb 0x7f392c0ab6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:15:40.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.902+0000 7f3941ad2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45dab0a0 0x55fb46190ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:40.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.902+0000 7f3941ad2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45dab0a0 0x55fb46190ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52270/0 (socket says 172.21.15.67:52270) 2024-07-27T03:15:40.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.902+0000 7f3941ad2640 1 -- 172.21.15.67:0/798776790 learned_addr learned my addr 172.21.15.67:0/798776790 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:40.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.902+0000 7f39412d1640 1 --2- 172.21.15.67:0/798776790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb45db3280 0x55fb4618fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:40.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.902+0000 7f3941ad2640 1 -- 172.21.15.67:0/798776790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb45dd3f60 msgr2=0x55fb46190010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:40.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.902+0000 7f3941ad2640 1 --2- 172.21.15.67:0/798776790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb45dd3f60 0x55fb46190010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.902+0000 7f3941ad2640 1 -- 172.21.15.67:0/798776790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb45db3280 msgr2=0x55fb4618fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:40.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.902+0000 7f3941ad2640 1 --2- 172.21.15.67:0/798776790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb45db3280 0x55fb4618fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.902+0000 7f3941ad2640 1 -- 172.21.15.67:0/798776790 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb46190550 con 0x55fb45dab0a0 2024-07-27T03:15:40.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.903+0000 7f3941ad2640 1 --2- 172.21.15.67:0/798776790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45dab0a0 0x55fb46190ed0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f393800fc30 tx=0x7f393800fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6fd51890602899 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:40.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.903+0000 7f39257fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:15:40.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.903+0000 7f39257fa640 1 -- 172.21.15.67:0/798776790 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938004250 con 0x55fb45dab0a0 2024-07-27T03:15:40.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.903+0000 7f39257fa640 1 -- 172.21.15.67:0/798776790 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f39380043f0 con 0x55fb45dab0a0 2024-07-27T03:15:40.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.903+0000 7f394a2bb740 1 -- 172.21.15.67:0/798776790 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f392c0b4e40 con 0x55fb45dab0a0 2024-07-27T03:15:40.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.903+0000 7f39257fa640 1 -- 172.21.15.67:0/798776790 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f393803b050 con 0x55fb45dab0a0 2024-07-27T03:15:40.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.903+0000 7f394a2bb740 10 client.15210 fetch_fsmap learned FSMap version 29 2024-07-27T03:15:40.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.903+0000 7f394a2bb740 1 -- 172.21.15.67:0/798776790 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fb461a3310 con 0x55fb45dab0a0 2024-07-27T03:15:40.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.903+0000 7f39257fa640 1 -- 172.21.15.67:0/798776790 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f392c0b4e40 con 0x55fb45dab0a0 2024-07-27T03:15:40.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.904+0000 7f39257fa640 1 -- 172.21.15.67:0/798776790 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f39380116e0 con 0x55fb45dab0a0 2024-07-27T03:15:40.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.904+0000 7f394a2bb740 10 client.15210 fetch_fsmap finished waiting for FSMap version 29 2024-07-27T03:15:40.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.904+0000 7f394a2bb740 10 client.15210 resolve_mds: resolved name 'e' to daemon mds.e 2024-07-27T03:15:40.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.904+0000 7f394a2bb740 20 client.15210 populate_metadata read hostname 'smithi067' 2024-07-27T03:15:40.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.904+0000 7f394a2bb740 1 --2- 172.21.15.67:0/798776790 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55fb45d67c90 0x55fb46206a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:40.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.904+0000 7f394a2bb740 4 client.15210 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-27T03:15:40.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.904+0000 7f394a2bb740 1 -- 172.21.15.67:0/798776790 --> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] -- command(tid 0: {"prefix": "status"}) -- 0x55fb460ed2a0 con 0x55fb45d67c90 2024-07-27T03:15:40.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.904+0000 7f39412d1640 1 --2- 172.21.15.67:0/798776790 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55fb45d67c90 0x55fb46206a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:40.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.904+0000 7f39257fa640 10 client.15210 ms_handle_connect on v2:172.21.15.149:6834/3730964806 2024-07-27T03:15:40.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.904+0000 7f39412d1640 1 --2- 172.21.15.67:0/798776790 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55fb45d67c90 0x55fb46206a90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=47c301e208568189 server_cookie=17e2a715e304e883 in_seq=0 out_seq=0 2024-07-27T03:15:40.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.905+0000 7f39257fa640 1 -- 172.21.15.67:0/798776790 <== mds.4 v2:172.21.15.149:6834/3730964806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55fb460ed2a0 con 0x55fb45d67c90 2024-07-27T03:15:40.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.905+0000 7f39257fa640 10 client.15210 handle_command_reply: tid=0 2024-07-27T03:15:40.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.905+0000 7f394a2bb740 1 client.15210 shutdown 2024-07-27T03:15:40.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.905+0000 7f394a2bb740 1 -- 172.21.15.67:0/798776790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45dab0a0 msgr2=0x55fb46190ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:40.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.905+0000 7f394a2bb740 1 --2- 172.21.15.67:0/798776790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45dab0a0 0x55fb46190ed0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f393800fc30 tx=0x7f393800fd60 comp rx=0 tx=0).stop 2024-07-27T03:15:40.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.905+0000 7f394a2bb740 1 -- 172.21.15.67:0/798776790 shutdown_connections 2024-07-27T03:15:40.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.905+0000 7f394a2bb740 1 --2- 172.21.15.67:0/798776790 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55fb45d67c90 0x55fb46206a90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.905+0000 7f394a2bb740 1 --2- 172.21.15.67:0/798776790 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb45dd3f60 0x55fb46190010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.905+0000 7f394a2bb740 1 --2- 172.21.15.67:0/798776790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb45db3280 0x55fb4618fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.905+0000 7f394a2bb740 1 --2- 172.21.15.67:0/798776790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb45dab0a0 0x55fb46190ed0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.905+0000 7f39257fa640 0 client.15210 ms_handle_reset on v2:172.21.15.149:6834/3730964806 2024-07-27T03:15:40.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.905+0000 7f394a2bb740 1 -- 172.21.15.67:0/798776790 >> 172.21.15.67:0/798776790 conn(0x55fb45dc8350 msgr2=0x55fb46172060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:40.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.905+0000 7f394a2bb740 1 -- 172.21.15.67:0/798776790 shutdown_connections 2024-07-27T03:15:40.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.905+0000 7f394a2bb740 1 -- 172.21.15.67:0/798776790 wait complete. 2024-07-27T03:15:40.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.905+0000 7f394a2bb740 20 client.15210 trim_cache size 0 max 16384 2024-07-27T03:15:40.913 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:15:40.913 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "c0782a00-4bc4-11ef-bcc7-c7b262605968", 2024-07-27T03:15:40.913 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 4, 2024-07-27T03:15:40.913 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24563, 2024-07-27T03:15:40.913 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-27T03:15:40.913 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-27T03:15:40.914 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-27T03:15:40.914 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 121.667185116, 2024-07-27T03:15:40.914 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 29, 2024-07-27T03:15:40.914 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 91, 2024-07-27T03:15:40.914 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-27T03:15:40.914 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 220.49706728500001 2024-07-27T03:15:40.914 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:15:40.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.906+0000 7f3943d36640 1 -- 172.21.15.67:0/1079144287 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f39280b3bd0 msgr2=0x7f39280b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:40.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.906+0000 7f3943d36640 1 --2- 172.21.15.67:0/1079144287 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f39280b3bd0 0x7f39280b60e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f393402f500 tx=0x7f3934005d80 comp rx=0 tx=0).stop 2024-07-27T03:15:40.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.906+0000 7f3943d36640 1 -- 172.21.15.67:0/1079144287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f393c107e10 msgr2=0x7f393c199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:40.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.906+0000 7f3943d36640 1 --2- 172.21.15.67:0/1079144287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f393c107e10 0x7f393c199410 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f392c009860 tx=0x7f392c009d20 comp rx=0 tx=0).stop 2024-07-27T03:15:40.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.906+0000 7f3943d36640 1 -- 172.21.15.67:0/1079144287 shutdown_connections 2024-07-27T03:15:40.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.906+0000 7f3943d36640 1 --2- 172.21.15.67:0/1079144287 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f39280b3bd0 0x7f39280b60e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.906+0000 7f3943d36640 1 --2- 172.21.15.67:0/1079144287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f393c108760 0x7f393c19e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.906+0000 7f3943d36640 1 --2- 172.21.15.67:0/1079144287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f393c107e10 0x7f393c199410 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.906+0000 7f3943d36640 1 --2- 172.21.15.67:0/1079144287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f393c1024d0 0x7f393c198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:40.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.906+0000 7f3943d36640 1 -- 172.21.15.67:0/1079144287 >> 172.21.15.67:0/1079144287 conn(0x7f393c0791a0 msgr2=0x7f393c104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:40.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.906+0000 7f3943d36640 1 -- 172.21.15.67:0/1079144287 shutdown_connections 2024-07-27T03:15:40.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:40.906+0000 7f3943d36640 1 -- 172.21.15.67:0/1079144287 wait complete. 2024-07-27T03:15:41.694 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.c status 2024-07-27T03:15:41.969 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:15:42.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:42 smithi067 ceph-mon[32814]: pgmap v404: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-07-27T03:15:42.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:42 smithi149 ceph-mon[37173]: pgmap v404: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-07-27T03:15:42.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:42 smithi175 ceph-mon[37125]: pgmap v404: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-07-27T03:15:43.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.778+0000 7f0be2bf2640 1 -- 172.21.15.67:0/4074928334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bdc105550 msgr2=0x7f0bdc10efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:43.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.778+0000 7f0be2bf2640 1 --2- 172.21.15.67:0/4074928334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bdc105550 0x7f0bdc10efa0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009900 tx=0x7f0bc402a410 comp rx=0 tx=0).stop 2024-07-27T03:15:43.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.778+0000 7f0be2bf2640 1 -- 172.21.15.67:0/4074928334 shutdown_connections 2024-07-27T03:15:43.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.778+0000 7f0be2bf2640 1 --2- 172.21.15.67:0/4074928334 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bdc105550 0x7f0bdc10efa0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.778+0000 7f0be2bf2640 1 --2- 172.21.15.67:0/4074928334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bdc104c00 0x7f0bdc105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.778+0000 7f0be2bf2640 1 --2- 172.21.15.67:0/4074928334 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bdc102c30 0x7f0bdc103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.778+0000 7f0be2bf2640 1 -- 172.21.15.67:0/4074928334 >> 172.21.15.67:0/4074928334 conn(0x7f0bdc0776d0 msgr2=0x7f0bdc077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:43.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.778+0000 7f0be2bf2640 1 -- 172.21.15.67:0/4074928334 shutdown_connections 2024-07-27T03:15:43.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.778+0000 7f0be2bf2640 1 -- 172.21.15.67:0/4074928334 wait complete. 2024-07-27T03:15:43.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.779+0000 7f0be2bf2640 1 Processor -- start 2024-07-27T03:15:43.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.779+0000 7f0be2bf2640 1 -- start start 2024-07-27T03:15:43.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.779+0000 7f0be2bf2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bdc102c30 0x7f0bdc199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:43.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.779+0000 7f0be2bf2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bdc104c00 0x7f0bdc19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:43.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.779+0000 7f0be2bf2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bdc105550 0x7f0bdc19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:43.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.779+0000 7f0be2bf2640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc113510 con 0x7f0bdc105550 2024-07-27T03:15:43.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.779+0000 7f0be2bf2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc113390 con 0x7f0bdc104c00 2024-07-27T03:15:43.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.779+0000 7f0be2bf2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc113690 con 0x7f0bdc102c30 2024-07-27T03:15:43.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.780+0000 7f0be118f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bdc105550 0x7f0bdc19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:43.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.780+0000 7f0be118f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bdc105550 0x7f0bdc19f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52296/0 (socket says 172.21.15.67:52296) 2024-07-27T03:15:43.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.780+0000 7f0be118f640 1 -- 172.21.15.67:0/3675028889 learned_addr learned my addr 172.21.15.67:0/3675028889 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:43.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.780+0000 7f0be098e640 1 --2- 172.21.15.67:0/3675028889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bdc102c30 0x7f0bdc199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:43.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.780+0000 7f0bdbfff640 1 --2- 172.21.15.67:0/3675028889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bdc104c00 0x7f0bdc19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:43.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.780+0000 7f0be118f640 1 -- 172.21.15.67:0/3675028889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bdc102c30 msgr2=0x7f0bdc199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:43.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.780+0000 7f0be118f640 1 --2- 172.21.15.67:0/3675028889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bdc102c30 0x7f0bdc199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.780+0000 7f0be118f640 1 -- 172.21.15.67:0/3675028889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bdc104c00 msgr2=0x7f0bdc19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:43.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.780+0000 7f0be118f640 1 --2- 172.21.15.67:0/3675028889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bdc104c00 0x7f0bdc19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.780+0000 7f0be118f640 1 -- 172.21.15.67:0/3675028889 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bdc19fe90 con 0x7f0bdc105550 2024-07-27T03:15:43.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.780+0000 7f0be098e640 1 --2- 172.21.15.67:0/3675028889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bdc102c30 0x7f0bdc199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:15:43.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.780+0000 7f0be118f640 1 --2- 172.21.15.67:0/3675028889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bdc105550 0x7f0bdc19f760 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4004770 tx=0x7f0bc4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:43.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.780+0000 7f0bd9ffb640 1 -- 172.21.15.67:0/3675028889 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc403f030 con 0x7f0bdc105550 2024-07-27T03:15:43.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.780+0000 7f0bd9ffb640 1 -- 172.21.15.67:0/3675028889 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bc4003aa0 con 0x7f0bdc105550 2024-07-27T03:15:43.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.780+0000 7f0be2bf2640 1 -- 172.21.15.67:0/3675028889 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bdc1a0120 con 0x7f0bdc105550 2024-07-27T03:15:43.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.780+0000 7f0bd9ffb640 1 -- 172.21.15.67:0/3675028889 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4035070 con 0x7f0bdc105550 2024-07-27T03:15:43.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.781+0000 7f0be2bf2640 1 -- 172.21.15.67:0/3675028889 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bdc1a68e0 con 0x7f0bdc105550 2024-07-27T03:15:43.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.783+0000 7f0be9177740 1 Processor -- start 2024-07-27T03:15:43.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.783+0000 7f0bd9ffb640 1 -- 172.21.15.67:0/3675028889 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f0bc4042030 con 0x7f0bdc105550 2024-07-27T03:15:43.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.783+0000 7f0be9177740 1 -- start start 2024-07-27T03:15:43.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.784+0000 7f0be9177740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc93f60 0x55615d039200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:43.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.784+0000 7f0be9177740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55615cc6b0a0 0x55615d039740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:43.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.784+0000 7f0be9177740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55615cc73280 0x55615d039c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:43.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.784+0000 7f0be9177740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55615cf212b0 con 0x55615cc93f60 2024-07-27T03:15:43.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.784+0000 7f0bdbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc93f60 0x55615d039200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:43.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.784+0000 7f0bdbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc93f60 0x55615d039200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52308/0 (socket says 172.21.15.67:52308) 2024-07-27T03:15:43.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.784+0000 7f0bdbfff640 1 -- 172.21.15.67:0/3400342973 learned_addr learned my addr 172.21.15.67:0/3400342973 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:43.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.784+0000 7f0be9177740 1 -- 172.21.15.67:0/3400342973 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55615cfaf890 con 0x55615cc6b0a0 2024-07-27T03:15:43.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.784+0000 7f0be9177740 1 -- 172.21.15.67:0/3400342973 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55615cf9ed90 con 0x55615cc73280 2024-07-27T03:15:43.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.784+0000 7f0be098e640 1 --2- 172.21.15.67:0/3400342973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55615cc6b0a0 0x55615d039740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:43.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.784+0000 7f0bdbfff640 1 --2- 172.21.15.67:0/3400342973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55615cc73280 0x55615d039c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:43.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.785+0000 7f0bdbfff640 1 -- 172.21.15.67:0/3400342973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55615cc73280 msgr2=0x55615d039c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:43.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.785+0000 7f0bdbfff640 1 --2- 172.21.15.67:0/3400342973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55615cc73280 0x55615d039c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.785+0000 7f0bd9ffb640 1 --2- 172.21.15.67:0/3675028889 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0bc80b3bd0 0x7f0bc80b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:43.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.785+0000 7f0bdbfff640 1 -- 172.21.15.67:0/3400342973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55615cc6b0a0 msgr2=0x55615d039740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:43.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.785+0000 7f0bdbfff640 1 --2- 172.21.15.67:0/3400342973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55615cc6b0a0 0x55615d039740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.785+0000 7f0bdbfff640 1 -- 172.21.15.67:0/3400342973 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55615cfd8af0 con 0x55615cc93f60 2024-07-27T03:15:43.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.785+0000 7f0be098e640 1 --2- 172.21.15.67:0/3400342973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55615cc6b0a0 0x55615d039740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:15:43.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.785+0000 7f0be118f640 1 --2- 172.21.15.67:0/3675028889 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0bc80b3bd0 0x7f0bc80b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:43.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.785+0000 7f0bd9ffb640 1 -- 172.21.15.67:0/3675028889 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f0bc41047f0 con 0x7f0bdc105550 2024-07-27T03:15:43.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.785+0000 7f0bdbfff640 1 --2- 172.21.15.67:0/3400342973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc93f60 0x55615d039200 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00af50 tx=0x7f0bcc00a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819431a5ca405acc server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:43.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.785+0000 7f0bd2ffd640 1 -- 172.21.15.67:0/3400342973 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc00d730 con 0x55615cc93f60 2024-07-27T03:15:43.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.786+0000 7f0bd2ffd640 1 -- 172.21.15.67:0/3400342973 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bcc00dd80 con 0x55615cc93f60 2024-07-27T03:15:43.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.786+0000 7f0bd2ffd640 1 -- 172.21.15.67:0/3400342973 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc00f3d0 con 0x55615cc93f60 2024-07-27T03:15:43.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.786+0000 7f0be9177740 1 -- 172.21.15.67:0/3400342973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc93f60 msgr2=0x55615d039200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:43.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.786+0000 7f0be9177740 1 --2- 172.21.15.67:0/3400342973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc93f60 0x55615d039200 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00af50 tx=0x7f0bcc00a560 comp rx=0 tx=0).stop 2024-07-27T03:15:43.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.786+0000 7f0be118f640 1 --2- 172.21.15.67:0/3675028889 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0bc80b3bd0 0x7f0bc80b60e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4032fd0 tx=0x7f0bc402a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:43.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.787+0000 7f0be9177740 1 -- 172.21.15.67:0/3400342973 shutdown_connections 2024-07-27T03:15:43.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.787+0000 7f0be9177740 1 --2- 172.21.15.67:0/3400342973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55615cc73280 0x55615d039c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.787+0000 7f0be9177740 1 --2- 172.21.15.67:0/3400342973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55615cc6b0a0 0x55615d039740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.787+0000 7f0be9177740 1 --2- 172.21.15.67:0/3400342973 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc93f60 0x55615d039200 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.787+0000 7f0be9177740 1 -- 172.21.15.67:0/3400342973 >> 172.21.15.67:0/3400342973 conn(0x55615cc88350 msgr2=0x55615d0370f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:43.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.787+0000 7f0be9177740 1 -- 172.21.15.67:0/3400342973 shutdown_connections 2024-07-27T03:15:43.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.787+0000 7f0be9177740 1 -- 172.21.15.67:0/3400342973 wait complete. 2024-07-27T03:15:43.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.787+0000 7f0be9177740 1 -- start start 2024-07-27T03:15:43.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.787+0000 7f0be9177740 1 Processor -- start 2024-07-27T03:15:43.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.788+0000 7f0be9177740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55615cc6b0a0 0x55615d044e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:43.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.788+0000 7f0be9177740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc73280 0x55615d0453d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:43.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.788+0000 7f0be9177740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55615cc93f60 0x55615d054f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:43.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.788+0000 7f0be9177740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55615cf417b0 con 0x55615cc73280 2024-07-27T03:15:43.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.788+0000 7f0be9177740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55615cfa2dd0 con 0x55615cc93f60 2024-07-27T03:15:43.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.788+0000 7f0be9177740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55615ce025f0 con 0x55615cc6b0a0 2024-07-27T03:15:43.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.788+0000 7f0be9177740 10 client.0 _ll_register_callbacks cb 0x55615cf497e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:15:43.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.788+0000 7f0be098e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc73280 0x55615d0453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:43.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.788+0000 7f0be098e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc73280 0x55615d0453d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52318/0 (socket says 172.21.15.67:52318) 2024-07-27T03:15:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.788+0000 7f0be098e640 1 -- 172.21.15.67:0/2954129255 learned_addr learned my addr 172.21.15.67:0/2954129255 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.788+0000 7f0bdbfff640 1 --2- 172.21.15.67:0/2954129255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55615cc6b0a0 0x55615d044e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.788+0000 7f0bdbfff640 1 --2- 172.21.15.67:0/2954129255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55615cc93f60 0x55615d054f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.788+0000 7f0be098e640 1 -- 172.21.15.67:0/2954129255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55615cc6b0a0 msgr2=0x55615d044e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.788+0000 7f0be098e640 1 --2- 172.21.15.67:0/2954129255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55615cc6b0a0 0x55615d044e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.788+0000 7f0be098e640 1 -- 172.21.15.67:0/2954129255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55615cc93f60 msgr2=0x55615d054f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.788+0000 7f0be098e640 1 --2- 172.21.15.67:0/2954129255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55615cc93f60 0x55615d054f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.788+0000 7f0be098e640 1 -- 172.21.15.67:0/2954129255 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55615cf34620 con 0x55615cc73280 2024-07-27T03:15:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.789+0000 7f0be098e640 1 --2- 172.21.15.67:0/2954129255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc73280 0x55615d0453d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400bfb0 tx=0x7f0bd40128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaede2a023be1ec5 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.789+0000 7f0bbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:15:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.789+0000 7f0bbbfff640 1 -- 172.21.15.67:0/2954129255 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4004230 con 0x55615cc73280 2024-07-27T03:15:43.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.789+0000 7f0bbbfff640 1 -- 172.21.15.67:0/2954129255 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f0bd40043d0 con 0x55615cc73280 2024-07-27T03:15:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.789+0000 7f0be9177740 1 -- 172.21.15.67:0/2954129255 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bcc00dd80 con 0x55615cc73280 2024-07-27T03:15:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.789+0000 7f0bbbfff640 1 -- 172.21.15.67:0/2954129255 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0bd40107c0 con 0x55615cc73280 2024-07-27T03:15:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.789+0000 7f0be9177740 10 client.15222 fetch_fsmap learned FSMap version 29 2024-07-27T03:15:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.789+0000 7f0be9177740 1 -- 172.21.15.67:0/2954129255 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55615cf45aa0 con 0x55615cc73280 2024-07-27T03:15:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.789+0000 7f0bbbfff640 1 -- 172.21.15.67:0/2954129255 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bcc00dd80 con 0x55615cc73280 2024-07-27T03:15:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.790+0000 7f0bbbfff640 1 -- 172.21.15.67:0/2954129255 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f0bd4019070 con 0x55615cc73280 2024-07-27T03:15:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.790+0000 7f0be9177740 10 client.15222 fetch_fsmap finished waiting for FSMap version 29 2024-07-27T03:15:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.790+0000 7f0be9177740 10 client.15222 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-27T03:15:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.790+0000 7f0be9177740 20 client.15222 populate_metadata read hostname 'smithi067' 2024-07-27T03:15:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.790+0000 7f0be9177740 1 --2- 172.21.15.67:0/2954129255 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55615cc528e0 0x55615d05b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:43.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.790+0000 7f0be9177740 4 client.15222 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:15:43.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.790+0000 7f0be9177740 1 -- 172.21.15.67:0/2954129255 --> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55615cfb17e0 con 0x55615cc528e0 2024-07-27T03:15:43.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.790+0000 7f0bdbfff640 1 --2- 172.21.15.67:0/2954129255 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55615cc528e0 0x55615d05b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:43.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.791+0000 7f0bdbfff640 1 --2- 172.21.15.67:0/2954129255 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55615cc528e0 0x55615d05b660 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=672315b1c43fcbd8 server_cookie=5ea872770f45a4bd in_seq=0 out_seq=0 2024-07-27T03:15:43.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.791+0000 7f0bbbfff640 10 client.15222 ms_handle_connect on v2:172.21.15.175:6836/1304803502 2024-07-27T03:15:43.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.791+0000 7f0bbbfff640 1 -- 172.21.15.67:0/2954129255 <== mds.3 v2:172.21.15.175:6836/1304803502 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55615cfb17e0 con 0x55615cc528e0 2024-07-27T03:15:43.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.791+0000 7f0bbbfff640 10 client.15222 handle_command_reply: tid=0 2024-07-27T03:15:43.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.791+0000 7f0be9177740 1 client.15222 shutdown 2024-07-27T03:15:43.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.792+0000 7f0be9177740 1 -- 172.21.15.67:0/2954129255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc73280 msgr2=0x55615d0453d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:43.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.792+0000 7f0be9177740 1 --2- 172.21.15.67:0/2954129255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc73280 0x55615d0453d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400bfb0 tx=0x7f0bd40128e0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.792+0000 7f0be9177740 1 -- 172.21.15.67:0/2954129255 shutdown_connections 2024-07-27T03:15:43.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.792+0000 7f0be9177740 1 --2- 172.21.15.67:0/2954129255 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55615cc528e0 0x55615d05b660 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.802 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.792+0000 7f0be9177740 1 --2- 172.21.15.67:0/2954129255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55615cc93f60 0x55615d054f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.792+0000 7f0be9177740 1 --2- 172.21.15.67:0/2954129255 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc73280 0x55615d0453d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.792+0000 7f0bbbfff640 0 client.15222 ms_handle_reset on v2:172.21.15.175:6836/1304803502 2024-07-27T03:15:43.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.792+0000 7f0be9177740 1 --2- 172.21.15.67:0/2954129255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55615cc6b0a0 0x55615d044e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.792+0000 7f0be9177740 1 -- 172.21.15.67:0/2954129255 >> 172.21.15.67:0/2954129255 conn(0x55615cc88350 msgr2=0x55615d03c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:43.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.792+0000 7f0be9177740 1 -- 172.21.15.67:0/2954129255 shutdown_connections 2024-07-27T03:15:43.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.792+0000 7f0be9177740 1 -- 172.21.15.67:0/2954129255 wait complete. 2024-07-27T03:15:43.803 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.792+0000 7f0be9177740 20 client.15222 trim_cache size 0 max 16384 2024-07-27T03:15:43.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.810+0000 7f0be9177740 1 Processor -- start 2024-07-27T03:15:43.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.810+0000 7f0be9177740 1 -- start start 2024-07-27T03:15:43.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.810+0000 7f0be9177740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55615cc528e0 0x55615d05b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:43.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.810+0000 7f0be9177740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55615cc6b0a0 0x55615d054f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:43.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.810+0000 7f0be9177740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc73280 0x55615d044e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:43.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.810+0000 7f0be9177740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55615d0599a0 con 0x55615cc73280 2024-07-27T03:15:43.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.810+0000 7f0be9177740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55615d05a070 con 0x55615cc528e0 2024-07-27T03:15:43.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.810+0000 7f0be9177740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55615d05a740 con 0x55615cc6b0a0 2024-07-27T03:15:43.814 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.810+0000 7f0bdbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc73280 0x55615d044e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:43.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.810+0000 7f0bdbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc73280 0x55615d044e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52328/0 (socket says 172.21.15.67:52328) 2024-07-27T03:15:43.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.810+0000 7f0bdbfff640 1 -- 172.21.15.67:0/2568323110 learned_addr learned my addr 172.21.15.67:0/2568323110 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:43.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.811+0000 7f0bdbfff640 1 --2- 172.21.15.67:0/2568323110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55615cc528e0 0x55615d05b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:43.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.811+0000 7f0be098e640 1 --2- 172.21.15.67:0/2568323110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55615cc6b0a0 0x55615d054f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:43.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.811+0000 7f0bdbfff640 1 -- 172.21.15.67:0/2568323110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55615cc6b0a0 msgr2=0x55615d054f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:43.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.811+0000 7f0bdbfff640 1 --2- 172.21.15.67:0/2568323110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55615cc6b0a0 0x55615d054f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.811+0000 7f0bdbfff640 1 -- 172.21.15.67:0/2568323110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55615cc528e0 msgr2=0x55615d05b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:43.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.811+0000 7f0bdbfff640 1 --2- 172.21.15.67:0/2568323110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55615cc528e0 0x55615d05b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.811+0000 7f0bdbfff640 1 -- 172.21.15.67:0/2568323110 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55615ccff540 con 0x55615cc73280 2024-07-27T03:15:43.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.811+0000 7f0bdbfff640 1 --2- 172.21.15.67:0/2568323110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc73280 0x55615d044e30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc005cc0 tx=0x7f0bcc0109c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd662a86079474 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:43.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.811+0000 7f0bd27fc640 1 -- 172.21.15.67:0/2568323110 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc032070 con 0x55615cc73280 2024-07-27T03:15:43.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.811+0000 7f0bd27fc640 1 -- 172.21.15.67:0/2568323110 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bcc010470 con 0x55615cc73280 2024-07-27T03:15:43.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.811+0000 7f0bd27fc640 1 -- 172.21.15.67:0/2568323110 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc00dd60 con 0x55615cc73280 2024-07-27T03:15:43.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.812+0000 7f0be9177740 1 -- 172.21.15.67:0/2568323110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc73280 msgr2=0x55615d044e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:43.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.812+0000 7f0be9177740 1 --2- 172.21.15.67:0/2568323110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc73280 0x55615d044e30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc005cc0 tx=0x7f0bcc0109c0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.812+0000 7f0be9177740 1 -- 172.21.15.67:0/2568323110 shutdown_connections 2024-07-27T03:15:43.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.812+0000 7f0be9177740 1 --2- 172.21.15.67:0/2568323110 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc73280 0x55615d044e30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.812+0000 7f0be9177740 1 --2- 172.21.15.67:0/2568323110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55615cc6b0a0 0x55615d054f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.812+0000 7f0be9177740 1 --2- 172.21.15.67:0/2568323110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55615cc528e0 0x55615d05b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.812+0000 7f0be9177740 1 -- 172.21.15.67:0/2568323110 >> 172.21.15.67:0/2568323110 conn(0x55615cc88350 msgr2=0x55615d036a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:43.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.812+0000 7f0be9177740 1 -- 172.21.15.67:0/2568323110 shutdown_connections 2024-07-27T03:15:43.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.812+0000 7f0be9177740 1 -- 172.21.15.67:0/2568323110 wait complete. 2024-07-27T03:15:43.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.812+0000 7f0be9177740 1 -- start start 2024-07-27T03:15:43.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.813+0000 7f0be9177740 1 Processor -- start 2024-07-27T03:15:43.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.813+0000 7f0be9177740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55615cc528e0 0x55615d05c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:43.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.813+0000 7f0be9177740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55615cc6b0a0 0x55615d05cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:43.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.813+0000 7f0be9177740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc73280 0x55615d05d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:43.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.813+0000 7f0be9177740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55615d05b4e0 con 0x55615cc73280 2024-07-27T03:15:43.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.813+0000 7f0be9177740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55615d05ae10 con 0x55615cc6b0a0 2024-07-27T03:15:43.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.813+0000 7f0be9177740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55615d05bbb0 con 0x55615cc528e0 2024-07-27T03:15:43.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.813+0000 7f0be9177740 10 client.0 _ll_register_callbacks cb 0x7f0bd400b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:15:43.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.813+0000 7f0bdbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc73280 0x55615d05d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:43.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.813+0000 7f0bdbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc73280 0x55615d05d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52340/0 (socket says 172.21.15.67:52340) 2024-07-27T03:15:43.822 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.813+0000 7f0bdbfff640 1 -- 172.21.15.67:0/1499845391 learned_addr learned my addr 172.21.15.67:0/1499845391 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:43.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.813+0000 7f0bdbfff640 1 --2- 172.21.15.67:0/1499845391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55615cc528e0 0x55615d05c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:43.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.813+0000 7f0be098e640 1 --2- 172.21.15.67:0/1499845391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55615cc6b0a0 0x55615d05cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:43.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.813+0000 7f0bdbfff640 1 -- 172.21.15.67:0/1499845391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55615cc528e0 msgr2=0x55615d05c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:43.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.813+0000 7f0bdbfff640 1 --2- 172.21.15.67:0/1499845391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55615cc528e0 0x55615d05c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.813+0000 7f0bdbfff640 1 -- 172.21.15.67:0/1499845391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55615cc6b0a0 msgr2=0x55615d05cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:43.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.814+0000 7f0bdbfff640 1 --2- 172.21.15.67:0/1499845391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55615cc6b0a0 0x55615d05cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.814+0000 7f0bdbfff640 1 -- 172.21.15.67:0/1499845391 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55615d0c28f0 con 0x55615cc73280 2024-07-27T03:15:43.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.814+0000 7f0be098e640 1 --2- 172.21.15.67:0/1499845391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55615cc6b0a0 0x55615d05cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:15:43.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.814+0000 7f0bdbfff640 1 --2- 172.21.15.67:0/1499845391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc73280 0x55615d05d2e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00a440 tx=0x7f0bcc020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4854712b1e530838 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:43.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.814+0000 7f0bbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:15:43.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.814+0000 7f0bbbfff640 1 -- 172.21.15.67:0/1499845391 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc03b030 con 0x55615cc73280 2024-07-27T03:15:43.823 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.814+0000 7f0bbbfff640 1 -- 172.21.15.67:0/1499845391 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f0bcc01a040 con 0x55615cc73280 2024-07-27T03:15:43.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.814+0000 7f0be9177740 1 -- 172.21.15.67:0/1499845391 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bcc010470 con 0x55615cc73280 2024-07-27T03:15:43.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.814+0000 7f0bbbfff640 1 -- 172.21.15.67:0/1499845391 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0bcc045050 con 0x55615cc73280 2024-07-27T03:15:43.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.814+0000 7f0be9177740 10 client.15228 fetch_fsmap learned FSMap version 29 2024-07-27T03:15:43.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.814+0000 7f0be9177740 1 -- 172.21.15.67:0/1499845391 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55615d0c3b10 con 0x55615cc73280 2024-07-27T03:15:43.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.814+0000 7f0bbbfff640 1 -- 172.21.15.67:0/1499845391 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bcc010470 con 0x55615cc73280 2024-07-27T03:15:43.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.815+0000 7f0bbbfff640 1 -- 172.21.15.67:0/1499845391 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f0bcc02b820 con 0x55615cc73280 2024-07-27T03:15:43.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.815+0000 7f0be9177740 10 client.15228 fetch_fsmap finished waiting for FSMap version 29 2024-07-27T03:15:43.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.815+0000 7f0be9177740 10 client.15228 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-27T03:15:43.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.815+0000 7f0be9177740 20 client.15228 populate_metadata read hostname 'smithi067' 2024-07-27T03:15:43.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.815+0000 7f0be9177740 1 --2- 172.21.15.67:0/1499845391 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55615cc93f60 0x55615d0c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:43.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.815+0000 7f0be9177740 4 client.15228 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-27T03:15:43.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.815+0000 7f0be9177740 1 -- 172.21.15.67:0/1499845391 --> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] -- command(tid 0: {"prefix": "status"}) -- 0x55615cfad280 con 0x55615cc93f60 2024-07-27T03:15:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.815+0000 7f0be098e640 1 --2- 172.21.15.67:0/1499845391 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55615cc93f60 0x55615d0c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.815+0000 7f0be098e640 1 --2- 172.21.15.67:0/1499845391 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55615cc93f60 0x55615d0c6dc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bc4efde5dac4e4d5 server_cookie=9613557ac6570c0a in_seq=0 out_seq=0 2024-07-27T03:15:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.815+0000 7f0bbbfff640 10 client.15228 ms_handle_connect on v2:172.21.15.175:6836/1304803502 2024-07-27T03:15:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.816+0000 7f0bbbfff640 1 -- 172.21.15.67:0/1499845391 <== mds.3 v2:172.21.15.175:6836/1304803502 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55615cfad280 con 0x55615cc93f60 2024-07-27T03:15:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.816+0000 7f0bbbfff640 10 client.15228 handle_command_reply: tid=0 2024-07-27T03:15:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.816+0000 7f0be9177740 1 client.15228 shutdown 2024-07-27T03:15:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.816+0000 7f0be9177740 1 -- 172.21.15.67:0/1499845391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc73280 msgr2=0x55615d05d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.816+0000 7f0be9177740 1 --2- 172.21.15.67:0/1499845391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc73280 0x55615d05d2e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00a440 tx=0x7f0bcc020040 comp rx=0 tx=0).stop 2024-07-27T03:15:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.816+0000 7f0be9177740 1 -- 172.21.15.67:0/1499845391 shutdown_connections 2024-07-27T03:15:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.816+0000 7f0be9177740 1 --2- 172.21.15.67:0/1499845391 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55615cc93f60 0x55615d0c6dc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.816+0000 7f0be9177740 1 --2- 172.21.15.67:0/1499845391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55615cc73280 0x55615d05d2e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.816+0000 7f0bbbfff640 0 client.15228 ms_handle_reset on v2:172.21.15.175:6836/1304803502 2024-07-27T03:15:43.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.816+0000 7f0be9177740 1 --2- 172.21.15.67:0/1499845391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55615cc6b0a0 0x55615d05cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.816+0000 7f0be9177740 1 --2- 172.21.15.67:0/1499845391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55615cc528e0 0x55615d05c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.816+0000 7f0be9177740 1 -- 172.21.15.67:0/1499845391 >> 172.21.15.67:0/1499845391 conn(0x55615cc88350 msgr2=0x55615d046100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:43.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.816+0000 7f0be9177740 1 -- 172.21.15.67:0/1499845391 shutdown_connections 2024-07-27T03:15:43.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.816+0000 7f0be9177740 1 -- 172.21.15.67:0/1499845391 wait complete. 2024-07-27T03:15:43.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.816+0000 7f0be9177740 20 client.15228 trim_cache size 0 max 16384 2024-07-27T03:15:43.826 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:15:43.826 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "c0782a00-4bc4-11ef-bcc7-c7b262605968", 2024-07-27T03:15:43.826 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 3, 2024-07-27T03:15:43.827 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24581, 2024-07-27T03:15:43.827 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-27T03:15:43.827 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-27T03:15:43.827 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-27T03:15:43.827 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 125.594890112, 2024-07-27T03:15:43.827 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 29, 2024-07-27T03:15:43.827 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 91, 2024-07-27T03:15:43.827 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-27T03:15:43.827 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 218.26745343100001 2024-07-27T03:15:43.827 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:15:43.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.817+0000 7f0be2bf2640 1 -- 172.21.15.67:0/3675028889 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0bc80b3bd0 msgr2=0x7f0bc80b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:43.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.817+0000 7f0be2bf2640 1 --2- 172.21.15.67:0/3675028889 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0bc80b3bd0 0x7f0bc80b60e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4032fd0 tx=0x7f0bc402a410 comp rx=0 tx=0).stop 2024-07-27T03:15:43.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.817+0000 7f0be2bf2640 1 -- 172.21.15.67:0/3675028889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bdc105550 msgr2=0x7f0bdc19f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:43.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.817+0000 7f0be2bf2640 1 --2- 172.21.15.67:0/3675028889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bdc105550 0x7f0bdc19f760 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4004770 tx=0x7f0bc4004270 comp rx=0 tx=0).stop 2024-07-27T03:15:43.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.817+0000 7f0be2bf2640 1 -- 172.21.15.67:0/3675028889 shutdown_connections 2024-07-27T03:15:43.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.817+0000 7f0be2bf2640 1 --2- 172.21.15.67:0/3675028889 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0bc80b3bd0 0x7f0bc80b60e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.817+0000 7f0be2bf2640 1 --2- 172.21.15.67:0/3675028889 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bdc105550 0x7f0bdc19f760 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.817+0000 7f0be2bf2640 1 --2- 172.21.15.67:0/3675028889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bdc104c00 0x7f0bdc19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.817+0000 7f0be2bf2640 1 --2- 172.21.15.67:0/3675028889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bdc102c30 0x7f0bdc199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:43.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.817+0000 7f0be2bf2640 1 -- 172.21.15.67:0/3675028889 >> 172.21.15.67:0/3675028889 conn(0x7f0bdc0776d0 msgr2=0x7f0bdc105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:43.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.817+0000 7f0be2bf2640 1 -- 172.21.15.67:0/3675028889 shutdown_connections 2024-07-27T03:15:43.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:43.817+0000 7f0be2bf2640 1 -- 172.21.15.67:0/3675028889 wait complete. 2024-07-27T03:15:44.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:44 smithi149 ceph-mon[37173]: pgmap v405: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-07-27T03:15:44.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:44 smithi175 ceph-mon[37125]: pgmap v405: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-07-27T03:15:44.627 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:44 smithi067 ceph-mon[32814]: pgmap v405: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-07-27T03:15:44.659 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.i status 2024-07-27T03:15:44.932 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:15:46.330 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:46 smithi067 ceph-mon[32814]: pgmap v406: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:46.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:46 smithi149 ceph-mon[37173]: pgmap v406: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:46.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.555+0000 7fe740660640 1 -- 172.21.15.67:0/4260592702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe738108f10 msgr2=0x7fe738109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:46.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.555+0000 7fe740660640 1 --2- 172.21.15.67:0/4260592702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe738108f10 0x7fe738109320 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe728009900 tx=0x7fe72802a3f0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.555+0000 7fe740660640 1 -- 172.21.15.67:0/4260592702 shutdown_connections 2024-07-27T03:15:46.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.555+0000 7fe740660640 1 --2- 172.21.15.67:0/4260592702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe738109860 0x7fe738110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.555+0000 7fe740660640 1 --2- 172.21.15.67:0/4260592702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe738108f10 0x7fe738109320 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.555+0000 7fe740660640 1 --2- 172.21.15.67:0/4260592702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7381035d0 0x7fe7381039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.555+0000 7fe740660640 1 -- 172.21.15.67:0/4260592702 >> 172.21.15.67:0/4260592702 conn(0x7fe73807b2c0 msgr2=0x7fe73807b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:46.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.555+0000 7fe740660640 1 -- 172.21.15.67:0/4260592702 shutdown_connections 2024-07-27T03:15:46.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.555+0000 7fe740660640 1 -- 172.21.15.67:0/4260592702 wait complete. 2024-07-27T03:15:46.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.556+0000 7fe740660640 1 Processor -- start 2024-07-27T03:15:46.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.556+0000 7fe740660640 1 -- start start 2024-07-27T03:15:46.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.556+0000 7fe740660640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7381035d0 0x7fe738199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:46.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.556+0000 7fe740660640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe738108f10 0x7fe73819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:46.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.556+0000 7fe740660640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe738109860 0x7fe73819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:46.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.556+0000 7fe740660640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe738113590 con 0x7fe738109860 2024-07-27T03:15:46.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.556+0000 7fe740660640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe738113410 con 0x7fe7381035d0 2024-07-27T03:15:46.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.556+0000 7fe740660640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe738113710 con 0x7fe738108f10 2024-07-27T03:15:46.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.557+0000 7fe73ebfd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe738109860 0x7fe73819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:46.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.557+0000 7fe73ebfd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe738109860 0x7fe73819f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52216/0 (socket says 172.21.15.67:52216) 2024-07-27T03:15:46.561 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.557+0000 7fe73ebfd640 1 -- 172.21.15.67:0/626366471 learned_addr learned my addr 172.21.15.67:0/626366471 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:46.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.557+0000 7fe73dbfb640 1 --2- 172.21.15.67:0/626366471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe738108f10 0x7fe73819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:46.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.557+0000 7fe73e3fc640 1 --2- 172.21.15.67:0/626366471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7381035d0 0x7fe738199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:46.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.557+0000 7fe73ebfd640 1 -- 172.21.15.67:0/626366471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe738108f10 msgr2=0x7fe73819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:46.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.557+0000 7fe73ebfd640 1 --2- 172.21.15.67:0/626366471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe738108f10 0x7fe73819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.557+0000 7fe73ebfd640 1 -- 172.21.15.67:0/626366471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7381035d0 msgr2=0x7fe738199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:46.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.557+0000 7fe73ebfd640 1 --2- 172.21.15.67:0/626366471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7381035d0 0x7fe738199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.557+0000 7fe73ebfd640 1 -- 172.21.15.67:0/626366471 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73819fe70 con 0x7fe738109860 2024-07-27T03:15:46.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.557+0000 7fe73e3fc640 1 --2- 172.21.15.67:0/626366471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7381035d0 0x7fe738199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:15:46.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.557+0000 7fe73ebfd640 1 --2- 172.21.15.67:0/626366471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe738109860 0x7fe73819f800 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe73000cce0 tx=0x7fe73000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:46.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.557+0000 7fe72f7fe640 1 -- 172.21.15.67:0/626366471 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73000f450 con 0x7fe738109860 2024-07-27T03:15:46.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.557+0000 7fe72f7fe640 1 -- 172.21.15.67:0/626366471 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe73000faa0 con 0x7fe738109860 2024-07-27T03:15:46.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.557+0000 7fe740660640 1 -- 172.21.15.67:0/626366471 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7381a0160 con 0x7fe738109860 2024-07-27T03:15:46.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.557+0000 7fe72f7fe640 1 -- 172.21.15.67:0/626366471 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe730018b70 con 0x7fe738109860 2024-07-27T03:15:46.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.557+0000 7fe740660640 1 -- 172.21.15.67:0/626366471 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7381a6980 con 0x7fe738109860 2024-07-27T03:15:46.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.560+0000 7fe746be5740 1 Processor -- start 2024-07-27T03:15:46.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.560+0000 7fe72f7fe640 1 -- 172.21.15.67:0/626366471 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7fe730018d10 con 0x7fe738109860 2024-07-27T03:15:46.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.560+0000 7fe746be5740 1 -- start start 2024-07-27T03:15:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.561+0000 7fe746be5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565368981f60 0x565368d27200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.561+0000 7fe746be5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653689590a0 0x565368d27740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.561+0000 7fe746be5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565368961280 0x565368d27c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.561+0000 7fe746be5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x565368c0f2b0 con 0x565368981f60 2024-07-27T03:15:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.561+0000 7fe746be5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x565368c9d890 con 0x565368961280 2024-07-27T03:15:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.561+0000 7fe746be5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x565368c8cd90 con 0x5653689590a0 2024-07-27T03:15:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.561+0000 7fe73dbfb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565368981f60 0x565368d27200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.561+0000 7fe73dbfb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565368981f60 0x565368d27200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52218/0 (socket says 172.21.15.67:52218) 2024-07-27T03:15:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.561+0000 7fe73dbfb640 1 -- 172.21.15.67:0/3538314048 learned_addr learned my addr 172.21.15.67:0/3538314048 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.561+0000 7fe73e3fc640 1 --2- 172.21.15.67:0/3538314048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653689590a0 0x565368d27740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.561+0000 7fe73dbfb640 1 --2- 172.21.15.67:0/3538314048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565368961280 0x565368d27c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.561+0000 7fe72f7fe640 1 --2- 172.21.15.67:0/626366471 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe7240b3c00 0x7fe7240b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.561+0000 7fe73dbfb640 1 -- 172.21.15.67:0/3538314048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653689590a0 msgr2=0x565368d27740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:46.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.561+0000 7fe73dbfb640 1 --2- 172.21.15.67:0/3538314048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653689590a0 0x565368d27740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.561+0000 7fe73dbfb640 1 -- 172.21.15.67:0/3538314048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565368961280 msgr2=0x565368d27c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:46.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.561+0000 7fe73dbfb640 1 --2- 172.21.15.67:0/3538314048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565368961280 0x565368d27c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.561+0000 7fe73dbfb640 1 -- 172.21.15.67:0/3538314048 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565368cc6af0 con 0x565368981f60 2024-07-27T03:15:46.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.561+0000 7fe73ebfd640 1 --2- 172.21.15.67:0/626366471 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe7240b3c00 0x7fe7240b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:46.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.561+0000 7fe73dbfb640 1 --2- 172.21.15.67:0/3538314048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565368981f60 0x565368d27200 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe728031440 tx=0x7fe728000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaec34d7475407a9 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:46.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.561+0000 7fe72f7fe640 1 -- 172.21.15.67:0/626366471 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fe7300e43b0 con 0x7fe738109860 2024-07-27T03:15:46.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.562+0000 7fe72cff9640 1 -- 172.21.15.67:0/3538314048 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe728035040 con 0x565368981f60 2024-07-27T03:15:46.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.562+0000 7fe72cff9640 1 -- 172.21.15.67:0/3538314048 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7280054d0 con 0x565368981f60 2024-07-27T03:15:46.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.562+0000 7fe72cff9640 1 -- 172.21.15.67:0/3538314048 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe728003970 con 0x565368981f60 2024-07-27T03:15:46.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.563+0000 7fe746be5740 1 -- 172.21.15.67:0/3538314048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565368981f60 msgr2=0x565368d27200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:46.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.563+0000 7fe73ebfd640 1 --2- 172.21.15.67:0/626366471 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe7240b3c00 0x7fe7240b6110 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe7300a6fd0 tx=0x7fe730092040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:46.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.563+0000 7fe746be5740 1 --2- 172.21.15.67:0/3538314048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565368981f60 0x565368d27200 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe728031440 tx=0x7fe728000bc0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.563+0000 7fe746be5740 1 -- 172.21.15.67:0/3538314048 shutdown_connections 2024-07-27T03:15:46.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.563+0000 7fe746be5740 1 --2- 172.21.15.67:0/3538314048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565368961280 0x565368d27c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.563+0000 7fe746be5740 1 --2- 172.21.15.67:0/3538314048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5653689590a0 0x565368d27740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.563+0000 7fe746be5740 1 --2- 172.21.15.67:0/3538314048 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565368981f60 0x565368d27200 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.563+0000 7fe746be5740 1 -- 172.21.15.67:0/3538314048 >> 172.21.15.67:0/3538314048 conn(0x565368976350 msgr2=0x565368d250f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:46.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.563+0000 7fe746be5740 1 -- 172.21.15.67:0/3538314048 shutdown_connections 2024-07-27T03:15:46.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.563+0000 7fe746be5740 1 -- 172.21.15.67:0/3538314048 wait complete. 2024-07-27T03:15:46.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.563+0000 7fe746be5740 1 -- start start 2024-07-27T03:15:46.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.563+0000 7fe746be5740 1 Processor -- start 2024-07-27T03:15:46.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.564+0000 7fe746be5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653689590a0 0x565368d32e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:46.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.564+0000 7fe746be5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565368961280 0x565368d33400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:46.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.564+0000 7fe746be5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565368981f60 0x565368d3eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:46.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.564+0000 7fe746be5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x565368c2f7b0 con 0x565368961280 2024-07-27T03:15:46.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.564+0000 7fe746be5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x565368c90dd0 con 0x5653689590a0 2024-07-27T03:15:46.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.564+0000 7fe746be5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x565368af05f0 con 0x565368981f60 2024-07-27T03:15:46.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.564+0000 7fe746be5740 10 client.0 _ll_register_callbacks cb 0x565368c377e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:15:46.571 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.564+0000 7fe73e3fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565368961280 0x565368d33400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:46.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.564+0000 7fe73e3fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565368961280 0x565368d33400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52228/0 (socket says 172.21.15.67:52228) 2024-07-27T03:15:46.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.564+0000 7fe73e3fc640 1 -- 172.21.15.67:0/4197458059 learned_addr learned my addr 172.21.15.67:0/4197458059 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:46.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.564+0000 7fe73dbfb640 1 --2- 172.21.15.67:0/4197458059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565368981f60 0x565368d3eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:46.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.564+0000 7fe73dbfb640 1 --2- 172.21.15.67:0/4197458059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653689590a0 0x565368d32e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:46.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.564+0000 7fe73e3fc640 1 -- 172.21.15.67:0/4197458059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565368981f60 msgr2=0x565368d3eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:46.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.564+0000 7fe73e3fc640 1 --2- 172.21.15.67:0/4197458059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565368981f60 0x565368d3eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.564+0000 7fe73e3fc640 1 -- 172.21.15.67:0/4197458059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653689590a0 msgr2=0x565368d32e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:46.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.564+0000 7fe73e3fc640 1 --2- 172.21.15.67:0/4197458059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653689590a0 0x565368d32e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.564+0000 7fe73e3fc640 1 -- 172.21.15.67:0/4197458059 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565368c22620 con 0x565368961280 2024-07-27T03:15:46.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.564+0000 7fe73dbfb640 1 --2- 172.21.15.67:0/4197458059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565368981f60 0x565368d3eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:15:46.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.564+0000 7fe73e3fc640 1 --2- 172.21.15.67:0/4197458059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565368961280 0x565368d33400 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe734010500 tx=0x7fe7340109c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd38c549df614863 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:46.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.565+0000 7fe721ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:15:46.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.565+0000 7fe721ffb640 1 -- 172.21.15.67:0/4197458059 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe734018070 con 0x565368961280 2024-07-27T03:15:46.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.565+0000 7fe721ffb640 1 -- 172.21.15.67:0/4197458059 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fe734014640 con 0x565368961280 2024-07-27T03:15:46.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.565+0000 7fe746be5740 1 -- 172.21.15.67:0/4197458059 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7280054d0 con 0x565368961280 2024-07-27T03:15:46.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.565+0000 7fe721ffb640 1 -- 172.21.15.67:0/4197458059 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe734024050 con 0x565368961280 2024-07-27T03:15:46.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.565+0000 7fe746be5740 10 client.15240 fetch_fsmap learned FSMap version 29 2024-07-27T03:15:46.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.565+0000 7fe746be5740 1 -- 172.21.15.67:0/4197458059 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565368c33aa0 con 0x565368961280 2024-07-27T03:15:46.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.565+0000 7fe721ffb640 1 -- 172.21.15.67:0/4197458059 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7280054d0 con 0x565368961280 2024-07-27T03:15:46.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.565+0000 7fe721ffb640 1 -- 172.21.15.67:0/4197458059 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fe7340129c0 con 0x565368961280 2024-07-27T03:15:46.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.565+0000 7fe746be5740 10 client.15240 fetch_fsmap finished waiting for FSMap version 29 2024-07-27T03:15:46.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.565+0000 7fe746be5740 10 client.15240 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-27T03:15:46.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.565+0000 7fe746be5740 20 client.15240 populate_metadata read hostname 'smithi067' 2024-07-27T03:15:46.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.565+0000 7fe746be5740 1 --2- 172.21.15.67:0/4197458059 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x5653689408e0 0x565368d451d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:46.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.565+0000 7fe746be5740 4 client.15240 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:15:46.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.565+0000 7fe746be5740 1 -- 172.21.15.67:0/4197458059 --> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565368c9f7e0 con 0x5653689408e0 2024-07-27T03:15:46.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.566+0000 7fe73dbfb640 1 --2- 172.21.15.67:0/4197458059 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x5653689408e0 0x565368d451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:46.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.566+0000 7fe73dbfb640 1 --2- 172.21.15.67:0/4197458059 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x5653689408e0 0x565368d451d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf6f5dc5532ffd25 server_cookie=be77d9d663235e4 in_seq=0 out_seq=0 2024-07-27T03:15:46.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.566+0000 7fe721ffb640 10 client.15240 ms_handle_connect on v2:172.21.15.175:6838/1670568853 2024-07-27T03:15:46.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.567+0000 7fe721ffb640 1 -- 172.21.15.67:0/4197458059 <== mds.2 v2:172.21.15.175:6838/1670568853 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565368c9f7e0 con 0x5653689408e0 2024-07-27T03:15:46.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.567+0000 7fe721ffb640 10 client.15240 handle_command_reply: tid=0 2024-07-27T03:15:46.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.567+0000 7fe746be5740 1 client.15240 shutdown 2024-07-27T03:15:46.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.567+0000 7fe746be5740 1 -- 172.21.15.67:0/4197458059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565368961280 msgr2=0x565368d33400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:46.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.567+0000 7fe746be5740 1 --2- 172.21.15.67:0/4197458059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565368961280 0x565368d33400 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe734010500 tx=0x7fe7340109c0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.567+0000 7fe746be5740 1 -- 172.21.15.67:0/4197458059 shutdown_connections 2024-07-27T03:15:46.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.567+0000 7fe746be5740 1 --2- 172.21.15.67:0/4197458059 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x5653689408e0 0x565368d451d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.567+0000 7fe746be5740 1 --2- 172.21.15.67:0/4197458059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565368981f60 0x565368d3eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.567+0000 7fe746be5740 1 --2- 172.21.15.67:0/4197458059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565368961280 0x565368d33400 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.567+0000 7fe746be5740 1 --2- 172.21.15.67:0/4197458059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653689590a0 0x565368d32e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.567+0000 7fe746be5740 1 -- 172.21.15.67:0/4197458059 >> 172.21.15.67:0/4197458059 conn(0x565368976350 msgr2=0x565368d2a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:46.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.568+0000 7fe721ffb640 0 client.15240 ms_handle_reset on v2:172.21.15.175:6838/1670568853 2024-07-27T03:15:46.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.568+0000 7fe746be5740 1 -- 172.21.15.67:0/4197458059 shutdown_connections 2024-07-27T03:15:46.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.568+0000 7fe746be5740 1 -- 172.21.15.67:0/4197458059 wait complete. 2024-07-27T03:15:46.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.568+0000 7fe746be5740 20 client.15240 trim_cache size 0 max 16384 2024-07-27T03:15:46.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.586+0000 7fe746be5740 1 Processor -- start 2024-07-27T03:15:46.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.586+0000 7fe746be5740 1 -- start start 2024-07-27T03:15:46.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.586+0000 7fe746be5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653689590a0 0x565368d44b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:46.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.586+0000 7fe746be5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565368961280 0x565368d3ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:46.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.586+0000 7fe746be5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565368981f60 0x565368d314c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:46.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.586+0000 7fe746be5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x565368d43510 con 0x5653689590a0 2024-07-27T03:15:46.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.586+0000 7fe746be5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x565368d43be0 con 0x565368961280 2024-07-27T03:15:46.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.586+0000 7fe746be5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x565368d442b0 con 0x565368981f60 2024-07-27T03:15:46.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.586+0000 7fe73dbfb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653689590a0 0x565368d44b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:46.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.587+0000 7fe73dbfb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653689590a0 0x565368d44b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52232/0 (socket says 172.21.15.67:52232) 2024-07-27T03:15:46.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.587+0000 7fe73dbfb640 1 -- 172.21.15.67:0/3252966511 learned_addr learned my addr 172.21.15.67:0/3252966511 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:46.590 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.587+0000 7fe73e3fc640 1 --2- 172.21.15.67:0/3252966511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565368961280 0x565368d3ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:46.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.587+0000 7fe73dbfb640 1 -- 172.21.15.67:0/3252966511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565368981f60 msgr2=0x565368d314c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:46.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.587+0000 7fe73dbfb640 1 --2- 172.21.15.67:0/3252966511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565368981f60 0x565368d314c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.587+0000 7fe73dbfb640 1 -- 172.21.15.67:0/3252966511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565368961280 msgr2=0x565368d3ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:46.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.587+0000 7fe73dbfb640 1 --2- 172.21.15.67:0/3252966511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565368961280 0x565368d3ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.587+0000 7fe73dbfb640 1 -- 172.21.15.67:0/3252966511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653689ed540 con 0x5653689590a0 2024-07-27T03:15:46.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.587+0000 7fe73dbfb640 1 --2- 172.21.15.67:0/3252966511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653689590a0 0x565368d44b00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe7280096a0 tx=0x7fe728000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb82795717d1f963 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:46.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.587+0000 7fe723fff640 1 -- 172.21.15.67:0/3252966511 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe728055020 con 0x5653689590a0 2024-07-27T03:15:46.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.587+0000 7fe723fff640 1 -- 172.21.15.67:0/3252966511 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe728003e00 con 0x5653689590a0 2024-07-27T03:15:46.591 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.587+0000 7fe723fff640 1 -- 172.21.15.67:0/3252966511 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe728035040 con 0x5653689590a0 2024-07-27T03:15:46.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.588+0000 7fe746be5740 1 -- 172.21.15.67:0/3252966511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653689590a0 msgr2=0x565368d44b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.588+0000 7fe746be5740 1 --2- 172.21.15.67:0/3252966511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653689590a0 0x565368d44b00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe7280096a0 tx=0x7fe728000bc0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.588+0000 7fe746be5740 1 -- 172.21.15.67:0/3252966511 shutdown_connections 2024-07-27T03:15:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.588+0000 7fe746be5740 1 --2- 172.21.15.67:0/3252966511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565368981f60 0x565368d314c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.588+0000 7fe746be5740 1 --2- 172.21.15.67:0/3252966511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565368961280 0x565368d3ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.588+0000 7fe746be5740 1 --2- 172.21.15.67:0/3252966511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5653689590a0 0x565368d44b00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.588+0000 7fe746be5740 1 -- 172.21.15.67:0/3252966511 >> 172.21.15.67:0/3252966511 conn(0x565368976350 msgr2=0x565368d2dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.588+0000 7fe746be5740 1 -- 172.21.15.67:0/3252966511 shutdown_connections 2024-07-27T03:15:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.588+0000 7fe746be5740 1 -- 172.21.15.67:0/3252966511 wait complete. 2024-07-27T03:15:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.588+0000 7fe746be5740 1 -- start start 2024-07-27T03:15:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.589+0000 7fe746be5740 1 Processor -- start 2024-07-27T03:15:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.589+0000 7fe746be5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653689590a0 0x565368d4d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.589+0000 7fe746be5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565368961280 0x565368d4c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:46.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.589+0000 7fe746be5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565368981f60 0x565368d4ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:46.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.589+0000 7fe746be5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x565368d45050 con 0x565368981f60 2024-07-27T03:15:46.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.589+0000 7fe746be5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x565368d44980 con 0x5653689590a0 2024-07-27T03:15:46.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.589+0000 7fe746be5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x565368d45720 con 0x565368961280 2024-07-27T03:15:46.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.589+0000 7fe746be5740 10 client.0 _ll_register_callbacks cb 0x7fe734007520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:15:46.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.589+0000 7fe73dbfb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565368981f60 0x565368d4ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:46.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.589+0000 7fe73dbfb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565368981f60 0x565368d4ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52238/0 (socket says 172.21.15.67:52238) 2024-07-27T03:15:46.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.589+0000 7fe73dbfb640 1 -- 172.21.15.67:0/2266311632 learned_addr learned my addr 172.21.15.67:0/2266311632 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:46.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.589+0000 7fe73dbfb640 1 --2- 172.21.15.67:0/2266311632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653689590a0 0x565368d4d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:46.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.589+0000 7fe73e3fc640 1 --2- 172.21.15.67:0/2266311632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565368961280 0x565368d4c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:46.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.589+0000 7fe73dbfb640 1 -- 172.21.15.67:0/2266311632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565368961280 msgr2=0x565368d4c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:46.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.589+0000 7fe73dbfb640 1 --2- 172.21.15.67:0/2266311632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565368961280 0x565368d4c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.589+0000 7fe73dbfb640 1 -- 172.21.15.67:0/2266311632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653689590a0 msgr2=0x565368d4d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:46.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.589+0000 7fe73dbfb640 1 --2- 172.21.15.67:0/2266311632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653689590a0 0x565368d4d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.589+0000 7fe73dbfb640 1 -- 172.21.15.67:0/2266311632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565368d4cfd0 con 0x565368981f60 2024-07-27T03:15:46.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.590+0000 7fe73dbfb640 1 --2- 172.21.15.67:0/2266311632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565368981f60 0x565368d4ca90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe728009670 tx=0x7fe728040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832c761eab4e1269 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:46.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.590+0000 7fe721ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:15:46.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.590+0000 7fe721ffb640 1 -- 172.21.15.67:0/2266311632 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe728035030 con 0x565368981f60 2024-07-27T03:15:46.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.590+0000 7fe746be5740 1 -- 172.21.15.67:0/2266311632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe728003e00 con 0x565368981f60 2024-07-27T03:15:46.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.590+0000 7fe721ffb640 1 -- 172.21.15.67:0/2266311632 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fe72805b040 con 0x565368981f60 2024-07-27T03:15:46.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.590+0000 7fe721ffb640 1 -- 172.21.15.67:0/2266311632 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe728055070 con 0x565368981f60 2024-07-27T03:15:46.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.590+0000 7fe746be5740 10 client.15246 fetch_fsmap learned FSMap version 29 2024-07-27T03:15:46.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.590+0000 7fe746be5740 1 -- 172.21.15.67:0/2266311632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565368dad020 con 0x565368981f60 2024-07-27T03:15:46.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.590+0000 7fe721ffb640 1 -- 172.21.15.67:0/2266311632 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe728003e00 con 0x565368981f60 2024-07-27T03:15:46.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.590+0000 7fe721ffb640 1 -- 172.21.15.67:0/2266311632 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fe72804a540 con 0x565368981f60 2024-07-27T03:15:46.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.591+0000 7fe746be5740 10 client.15246 fetch_fsmap finished waiting for FSMap version 29 2024-07-27T03:15:46.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.591+0000 7fe746be5740 10 client.15246 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-27T03:15:46.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.591+0000 7fe746be5740 20 client.15246 populate_metadata read hostname 'smithi067' 2024-07-27T03:15:46.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.591+0000 7fe746be5740 1 --2- 172.21.15.67:0/2266311632 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x565368915c90 0x565368db02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:46.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.591+0000 7fe746be5740 4 client.15246 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-27T03:15:46.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.591+0000 7fe746be5740 1 -- 172.21.15.67:0/2266311632 --> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] -- command(tid 0: {"prefix": "status"}) -- 0x565368c9b280 con 0x565368915c90 2024-07-27T03:15:46.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.591+0000 7fe73e3fc640 1 --2- 172.21.15.67:0/2266311632 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x565368915c90 0x565368db02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:46.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.591+0000 7fe73e3fc640 1 --2- 172.21.15.67:0/2266311632 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x565368915c90 0x565368db02f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b23fe784b9f4bf7a server_cookie=3490562c0dd225ef in_seq=0 out_seq=0 2024-07-27T03:15:46.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.591+0000 7fe721ffb640 10 client.15246 ms_handle_connect on v2:172.21.15.175:6838/1670568853 2024-07-27T03:15:46.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.592+0000 7fe721ffb640 1 -- 172.21.15.67:0/2266311632 <== mds.2 v2:172.21.15.175:6838/1670568853 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x565368c9b280 con 0x565368915c90 2024-07-27T03:15:46.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.592+0000 7fe721ffb640 10 client.15246 handle_command_reply: tid=0 2024-07-27T03:15:46.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.592+0000 7fe746be5740 1 client.15246 shutdown 2024-07-27T03:15:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.592+0000 7fe746be5740 1 -- 172.21.15.67:0/2266311632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565368981f60 msgr2=0x565368d4ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.592+0000 7fe746be5740 1 --2- 172.21.15.67:0/2266311632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565368981f60 0x565368d4ca90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe728009670 tx=0x7fe728040040 comp rx=0 tx=0).stop 2024-07-27T03:15:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.592+0000 7fe746be5740 1 -- 172.21.15.67:0/2266311632 shutdown_connections 2024-07-27T03:15:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.592+0000 7fe746be5740 1 --2- 172.21.15.67:0/2266311632 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x565368915c90 0x565368db02f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.592+0000 7fe746be5740 1 --2- 172.21.15.67:0/2266311632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x565368981f60 0x565368d4ca90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.592+0000 7fe746be5740 1 --2- 172.21.15.67:0/2266311632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x565368961280 0x565368d4c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.592+0000 7fe721ffb640 0 client.15246 ms_handle_reset on v2:172.21.15.175:6838/1670568853 2024-07-27T03:15:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.592+0000 7fe746be5740 1 --2- 172.21.15.67:0/2266311632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5653689590a0 0x565368d4d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.592+0000 7fe746be5740 1 -- 172.21.15.67:0/2266311632 >> 172.21.15.67:0/2266311632 conn(0x565368976350 msgr2=0x565368d32e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.592+0000 7fe746be5740 1 -- 172.21.15.67:0/2266311632 shutdown_connections 2024-07-27T03:15:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.592+0000 7fe746be5740 1 -- 172.21.15.67:0/2266311632 wait complete. 2024-07-27T03:15:46.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.592+0000 7fe746be5740 20 client.15246 trim_cache size 0 max 16384 2024-07-27T03:15:46.600 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:15:46.600 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "c0782a00-4bc4-11ef-bcc7-c7b262605968", 2024-07-27T03:15:46.600 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 2, 2024-07-27T03:15:46.600 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24583, 2024-07-27T03:15:46.600 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-27T03:15:46.600 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-27T03:15:46.600 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-27T03:15:46.600 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 129.37655320900001, 2024-07-27T03:15:46.600 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 29, 2024-07-27T03:15:46.600 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 91, 2024-07-27T03:15:46.601 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-27T03:15:46.601 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 215.82251858500001 2024-07-27T03:15:46.601 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:15:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.593+0000 7fe740660640 1 -- 172.21.15.67:0/626366471 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe7240b3c00 msgr2=0x7fe7240b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.593+0000 7fe740660640 1 --2- 172.21.15.67:0/626366471 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe7240b3c00 0x7fe7240b6110 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe7300a6fd0 tx=0x7fe730092040 comp rx=0 tx=0).stop 2024-07-27T03:15:46.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.593+0000 7fe740660640 1 -- 172.21.15.67:0/626366471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe738109860 msgr2=0x7fe73819f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.593+0000 7fe740660640 1 --2- 172.21.15.67:0/626366471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe738109860 0x7fe73819f800 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe73000cce0 tx=0x7fe73000a620 comp rx=0 tx=0).stop 2024-07-27T03:15:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.593+0000 7fe740660640 1 -- 172.21.15.67:0/626366471 shutdown_connections 2024-07-27T03:15:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.593+0000 7fe740660640 1 --2- 172.21.15.67:0/626366471 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe7240b3c00 0x7fe7240b6110 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.593+0000 7fe740660640 1 --2- 172.21.15.67:0/626366471 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe738109860 0x7fe73819f800 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.593+0000 7fe740660640 1 --2- 172.21.15.67:0/626366471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe738108f10 0x7fe73819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.593+0000 7fe740660640 1 --2- 172.21.15.67:0/626366471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7381035d0 0x7fe738199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.593+0000 7fe740660640 1 -- 172.21.15.67:0/626366471 >> 172.21.15.67:0/626366471 conn(0x7fe73807b2c0 msgr2=0x7fe738107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.593+0000 7fe740660640 1 -- 172.21.15.67:0/626366471 shutdown_connections 2024-07-27T03:15:46.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:46.593+0000 7fe740660640 1 -- 172.21.15.67:0/626366471 wait complete. 2024-07-27T03:15:46.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:46 smithi175 ceph-mon[37125]: pgmap v406: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:47.515 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.h status 2024-07-27T03:15:47.790 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:15:48.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:48 smithi149 ceph-mon[37173]: pgmap v407: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:48.551 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:48 smithi067 ceph-mon[32814]: pgmap v407: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:48.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:48 smithi175 ceph-mon[37125]: pgmap v407: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:49.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.381+0000 7fe1c7b39640 1 -- 172.21.15.67:0/1787951876 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1c0109860 msgr2=0x7fe1c0110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:49.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.381+0000 7fe1c7b39640 1 --2- 172.21.15.67:0/1787951876 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1c0109860 0x7fe1c0110090 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009ca0 tx=0x7fe1b802a8c0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.381+0000 7fe1c7b39640 1 -- 172.21.15.67:0/1787951876 shutdown_connections 2024-07-27T03:15:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.381+0000 7fe1c7b39640 1 --2- 172.21.15.67:0/1787951876 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1c0109860 0x7fe1c0110090 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.381+0000 7fe1c7b39640 1 --2- 172.21.15.67:0/1787951876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1c0108f10 0x7fe1c0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.381+0000 7fe1c7b39640 1 --2- 172.21.15.67:0/1787951876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1c01035d0 0x7fe1c01039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.381+0000 7fe1c7b39640 1 -- 172.21.15.67:0/1787951876 >> 172.21.15.67:0/1787951876 conn(0x7fe1c007b2c0 msgr2=0x7fe1c007b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.381+0000 7fe1c7b39640 1 -- 172.21.15.67:0/1787951876 shutdown_connections 2024-07-27T03:15:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.381+0000 7fe1c7b39640 1 -- 172.21.15.67:0/1787951876 wait complete. 2024-07-27T03:15:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.382+0000 7fe1c7b39640 1 Processor -- start 2024-07-27T03:15:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.382+0000 7fe1c7b39640 1 -- start start 2024-07-27T03:15:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.382+0000 7fe1c7b39640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1c01035d0 0x7fe1c0199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.382+0000 7fe1c7b39640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1c0108f10 0x7fe1c019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.382+0000 7fe1c7b39640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1c0109860 0x7fe1c019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:49.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.382+0000 7fe1c7b39640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c0113530 con 0x7fe1c01035d0 2024-07-27T03:15:49.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.382+0000 7fe1c7b39640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c01133b0 con 0x7fe1c0108f10 2024-07-27T03:15:49.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.382+0000 7fe1c7b39640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c01136b0 con 0x7fe1c0109860 2024-07-27T03:15:49.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.382+0000 7fe1c58d5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1c01035d0 0x7fe1c0199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:49.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.382+0000 7fe1c58d5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1c01035d0 0x7fe1c0199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52248/0 (socket says 172.21.15.67:52248) 2024-07-27T03:15:49.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.382+0000 7fe1c58d5640 1 -- 172.21.15.67:0/245385 learned_addr learned my addr 172.21.15.67:0/245385 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:49.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.382+0000 7fe1c50d4640 1 --2- 172.21.15.67:0/245385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1c0108f10 0x7fe1c019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:49.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.383+0000 7fe1c60d6640 1 --2- 172.21.15.67:0/245385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1c0109860 0x7fe1c019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:49.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.383+0000 7fe1c58d5640 1 -- 172.21.15.67:0/245385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1c0109860 msgr2=0x7fe1c019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:49.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.383+0000 7fe1c58d5640 1 --2- 172.21.15.67:0/245385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1c0109860 0x7fe1c019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.383+0000 7fe1c58d5640 1 -- 172.21.15.67:0/245385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1c0108f10 msgr2=0x7fe1c019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:49.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.383+0000 7fe1c58d5640 1 --2- 172.21.15.67:0/245385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1c0108f10 0x7fe1c019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.383+0000 7fe1c58d5640 1 -- 172.21.15.67:0/245385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c019fe30 con 0x7fe1c01035d0 2024-07-27T03:15:49.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.383+0000 7fe1c50d4640 1 --2- 172.21.15.67:0/245385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1c0108f10 0x7fe1c019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:15:49.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.383+0000 7fe1c58d5640 1 --2- 172.21.15.67:0/245385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1c01035d0 0x7fe1c0199ec0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00d950 tx=0x7fe1bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:49.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.383+0000 7fe1b6ffd640 1 -- 172.21.15.67:0/245385 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc017030 con 0x7fe1c01035d0 2024-07-27T03:15:49.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.383+0000 7fe1c7b39640 1 -- 172.21.15.67:0/245385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c01a0120 con 0x7fe1c01035d0 2024-07-27T03:15:49.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.383+0000 7fe1b6ffd640 1 -- 172.21.15.67:0/245385 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1bc004020 con 0x7fe1c01035d0 2024-07-27T03:15:49.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.383+0000 7fe1b6ffd640 1 -- 172.21.15.67:0/245385 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc0047a0 con 0x7fe1c01035d0 2024-07-27T03:15:49.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.383+0000 7fe1c7b39640 1 -- 172.21.15.67:0/245385 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c01a6880 con 0x7fe1c01035d0 2024-07-27T03:15:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.386+0000 7fe1ce0be740 1 Processor -- start 2024-07-27T03:15:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.386+0000 7fe1ce0be740 1 -- start start 2024-07-27T03:15:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.386+0000 7fe1b6ffd640 1 -- 172.21.15.67:0/245385 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7fe1bc009a30 con 0x7fe1c01035d0 2024-07-27T03:15:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.386+0000 7fe1ce0be740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206376f60 0x55d20671c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.386+0000 7fe1ce0be740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d20634e0a0 0x55d20671c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.386+0000 7fe1ce0be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d206356280 0x55d20671cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.386+0000 7fe1ce0be740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d2066042b0 con 0x55d206376f60 2024-07-27T03:15:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.386+0000 7fe1ce0be740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d206692890 con 0x55d20634e0a0 2024-07-27T03:15:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.386+0000 7fe1ce0be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d206681d90 con 0x55d206356280 2024-07-27T03:15:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.387+0000 7fe1c60d6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206376f60 0x55d20671c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.387+0000 7fe1c60d6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206376f60 0x55d20671c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52262/0 (socket says 172.21.15.67:52262) 2024-07-27T03:15:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.387+0000 7fe1c60d6640 1 -- 172.21.15.67:0/1612800019 learned_addr learned my addr 172.21.15.67:0/1612800019 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.387+0000 7fe1c58d5640 1 --2- 172.21.15.67:0/1612800019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d20634e0a0 0x55d20671c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.387+0000 7fe1c50d4640 1 --2- 172.21.15.67:0/1612800019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d206356280 0x55d20671cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.387+0000 7fe1c60d6640 1 -- 172.21.15.67:0/1612800019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d206356280 msgr2=0x55d20671cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.387+0000 7fe1c60d6640 1 --2- 172.21.15.67:0/1612800019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d206356280 0x55d20671cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.387+0000 7fe1c60d6640 1 -- 172.21.15.67:0/1612800019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d20634e0a0 msgr2=0x55d20671c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.387+0000 7fe1c60d6640 1 --2- 172.21.15.67:0/1612800019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d20634e0a0 0x55d20671c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.387+0000 7fe1c60d6640 1 -- 172.21.15.67:0/1612800019 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2066bbaf0 con 0x55d206376f60 2024-07-27T03:15:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.387+0000 7fe1c50d4640 1 --2- 172.21.15.67:0/1612800019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d206356280 0x55d20671cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:15:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.387+0000 7fe1c60d6640 1 --2- 172.21.15.67:0/1612800019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206376f60 0x55d20671c200 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8005cc0 tx=0x7fe1b8009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572ffe64c30e983a server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.387+0000 7fe1b6ffd640 1 --2- 172.21.15.67:0/245385 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe1ac0b3c50 0x7fe1ac0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.387+0000 7fe1abfff640 1 -- 172.21.15.67:0/1612800019 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8031cf0 con 0x55d206376f60 2024-07-27T03:15:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.388+0000 7fe1c58d5640 1 --2- 172.21.15.67:0/245385 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe1ac0b3c50 0x7fe1ac0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.388+0000 7fe1abfff640 1 -- 172.21.15.67:0/1612800019 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1b8036040 con 0x55d206376f60 2024-07-27T03:15:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.388+0000 7fe1abfff640 1 -- 172.21.15.67:0/1612800019 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b80042a0 con 0x55d206376f60 2024-07-27T03:15:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.388+0000 7fe1b6ffd640 1 -- 172.21.15.67:0/245385 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fe1bc0e3b00 con 0x7fe1c01035d0 2024-07-27T03:15:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.389+0000 7fe1ce0be740 1 -- 172.21.15.67:0/1612800019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206376f60 msgr2=0x55d20671c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.389+0000 7fe1c58d5640 1 --2- 172.21.15.67:0/245385 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe1ac0b3c50 0x7fe1ac0b6160 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55d20671ae80 tx=0x7fe1bc004590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.389+0000 7fe1ce0be740 1 --2- 172.21.15.67:0/1612800019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206376f60 0x55d20671c200 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8005cc0 tx=0x7fe1b8009800 comp rx=0 tx=0).stop 2024-07-27T03:15:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.389+0000 7fe1ce0be740 1 -- 172.21.15.67:0/1612800019 shutdown_connections 2024-07-27T03:15:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.389+0000 7fe1ce0be740 1 --2- 172.21.15.67:0/1612800019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d206356280 0x55d20671cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.389+0000 7fe1ce0be740 1 --2- 172.21.15.67:0/1612800019 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d20634e0a0 0x55d20671c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.389+0000 7fe1ce0be740 1 --2- 172.21.15.67:0/1612800019 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206376f60 0x55d20671c200 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.389+0000 7fe1ce0be740 1 -- 172.21.15.67:0/1612800019 >> 172.21.15.67:0/1612800019 conn(0x55d20636b350 msgr2=0x55d20671a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.389+0000 7fe1ce0be740 1 -- 172.21.15.67:0/1612800019 shutdown_connections 2024-07-27T03:15:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.389+0000 7fe1ce0be740 1 -- 172.21.15.67:0/1612800019 wait complete. 2024-07-27T03:15:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.389+0000 7fe1ce0be740 1 -- start start 2024-07-27T03:15:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.389+0000 7fe1ce0be740 1 Processor -- start 2024-07-27T03:15:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.390+0000 7fe1ce0be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d20634e0a0 0x55d206727d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.390+0000 7fe1ce0be740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206356280 0x55d2067282d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.390+0000 7fe1ce0be740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d206376f60 0x55d206733b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.390+0000 7fe1ce0be740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d2066247b0 con 0x55d206356280 2024-07-27T03:15:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.390+0000 7fe1ce0be740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d206685dd0 con 0x55d206376f60 2024-07-27T03:15:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.390+0000 7fe1ce0be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d2064e55f0 con 0x55d20634e0a0 2024-07-27T03:15:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.390+0000 7fe1c50d4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206356280 0x55d2067282d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.390+0000 7fe1c50d4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206356280 0x55d2067282d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52272/0 (socket says 172.21.15.67:52272) 2024-07-27T03:15:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.390+0000 7fe1c50d4640 1 -- 172.21.15.67:0/937585968 learned_addr learned my addr 172.21.15.67:0/937585968 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.390+0000 7fe1ce0be740 10 client.0 _ll_register_callbacks cb 0x55d20662c7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:15:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.390+0000 7fe1c60d6640 1 --2- 172.21.15.67:0/937585968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d20634e0a0 0x55d206727d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.390+0000 7fe1c60d6640 1 --2- 172.21.15.67:0/937585968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d206376f60 0x55d206733b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.390+0000 7fe1c50d4640 1 -- 172.21.15.67:0/937585968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d20634e0a0 msgr2=0x55d206727d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.390+0000 7fe1c50d4640 1 --2- 172.21.15.67:0/937585968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d20634e0a0 0x55d206727d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.390+0000 7fe1c50d4640 1 -- 172.21.15.67:0/937585968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d206376f60 msgr2=0x55d206733b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.390+0000 7fe1c50d4640 1 --2- 172.21.15.67:0/937585968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d206376f60 0x55d206733b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.390+0000 7fe1c50d4640 1 -- 172.21.15.67:0/937585968 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d206617620 con 0x55d206356280 2024-07-27T03:15:49.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.390+0000 7fe1c60d6640 1 --2- 172.21.15.67:0/937585968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d206376f60 0x55d206733b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:15:49.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.391+0000 7fe1c50d4640 1 --2- 172.21.15.67:0/937585968 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206356280 0x55d2067282d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0010670 tx=0x7fe1b0010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8f41690de35e07 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:49.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.391+0000 7fe1a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:15:49.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.391+0000 7fe1a97fa640 1 -- 172.21.15.67:0/937585968 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b000ead0 con 0x55d206356280 2024-07-27T03:15:49.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.391+0000 7fe1a97fa640 1 -- 172.21.15.67:0/937585968 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fe1b000ec70 con 0x55d206356280 2024-07-27T03:15:49.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.391+0000 7fe1ce0be740 1 -- 172.21.15.67:0/937585968 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1b8036040 con 0x55d206356280 2024-07-27T03:15:49.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.391+0000 7fe1a97fa640 1 -- 172.21.15.67:0/937585968 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe1b0025050 con 0x55d206356280 2024-07-27T03:15:49.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.391+0000 7fe1ce0be740 10 client.15258 fetch_fsmap learned FSMap version 29 2024-07-27T03:15:49.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.391+0000 7fe1ce0be740 1 -- 172.21.15.67:0/937585968 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d206628aa0 con 0x55d206356280 2024-07-27T03:15:49.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.391+0000 7fe1a97fa640 1 -- 172.21.15.67:0/937585968 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1b8036040 con 0x55d206356280 2024-07-27T03:15:49.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.391+0000 7fe1a97fa640 1 -- 172.21.15.67:0/937585968 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fe1b000f040 con 0x55d206356280 2024-07-27T03:15:49.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.392+0000 7fe1ce0be740 10 client.15258 fetch_fsmap finished waiting for FSMap version 29 2024-07-27T03:15:49.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.392+0000 7fe1ce0be740 10 client.15258 resolve_mds: resolved name 'h' to daemon mds.h 2024-07-27T03:15:49.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.392+0000 7fe1ce0be740 20 client.15258 populate_metadata read hostname 'smithi067' 2024-07-27T03:15:49.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.392+0000 7fe1ce0be740 1 --2- 172.21.15.67:0/937585968 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55d2063358e0 0x55d20673a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:49.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.392+0000 7fe1ce0be740 4 client.15258 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:15:49.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.392+0000 7fe1ce0be740 1 -- 172.21.15.67:0/937585968 --> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2066947e0 con 0x55d2063358e0 2024-07-27T03:15:49.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.392+0000 7fe1c60d6640 1 --2- 172.21.15.67:0/937585968 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55d2063358e0 0x55d20673a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:49.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.392+0000 7fe1a97fa640 10 client.15258 ms_handle_connect on v2:172.21.15.149:6836/2932951845 2024-07-27T03:15:49.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.393+0000 7fe1c60d6640 1 --2- 172.21.15.67:0/937585968 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55d2063358e0 0x55d20673a0b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=641df473c4c172d3 server_cookie=aa8994a3b2d75de1 in_seq=0 out_seq=0 2024-07-27T03:15:49.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.393+0000 7fe1a97fa640 1 -- 172.21.15.67:0/937585968 <== mds.1 v2:172.21.15.149:6836/2932951845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d2066947e0 con 0x55d2063358e0 2024-07-27T03:15:49.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.393+0000 7fe1a97fa640 10 client.15258 handle_command_reply: tid=0 2024-07-27T03:15:49.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.393+0000 7fe1ce0be740 1 client.15258 shutdown 2024-07-27T03:15:49.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.394+0000 7fe1ce0be740 1 -- 172.21.15.67:0/937585968 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206356280 msgr2=0x55d2067282d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:49.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.394+0000 7fe1ce0be740 1 --2- 172.21.15.67:0/937585968 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206356280 0x55d2067282d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0010670 tx=0x7fe1b0010b30 comp rx=0 tx=0).stop 2024-07-27T03:15:49.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.394+0000 7fe1ce0be740 1 -- 172.21.15.67:0/937585968 shutdown_connections 2024-07-27T03:15:49.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.394+0000 7fe1ce0be740 1 --2- 172.21.15.67:0/937585968 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55d2063358e0 0x55d20673a0b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.394+0000 7fe1ce0be740 1 --2- 172.21.15.67:0/937585968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d206376f60 0x55d206733b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.394+0000 7fe1ce0be740 1 --2- 172.21.15.67:0/937585968 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206356280 0x55d2067282d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.394+0000 7fe1ce0be740 1 --2- 172.21.15.67:0/937585968 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d20634e0a0 0x55d206727d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.394+0000 7fe1a97fa640 0 client.15258 ms_handle_reset on v2:172.21.15.149:6836/2932951845 2024-07-27T03:15:49.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.394+0000 7fe1ce0be740 1 -- 172.21.15.67:0/937585968 >> 172.21.15.67:0/937585968 conn(0x55d20636b350 msgr2=0x55d20671f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:49.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.394+0000 7fe1ce0be740 1 -- 172.21.15.67:0/937585968 shutdown_connections 2024-07-27T03:15:49.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.394+0000 7fe1ce0be740 1 -- 172.21.15.67:0/937585968 wait complete. 2024-07-27T03:15:49.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.394+0000 7fe1ce0be740 20 client.15258 trim_cache size 0 max 16384 2024-07-27T03:15:49.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.412+0000 7fe1ce0be740 1 Processor -- start 2024-07-27T03:15:49.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.412+0000 7fe1ce0be740 1 -- start start 2024-07-27T03:15:49.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.413+0000 7fe1ce0be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d20634e0a0 0x55d206733a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:49.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.413+0000 7fe1ce0be740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206356280 0x55d206733fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:49.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.413+0000 7fe1ce0be740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d206376f60 0x55d206721310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:49.416 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.413+0000 7fe1ce0be740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d2067383f0 con 0x55d206356280 2024-07-27T03:15:49.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.413+0000 7fe1ce0be740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d206738ac0 con 0x55d206376f60 2024-07-27T03:15:49.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.413+0000 7fe1ce0be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d206739190 con 0x55d20634e0a0 2024-07-27T03:15:49.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.413+0000 7fe1c50d4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206356280 0x55d206733fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:49.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.413+0000 7fe1c50d4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206356280 0x55d206733fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52278/0 (socket says 172.21.15.67:52278) 2024-07-27T03:15:49.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.413+0000 7fe1c50d4640 1 -- 172.21.15.67:0/3167553978 learned_addr learned my addr 172.21.15.67:0/3167553978 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:49.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.413+0000 7fe1c60d6640 1 --2- 172.21.15.67:0/3167553978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d20634e0a0 0x55d206733a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:49.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.413+0000 7fe1c60d6640 1 --2- 172.21.15.67:0/3167553978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d206376f60 0x55d206721310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:49.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.413+0000 7fe1c50d4640 1 -- 172.21.15.67:0/3167553978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d20634e0a0 msgr2=0x55d206733a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:49.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.413+0000 7fe1c50d4640 1 --2- 172.21.15.67:0/3167553978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d20634e0a0 0x55d206733a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.413+0000 7fe1c50d4640 1 -- 172.21.15.67:0/3167553978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d206376f60 msgr2=0x55d206721310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:49.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.413+0000 7fe1c50d4640 1 --2- 172.21.15.67:0/3167553978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d206376f60 0x55d206721310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.417 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.413+0000 7fe1c50d4640 1 -- 172.21.15.67:0/3167553978 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2063e2540 con 0x55d206356280 2024-07-27T03:15:49.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.413+0000 7fe1c50d4640 1 --2- 172.21.15.67:0/3167553978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206356280 0x55d206733fd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0012ed0 tx=0x7fe1b00100d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8892f6acaeb7eaff server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:49.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.413+0000 7fe1ab7fe640 1 -- 172.21.15.67:0/3167553978 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b002d070 con 0x55d206356280 2024-07-27T03:15:49.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.414+0000 7fe1ab7fe640 1 -- 172.21.15.67:0/3167553978 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1b001ae40 con 0x55d206356280 2024-07-27T03:15:49.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.414+0000 7fe1ab7fe640 1 -- 172.21.15.67:0/3167553978 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0005040 con 0x55d206356280 2024-07-27T03:15:49.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.414+0000 7fe1ce0be740 1 -- 172.21.15.67:0/3167553978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206356280 msgr2=0x55d206733fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:49.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.414+0000 7fe1ce0be740 1 --2- 172.21.15.67:0/3167553978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206356280 0x55d206733fd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0012ed0 tx=0x7fe1b00100d0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.414+0000 7fe1ce0be740 1 -- 172.21.15.67:0/3167553978 shutdown_connections 2024-07-27T03:15:49.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.414+0000 7fe1ce0be740 1 --2- 172.21.15.67:0/3167553978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d206376f60 0x55d206721310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.414+0000 7fe1ce0be740 1 --2- 172.21.15.67:0/3167553978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206356280 0x55d206733fd0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.414+0000 7fe1ce0be740 1 --2- 172.21.15.67:0/3167553978 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d20634e0a0 0x55d206733a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.414+0000 7fe1ce0be740 1 -- 172.21.15.67:0/3167553978 >> 172.21.15.67:0/3167553978 conn(0x55d20636b350 msgr2=0x55d206722d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:49.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.414+0000 7fe1ce0be740 1 -- 172.21.15.67:0/3167553978 shutdown_connections 2024-07-27T03:15:49.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.414+0000 7fe1ce0be740 1 -- 172.21.15.67:0/3167553978 wait complete. 2024-07-27T03:15:49.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.415+0000 7fe1ce0be740 1 -- start start 2024-07-27T03:15:49.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.415+0000 7fe1ce0be740 1 Processor -- start 2024-07-27T03:15:49.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.415+0000 7fe1ce0be740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d20634e0a0 0x55d20673b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:49.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.415+0000 7fe1ce0be740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206356280 0x55d20673be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:49.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.415+0000 7fe1ce0be740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d206376f60 0x55d206740e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:49.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.415+0000 7fe1ce0be740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d206739f30 con 0x55d206356280 2024-07-27T03:15:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.415+0000 7fe1ce0be740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d206739860 con 0x55d20634e0a0 2024-07-27T03:15:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.415+0000 7fe1ce0be740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d20673a600 con 0x55d206376f60 2024-07-27T03:15:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.415+0000 7fe1ce0be740 10 client.0 _ll_register_callbacks cb 0x7fe1b000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:15:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.416+0000 7fe1c50d4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206356280 0x55d20673be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.416+0000 7fe1c50d4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206356280 0x55d20673be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52286/0 (socket says 172.21.15.67:52286) 2024-07-27T03:15:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.416+0000 7fe1c50d4640 1 -- 172.21.15.67:0/58334328 learned_addr learned my addr 172.21.15.67:0/58334328 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.416+0000 7fe1c60d6640 1 --2- 172.21.15.67:0/58334328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d20634e0a0 0x55d20673b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.416+0000 7fe1c60d6640 1 --2- 172.21.15.67:0/58334328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d206376f60 0x55d206740e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.416+0000 7fe1c50d4640 1 -- 172.21.15.67:0/58334328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d206376f60 msgr2=0x55d206740e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.416+0000 7fe1c50d4640 1 --2- 172.21.15.67:0/58334328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d206376f60 0x55d206740e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.416+0000 7fe1c50d4640 1 -- 172.21.15.67:0/58334328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d20634e0a0 msgr2=0x55d20673b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.416+0000 7fe1c50d4640 1 --2- 172.21.15.67:0/58334328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d20634e0a0 0x55d20673b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.416+0000 7fe1c50d4640 1 -- 172.21.15.67:0/58334328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d206741460 con 0x55d206356280 2024-07-27T03:15:49.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.416+0000 7fe1c50d4640 1 --2- 172.21.15.67:0/58334328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206356280 0x55d20673be80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000bfe0 tx=0x7fe1b000ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feea0fc275bced56 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:49.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.416+0000 7fe1a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:15:49.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.416+0000 7fe1a97fa640 1 -- 172.21.15.67:0/58334328 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b002d040 con 0x55d206356280 2024-07-27T03:15:49.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.416+0000 7fe1a97fa640 1 -- 172.21.15.67:0/58334328 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fe1b001a990 con 0x55d206356280 2024-07-27T03:15:49.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.416+0000 7fe1ce0be740 1 -- 172.21.15.67:0/58334328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d206690280 con 0x55d206356280 2024-07-27T03:15:49.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.417+0000 7fe1a97fa640 1 -- 172.21.15.67:0/58334328 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe1b0043050 con 0x55d206356280 2024-07-27T03:15:49.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.417+0000 7fe1ce0be740 10 client.15264 fetch_fsmap learned FSMap version 29 2024-07-27T03:15:49.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.417+0000 7fe1ce0be740 1 -- 172.21.15.67:0/58334328 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d206742840 con 0x55d206356280 2024-07-27T03:15:49.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.417+0000 7fe1a97fa640 1 -- 172.21.15.67:0/58334328 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d206690280 con 0x55d206356280 2024-07-27T03:15:49.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.417+0000 7fe1a97fa640 1 -- 172.21.15.67:0/58334328 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fe1b0004680 con 0x55d206356280 2024-07-27T03:15:49.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.417+0000 7fe1ce0be740 10 client.15264 fetch_fsmap finished waiting for FSMap version 29 2024-07-27T03:15:49.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.417+0000 7fe1ce0be740 10 client.15264 resolve_mds: resolved name 'h' to daemon mds.h 2024-07-27T03:15:49.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.417+0000 7fe1ce0be740 20 client.15264 populate_metadata read hostname 'smithi067' 2024-07-27T03:15:49.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.417+0000 7fe1ce0be740 1 --2- 172.21.15.67:0/58334328 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55d20630ac90 0x55d2067a5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:49.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.417+0000 7fe1ce0be740 4 client.15264 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-27T03:15:49.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.417+0000 7fe1ce0be740 1 -- 172.21.15.67:0/58334328 --> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] -- command(tid 0: {"prefix": "status"}) -- 0x55d20668d9c0 con 0x55d20630ac90 2024-07-27T03:15:49.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.417+0000 7fe1c60d6640 1 --2- 172.21.15.67:0/58334328 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55d20630ac90 0x55d2067a5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:49.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.418+0000 7fe1a97fa640 10 client.15264 ms_handle_connect on v2:172.21.15.149:6836/2932951845 2024-07-27T03:15:49.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.418+0000 7fe1c60d6640 1 --2- 172.21.15.67:0/58334328 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55d20630ac90 0x55d2067a5f50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bfd0d70c506c610f server_cookie=1317f275c5fecd59 in_seq=0 out_seq=0 2024-07-27T03:15:49.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.418+0000 7fe1a97fa640 1 -- 172.21.15.67:0/58334328 <== mds.1 v2:172.21.15.149:6836/2932951845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x7fe1b8036160 con 0x55d20630ac90 2024-07-27T03:15:49.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.418+0000 7fe1a97fa640 10 client.15264 handle_command_reply: tid=0 2024-07-27T03:15:49.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.418+0000 7fe1ce0be740 1 client.15264 shutdown 2024-07-27T03:15:49.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.418+0000 7fe1ce0be740 1 -- 172.21.15.67:0/58334328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206356280 msgr2=0x55d20673be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:49.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.418+0000 7fe1ce0be740 1 --2- 172.21.15.67:0/58334328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206356280 0x55d20673be80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000bfe0 tx=0x7fe1b000ef50 comp rx=0 tx=0).stop 2024-07-27T03:15:49.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.418+0000 7fe1ce0be740 1 -- 172.21.15.67:0/58334328 shutdown_connections 2024-07-27T03:15:49.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.418+0000 7fe1ce0be740 1 --2- 172.21.15.67:0/58334328 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55d20630ac90 0x55d2067a5f50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.418+0000 7fe1ce0be740 1 --2- 172.21.15.67:0/58334328 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d206376f60 0x55d206740e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.418+0000 7fe1ce0be740 1 --2- 172.21.15.67:0/58334328 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d206356280 0x55d20673be80 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.418+0000 7fe1ce0be740 1 --2- 172.21.15.67:0/58334328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d20634e0a0 0x55d20673b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.418+0000 7fe1a97fa640 0 client.15264 ms_handle_reset on v2:172.21.15.149:6836/2932951845 2024-07-27T03:15:49.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.418+0000 7fe1ce0be740 1 -- 172.21.15.67:0/58334328 >> 172.21.15.67:0/58334328 conn(0x55d20636b350 msgr2=0x55d206722d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:49.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.418+0000 7fe1ce0be740 1 -- 172.21.15.67:0/58334328 shutdown_connections 2024-07-27T03:15:49.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.418+0000 7fe1ce0be740 1 -- 172.21.15.67:0/58334328 wait complete. 2024-07-27T03:15:49.427 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.419+0000 7fe1ce0be740 20 client.15264 trim_cache size 0 max 16384 2024-07-27T03:15:49.427 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:15:49.427 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "c0782a00-4bc4-11ef-bcc7-c7b262605968", 2024-07-27T03:15:49.427 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 1, 2024-07-27T03:15:49.428 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24587, 2024-07-27T03:15:49.428 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-27T03:15:49.428 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-27T03:15:49.428 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-27T03:15:49.428 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 133.21710237600001, 2024-07-27T03:15:49.428 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 29, 2024-07-27T03:15:49.428 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 91, 2024-07-27T03:15:49.428 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-27T03:15:49.428 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 221.31281049099999 2024-07-27T03:15:49.428 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:15:49.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.419+0000 7fe1c7b39640 1 -- 172.21.15.67:0/245385 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe1ac0b3c50 msgr2=0x7fe1ac0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:49.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.419+0000 7fe1c7b39640 1 --2- 172.21.15.67:0/245385 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe1ac0b3c50 0x7fe1ac0b6160 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55d20671ae80 tx=0x7fe1bc004590 comp rx=0 tx=0).stop 2024-07-27T03:15:49.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.419+0000 7fe1c7b39640 1 -- 172.21.15.67:0/245385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1c01035d0 msgr2=0x7fe1c0199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:49.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.419+0000 7fe1c7b39640 1 --2- 172.21.15.67:0/245385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1c01035d0 0x7fe1c0199ec0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00d950 tx=0x7fe1bc00de10 comp rx=0 tx=0).stop 2024-07-27T03:15:49.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.419+0000 7fe1c7b39640 1 -- 172.21.15.67:0/245385 shutdown_connections 2024-07-27T03:15:49.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.419+0000 7fe1c7b39640 1 --2- 172.21.15.67:0/245385 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe1ac0b3c50 0x7fe1ac0b6160 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.419+0000 7fe1c7b39640 1 --2- 172.21.15.67:0/245385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1c0109860 0x7fe1c019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.419+0000 7fe1c7b39640 1 --2- 172.21.15.67:0/245385 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1c0108f10 0x7fe1c019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.429 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.419+0000 7fe1c7b39640 1 --2- 172.21.15.67:0/245385 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe1c01035d0 0x7fe1c0199ec0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:49.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.419+0000 7fe1c7b39640 1 -- 172.21.15.67:0/245385 >> 172.21.15.67:0/245385 conn(0x7fe1c007b2c0 msgr2=0x7fe1c0106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:49.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.419+0000 7fe1c7b39640 1 -- 172.21.15.67:0/245385 shutdown_connections 2024-07-27T03:15:49.430 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:49.419+0000 7fe1c7b39640 1 -- 172.21.15.67:0/245385 wait complete. 2024-07-27T03:15:50.087 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.b status 2024-07-27T03:15:50.358 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:15:50.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:50 smithi067 ceph-mon[32814]: pgmap v408: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:15:50.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:50 smithi149 ceph-mon[37173]: pgmap v408: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:15:50.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:50 smithi175 ceph-mon[37125]: pgmap v408: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:15:52.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.084+0000 7f3ed8348640 1 -- 172.21.15.67:0/791849086 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ed01024f0 msgr2=0x7f3ed0102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:52.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.084+0000 7f3ed8348640 1 --2- 172.21.15.67:0/791849086 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ed01024f0 0x7f3ed0102900 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009900 tx=0x7f3ecc02a410 comp rx=0 tx=0).stop 2024-07-27T03:15:52.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.084+0000 7f3ed8348640 1 -- 172.21.15.67:0/791849086 shutdown_connections 2024-07-27T03:15:52.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.084+0000 7f3ed8348640 1 --2- 172.21.15.67:0/791849086 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ed0108780 0x7f3ed010efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.084+0000 7f3ed8348640 1 --2- 172.21.15.67:0/791849086 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ed0107e30 0x7f3ed0108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.084+0000 7f3ed8348640 1 --2- 172.21.15.67:0/791849086 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ed01024f0 0x7f3ed0102900 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.084+0000 7f3ed8348640 1 -- 172.21.15.67:0/791849086 >> 172.21.15.67:0/791849086 conn(0x7f3ed00791b0 msgr2=0x7f3ed00795c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:52.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.084+0000 7f3ed8348640 1 -- 172.21.15.67:0/791849086 shutdown_connections 2024-07-27T03:15:52.091 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.084+0000 7f3ed8348640 1 -- 172.21.15.67:0/791849086 wait complete. 2024-07-27T03:15:52.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.085+0000 7f3ed8348640 1 Processor -- start 2024-07-27T03:15:52.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.085+0000 7f3ed8348640 1 -- start start 2024-07-27T03:15:52.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.085+0000 7f3ed8348640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ed01024f0 0x7f3ed0199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:52.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.085+0000 7f3ed8348640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ed0107e30 0x7f3ed019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:52.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.085+0000 7f3ed8348640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ed0108780 0x7f3ed019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:52.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.085+0000 7f3ed8348640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed0113510 con 0x7f3ed0108780 2024-07-27T03:15:52.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.085+0000 7f3ed8348640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed0113390 con 0x7f3ed01024f0 2024-07-27T03:15:52.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.085+0000 7f3ed8348640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed0113690 con 0x7f3ed0107e30 2024-07-27T03:15:52.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.085+0000 7f3ed68e5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ed0108780 0x7f3ed019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:52.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.085+0000 7f3ed68e5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ed0108780 0x7f3ed019f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52294/0 (socket says 172.21.15.67:52294) 2024-07-27T03:15:52.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.085+0000 7f3ed68e5640 1 -- 172.21.15.67:0/2150205066 learned_addr learned my addr 172.21.15.67:0/2150205066 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:52.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.085+0000 7f3ed60e4640 1 --2- 172.21.15.67:0/2150205066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ed01024f0 0x7f3ed0199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:52.092 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.086+0000 7f3ed58e3640 1 --2- 172.21.15.67:0/2150205066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ed0107e30 0x7f3ed019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.086+0000 7f3ed68e5640 1 -- 172.21.15.67:0/2150205066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ed0107e30 msgr2=0x7f3ed019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.086+0000 7f3ed68e5640 1 --2- 172.21.15.67:0/2150205066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ed0107e30 0x7f3ed019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.086+0000 7f3ed68e5640 1 -- 172.21.15.67:0/2150205066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ed01024f0 msgr2=0x7f3ed0199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.086+0000 7f3ed68e5640 1 --2- 172.21.15.67:0/2150205066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ed01024f0 0x7f3ed0199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.086+0000 7f3ed68e5640 1 -- 172.21.15.67:0/2150205066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed019fef0 con 0x7f3ed0108780 2024-07-27T03:15:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.086+0000 7f3ed58e3640 1 --2- 172.21.15.67:0/2150205066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ed0107e30 0x7f3ed019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:15:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.086+0000 7f3ed68e5640 1 --2- 172.21.15.67:0/2150205066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ed0108780 0x7f3ed019f7c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800cad0 tx=0x7f3ec800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.086+0000 7f3ec37fe640 1 -- 172.21.15.67:0/2150205066 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec800ae20 con 0x7f3ed0108780 2024-07-27T03:15:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.086+0000 7f3ec37fe640 1 -- 172.21.15.67:0/2150205066 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ec8005020 con 0x7f3ed0108780 2024-07-27T03:15:52.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.086+0000 7f3ed8348640 1 -- 172.21.15.67:0/2150205066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed01a01e0 con 0x7f3ed0108780 2024-07-27T03:15:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.086+0000 7f3ec37fe640 1 -- 172.21.15.67:0/2150205066 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec800f6b0 con 0x7f3ed0108780 2024-07-27T03:15:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.086+0000 7f3ed8348640 1 -- 172.21.15.67:0/2150205066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed01a6940 con 0x7f3ed0108780 2024-07-27T03:15:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.089+0000 7f3ede8cd740 1 Processor -- start 2024-07-27T03:15:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.090+0000 7f3ec37fe640 1 -- 172.21.15.67:0/2150205066 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f3ec80051d0 con 0x7f3ed0108780 2024-07-27T03:15:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.090+0000 7f3ede8cd740 1 -- start start 2024-07-27T03:15:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.090+0000 7f3ede8cd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641d669ff60 0x5641d6a45220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.090+0000 7f3ede8cd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d66770a0 0x5641d6a45760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.090+0000 7f3ede8cd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d667f280 0x5641d6a45ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.090+0000 7f3ede8cd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5641d692d2d0 con 0x5641d667f280 2024-07-27T03:15:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.090+0000 7f3ede8cd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5641d69bb8b0 con 0x5641d66770a0 2024-07-27T03:15:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.090+0000 7f3ede8cd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5641d69aadb0 con 0x5641d669ff60 2024-07-27T03:15:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.091+0000 7f3ed58e3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d667f280 0x5641d6a45ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:52.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.091+0000 7f3ed60e4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d66770a0 0x5641d6a45760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:52.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.091+0000 7f3ed60e4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d66770a0 0x5641d6a45760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.67:59224/0 (socket says 172.21.15.67:59224) 2024-07-27T03:15:52.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.091+0000 7f3ed58e3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d667f280 0x5641d6a45ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52304/0 (socket says 172.21.15.67:52304) 2024-07-27T03:15:52.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.091+0000 7f3ed60e4640 1 -- 172.21.15.67:0/2625442208 learned_addr learned my addr 172.21.15.67:0/2625442208 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:52.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.091+0000 7f3ed58e3640 1 --2- 172.21.15.67:0/2625442208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641d669ff60 0x5641d6a45220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:52.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.091+0000 7f3ec37fe640 1 --2- 172.21.15.67:0/2150205066 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3ebc0b3bd0 0x7f3ebc0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:52.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.091+0000 7f3ed60e4640 1 -- 172.21.15.67:0/2625442208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641d669ff60 msgr2=0x5641d6a45220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:52.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.091+0000 7f3ed60e4640 1 --2- 172.21.15.67:0/2625442208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641d669ff60 0x5641d6a45220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.091+0000 7f3ed60e4640 1 -- 172.21.15.67:0/2625442208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d667f280 msgr2=0x5641d6a45ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:52.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.091+0000 7f3ed60e4640 1 --2- 172.21.15.67:0/2625442208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d667f280 0x5641d6a45ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.091+0000 7f3ed60e4640 1 -- 172.21.15.67:0/2625442208 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641d69e4b10 con 0x5641d66770a0 2024-07-27T03:15:52.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.091+0000 7f3ed68e5640 1 --2- 172.21.15.67:0/2150205066 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3ebc0b3bd0 0x7f3ebc0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:52.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.091+0000 7f3ec37fe640 1 -- 172.21.15.67:0/2150205066 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f3ec80e3f40 con 0x7f3ed0108780 2024-07-27T03:15:52.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.091+0000 7f3ed60e4640 1 --2- 172.21.15.67:0/2625442208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d66770a0 0x5641d6a45760 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009900 tx=0x7f3ecc032e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5726fd34250f912 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:52.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.092+0000 7f3ec0d9a640 1 -- 172.21.15.67:0/2625442208 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc040070 con 0x5641d66770a0 2024-07-27T03:15:52.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.092+0000 7f3ed68e5640 1 --2- 172.21.15.67:0/2150205066 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3ebc0b3bd0 0x7f3ebc0b60e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3ec806a960 tx=0x7f3ec806fa00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:52.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.092+0000 7f3ec0d9a640 1 -- 172.21.15.67:0/2625442208 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ecc0316a0 con 0x5641d66770a0 2024-07-27T03:15:52.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.092+0000 7f3ec0d9a640 1 -- 172.21.15.67:0/2625442208 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc035040 con 0x5641d66770a0 2024-07-27T03:15:52.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.093+0000 7f3ede8cd740 1 -- 172.21.15.67:0/2625442208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d66770a0 msgr2=0x5641d6a45760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.093+0000 7f3ede8cd740 1 --2- 172.21.15.67:0/2625442208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d66770a0 0x5641d6a45760 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009900 tx=0x7f3ecc032e60 comp rx=0 tx=0).stop 2024-07-27T03:15:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.093+0000 7f3ede8cd740 1 -- 172.21.15.67:0/2625442208 shutdown_connections 2024-07-27T03:15:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.093+0000 7f3ede8cd740 1 --2- 172.21.15.67:0/2625442208 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d667f280 0x5641d6a45ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.093+0000 7f3ede8cd740 1 --2- 172.21.15.67:0/2625442208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d66770a0 0x5641d6a45760 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.093+0000 7f3ede8cd740 1 --2- 172.21.15.67:0/2625442208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641d669ff60 0x5641d6a45220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.093+0000 7f3ede8cd740 1 -- 172.21.15.67:0/2625442208 >> 172.21.15.67:0/2625442208 conn(0x5641d6694350 msgr2=0x5641d6a43110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.093+0000 7f3ede8cd740 1 -- 172.21.15.67:0/2625442208 shutdown_connections 2024-07-27T03:15:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.093+0000 7f3ede8cd740 1 -- 172.21.15.67:0/2625442208 wait complete. 2024-07-27T03:15:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.093+0000 7f3ede8cd740 1 -- start start 2024-07-27T03:15:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.093+0000 7f3ede8cd740 1 Processor -- start 2024-07-27T03:15:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.094+0000 7f3ede8cd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d66770a0 0x5641d6a55270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:52.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.094+0000 7f3ede8cd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d667f280 0x5641d6a557f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:52.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.094+0000 7f3ede8cd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641d669ff60 0x5641d6a60e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:52.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.094+0000 7f3ede8cd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5641d694d7d0 con 0x5641d66770a0 2024-07-27T03:15:52.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.094+0000 7f3ede8cd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5641d69aedf0 con 0x5641d667f280 2024-07-27T03:15:52.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.094+0000 7f3ede8cd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5641d680e5f0 con 0x5641d669ff60 2024-07-27T03:15:52.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.094+0000 7f3ede8cd740 10 client.0 _ll_register_callbacks cb 0x5641d6955800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:15:52.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.094+0000 7f3ed58e3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d66770a0 0x5641d6a55270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:52.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.094+0000 7f3ed58e3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d66770a0 0x5641d6a55270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52316/0 (socket says 172.21.15.67:52316) 2024-07-27T03:15:52.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.094+0000 7f3ed58e3640 1 -- 172.21.15.67:0/1984760701 learned_addr learned my addr 172.21.15.67:0/1984760701 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:52.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.094+0000 7f3ed60e4640 1 --2- 172.21.15.67:0/1984760701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d667f280 0x5641d6a557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:52.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.094+0000 7f3ed58e3640 1 --2- 172.21.15.67:0/1984760701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641d669ff60 0x5641d6a60e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:52.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.094+0000 7f3ed58e3640 1 -- 172.21.15.67:0/1984760701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641d669ff60 msgr2=0x5641d6a60e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:52.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.094+0000 7f3ed58e3640 1 --2- 172.21.15.67:0/1984760701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641d669ff60 0x5641d6a60e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.094+0000 7f3ed58e3640 1 -- 172.21.15.67:0/1984760701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d667f280 msgr2=0x5641d6a557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:52.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.094+0000 7f3ed58e3640 1 --2- 172.21.15.67:0/1984760701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d667f280 0x5641d6a557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.094+0000 7f3ed58e3640 1 -- 172.21.15.67:0/1984760701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641d6940640 con 0x5641d66770a0 2024-07-27T03:15:52.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.094+0000 7f3ed58e3640 1 --2- 172.21.15.67:0/1984760701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d66770a0 0x5641d6a55270 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4012d90 tx=0x7f3ec4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38193a5fa02abc98 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:52.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.094+0000 7f3eb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:15:52.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.095+0000 7f3eb9ffb640 1 -- 172.21.15.67:0/1984760701 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec401c040 con 0x5641d66770a0 2024-07-27T03:15:52.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.095+0000 7f3ede8cd740 1 -- 172.21.15.67:0/1984760701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ecc0316a0 con 0x5641d66770a0 2024-07-27T03:15:52.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.095+0000 7f3eb9ffb640 1 -- 172.21.15.67:0/1984760701 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f3ec4005220 con 0x5641d66770a0 2024-07-27T03:15:52.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.095+0000 7f3eb9ffb640 1 -- 172.21.15.67:0/1984760701 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3ec4028050 con 0x5641d66770a0 2024-07-27T03:15:52.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.095+0000 7f3ede8cd740 10 client.15276 fetch_fsmap learned FSMap version 29 2024-07-27T03:15:52.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.095+0000 7f3ede8cd740 1 -- 172.21.15.67:0/1984760701 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5641d6951ac0 con 0x5641d66770a0 2024-07-27T03:15:52.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.095+0000 7f3eb9ffb640 1 -- 172.21.15.67:0/1984760701 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ecc0316a0 con 0x5641d66770a0 2024-07-27T03:15:52.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.095+0000 7f3eb9ffb640 1 -- 172.21.15.67:0/1984760701 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f3ec400f580 con 0x5641d66770a0 2024-07-27T03:15:52.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.095+0000 7f3ede8cd740 10 client.15276 fetch_fsmap finished waiting for FSMap version 29 2024-07-27T03:15:52.104 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.095+0000 7f3ede8cd740 10 client.15276 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-27T03:15:52.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.095+0000 7f3ede8cd740 20 client.15276 populate_metadata read hostname 'smithi067' 2024-07-27T03:15:52.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.095+0000 7f3ede8cd740 1 --2- 172.21.15.67:0/1984760701 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5641d665e8e0 0x5641d6a67500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:52.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.095+0000 7f3ede8cd740 4 client.15276 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:15:52.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.095+0000 7f3ede8cd740 1 -- 172.21.15.67:0/1984760701 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641d69bd800 con 0x5641d665e8e0 2024-07-27T03:15:52.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.096+0000 7f3ed60e4640 1 --2- 172.21.15.67:0/1984760701 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5641d665e8e0 0x5641d6a67500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:52.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.096+0000 7f3eb9ffb640 10 client.15276 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:15:52.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.096+0000 7f3ed60e4640 1 --2- 172.21.15.67:0/1984760701 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5641d665e8e0 0x5641d6a67500 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1631e23ef131fba1 server_cookie=523a9c2442d92aff in_seq=0 out_seq=0 2024-07-27T03:15:52.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.097+0000 7f3eb9ffb640 1 -- 172.21.15.67:0/1984760701 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641d69bd800 con 0x5641d665e8e0 2024-07-27T03:15:52.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.097+0000 7f3eb9ffb640 10 client.15276 handle_command_reply: tid=0 2024-07-27T03:15:52.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.097+0000 7f3ede8cd740 1 client.15276 shutdown 2024-07-27T03:15:52.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.097+0000 7f3ede8cd740 1 -- 172.21.15.67:0/1984760701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d66770a0 msgr2=0x5641d6a55270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:52.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.097+0000 7f3ede8cd740 1 --2- 172.21.15.67:0/1984760701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d66770a0 0x5641d6a55270 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4012d90 tx=0x7f3ec4004730 comp rx=0 tx=0).stop 2024-07-27T03:15:52.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.097+0000 7f3ede8cd740 1 -- 172.21.15.67:0/1984760701 shutdown_connections 2024-07-27T03:15:52.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.097+0000 7f3ede8cd740 1 --2- 172.21.15.67:0/1984760701 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5641d665e8e0 0x5641d6a67500 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.097+0000 7f3ede8cd740 1 --2- 172.21.15.67:0/1984760701 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641d669ff60 0x5641d6a60e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.097+0000 7f3ede8cd740 1 --2- 172.21.15.67:0/1984760701 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d667f280 0x5641d6a557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.097+0000 7f3ede8cd740 1 --2- 172.21.15.67:0/1984760701 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d66770a0 0x5641d6a55270 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.097+0000 7f3ede8cd740 1 -- 172.21.15.67:0/1984760701 >> 172.21.15.67:0/1984760701 conn(0x5641d6694350 msgr2=0x5641d6a41710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:52.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.097+0000 7f3eb9ffb640 0 client.15276 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:15:52.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.097+0000 7f3ede8cd740 1 -- 172.21.15.67:0/1984760701 shutdown_connections 2024-07-27T03:15:52.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.097+0000 7f3ede8cd740 1 -- 172.21.15.67:0/1984760701 wait complete. 2024-07-27T03:15:52.106 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.098+0000 7f3ede8cd740 20 client.15276 trim_cache size 0 max 16384 2024-07-27T03:15:52.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.115+0000 7f3ede8cd740 1 Processor -- start 2024-07-27T03:15:52.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.115+0000 7f3ede8cd740 1 -- start start 2024-07-27T03:15:52.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.115+0000 7f3ede8cd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641d66770a0 0x5641d6a67500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:52.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.115+0000 7f3ede8cd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d667f280 0x5641d6a55250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:52.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.116+0000 7f3ede8cd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d669ff60 0x5641d6a55790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:52.119 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.116+0000 7f3ede8cd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5641d6a65840 con 0x5641d669ff60 2024-07-27T03:15:52.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.116+0000 7f3ede8cd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5641d6a65f10 con 0x5641d667f280 2024-07-27T03:15:52.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.116+0000 7f3ede8cd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5641d6a665e0 con 0x5641d66770a0 2024-07-27T03:15:52.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.116+0000 7f3ed58e3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d669ff60 0x5641d6a55790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:52.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.116+0000 7f3ed58e3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d669ff60 0x5641d6a55790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52326/0 (socket says 172.21.15.67:52326) 2024-07-27T03:15:52.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.116+0000 7f3ed58e3640 1 -- 172.21.15.67:0/857719981 learned_addr learned my addr 172.21.15.67:0/857719981 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:52.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.116+0000 7f3ed60e4640 1 --2- 172.21.15.67:0/857719981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d667f280 0x5641d6a55250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:52.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.116+0000 7f3ed58e3640 1 --2- 172.21.15.67:0/857719981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641d66770a0 0x5641d6a67500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:52.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.116+0000 7f3ed58e3640 1 -- 172.21.15.67:0/857719981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641d66770a0 msgr2=0x5641d6a67500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:52.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.116+0000 7f3ed58e3640 1 --2- 172.21.15.67:0/857719981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641d66770a0 0x5641d6a67500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.116+0000 7f3ed58e3640 1 -- 172.21.15.67:0/857719981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d667f280 msgr2=0x5641d6a55250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:52.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.116+0000 7f3ed58e3640 1 --2- 172.21.15.67:0/857719981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d667f280 0x5641d6a55250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.116+0000 7f3ed58e3640 1 -- 172.21.15.67:0/857719981 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641d670b540 con 0x5641d669ff60 2024-07-27T03:15:52.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.116+0000 7f3ed60e4640 1 --2- 172.21.15.67:0/857719981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d667f280 0x5641d6a55250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:15:52.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.116+0000 7f3ed58e3640 1 --2- 172.21.15.67:0/857719981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d669ff60 0x5641d6a55790 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4001eb0 tx=0x7f3ec40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fabb9ba87c6de3 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:52.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.116+0000 7f3ebbfff640 1 -- 172.21.15.67:0/857719981 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4030030 con 0x5641d669ff60 2024-07-27T03:15:52.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.117+0000 7f3ebbfff640 1 -- 172.21.15.67:0/857719981 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ec400fb40 con 0x5641d669ff60 2024-07-27T03:15:52.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.117+0000 7f3ebbfff640 1 -- 172.21.15.67:0/857719981 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4021b30 con 0x5641d669ff60 2024-07-27T03:15:52.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.117+0000 7f3ede8cd740 1 -- 172.21.15.67:0/857719981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d669ff60 msgr2=0x5641d6a55790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:52.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.117+0000 7f3ede8cd740 1 --2- 172.21.15.67:0/857719981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d669ff60 0x5641d6a55790 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4001eb0 tx=0x7f3ec40046c0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.117+0000 7f3ede8cd740 1 -- 172.21.15.67:0/857719981 shutdown_connections 2024-07-27T03:15:52.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.117+0000 7f3ede8cd740 1 --2- 172.21.15.67:0/857719981 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d669ff60 0x5641d6a55790 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.117+0000 7f3ede8cd740 1 --2- 172.21.15.67:0/857719981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d667f280 0x5641d6a55250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.117+0000 7f3ede8cd740 1 --2- 172.21.15.67:0/857719981 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641d66770a0 0x5641d6a67500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.117+0000 7f3ede8cd740 1 -- 172.21.15.67:0/857719981 >> 172.21.15.67:0/857719981 conn(0x5641d6694350 msgr2=0x5641d6a50250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:52.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.117+0000 7f3ede8cd740 1 -- 172.21.15.67:0/857719981 shutdown_connections 2024-07-27T03:15:52.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.117+0000 7f3ede8cd740 1 -- 172.21.15.67:0/857719981 wait complete. 2024-07-27T03:15:52.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.118+0000 7f3ede8cd740 1 -- start start 2024-07-27T03:15:52.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.118+0000 7f3ede8cd740 1 Processor -- start 2024-07-27T03:15:52.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.118+0000 7f3ede8cd740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d66770a0 0x5641d6a68660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:52.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.118+0000 7f3ede8cd740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d667f280 0x5641d6a68c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:52.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.118+0000 7f3ede8cd740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641d669ff60 0x5641d6a6dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:52.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.118+0000 7f3ede8cd740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5641d6a67380 con 0x5641d667f280 2024-07-27T03:15:52.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.118+0000 7f3ede8cd740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5641d6a66cb0 con 0x5641d66770a0 2024-07-27T03:15:52.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.118+0000 7f3ede8cd740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5641d6a67a50 con 0x5641d669ff60 2024-07-27T03:15:52.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.118+0000 7f3ede8cd740 10 client.0 _ll_register_callbacks cb 0x7f3ec400cde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:15:52.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.118+0000 7f3ed60e4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d667f280 0x5641d6a68c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:52.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.118+0000 7f3ed60e4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d667f280 0x5641d6a68c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52332/0 (socket says 172.21.15.67:52332) 2024-07-27T03:15:52.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.118+0000 7f3ed60e4640 1 -- 172.21.15.67:0/2711290338 learned_addr learned my addr 172.21.15.67:0/2711290338 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:52.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.119+0000 7f3ed58e3640 1 --2- 172.21.15.67:0/2711290338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d66770a0 0x5641d6a68660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:52.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.119+0000 7f3ed60e4640 1 -- 172.21.15.67:0/2711290338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641d669ff60 msgr2=0x5641d6a6dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:52.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.119+0000 7f3ed58e3640 1 --2- 172.21.15.67:0/2711290338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641d669ff60 0x5641d6a6dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:52.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.119+0000 7f3ed60e4640 1 --2- 172.21.15.67:0/2711290338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641d669ff60 0x5641d6a6dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.119+0000 7f3ed60e4640 1 -- 172.21.15.67:0/2711290338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d66770a0 msgr2=0x5641d6a68660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:15:52.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.119+0000 7f3ed60e4640 1 --2- 172.21.15.67:0/2711290338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d66770a0 0x5641d6a68660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.119+0000 7f3ed60e4640 1 -- 172.21.15.67:0/2711290338 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641d6a6e1e0 con 0x5641d667f280 2024-07-27T03:15:52.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.119+0000 7f3ed60e4640 1 --2- 172.21.15.67:0/2711290338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d667f280 0x5641d6a68c00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0053b0 tx=0x7f3ecc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95fbee5b45ce1e57 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:52.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.119+0000 7f3eb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:15:52.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.119+0000 7f3eb9ffb640 1 -- 172.21.15.67:0/2711290338 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc035040 con 0x5641d667f280 2024-07-27T03:15:52.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.119+0000 7f3eb9ffb640 1 -- 172.21.15.67:0/2711290338 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f3ecc031b10 con 0x5641d667f280 2024-07-27T03:15:52.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.119+0000 7f3ede8cd740 1 -- 172.21.15.67:0/2711290338 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ec400fb40 con 0x5641d667f280 2024-07-27T03:15:52.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.119+0000 7f3eb9ffb640 1 -- 172.21.15.67:0/2711290338 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3ecc05d050 con 0x5641d667f280 2024-07-27T03:15:52.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.119+0000 7f3ede8cd740 10 client.15282 fetch_fsmap learned FSMap version 29 2024-07-27T03:15:52.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.119+0000 7f3ede8cd740 1 -- 172.21.15.67:0/2711290338 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5641d6a6f5c0 con 0x5641d667f280 2024-07-27T03:15:52.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.120+0000 7f3eb9ffb640 1 -- 172.21.15.67:0/2711290338 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ec400fb40 con 0x5641d667f280 2024-07-27T03:15:52.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.120+0000 7f3eb9ffb640 1 -- 172.21.15.67:0/2711290338 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f3ecc0035f0 con 0x5641d667f280 2024-07-27T03:15:52.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.120+0000 7f3ede8cd740 10 client.15282 fetch_fsmap finished waiting for FSMap version 29 2024-07-27T03:15:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.120+0000 7f3ede8cd740 10 client.15282 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-27T03:15:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.120+0000 7f3ede8cd740 20 client.15282 populate_metadata read hostname 'smithi067' 2024-07-27T03:15:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.120+0000 7f3ede8cd740 1 --2- 172.21.15.67:0/2711290338 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5641d6633c90 0x5641d6ad2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.120+0000 7f3ede8cd740 4 client.15282 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-27T03:15:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.120+0000 7f3ede8cd740 1 -- 172.21.15.67:0/2711290338 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "status"}) -- 0x5641d69b92a0 con 0x5641d6633c90 2024-07-27T03:15:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.120+0000 7f3ed58e3640 1 --2- 172.21.15.67:0/2711290338 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5641d6633c90 0x5641d6ad2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.120+0000 7f3eb9ffb640 10 client.15282 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:15:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.120+0000 7f3ed58e3640 1 --2- 172.21.15.67:0/2711290338 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5641d6633c90 0x5641d6ad2c20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36809e3692245a75 server_cookie=e23fae8dfcbed942 in_seq=0 out_seq=0 2024-07-27T03:15:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.121+0000 7f3eb9ffb640 1 -- 172.21.15.67:0/2711290338 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5641d69b92a0 con 0x5641d6633c90 2024-07-27T03:15:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.121+0000 7f3eb9ffb640 10 client.15282 handle_command_reply: tid=0 2024-07-27T03:15:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.121+0000 7f3ede8cd740 1 client.15282 shutdown 2024-07-27T03:15:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.121+0000 7f3ede8cd740 1 -- 172.21.15.67:0/2711290338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d667f280 msgr2=0x5641d6a68c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:52.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.121+0000 7f3ede8cd740 1 --2- 172.21.15.67:0/2711290338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d667f280 0x5641d6a68c00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0053b0 tx=0x7f3ecc02a410 comp rx=0 tx=0).stop 2024-07-27T03:15:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.121+0000 7f3ede8cd740 1 -- 172.21.15.67:0/2711290338 shutdown_connections 2024-07-27T03:15:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.121+0000 7f3ede8cd740 1 --2- 172.21.15.67:0/2711290338 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5641d6633c90 0x5641d6ad2c20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.121+0000 7f3ede8cd740 1 --2- 172.21.15.67:0/2711290338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5641d669ff60 0x5641d6a6dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.121+0000 7f3ede8cd740 1 --2- 172.21.15.67:0/2711290338 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5641d667f280 0x5641d6a68c00 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.121+0000 7f3ede8cd740 1 --2- 172.21.15.67:0/2711290338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641d66770a0 0x5641d6a68660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.121+0000 7f3ede8cd740 1 -- 172.21.15.67:0/2711290338 >> 172.21.15.67:0/2711290338 conn(0x5641d6694350 msgr2=0x5641d6a50250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.121+0000 7f3eb9ffb640 0 client.15282 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:15:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.121+0000 7f3ede8cd740 1 -- 172.21.15.67:0/2711290338 shutdown_connections 2024-07-27T03:15:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.121+0000 7f3ede8cd740 1 -- 172.21.15.67:0/2711290338 wait complete. 2024-07-27T03:15:52.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.121+0000 7f3ede8cd740 20 client.15282 trim_cache size 0 max 16384 2024-07-27T03:15:52.129 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:15:52.130 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "c0782a00-4bc4-11ef-bcc7-c7b262605968", 2024-07-27T03:15:52.130 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-07-27T03:15:52.130 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24620, 2024-07-27T03:15:52.130 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-27T03:15:52.130 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-27T03:15:52.130 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-27T03:15:52.130 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 197.55811258200001, 2024-07-27T03:15:52.130 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 29, 2024-07-27T03:15:52.130 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 91, 2024-07-27T03:15:52.130 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-27T03:15:52.130 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 211.62535949900001 2024-07-27T03:15:52.131 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:15:52.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.122+0000 7f3ed8348640 1 -- 172.21.15.67:0/2150205066 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3ebc0b3bd0 msgr2=0x7f3ebc0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:52.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.122+0000 7f3ed8348640 1 --2- 172.21.15.67:0/2150205066 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3ebc0b3bd0 0x7f3ebc0b60e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3ec806a960 tx=0x7f3ec806fa00 comp rx=0 tx=0).stop 2024-07-27T03:15:52.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.122+0000 7f3ed8348640 1 -- 172.21.15.67:0/2150205066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ed0108780 msgr2=0x7f3ed019f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:52.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.122+0000 7f3ed8348640 1 --2- 172.21.15.67:0/2150205066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ed0108780 0x7f3ed019f7c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800cad0 tx=0x7f3ec800cf90 comp rx=0 tx=0).stop 2024-07-27T03:15:52.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.122+0000 7f3ed8348640 1 -- 172.21.15.67:0/2150205066 shutdown_connections 2024-07-27T03:15:52.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.122+0000 7f3ed8348640 1 --2- 172.21.15.67:0/2150205066 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3ebc0b3bd0 0x7f3ebc0b60e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.122+0000 7f3ed8348640 1 --2- 172.21.15.67:0/2150205066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3ed0108780 0x7f3ed019f7c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.122+0000 7f3ed8348640 1 --2- 172.21.15.67:0/2150205066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ed0107e30 0x7f3ed019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.122+0000 7f3ed8348640 1 --2- 172.21.15.67:0/2150205066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ed01024f0 0x7f3ed0199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:52.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.122+0000 7f3ed8348640 1 -- 172.21.15.67:0/2150205066 >> 172.21.15.67:0/2150205066 conn(0x7f3ed00791b0 msgr2=0x7f3ed0100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:52.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.122+0000 7f3ed8348640 1 -- 172.21.15.67:0/2150205066 shutdown_connections 2024-07-27T03:15:52.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:52.122+0000 7f3ed8348640 1 -- 172.21.15.67:0/2150205066 wait complete. 2024-07-27T03:15:52.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:52 smithi067 ceph-mon[32814]: pgmap v409: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:52.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:52 smithi149 ceph-mon[37173]: pgmap v409: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:52.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:52 smithi175 ceph-mon[37125]: pgmap v409: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:52.802 INFO:tasks.cephadm:Setup complete, yielding 2024-07-27T03:15:52.802 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-27T03:15:52.813 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-07-27T03:15:52.814 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- bash -c 'ceph orch status' 2024-07-27T03:15:53.080 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:15:54.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:54 smithi067 ceph-mon[32814]: pgmap v410: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:54.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:54 smithi149 ceph-mon[37173]: pgmap v410: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:54.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:54 smithi175 ceph-mon[37125]: pgmap v410: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:54.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.780+0000 7f0bd8ce3640 1 -- 172.21.15.67:0/37656660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bd4103af0 msgr2=0x7f0bd410a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:54.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.780+0000 7f0bd8ce3640 1 --2- 172.21.15.67:0/37656660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bd4103af0 0x7f0bd410a490 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009ca0 tx=0x7f0bcc02a950 comp rx=0 tx=0).stop 2024-07-27T03:15:54.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.780+0000 7f0bd8ce3640 1 -- 172.21.15.67:0/37656660 shutdown_connections 2024-07-27T03:15:54.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.780+0000 7f0bd8ce3640 1 --2- 172.21.15.67:0/37656660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bd4103af0 0x7f0bd410a490 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:54.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.780+0000 7f0bd8ce3640 1 --2- 172.21.15.67:0/37656660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd4103150 0x7f0bd4103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:54.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.780+0000 7f0bd8ce3640 1 --2- 172.21.15.67:0/37656660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bd41020c0 0x7f0bd4102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:54.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.780+0000 7f0bd8ce3640 1 -- 172.21.15.67:0/37656660 >> 172.21.15.67:0/37656660 conn(0x7f0bd40fda50 msgr2=0x7f0bd40ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:54.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.780+0000 7f0bd8ce3640 1 -- 172.21.15.67:0/37656660 shutdown_connections 2024-07-27T03:15:54.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.780+0000 7f0bd8ce3640 1 -- 172.21.15.67:0/37656660 wait complete. 2024-07-27T03:15:54.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.781+0000 7f0bd8ce3640 1 Processor -- start 2024-07-27T03:15:54.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.781+0000 7f0bd8ce3640 1 -- start start 2024-07-27T03:15:54.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.781+0000 7f0bd8ce3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bd41020c0 0x7f0bd419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:54.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.781+0000 7f0bd8ce3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bd4103150 0x7f0bd419c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:54.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.781+0000 7f0bd8ce3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd4103af0 0x7f0bd419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:54.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.781+0000 7f0bd8ce3640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd410ee20 con 0x7f0bd41020c0 2024-07-27T03:15:54.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.781+0000 7f0bd8ce3640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd410eca0 con 0x7f0bd4103150 2024-07-27T03:15:54.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.781+0000 7f0bd8ce3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd410efa0 con 0x7f0bd4103af0 2024-07-27T03:15:54.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.781+0000 7f0bd259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bd41020c0 0x7f0bd419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:54.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.781+0000 7f0bd259c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bd41020c0 0x7f0bd419bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52352/0 (socket says 172.21.15.67:52352) 2024-07-27T03:15:54.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.781+0000 7f0bd259c640 1 -- 172.21.15.67:0/2482567557 learned_addr learned my addr 172.21.15.67:0/2482567557 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:54.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.782+0000 7f0bd1d9b640 1 --2- 172.21.15.67:0/2482567557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bd4103150 0x7f0bd419c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:54.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.782+0000 7f0bd2d9d640 1 --2- 172.21.15.67:0/2482567557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd4103af0 0x7f0bd419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:54.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.782+0000 7f0bd259c640 1 -- 172.21.15.67:0/2482567557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd4103af0 msgr2=0x7f0bd419cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:54.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.782+0000 7f0bd259c640 1 --2- 172.21.15.67:0/2482567557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd4103af0 0x7f0bd419cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:54.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.782+0000 7f0bd259c640 1 -- 172.21.15.67:0/2482567557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bd4103150 msgr2=0x7f0bd419c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:54.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.782+0000 7f0bd259c640 1 --2- 172.21.15.67:0/2482567557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bd4103150 0x7f0bd419c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:54.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.782+0000 7f0bd259c640 1 -- 172.21.15.67:0/2482567557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd419d610 con 0x7f0bd41020c0 2024-07-27T03:15:54.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.782+0000 7f0bd259c640 1 --2- 172.21.15.67:0/2482567557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bd41020c0 0x7f0bd419bbe0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400e9f0 tx=0x7f0bc400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:54.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.782+0000 7f0bc37fe640 1 -- 172.21.15.67:0/2482567557 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc400cd80 con 0x7f0bd41020c0 2024-07-27T03:15:54.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.782+0000 7f0bc37fe640 1 -- 172.21.15.67:0/2482567557 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bc40108e0 con 0x7f0bd41020c0 2024-07-27T03:15:54.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.782+0000 7f0bd8ce3640 1 -- 172.21.15.67:0/2482567557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd419d900 con 0x7f0bd41020c0 2024-07-27T03:15:54.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.782+0000 7f0bc37fe640 1 -- 172.21.15.67:0/2482567557 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4018980 con 0x7f0bd41020c0 2024-07-27T03:15:54.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.782+0000 7f0bd8ce3640 1 -- 172.21.15.67:0/2482567557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd41a4000 con 0x7f0bd41020c0 2024-07-27T03:15:54.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.784+0000 7f0bd8ce3640 1 -- 172.21.15.67:0/2482567557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bd41a41d0 con 0x7f0bd41020c0 2024-07-27T03:15:54.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.785+0000 7f0bc37fe640 1 -- 172.21.15.67:0/2482567557 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f0bc4010430 con 0x7f0bd41020c0 2024-07-27T03:15:54.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.786+0000 7f0bc37fe640 1 --2- 172.21.15.67:0/2482567557 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0bbc0b3c50 0x7f0bbc0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:54.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.786+0000 7f0bd1d9b640 1 --2- 172.21.15.67:0/2482567557 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0bbc0b3c50 0x7f0bbc0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:54.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.786+0000 7f0bc37fe640 1 -- 172.21.15.67:0/2482567557 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f0bc40e4390 con 0x7f0bd41020c0 2024-07-27T03:15:54.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.786+0000 7f0bd1d9b640 1 --2- 172.21.15.67:0/2482567557 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0bbc0b3c50 0x7f0bbc0b6160 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8005fd0 tx=0x7f0bc8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:54.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.790+0000 7f0bc37fe640 1 -- 172.21.15.67:0/2482567557 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f0bc40ab630 con 0x7f0bd41020c0 2024-07-27T03:15:54.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.989+0000 7f0bd8ce3640 1 -- 172.21.15.67:0/2482567557 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f0bd41a43a0 con 0x7f0bbc0b3c50 2024-07-27T03:15:54.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.990+0000 7f0bc37fe640 1 -- 172.21.15.67:0/2482567557 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f0bd41a43a0 con 0x7f0bbc0b3c50 2024-07-27T03:15:54.993 INFO:teuthology.orchestra.run.smithi067.stdout:Backend: cephadm 2024-07-27T03:15:54.993 INFO:teuthology.orchestra.run.smithi067.stdout:Available: Yes 2024-07-27T03:15:54.993 INFO:teuthology.orchestra.run.smithi067.stdout:Paused: No 2024-07-27T03:15:54.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.992+0000 7f0bd8ce3640 1 -- 172.21.15.67:0/2482567557 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0bbc0b3c50 msgr2=0x7f0bbc0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:54.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.992+0000 7f0bd8ce3640 1 --2- 172.21.15.67:0/2482567557 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0bbc0b3c50 0x7f0bbc0b6160 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8005fd0 tx=0x7f0bc8005850 comp rx=0 tx=0).stop 2024-07-27T03:15:54.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.992+0000 7f0bd8ce3640 1 -- 172.21.15.67:0/2482567557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bd41020c0 msgr2=0x7f0bd419bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:54.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.992+0000 7f0bd8ce3640 1 --2- 172.21.15.67:0/2482567557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bd41020c0 0x7f0bd419bbe0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400e9f0 tx=0x7f0bc400eeb0 comp rx=0 tx=0).stop 2024-07-27T03:15:54.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.992+0000 7f0bd8ce3640 1 -- 172.21.15.67:0/2482567557 shutdown_connections 2024-07-27T03:15:54.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.992+0000 7f0bd8ce3640 1 --2- 172.21.15.67:0/2482567557 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f0bbc0b3c50 0x7f0bbc0b6160 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:54.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.992+0000 7f0bd8ce3640 1 --2- 172.21.15.67:0/2482567557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd4103af0 0x7f0bd419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:54.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.992+0000 7f0bd8ce3640 1 --2- 172.21.15.67:0/2482567557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0bd4103150 0x7f0bd419c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:54.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.992+0000 7f0bd8ce3640 1 --2- 172.21.15.67:0/2482567557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0bd41020c0 0x7f0bd419bbe0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:54.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.993+0000 7f0bd8ce3640 1 -- 172.21.15.67:0/2482567557 >> 172.21.15.67:0/2482567557 conn(0x7f0bd40fda50 msgr2=0x7f0bd410adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:54.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.993+0000 7f0bd8ce3640 1 -- 172.21.15.67:0/2482567557 shutdown_connections 2024-07-27T03:15:54.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:54.993+0000 7f0bd8ce3640 1 -- 172.21.15.67:0/2482567557 wait complete. 2024-07-27T03:15:55.627 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- bash -c 'ceph orch ps' 2024-07-27T03:15:55.893 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:15:56.275 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:56 smithi067 ceph-mon[32814]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:15:56.275 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:56 smithi067 ceph-mon[32814]: pgmap v411: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:56.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:56 smithi149 ceph-mon[37173]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:15:56.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:56 smithi149 ceph-mon[37173]: pgmap v411: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:56.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:56 smithi175 ceph-mon[37125]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:15:56.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:56 smithi175 ceph-mon[37125]: pgmap v411: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:15:57.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.551+0000 7fafbbc9a640 1 -- 172.21.15.67:0/1288669722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb41020b0 msgr2=0x7fafb4102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:57.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.551+0000 7fafbbc9a640 1 --2- 172.21.15.67:0/1288669722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb41020b0 0x7fafb4102520 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009900 tx=0x7fafb002a480 comp rx=0 tx=0).stop 2024-07-27T03:15:57.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.551+0000 7fafbbc9a640 1 -- 172.21.15.67:0/1288669722 shutdown_connections 2024-07-27T03:15:57.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.551+0000 7fafbbc9a640 1 --2- 172.21.15.67:0/1288669722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb4103ae0 0x7fafb410a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:57.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.551+0000 7fafbbc9a640 1 --2- 172.21.15.67:0/1288669722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafb4103140 0x7fafb4103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:57.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.551+0000 7fafbbc9a640 1 --2- 172.21.15.67:0/1288669722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb41020b0 0x7fafb4102520 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:57.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.551+0000 7fafbbc9a640 1 -- 172.21.15.67:0/1288669722 >> 172.21.15.67:0/1288669722 conn(0x7fafb40fda20 msgr2=0x7fafb40ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:57.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.551+0000 7fafbbc9a640 1 -- 172.21.15.67:0/1288669722 shutdown_connections 2024-07-27T03:15:57.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.551+0000 7fafbbc9a640 1 -- 172.21.15.67:0/1288669722 wait complete. 2024-07-27T03:15:57.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.551+0000 7fafbbc9a640 1 Processor -- start 2024-07-27T03:15:57.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.551+0000 7fafbbc9a640 1 -- start start 2024-07-27T03:15:57.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.552+0000 7fafbbc9a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafb41020b0 0x7fafb4105ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:57.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.552+0000 7fafbbc9a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb4103140 0x7fafb4104570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:57.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.552+0000 7fafbbc9a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb4103ae0 0x7fafb4104ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:57.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.552+0000 7fafbbc9a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fafb410dd10 con 0x7fafb4103140 2024-07-27T03:15:57.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.552+0000 7fafbbc9a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fafb410db90 con 0x7fafb4103ae0 2024-07-27T03:15:57.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.552+0000 7fafbbc9a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fafb410de90 con 0x7fafb41020b0 2024-07-27T03:15:57.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.552+0000 7fafb9235640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb4103140 0x7fafb4104570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:57.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.552+0000 7fafb9235640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb4103140 0x7fafb4104570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52092/0 (socket says 172.21.15.67:52092) 2024-07-27T03:15:57.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.552+0000 7fafb9235640 1 -- 172.21.15.67:0/345130841 learned_addr learned my addr 172.21.15.67:0/345130841 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:15:57.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.552+0000 7fafba237640 1 --2- 172.21.15.67:0/345130841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb4103ae0 0x7fafb4104ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:57.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.552+0000 7fafb9a36640 1 --2- 172.21.15.67:0/345130841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafb41020b0 0x7fafb4105ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:57.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.552+0000 7fafb9235640 1 -- 172.21.15.67:0/345130841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafb41020b0 msgr2=0x7fafb4105ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:57.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.552+0000 7fafb9235640 1 --2- 172.21.15.67:0/345130841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafb41020b0 0x7fafb4105ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:57.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.552+0000 7fafb9235640 1 -- 172.21.15.67:0/345130841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb4103ae0 msgr2=0x7fafb4104ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:57.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.552+0000 7fafb9235640 1 --2- 172.21.15.67:0/345130841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb4103ae0 0x7fafb4104ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:57.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.552+0000 7fafb9235640 1 -- 172.21.15.67:0/345130841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb419d850 con 0x7fafb4103140 2024-07-27T03:15:57.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.552+0000 7fafb9235640 1 --2- 172.21.15.67:0/345130841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb4103140 0x7fafb4104570 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fafa400e9f0 tx=0x7fafa400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:57.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.553+0000 7fafaaffd640 1 -- 172.21.15.67:0/345130841 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa400cd80 con 0x7fafb4103140 2024-07-27T03:15:57.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.553+0000 7fafaaffd640 1 -- 172.21.15.67:0/345130841 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafa40054d0 con 0x7fafb4103140 2024-07-27T03:15:57.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.553+0000 7fafbbc9a640 1 -- 172.21.15.67:0/345130841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb41050b0 con 0x7fafb4103140 2024-07-27T03:15:57.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.553+0000 7fafaaffd640 1 -- 172.21.15.67:0/345130841 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4010690 con 0x7fafb4103140 2024-07-27T03:15:57.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.553+0000 7fafbbc9a640 1 -- 172.21.15.67:0/345130841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb41a8470 con 0x7fafb4103140 2024-07-27T03:15:57.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.554+0000 7fafbbc9a640 1 -- 172.21.15.67:0/345130841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb41a8640 con 0x7fafb4103140 2024-07-27T03:15:57.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.555+0000 7fafaaffd640 1 -- 172.21.15.67:0/345130841 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7fafa4004e30 con 0x7fafb4103140 2024-07-27T03:15:57.559 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.556+0000 7fafaaffd640 1 --2- 172.21.15.67:0/345130841 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fafa00b3c50 0x7fafa00b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:15:57.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.556+0000 7fafb9a36640 1 --2- 172.21.15.67:0/345130841 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fafa00b3c50 0x7fafa00b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:15:57.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.556+0000 7fafaaffd640 1 -- 172.21.15.67:0/345130841 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fafa40e4530 con 0x7fafb4103140 2024-07-27T03:15:57.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.557+0000 7fafb9a36640 1 --2- 172.21.15.67:0/345130841 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fafa00b3c50 0x7fafa00b6160 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fafb0000c00 tx=0x7fafb0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:15:57.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.559+0000 7fafaaffd640 1 -- 172.21.15.67:0/345130841 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fafa4055600 con 0x7fafb4103140 2024-07-27T03:15:57.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.753+0000 7fafbbc9a640 1 -- 172.21.15.67:0/345130841 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fafb40ffa00 con 0x7fafa00b3c50 2024-07-27T03:15:57.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.765+0000 7fafaaffd640 1 -- 172.21.15.67:0/345130841 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fafb40ffa00 con 0x7fafa00b3c50 2024-07-27T03:15:57.768 INFO:teuthology.orchestra.run.smithi067.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-07-27T03:15:57.768 INFO:teuthology.orchestra.run.smithi067.stdout:mds.a smithi067 running (3m) 45s ago 3m 85.7M - 19.0.0-5358-gaf8fdbf0 b2e977707047 ae35ee434933 2024-07-27T03:15:57.768 INFO:teuthology.orchestra.run.smithi067.stdout:mds.b smithi149 running (3m) 83s ago 3m 94.5M - 19.0.0-5358-gaf8fdbf0 b2e977707047 e6c2c871b8ee 2024-07-27T03:15:57.768 INFO:teuthology.orchestra.run.smithi067.stdout:mds.c smithi175 running (3m) 83s ago 3m 85.5M - 19.0.0-5358-gaf8fdbf0 b2e977707047 443e7d24d3cd 2024-07-27T03:15:57.768 INFO:teuthology.orchestra.run.smithi067.stdout:mds.d smithi067 running (4m) 45s ago 3m 82.7M - 19.0.0-5358-gaf8fdbf0 b2e977707047 a30b577e5ac2 2024-07-27T03:15:57.768 INFO:teuthology.orchestra.run.smithi067.stdout:mds.e smithi149 running (3m) 83s ago 3m 88.8M - 19.0.0-5358-gaf8fdbf0 b2e977707047 4ef3b65849c3 2024-07-27T03:15:57.769 INFO:teuthology.orchestra.run.smithi067.stdout:mds.f smithi175 running (3m) 83s ago 3m 86.0M - 19.0.0-5358-gaf8fdbf0 b2e977707047 73a994aac40f 2024-07-27T03:15:57.769 INFO:teuthology.orchestra.run.smithi067.stdout:mds.g smithi067 running (3m) 45s ago 3m 86.2M - 19.0.0-5358-gaf8fdbf0 b2e977707047 2799be9e5935 2024-07-27T03:15:57.769 INFO:teuthology.orchestra.run.smithi067.stdout:mds.h smithi149 running (3m) 83s ago 3m 85.9M - 19.0.0-5358-gaf8fdbf0 b2e977707047 7b2d25a41b91 2024-07-27T03:15:57.769 INFO:teuthology.orchestra.run.smithi067.stdout:mds.i smithi175 running (3m) 83s ago 3m 86.9M - 19.0.0-5358-gaf8fdbf0 b2e977707047 2cf721d54932 2024-07-27T03:15:57.769 INFO:teuthology.orchestra.run.smithi067.stdout:mds.j smithi067 running (4m) 45s ago 4m 82.6M - 19.0.0-5358-gaf8fdbf0 b2e977707047 c8f67424c9aa 2024-07-27T03:15:57.769 INFO:teuthology.orchestra.run.smithi067.stdout:mds.k smithi149 running (4m) 83s ago 4m 85.5M - 19.0.0-5358-gaf8fdbf0 b2e977707047 15e7bc64f44a 2024-07-27T03:15:57.769 INFO:teuthology.orchestra.run.smithi067.stdout:mds.l smithi175 running (4m) 83s ago 4m 85.4M - 19.0.0-5358-gaf8fdbf0 b2e977707047 c5738d3a87e5 2024-07-27T03:15:57.769 INFO:teuthology.orchestra.run.smithi067.stdout:mgr.x smithi067 *:9283,8765 running (11m) 45s ago 11m 592M - 19.0.0-5358-gaf8fdbf0 b2e977707047 b8e00c316ed9 2024-07-27T03:15:57.769 INFO:teuthology.orchestra.run.smithi067.stdout:mgr.y smithi149 *:8443,8765 running (9m) 83s ago 9m 525M - 19.0.0-5358-gaf8fdbf0 b2e977707047 7c5e4c54a309 2024-07-27T03:15:57.769 INFO:teuthology.orchestra.run.smithi067.stdout:mgr.z smithi175 *:8443,8765 running (9m) 83s ago 9m 525M - 19.0.0-5358-gaf8fdbf0 b2e977707047 8d5ca39b6dcd 2024-07-27T03:15:57.769 INFO:teuthology.orchestra.run.smithi067.stdout:mon.a smithi067 running (11m) 45s ago 11m 121M 2048M 19.0.0-5358-gaf8fdbf0 b2e977707047 23a97e917213 2024-07-27T03:15:57.769 INFO:teuthology.orchestra.run.smithi067.stdout:mon.b smithi149 running (10m) 83s ago 10m 109M 2048M 19.0.0-5358-gaf8fdbf0 b2e977707047 caccb2bcb96e 2024-07-27T03:15:57.769 INFO:teuthology.orchestra.run.smithi067.stdout:mon.c smithi175 running (10m) 83s ago 10m 108M 2048M 19.0.0-5358-gaf8fdbf0 b2e977707047 f9196a2808bc 2024-07-27T03:15:57.770 INFO:teuthology.orchestra.run.smithi067.stdout:osd.0 smithi067 running (8m) 45s ago 8m 121M 4272M 19.0.0-5358-gaf8fdbf0 b2e977707047 d36756a18466 2024-07-27T03:15:57.770 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1 smithi149 running (8m) 83s ago 8m 121M 1454M 19.0.0-5358-gaf8fdbf0 b2e977707047 956ef7224651 2024-07-27T03:15:57.770 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2 smithi175 running (8m) 83s ago 8m 123M 4272M 19.0.0-5358-gaf8fdbf0 b2e977707047 404238dc1400 2024-07-27T03:15:57.770 INFO:teuthology.orchestra.run.smithi067.stdout:osd.3 smithi067 running (7m) 45s ago 7m 112M 4272M 19.0.0-5358-gaf8fdbf0 b2e977707047 09a91e0de36f 2024-07-27T03:15:57.770 INFO:teuthology.orchestra.run.smithi067.stdout:osd.4 smithi149 running (7m) 83s ago 7m 119M 1454M 19.0.0-5358-gaf8fdbf0 b2e977707047 0fc36efd99f8 2024-07-27T03:15:57.770 INFO:teuthology.orchestra.run.smithi067.stdout:osd.5 smithi175 running (6m) 83s ago 6m 113M 4272M 19.0.0-5358-gaf8fdbf0 b2e977707047 9ad61b1b22df 2024-07-27T03:15:57.770 INFO:teuthology.orchestra.run.smithi067.stdout:osd.6 smithi067 running (6m) 45s ago 6m 110M 4272M 19.0.0-5358-gaf8fdbf0 b2e977707047 7def2980f6bf 2024-07-27T03:15:57.770 INFO:teuthology.orchestra.run.smithi067.stdout:osd.7 smithi149 running (6m) 83s ago 6m 112M 1454M 19.0.0-5358-gaf8fdbf0 b2e977707047 e61acfa75d8d 2024-07-27T03:15:57.770 INFO:teuthology.orchestra.run.smithi067.stdout:osd.8 smithi175 running (5m) 83s ago 5m 112M 4272M 19.0.0-5358-gaf8fdbf0 b2e977707047 76aabc85fd36 2024-07-27T03:15:57.770 INFO:teuthology.orchestra.run.smithi067.stdout:osd.9 smithi067 running (5m) 45s ago 5m 119M 4272M 19.0.0-5358-gaf8fdbf0 b2e977707047 075353a7d185 2024-07-27T03:15:57.770 INFO:teuthology.orchestra.run.smithi067.stdout:osd.10 smithi149 running (4m) 83s ago 4m 120M 1454M 19.0.0-5358-gaf8fdbf0 b2e977707047 0a250e1cdceb 2024-07-27T03:15:57.770 INFO:teuthology.orchestra.run.smithi067.stdout:osd.11 smithi175 running (4m) 83s ago 4m 109M 4272M 19.0.0-5358-gaf8fdbf0 b2e977707047 c10699581d89 2024-07-27T03:15:57.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.767+0000 7fafbbc9a640 1 -- 172.21.15.67:0/345130841 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fafa00b3c50 msgr2=0x7fafa00b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:57.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.767+0000 7fafbbc9a640 1 --2- 172.21.15.67:0/345130841 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fafa00b3c50 0x7fafa00b6160 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fafb0000c00 tx=0x7fafb0031400 comp rx=0 tx=0).stop 2024-07-27T03:15:57.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.767+0000 7fafbbc9a640 1 -- 172.21.15.67:0/345130841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb4103140 msgr2=0x7fafb4104570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:15:57.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.767+0000 7fafbbc9a640 1 --2- 172.21.15.67:0/345130841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb4103140 0x7fafb4104570 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fafa400e9f0 tx=0x7fafa400edf0 comp rx=0 tx=0).stop 2024-07-27T03:15:57.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.767+0000 7fafbbc9a640 1 -- 172.21.15.67:0/345130841 shutdown_connections 2024-07-27T03:15:57.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.767+0000 7fafbbc9a640 1 --2- 172.21.15.67:0/345130841 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fafa00b3c50 0x7fafa00b6160 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:57.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.767+0000 7fafbbc9a640 1 --2- 172.21.15.67:0/345130841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb4103ae0 0x7fafb4104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:57.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.767+0000 7fafbbc9a640 1 --2- 172.21.15.67:0/345130841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fafb4103140 0x7fafb4104570 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:57.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.767+0000 7fafbbc9a640 1 --2- 172.21.15.67:0/345130841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fafb41020b0 0x7fafb4105ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:15:57.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.767+0000 7fafbbc9a640 1 -- 172.21.15.67:0/345130841 >> 172.21.15.67:0/345130841 conn(0x7fafb40fda20 msgr2=0x7fafb40ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:15:57.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.768+0000 7fafbbc9a640 1 -- 172.21.15.67:0/345130841 shutdown_connections 2024-07-27T03:15:57.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:15:57.768+0000 7fafbbc9a640 1 -- 172.21.15.67:0/345130841 wait complete. 2024-07-27T03:15:58.420 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- bash -c 'ceph orch ls' 2024-07-27T03:15:58.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:58 smithi067 ceph-mon[32814]: pgmap v412: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:58.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:15:58 smithi067 ceph-mon[32814]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:15:58.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:58 smithi149 ceph-mon[37173]: pgmap v412: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:58.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:15:58 smithi149 ceph-mon[37173]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:15:58.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:58 smithi175 ceph-mon[37125]: pgmap v412: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:15:58.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:15:58 smithi175 ceph-mon[37125]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:15:58.687 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:16:00.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.329+0000 7f80d6387640 1 -- 172.21.15.67:0/1329526789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80d0103170 msgr2=0x7f80d0103580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:00.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.329+0000 7f80d6387640 1 --2- 172.21.15.67:0/1329526789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80d0103170 0x7f80d0103580 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009900 tx=0x7f80c402a480 comp rx=0 tx=0).stop 2024-07-27T03:16:00.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.329+0000 7f80d6387640 1 -- 172.21.15.67:0/1329526789 shutdown_connections 2024-07-27T03:16:00.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.329+0000 7f80d6387640 1 --2- 172.21.15.67:0/1329526789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80d0103b10 0x7f80d010a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:00.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.329+0000 7f80d6387640 1 --2- 172.21.15.67:0/1329526789 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80d0103170 0x7f80d0103580 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:00.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.329+0000 7f80d6387640 1 --2- 172.21.15.67:0/1329526789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80d01020e0 0x7f80d0102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:00.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.329+0000 7f80d6387640 1 -- 172.21.15.67:0/1329526789 >> 172.21.15.67:0/1329526789 conn(0x7f80d00fda50 msgr2=0x7f80d00fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:00.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.329+0000 7f80d6387640 1 -- 172.21.15.67:0/1329526789 shutdown_connections 2024-07-27T03:16:00.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.329+0000 7f80d6387640 1 -- 172.21.15.67:0/1329526789 wait complete. 2024-07-27T03:16:00.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.330+0000 7f80d6387640 1 Processor -- start 2024-07-27T03:16:00.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.330+0000 7f80d6387640 1 -- start start 2024-07-27T03:16:00.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.330+0000 7f80d6387640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80d01020e0 0x7f80d006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:00.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.330+0000 7f80d6387640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80d0103170 0x7f80d019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:00.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.330+0000 7f80d6387640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80d0103b10 0x7f80d01a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:00.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.330+0000 7f80d6387640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f80d010dd40 con 0x7f80d01020e0 2024-07-27T03:16:00.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.330+0000 7f80d6387640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f80d010dbc0 con 0x7f80d0103b10 2024-07-27T03:16:00.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.330+0000 7f80d6387640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80d010dec0 con 0x7f80d0103170 2024-07-27T03:16:00.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.331+0000 7f80cffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80d01020e0 0x7f80d006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:00.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.331+0000 7f80cffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80d01020e0 0x7f80d006bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52104/0 (socket says 172.21.15.67:52104) 2024-07-27T03:16:00.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.331+0000 7f80cffff640 1 -- 172.21.15.67:0/4228019632 learned_addr learned my addr 172.21.15.67:0/4228019632 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:00.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.331+0000 7f80cf7fe640 1 --2- 172.21.15.67:0/4228019632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80d0103170 0x7f80d019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:00.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.331+0000 7f80d4924640 1 --2- 172.21.15.67:0/4228019632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80d0103b10 0x7f80d01a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:00.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.331+0000 7f80cffff640 1 -- 172.21.15.67:0/4228019632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80d0103170 msgr2=0x7f80d019e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:00.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.331+0000 7f80cffff640 1 --2- 172.21.15.67:0/4228019632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80d0103170 0x7f80d019e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:00.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.331+0000 7f80cffff640 1 -- 172.21.15.67:0/4228019632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80d0103b10 msgr2=0x7f80d01a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:00.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.331+0000 7f80cffff640 1 --2- 172.21.15.67:0/4228019632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80d0103b10 0x7f80d01a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:00.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.331+0000 7f80cffff640 1 -- 172.21.15.67:0/4228019632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d01a14f0 con 0x7f80d01020e0 2024-07-27T03:16:00.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.331+0000 7f80cffff640 1 --2- 172.21.15.67:0/4228019632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80d01020e0 0x7f80d006bdf0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f80c000d950 tx=0x7f80c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:00.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.331+0000 7f80cd7fa640 1 -- 172.21.15.67:0/4228019632 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0017070 con 0x7f80d01020e0 2024-07-27T03:16:00.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.331+0000 7f80cd7fa640 1 -- 172.21.15.67:0/4228019632 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c0009c30 con 0x7f80d01020e0 2024-07-27T03:16:00.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.331+0000 7f80d6387640 1 -- 172.21.15.67:0/4228019632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d01a17e0 con 0x7f80d01020e0 2024-07-27T03:16:00.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.331+0000 7f80cd7fa640 1 -- 172.21.15.67:0/4228019632 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c000f620 con 0x7f80d01020e0 2024-07-27T03:16:00.340 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.332+0000 7f80d6387640 1 -- 172.21.15.67:0/4228019632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d01a1d20 con 0x7f80d01020e0 2024-07-27T03:16:00.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.332+0000 7f80d6387640 1 -- 172.21.15.67:0/4228019632 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80d01092a0 con 0x7f80d01020e0 2024-07-27T03:16:00.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.334+0000 7f80cd7fa640 1 -- 172.21.15.67:0/4228019632 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f80c0010460 con 0x7f80d01020e0 2024-07-27T03:16:00.342 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:00 smithi067 ceph-mon[32814]: pgmap v413: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:16:00.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.335+0000 7f80cd7fa640 1 --2- 172.21.15.67:0/4228019632 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f80b80b3bd0 0x7f80b80b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:00.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.335+0000 7f80cf7fe640 1 --2- 172.21.15.67:0/4228019632 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f80b80b3bd0 0x7f80b80b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:00.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.335+0000 7f80cd7fa640 1 -- 172.21.15.67:0/4228019632 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f80c00e3b20 con 0x7f80d01020e0 2024-07-27T03:16:00.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.336+0000 7f80cf7fe640 1 --2- 172.21.15.67:0/4228019632 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f80b80b3bd0 0x7f80b80b60e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f80c4000c00 tx=0x7f80c4031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:00.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.339+0000 7f80cd7fa640 1 -- 172.21.15.67:0/4228019632 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f80c0009de0 con 0x7f80d01020e0 2024-07-27T03:16:00.501 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:00 smithi149 ceph-mon[37173]: pgmap v413: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:16:00.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.531+0000 7f80d6387640 1 -- 172.21.15.67:0/4228019632 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f80d0063270 con 0x7f80b80b3bd0 2024-07-27T03:16:00.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.537+0000 7f80cd7fa640 1 -- 172.21.15.67:0/4228019632 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f80d0063270 con 0x7f80b80b3bd0 2024-07-27T03:16:00.540 INFO:teuthology.orchestra.run.smithi067.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-07-27T03:16:00.540 INFO:teuthology.orchestra.run.smithi067.stdout:mds.all 12/12 86s ago 4m smithi067=a;smithi067=d;smithi067=g;smithi067=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi175=c;smithi175=f;smithi175=i;smithi175=l;count:12 2024-07-27T03:16:00.540 INFO:teuthology.orchestra.run.smithi067.stdout:mgr 3/3 86s ago 10m smithi067=x;smithi149=y;smithi175=z;count:3 2024-07-27T03:16:00.540 INFO:teuthology.orchestra.run.smithi067.stdout:mon 3/3 86s ago 10m smithi067:172.21.15.67=a;smithi149:172.21.15.149=b;smithi175:172.21.15.175=c;count:3 2024-07-27T03:16:00.541 INFO:teuthology.orchestra.run.smithi067.stdout:osd 12 86s ago - 2024-07-27T03:16:00.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.539+0000 7f80d6387640 1 -- 172.21.15.67:0/4228019632 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f80b80b3bd0 msgr2=0x7f80b80b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:00.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.539+0000 7f80d6387640 1 --2- 172.21.15.67:0/4228019632 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f80b80b3bd0 0x7f80b80b60e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f80c4000c00 tx=0x7f80c4031400 comp rx=0 tx=0).stop 2024-07-27T03:16:00.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.539+0000 7f80d6387640 1 -- 172.21.15.67:0/4228019632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80d01020e0 msgr2=0x7f80d006bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:00.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.539+0000 7f80d6387640 1 --2- 172.21.15.67:0/4228019632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80d01020e0 0x7f80d006bdf0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f80c000d950 tx=0x7f80c000de10 comp rx=0 tx=0).stop 2024-07-27T03:16:00.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.539+0000 7f80d6387640 1 -- 172.21.15.67:0/4228019632 shutdown_connections 2024-07-27T03:16:00.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.539+0000 7f80d6387640 1 --2- 172.21.15.67:0/4228019632 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f80b80b3bd0 0x7f80b80b60e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:00.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.539+0000 7f80d6387640 1 --2- 172.21.15.67:0/4228019632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80d0103b10 0x7f80d01a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:00.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.539+0000 7f80d6387640 1 --2- 172.21.15.67:0/4228019632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80d0103170 0x7f80d019e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:00.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.539+0000 7f80d6387640 1 --2- 172.21.15.67:0/4228019632 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80d01020e0 0x7f80d006bdf0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:00.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.539+0000 7f80d6387640 1 -- 172.21.15.67:0/4228019632 >> 172.21.15.67:0/4228019632 conn(0x7f80d00fda50 msgr2=0x7f80d0106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:00.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.539+0000 7f80d6387640 1 -- 172.21.15.67:0/4228019632 shutdown_connections 2024-07-27T03:16:00.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:00.539+0000 7f80d6387640 1 -- 172.21.15.67:0/4228019632 wait complete. 2024-07-27T03:16:00.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:00 smithi175 ceph-mon[37125]: pgmap v413: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:16:01.223 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- bash -c 'ceph orch host ls' 2024-07-27T03:16:01.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:01 smithi067 ceph-mon[32814]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:16:01.497 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:16:01.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:01 smithi175 ceph-mon[37125]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:16:01.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:01 smithi149 ceph-mon[37173]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:16:02.582 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:02 smithi067 ceph-mon[32814]: pgmap v414: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:16:02.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:02 smithi175 ceph-mon[37125]: pgmap v414: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:16:02.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:02 smithi149 ceph-mon[37173]: pgmap v414: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:16:03.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.119+0000 7fcce2039640 1 -- 172.21.15.67:0/3594249880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccdc1020e0 msgr2=0x7fccdc102550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:03.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.119+0000 7fcce2039640 1 --2- 172.21.15.67:0/3594249880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccdc1020e0 0x7fccdc102550 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcccc009900 tx=0x7fcccc02a480 comp rx=0 tx=0).stop 2024-07-27T03:16:03.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.119+0000 7fcce2039640 1 -- 172.21.15.67:0/3594249880 shutdown_connections 2024-07-27T03:16:03.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.119+0000 7fcce2039640 1 --2- 172.21.15.67:0/3594249880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccdc103b10 0x7fccdc10a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:03.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.119+0000 7fcce2039640 1 --2- 172.21.15.67:0/3594249880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccdc103170 0x7fccdc103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:03.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.119+0000 7fcce2039640 1 --2- 172.21.15.67:0/3594249880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccdc1020e0 0x7fccdc102550 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:03.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.119+0000 7fcce2039640 1 -- 172.21.15.67:0/3594249880 >> 172.21.15.67:0/3594249880 conn(0x7fccdc0fda50 msgr2=0x7fccdc0fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:03.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.119+0000 7fcce2039640 1 -- 172.21.15.67:0/3594249880 shutdown_connections 2024-07-27T03:16:03.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.119+0000 7fcce2039640 1 -- 172.21.15.67:0/3594249880 wait complete. 2024-07-27T03:16:03.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.120+0000 7fcce2039640 1 Processor -- start 2024-07-27T03:16:03.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.120+0000 7fcce2039640 1 -- start start 2024-07-27T03:16:03.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.120+0000 7fcce2039640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccdc1020e0 0x7fccdc06bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:03.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.120+0000 7fcce2039640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccdc103170 0x7fccdc1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:03.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.120+0000 7fcce2039640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccdc103b10 0x7fccdc1a3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:03.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.120+0000 7fcce2039640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc10dd40 con 0x7fccdc1020e0 2024-07-27T03:16:03.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.120+0000 7fcce2039640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc10dbc0 con 0x7fccdc103b10 2024-07-27T03:16:03.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.120+0000 7fcce2039640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc10dec0 con 0x7fccdc103170 2024-07-27T03:16:03.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.120+0000 7fccdb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccdc1020e0 0x7fccdc06bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:03.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.120+0000 7fccdb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccdc1020e0 0x7fccdc06bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52124/0 (socket says 172.21.15.67:52124) 2024-07-27T03:16:03.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.120+0000 7fccdb7fe640 1 -- 172.21.15.67:0/4025421244 learned_addr learned my addr 172.21.15.67:0/4025421244 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:03.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.120+0000 7fccd3fff640 1 --2- 172.21.15.67:0/4025421244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccdc103170 0x7fccdc1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:03.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.120+0000 7fccdbfff640 1 --2- 172.21.15.67:0/4025421244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccdc103b10 0x7fccdc1a3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:03.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.120+0000 7fccdb7fe640 1 -- 172.21.15.67:0/4025421244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccdc103170 msgr2=0x7fccdc1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:03.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.120+0000 7fccdb7fe640 1 --2- 172.21.15.67:0/4025421244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccdc103170 0x7fccdc1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:03.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.120+0000 7fccdb7fe640 1 -- 172.21.15.67:0/4025421244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccdc103b10 msgr2=0x7fccdc1a3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:03.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.120+0000 7fccdb7fe640 1 --2- 172.21.15.67:0/4025421244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccdc103b10 0x7fccdc1a3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:03.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.120+0000 7fccdb7fe640 1 -- 172.21.15.67:0/4025421244 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccdc1a3670 con 0x7fccdc1020e0 2024-07-27T03:16:03.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.120+0000 7fccd3fff640 1 --2- 172.21.15.67:0/4025421244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccdc103170 0x7fccdc1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:16:03.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.121+0000 7fccdb7fe640 1 --2- 172.21.15.67:0/4025421244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccdc1020e0 0x7fccdc06bd50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcccc006010 tx=0x7fcccc0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:03.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.121+0000 7fccd97fa640 1 -- 172.21.15.67:0/4025421244 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc03f030 con 0x7fccdc1020e0 2024-07-27T03:16:03.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.121+0000 7fccd97fa640 1 -- 172.21.15.67:0/4025421244 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcccc003cb0 con 0x7fccdc1020e0 2024-07-27T03:16:03.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.121+0000 7fccd97fa640 1 -- 172.21.15.67:0/4025421244 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc035070 con 0x7fccdc1020e0 2024-07-27T03:16:03.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.121+0000 7fcce2039640 1 -- 172.21.15.67:0/4025421244 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccdc1a3900 con 0x7fccdc1020e0 2024-07-27T03:16:03.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.121+0000 7fcce2039640 1 -- 172.21.15.67:0/4025421244 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccdc1a3d90 con 0x7fccdc1020e0 2024-07-27T03:16:03.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.122+0000 7fcce2039640 1 -- 172.21.15.67:0/4025421244 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccdc1092c0 con 0x7fccdc1020e0 2024-07-27T03:16:03.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.123+0000 7fccd97fa640 1 -- 172.21.15.67:0/4025421244 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7fcccc042030 con 0x7fccdc1020e0 2024-07-27T03:16:03.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.124+0000 7fccd97fa640 1 --2- 172.21.15.67:0/4025421244 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fccc80b3c50 0x7fccc80b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:03.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.124+0000 7fccd3fff640 1 --2- 172.21.15.67:0/4025421244 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fccc80b3c50 0x7fccc80b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:03.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.124+0000 7fccd97fa640 1 -- 172.21.15.67:0/4025421244 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fcccc104dd0 con 0x7fccdc1020e0 2024-07-27T03:16:03.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.125+0000 7fccd3fff640 1 --2- 172.21.15.67:0/4025421244 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fccc80b3c50 0x7fccc80b6160 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fccc40058c0 tx=0x7fccc4005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:03.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.128+0000 7fccd97fa640 1 -- 172.21.15.67:0/4025421244 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fcccc0cc070 con 0x7fccdc1020e0 2024-07-27T03:16:03.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.317+0000 7fcce2039640 1 -- 172.21.15.67:0/4025421244 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fccdc063270 con 0x7fccc80b3c50 2024-07-27T03:16:03.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.319+0000 7fccd97fa640 1 -- 172.21.15.67:0/4025421244 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fccdc063270 con 0x7fccc80b3c50 2024-07-27T03:16:03.321 INFO:teuthology.orchestra.run.smithi067.stdout:HOST ADDR LABELS STATUS 2024-07-27T03:16:03.321 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067 172.21.15.67 2024-07-27T03:16:03.321 INFO:teuthology.orchestra.run.smithi067.stdout:smithi149 172.21.15.149 2024-07-27T03:16:03.321 INFO:teuthology.orchestra.run.smithi067.stdout:smithi175 172.21.15.175 2024-07-27T03:16:03.322 INFO:teuthology.orchestra.run.smithi067.stdout:3 hosts in cluster 2024-07-27T03:16:03.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.321+0000 7fcce2039640 1 -- 172.21.15.67:0/4025421244 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fccc80b3c50 msgr2=0x7fccc80b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:03.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.321+0000 7fcce2039640 1 --2- 172.21.15.67:0/4025421244 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fccc80b3c50 0x7fccc80b6160 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fccc40058c0 tx=0x7fccc4005d30 comp rx=0 tx=0).stop 2024-07-27T03:16:03.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.321+0000 7fcce2039640 1 -- 172.21.15.67:0/4025421244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccdc1020e0 msgr2=0x7fccdc06bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:03.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.321+0000 7fcce2039640 1 --2- 172.21.15.67:0/4025421244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccdc1020e0 0x7fccdc06bd50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcccc006010 tx=0x7fcccc0043b0 comp rx=0 tx=0).stop 2024-07-27T03:16:03.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.321+0000 7fcce2039640 1 -- 172.21.15.67:0/4025421244 shutdown_connections 2024-07-27T03:16:03.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.321+0000 7fcce2039640 1 --2- 172.21.15.67:0/4025421244 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fccc80b3c50 0x7fccc80b6160 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:03.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.321+0000 7fcce2039640 1 --2- 172.21.15.67:0/4025421244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fccdc103b10 0x7fccdc1a3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:03.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.321+0000 7fcce2039640 1 --2- 172.21.15.67:0/4025421244 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccdc103170 0x7fccdc1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:03.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.321+0000 7fcce2039640 1 --2- 172.21.15.67:0/4025421244 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccdc1020e0 0x7fccdc06bd50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:03.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.321+0000 7fcce2039640 1 -- 172.21.15.67:0/4025421244 >> 172.21.15.67:0/4025421244 conn(0x7fccdc0fda50 msgr2=0x7fccdc106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:03.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.321+0000 7fcce2039640 1 -- 172.21.15.67:0/4025421244 shutdown_connections 2024-07-27T03:16:03.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:03.321+0000 7fcce2039640 1 -- 172.21.15.67:0/4025421244 wait complete. 2024-07-27T03:16:04.047 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- bash -c 'ceph orch device ls' 2024-07-27T03:16:04.316 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:16:04.439 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:04 smithi067 ceph-mon[32814]: pgmap v415: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:04.440 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:04 smithi067 ceph-mon[32814]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:16:04.552 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:04 smithi149 ceph-mon[37173]: pgmap v415: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:04.552 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:04 smithi149 ceph-mon[37173]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:16:04.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:04 smithi175 ceph-mon[37125]: pgmap v415: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:04.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:04 smithi175 ceph-mon[37125]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:16:06.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.004+0000 7f8609d37640 1 -- 172.21.15.67:0/342058068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8604108f10 msgr2=0x7f860410b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:06.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.004+0000 7f8609d37640 1 --2- 172.21.15.67:0/342058068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8604108f10 0x7f860410b420 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009ca0 tx=0x7f85fc02a8c0 comp rx=0 tx=0).stop 2024-07-27T03:16:06.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.004+0000 7f8609d37640 1 -- 172.21.15.67:0/342058068 shutdown_connections 2024-07-27T03:16:06.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.004+0000 7f8609d37640 1 --2- 172.21.15.67:0/342058068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8604108f10 0x7f860410b420 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:06.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.004+0000 7f8609d37640 1 --2- 172.21.15.67:0/342058068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8604106370 0x7f86041087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:06.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.004+0000 7f8609d37640 1 --2- 172.21.15.67:0/342058068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f860406c0d0 0x7f8604105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:06.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.004+0000 7f8609d37640 1 -- 172.21.15.67:0/342058068 >> 172.21.15.67:0/342058068 conn(0x7f86040fda50 msgr2=0x7f86040fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:06.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.004+0000 7f8609d37640 1 -- 172.21.15.67:0/342058068 shutdown_connections 2024-07-27T03:16:06.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.004+0000 7f8609d37640 1 -- 172.21.15.67:0/342058068 wait complete. 2024-07-27T03:16:06.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.005+0000 7f8609d37640 1 Processor -- start 2024-07-27T03:16:06.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.005+0000 7f8609d37640 1 -- start start 2024-07-27T03:16:06.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.005+0000 7f8609d37640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f860406c0d0 0x7f86041999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.005+0000 7f8609d37640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8604106370 0x7f860419a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.005+0000 7f8609d37640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8604108f10 0x7f860419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.005+0000 7f8609d37640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f860410ee30 con 0x7f8604108f10 2024-07-27T03:16:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.005+0000 7f8609d37640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f860410ecb0 con 0x7f860406c0d0 2024-07-27T03:16:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.005+0000 7f8609d37640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f860410efb0 con 0x7f8604106370 2024-07-27T03:16:06.014 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.006+0000 7f8603fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8604108f10 0x7f860419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:06.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.006+0000 7f8603fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8604108f10 0x7f860419abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52350/0 (socket says 172.21.15.67:52350) 2024-07-27T03:16:06.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.006+0000 7f8603fff640 1 -- 172.21.15.67:0/363654763 learned_addr learned my addr 172.21.15.67:0/363654763 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:06.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.006+0000 7f86037fe640 1 --2- 172.21.15.67:0/363654763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f860406c0d0 0x7f86041999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:06.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.006+0000 7f8602ffd640 1 --2- 172.21.15.67:0/363654763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8604106370 0x7f860419a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:06.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.006+0000 7f8603fff640 1 -- 172.21.15.67:0/363654763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8604106370 msgr2=0x7f860419a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:06.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.006+0000 7f8603fff640 1 --2- 172.21.15.67:0/363654763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8604106370 0x7f860419a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:06.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.006+0000 7f8603fff640 1 -- 172.21.15.67:0/363654763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f860406c0d0 msgr2=0x7f86041999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:06.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.006+0000 7f8603fff640 1 --2- 172.21.15.67:0/363654763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f860406c0d0 0x7f86041999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:06.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.006+0000 7f8603fff640 1 -- 172.21.15.67:0/363654763 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860419b350 con 0x7f8604108f10 2024-07-27T03:16:06.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.006+0000 7f86037fe640 1 --2- 172.21.15.67:0/363654763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f860406c0d0 0x7f86041999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:16:06.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.006+0000 7f8603fff640 1 --2- 172.21.15.67:0/363654763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8604108f10 0x7f860419abc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0097b0 tx=0x7f85fc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:06.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.006+0000 7f8600ff9640 1 -- 172.21.15.67:0/363654763 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc004240 con 0x7f8604108f10 2024-07-27T03:16:06.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.006+0000 7f8600ff9640 1 -- 172.21.15.67:0/363654763 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85fc02f380 con 0x7f8604108f10 2024-07-27T03:16:06.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.006+0000 7f8600ff9640 1 -- 172.21.15.67:0/363654763 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc02fc30 con 0x7f8604108f10 2024-07-27T03:16:06.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.006+0000 7f8609d37640 1 -- 172.21.15.67:0/363654763 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f860419b5e0 con 0x7f8604108f10 2024-07-27T03:16:06.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.006+0000 7f8609d37640 1 -- 172.21.15.67:0/363654763 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86041a1c00 con 0x7f8604108f10 2024-07-27T03:16:06.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.007+0000 7f8609d37640 1 -- 172.21.15.67:0/363654763 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86040ff570 con 0x7f8604108f10 2024-07-27T03:16:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.009+0000 7f8600ff9640 1 -- 172.21.15.67:0/363654763 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f85fc0043e0 con 0x7f8604108f10 2024-07-27T03:16:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.010+0000 7f8600ff9640 1 --2- 172.21.15.67:0/363654763 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f85ec0b3c50 0x7f85ec0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.010+0000 7f86037fe640 1 --2- 172.21.15.67:0/363654763 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f85ec0b3c50 0x7f85ec0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:06.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.010+0000 7f8600ff9640 1 -- 172.21.15.67:0/363654763 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f85fc1058b0 con 0x7f8604108f10 2024-07-27T03:16:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.011+0000 7f86037fe640 1 --2- 172.21.15.67:0/363654763 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f85ec0b3c50 0x7f85ec0b6160 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009b40 tx=0x7f85f4005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:06.020 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.013+0000 7f8600ff9640 1 -- 172.21.15.67:0/363654763 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f85fc0ccb50 con 0x7f8604108f10 2024-07-27T03:16:06.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.204+0000 7f8609d37640 1 -- 172.21.15.67:0/363654763 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f8604063270 con 0x7f85ec0b3c50 2024-07-27T03:16:06.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.205+0000 7f8600ff9640 1 -- 172.21.15.67:0/363654763 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f8604063270 con 0x7f85ec0b3c50 2024-07-27T03:16:06.208 INFO:teuthology.orchestra.run.smithi067.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-07-27T03:16:06.208 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53440042400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-07-27T03:16:06.208 INFO:teuthology.orchestra.run.smithi067.stdout:smithi149 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CK400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-07-27T03:16:06.208 INFO:teuthology.orchestra.run.smithi067.stdout:smithi175 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300M0400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-07-27T03:16:06.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.208+0000 7f8609d37640 1 -- 172.21.15.67:0/363654763 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f85ec0b3c50 msgr2=0x7f85ec0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:06.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.208+0000 7f8609d37640 1 --2- 172.21.15.67:0/363654763 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f85ec0b3c50 0x7f85ec0b6160 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009b40 tx=0x7f85f4005e50 comp rx=0 tx=0).stop 2024-07-27T03:16:06.210 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.208+0000 7f8609d37640 1 -- 172.21.15.67:0/363654763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8604108f10 msgr2=0x7f860419abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:06.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.208+0000 7f8609d37640 1 --2- 172.21.15.67:0/363654763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8604108f10 0x7f860419abc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0097b0 tx=0x7f85fc003c20 comp rx=0 tx=0).stop 2024-07-27T03:16:06.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.208+0000 7f8609d37640 1 -- 172.21.15.67:0/363654763 shutdown_connections 2024-07-27T03:16:06.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.208+0000 7f8609d37640 1 --2- 172.21.15.67:0/363654763 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f85ec0b3c50 0x7f85ec0b6160 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:06.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.208+0000 7f8609d37640 1 --2- 172.21.15.67:0/363654763 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8604108f10 0x7f860419abc0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:06.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.208+0000 7f8609d37640 1 --2- 172.21.15.67:0/363654763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8604106370 0x7f860419a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:06.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.208+0000 7f8609d37640 1 --2- 172.21.15.67:0/363654763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f860406c0d0 0x7f86041999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:06.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.208+0000 7f8609d37640 1 -- 172.21.15.67:0/363654763 >> 172.21.15.67:0/363654763 conn(0x7f86040fda50 msgr2=0x7f8604104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:06.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.208+0000 7f8609d37640 1 -- 172.21.15.67:0/363654763 shutdown_connections 2024-07-27T03:16:06.211 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:06.208+0000 7f8609d37640 1 -- 172.21.15.67:0/363654763 wait complete. 2024-07-27T03:16:06.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:06 smithi067 ceph-mon[32814]: pgmap v416: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:16:06.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:06 smithi175 ceph-mon[37125]: pgmap v416: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:16:06.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:06 smithi149 ceph-mon[37173]: pgmap v416: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:16:06.888 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-07-27T03:16:06.898 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-07-27T03:16:06.898 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- bash -c 'ceph fs dump' 2024-07-27T03:16:07.166 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:16:07.528 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:07 smithi067 ceph-mon[32814]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:16:07.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:07 smithi175 ceph-mon[37125]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:16:07.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:07 smithi149 ceph-mon[37173]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-07-27T03:16:08.531 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:08 smithi067 ceph-mon[32814]: pgmap v417: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:08.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:08 smithi175 ceph-mon[37125]: pgmap v417: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:08.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:08 smithi149 ceph-mon[37173]: pgmap v417: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:08.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.767+0000 7fba2aede640 1 -- 172.21.15.67:0/1221072324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba24103150 msgr2=0x7fba24103560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.767+0000 7fba2aede640 1 --2- 172.21.15.67:0/1221072324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba24103150 0x7fba24103560 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fba14009ca0 tx=0x7fba1402a950 comp rx=0 tx=0).stop 2024-07-27T03:16:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.767+0000 7fba2aede640 1 -- 172.21.15.67:0/1221072324 shutdown_connections 2024-07-27T03:16:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.767+0000 7fba2aede640 1 --2- 172.21.15.67:0/1221072324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba24103af0 0x7fba2410a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.767+0000 7fba2aede640 1 --2- 172.21.15.67:0/1221072324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba24103150 0x7fba24103560 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.767+0000 7fba2aede640 1 --2- 172.21.15.67:0/1221072324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba241020c0 0x7fba24102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.767+0000 7fba2aede640 1 -- 172.21.15.67:0/1221072324 >> 172.21.15.67:0/1221072324 conn(0x7fba240fda50 msgr2=0x7fba240ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.767+0000 7fba2aede640 1 -- 172.21.15.67:0/1221072324 shutdown_connections 2024-07-27T03:16:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.767+0000 7fba2aede640 1 -- 172.21.15.67:0/1221072324 wait complete. 2024-07-27T03:16:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.768+0000 7fba2aede640 1 Processor -- start 2024-07-27T03:16:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.768+0000 7fba2aede640 1 -- start start 2024-07-27T03:16:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.768+0000 7fba2aede640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba241020c0 0x7fba24199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.768+0000 7fba2aede640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba24103150 0x7fba2419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.768+0000 7fba2aede640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba24103af0 0x7fba2419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.768+0000 7fba2aede640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fba2410dd20 con 0x7fba241020c0 2024-07-27T03:16:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.768+0000 7fba2aede640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fba2410dba0 con 0x7fba24103150 2024-07-27T03:16:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.768+0000 7fba2aede640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fba2410dea0 con 0x7fba24103af0 2024-07-27T03:16:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.768+0000 7fba28c7a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba241020c0 0x7fba24199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.768+0000 7fba28c7a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba241020c0 0x7fba24199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52374/0 (socket says 172.21.15.67:52374) 2024-07-27T03:16:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.768+0000 7fba28c7a640 1 -- 172.21.15.67:0/2165880189 learned_addr learned my addr 172.21.15.67:0/2165880189 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.769+0000 7fba23fff640 1 --2- 172.21.15.67:0/2165880189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba24103150 0x7fba2419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.769+0000 7fba2947b640 1 --2- 172.21.15.67:0/2165880189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba24103af0 0x7fba2419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.769+0000 7fba28c7a640 1 -- 172.21.15.67:0/2165880189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba24103af0 msgr2=0x7fba2419ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.769+0000 7fba28c7a640 1 --2- 172.21.15.67:0/2165880189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba24103af0 0x7fba2419ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:08.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.769+0000 7fba28c7a640 1 -- 172.21.15.67:0/2165880189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba24103150 msgr2=0x7fba2419a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:08.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.769+0000 7fba28c7a640 1 --2- 172.21.15.67:0/2165880189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba24103150 0x7fba2419a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:08.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.769+0000 7fba28c7a640 1 -- 172.21.15.67:0/2165880189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2419b450 con 0x7fba241020c0 2024-07-27T03:16:08.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.769+0000 7fba28c7a640 1 --2- 172.21.15.67:0/2165880189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba241020c0 0x7fba24199a20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00d950 tx=0x7fba0c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:08.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.769+0000 7fba21ffb640 1 -- 172.21.15.67:0/2165880189 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c017070 con 0x7fba241020c0 2024-07-27T03:16:08.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.769+0000 7fba21ffb640 1 -- 172.21.15.67:0/2165880189 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba0c009c30 con 0x7fba241020c0 2024-07-27T03:16:08.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.769+0000 7fba21ffb640 1 -- 172.21.15.67:0/2165880189 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c00f620 con 0x7fba241020c0 2024-07-27T03:16:08.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.769+0000 7fba2aede640 1 -- 172.21.15.67:0/2165880189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2419b740 con 0x7fba241020c0 2024-07-27T03:16:08.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.769+0000 7fba2aede640 1 -- 172.21.15.67:0/2165880189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba241a1e40 con 0x7fba241020c0 2024-07-27T03:16:08.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.770+0000 7fba2aede640 1 -- 172.21.15.67:0/2165880189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba24102530 con 0x7fba241020c0 2024-07-27T03:16:08.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.772+0000 7fba21ffb640 1 -- 172.21.15.67:0/2165880189 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7fba0c010460 con 0x7fba241020c0 2024-07-27T03:16:08.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.773+0000 7fba21ffb640 1 --2- 172.21.15.67:0/2165880189 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fba100b3bd0 0x7fba100b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:08.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.773+0000 7fba23fff640 1 --2- 172.21.15.67:0/2165880189 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fba100b3bd0 0x7fba100b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:08.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.773+0000 7fba21ffb640 1 -- 172.21.15.67:0/2165880189 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fba0c0e3be0 con 0x7fba241020c0 2024-07-27T03:16:08.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.773+0000 7fba23fff640 1 --2- 172.21.15.67:0/2165880189 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fba100b3bd0 0x7fba100b60e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fba1402f8d0 tx=0x7fba14009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:08.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:08.777+0000 7fba21ffb640 1 -- 172.21.15.67:0/2165880189 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fba0c0aae80 con 0x7fba241020c0 2024-07-27T03:16:09.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:09.002+0000 7fba2aede640 1 -- 172.21.15.67:0/2165880189 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fba241a21d0 con 0x7fba241020c0 2024-07-27T03:16:09.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:09.003+0000 7fba21ffb640 1 -- 172.21.15.67:0/2165880189 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) ==== 76+0+3136 (secure 0 0 0) 0x7fba0c009de0 con 0x7fba241020c0 2024-07-27T03:16:09.006 INFO:teuthology.orchestra.run.smithi067.stdout:e29 2024-07-27T03:16:09.006 INFO:teuthology.orchestra.run.smithi067.stdout:btime 2024-07-27T03:13:42:247804+0000 2024-07-27T03:16:09.006 INFO:teuthology.orchestra.run.smithi067.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-07-27T03:16:09.006 INFO:teuthology.orchestra.run.smithi067.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-27T03:16:09.006 INFO:teuthology.orchestra.run.smithi067.stdout:legacy client fscid: 1 2024-07-27T03:16:09.006 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:16:09.006 INFO:teuthology.orchestra.run.smithi067.stdout:Filesystem 'cephfs' (1) 2024-07-27T03:16:09.006 INFO:teuthology.orchestra.run.smithi067.stdout:fs_name cephfs 2024-07-27T03:16:09.006 INFO:teuthology.orchestra.run.smithi067.stdout:epoch 29 2024-07-27T03:16:09.006 INFO:teuthology.orchestra.run.smithi067.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-07-27T03:16:09.006 INFO:teuthology.orchestra.run.smithi067.stdout:created 2024-07-27T03:12:34.320208+0000 2024-07-27T03:16:09.006 INFO:teuthology.orchestra.run.smithi067.stdout:modified 2024-07-27T03:13:42.062021+0000 2024-07-27T03:16:09.007 INFO:teuthology.orchestra.run.smithi067.stdout:tableserver 0 2024-07-27T03:16:09.007 INFO:teuthology.orchestra.run.smithi067.stdout:root 0 2024-07-27T03:16:09.007 INFO:teuthology.orchestra.run.smithi067.stdout:session_timeout 300 2024-07-27T03:16:09.007 INFO:teuthology.orchestra.run.smithi067.stdout:session_autoclose 300 2024-07-27T03:16:09.007 INFO:teuthology.orchestra.run.smithi067.stdout:max_file_size 1099511627776 2024-07-27T03:16:09.007 INFO:teuthology.orchestra.run.smithi067.stdout:max_xattr_size 65536 2024-07-27T03:16:09.007 INFO:teuthology.orchestra.run.smithi067.stdout:required_client_features {} 2024-07-27T03:16:09.007 INFO:teuthology.orchestra.run.smithi067.stdout:last_failure 0 2024-07-27T03:16:09.007 INFO:teuthology.orchestra.run.smithi067.stdout:last_failure_osd_epoch 0 2024-07-27T03:16:09.007 INFO:teuthology.orchestra.run.smithi067.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-07-27T03:16:09.007 INFO:teuthology.orchestra.run.smithi067.stdout:max_mds 5 2024-07-27T03:16:09.007 INFO:teuthology.orchestra.run.smithi067.stdout:in 0,1,2,3,4 2024-07-27T03:16:09.007 INFO:teuthology.orchestra.run.smithi067.stdout:up {0=24620,1=24587,2=24583,3=24581,4=24563} 2024-07-27T03:16:09.007 INFO:teuthology.orchestra.run.smithi067.stdout:failed 2024-07-27T03:16:09.008 INFO:teuthology.orchestra.run.smithi067.stdout:damaged 2024-07-27T03:16:09.008 INFO:teuthology.orchestra.run.smithi067.stdout:stopped 2024-07-27T03:16:09.008 INFO:teuthology.orchestra.run.smithi067.stdout:data_pools [3,4] 2024-07-27T03:16:09.008 INFO:teuthology.orchestra.run.smithi067.stdout:metadata_pool 2 2024-07-27T03:16:09.008 INFO:teuthology.orchestra.run.smithi067.stdout:inline_data disabled 2024-07-27T03:16:09.008 INFO:teuthology.orchestra.run.smithi067.stdout:balancer 2024-07-27T03:16:09.008 INFO:teuthology.orchestra.run.smithi067.stdout:bal_rank_mask -1 2024-07-27T03:16:09.008 INFO:teuthology.orchestra.run.smithi067.stdout:standby_count_wanted 0 2024-07-27T03:16:09.008 INFO:teuthology.orchestra.run.smithi067.stdout:qdb_cluster leader: 24620 members: 24563,24581,24620,24583,24587 2024-07-27T03:16:09.008 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.b{0:24620} state up:active seq 5 addr [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] compat {c=[1],r=[1],i=[1fff]}] 2024-07-27T03:16:09.008 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.f{0:24562} state up:standby-replay seq 1 addr [v2:172.21.15.175:6834/2462749027,v1:172.21.15.175:6835/2462749027] compat {c=[1],r=[1],i=[1fff]}] 2024-07-27T03:16:09.008 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.h{1:24587} state up:active seq 24 addr [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] compat {c=[1],r=[1],i=[1fff]}] 2024-07-27T03:16:09.008 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.k{1:24533} state up:standby-replay seq 1 addr [v2:172.21.15.149:6832/3868325215,v1:172.21.15.149:6833/3868325215] compat {c=[1],r=[1],i=[1fff]}] 2024-07-27T03:16:09.008 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.i{2:24583} state up:active seq 23 addr [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] compat {c=[1],r=[1],i=[1fff]}] 2024-07-27T03:16:09.009 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.l{2:24526} state up:standby-replay seq 1 addr [v2:172.21.15.175:6832/3804638701,v1:172.21.15.175:6833/3804638701] compat {c=[1],r=[1],i=[1fff]}] 2024-07-27T03:16:09.009 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.c{3:24581} state up:active seq 25 addr [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] compat {c=[1],r=[1],i=[1fff]}] 2024-07-27T03:16:09.009 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.a{3:14748} state up:standby-replay seq 1 addr [v2:172.21.15.67:6840/795704244,v1:172.21.15.67:6841/795704244] compat {c=[1],r=[1],i=[1fff]}] 2024-07-27T03:16:09.009 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.e{4:24563} state up:active seq 26 addr [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] compat {c=[1],r=[1],i=[1fff]}] 2024-07-27T03:16:09.009 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.g{4:14736} state up:standby-replay seq 1 addr [v2:172.21.15.67:6838/2162708317,v1:172.21.15.67:6839/2162708317] compat {c=[1],r=[1],i=[1fff]}] 2024-07-27T03:16:09.009 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:16:09.009 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:16:09.009 INFO:teuthology.orchestra.run.smithi067.stdout:Standby daemons: 2024-07-27T03:16:09.009 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:16:09.009 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.j{-1:14679} state up:standby seq 1 addr [v2:172.21.15.67:6834/252686369,v1:172.21.15.67:6835/252686369] compat {c=[1],r=[1],i=[1fff]}] 2024-07-27T03:16:09.010 INFO:teuthology.orchestra.run.smithi067.stdout:[mds.d{-1:14685} state up:standby seq 1 addr [v2:172.21.15.67:6836/3358832762,v1:172.21.15.67:6837/3358832762] compat {c=[1],r=[1],i=[1fff]}] 2024-07-27T03:16:09.010 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-07-27T03:16:09.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:09.005+0000 7fba2aede640 1 -- 172.21.15.67:0/2165880189 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fba100b3bd0 msgr2=0x7fba100b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:09.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:09.005+0000 7fba2aede640 1 --2- 172.21.15.67:0/2165880189 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fba100b3bd0 0x7fba100b60e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fba1402f8d0 tx=0x7fba14009360 comp rx=0 tx=0).stop 2024-07-27T03:16:09.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:09.005+0000 7fba2aede640 1 -- 172.21.15.67:0/2165880189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba241020c0 msgr2=0x7fba24199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:09.010 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:09.005+0000 7fba2aede640 1 --2- 172.21.15.67:0/2165880189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba241020c0 0x7fba24199a20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00d950 tx=0x7fba0c00de10 comp rx=0 tx=0).stop 2024-07-27T03:16:09.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:09.005+0000 7fba2aede640 1 -- 172.21.15.67:0/2165880189 shutdown_connections 2024-07-27T03:16:09.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:09.005+0000 7fba2aede640 1 --2- 172.21.15.67:0/2165880189 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fba100b3bd0 0x7fba100b60e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:09.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:09.005+0000 7fba2aede640 1 --2- 172.21.15.67:0/2165880189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba24103af0 0x7fba2419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:09.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:09.005+0000 7fba2aede640 1 --2- 172.21.15.67:0/2165880189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba24103150 0x7fba2419a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:09.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:09.005+0000 7fba2aede640 1 --2- 172.21.15.67:0/2165880189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fba241020c0 0x7fba24199a20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:09.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:09.005+0000 7fba2aede640 1 -- 172.21.15.67:0/2165880189 >> 172.21.15.67:0/2165880189 conn(0x7fba240fda50 msgr2=0x7fba240ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:09.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:09.005+0000 7fba2aede640 1 -- 172.21.15.67:0/2165880189 shutdown_connections 2024-07-27T03:16:09.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:09.006+0000 7fba2aede640 1 -- 172.21.15.67:0/2165880189 wait complete. 2024-07-27T03:16:09.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:09 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2165880189' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-27T03:16:09.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:09 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2165880189' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-27T03:16:09.695 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- bash -c 'ceph osd dump' 2024-07-27T03:16:09.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:09 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2165880189' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-07-27T03:16:09.961 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:16:10.336 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:10 smithi067 ceph-mon[32814]: pgmap v418: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:16:10.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:10 smithi175 ceph-mon[37125]: pgmap v418: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:16:10.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:10 smithi149 ceph-mon[37173]: pgmap v418: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:16:11.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.621+0000 7f27964cd640 1 -- 172.21.15.67:0/3591549679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2790106400 msgr2=0x7f2790108820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:11.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.621+0000 7f27964cd640 1 --2- 172.21.15.67:0/3591549679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2790106400 0x7f2790108820 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2778009900 tx=0x7f277802a410 comp rx=0 tx=0).stop 2024-07-27T03:16:11.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.621+0000 7f27964cd640 1 -- 172.21.15.67:0/3591549679 shutdown_connections 2024-07-27T03:16:11.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.621+0000 7f27964cd640 1 --2- 172.21.15.67:0/3591549679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2790108ee0 0x7f279010b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:11.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.621+0000 7f27964cd640 1 --2- 172.21.15.67:0/3591549679 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2790106400 0x7f2790108820 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:11.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.621+0000 7f27964cd640 1 --2- 172.21.15.67:0/3591549679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f279006b330 0x7f2790105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:11.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.621+0000 7f27964cd640 1 -- 172.21.15.67:0/3591549679 >> 172.21.15.67:0/3591549679 conn(0x7f27900fda20 msgr2=0x7f27900ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:11.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.621+0000 7f27964cd640 1 -- 172.21.15.67:0/3591549679 shutdown_connections 2024-07-27T03:16:11.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.621+0000 7f27964cd640 1 -- 172.21.15.67:0/3591549679 wait complete. 2024-07-27T03:16:11.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.622+0000 7f27964cd640 1 Processor -- start 2024-07-27T03:16:11.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.622+0000 7f27964cd640 1 -- start start 2024-07-27T03:16:11.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.622+0000 7f27964cd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f279006b330 0x7f2790199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:11.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.622+0000 7f27964cd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2790106400 0x7f279019a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:11.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.622+0000 7f27964cd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2790108ee0 0x7f279019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:11.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.622+0000 7f27964cd640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f279010ee00 con 0x7f2790106400 2024-07-27T03:16:11.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.622+0000 7f27964cd640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f279010ec80 con 0x7f279006b330 2024-07-27T03:16:11.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.622+0000 7f27964cd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f279010ef80 con 0x7f2790108ee0 2024-07-27T03:16:11.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.622+0000 7f2787fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2790106400 0x7f279019a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:11.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.622+0000 7f2787fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2790106400 0x7f279019a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52402/0 (socket says 172.21.15.67:52402) 2024-07-27T03:16:11.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.622+0000 7f2787fff640 1 -- 172.21.15.67:0/1112148703 learned_addr learned my addr 172.21.15.67:0/1112148703 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:11.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.622+0000 7f278ffff640 1 --2- 172.21.15.67:0/1112148703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f279006b330 0x7f2790199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:11.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.623+0000 7f2794a6a640 1 --2- 172.21.15.67:0/1112148703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2790108ee0 0x7f279019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:11.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.623+0000 7f2787fff640 1 -- 172.21.15.67:0/1112148703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2790108ee0 msgr2=0x7f279019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:11.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.623+0000 7f2787fff640 1 --2- 172.21.15.67:0/1112148703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2790108ee0 0x7f279019ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:11.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.623+0000 7f2787fff640 1 -- 172.21.15.67:0/1112148703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f279006b330 msgr2=0x7f2790199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:11.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.623+0000 7f2787fff640 1 --2- 172.21.15.67:0/1112148703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f279006b330 0x7f2790199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:11.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.623+0000 7f2787fff640 1 -- 172.21.15.67:0/1112148703 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279019b410 con 0x7f2790106400 2024-07-27T03:16:11.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.623+0000 7f2794a6a640 1 --2- 172.21.15.67:0/1112148703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2790108ee0 0x7f279019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:16:11.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.623+0000 7f2787fff640 1 --2- 172.21.15.67:0/1112148703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2790106400 0x7f279019a0e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2778006010 tx=0x7f27780043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:11.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.623+0000 7f278dffb640 1 -- 172.21.15.67:0/1112148703 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277803f030 con 0x7f2790106400 2024-07-27T03:16:11.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.623+0000 7f278dffb640 1 -- 172.21.15.67:0/1112148703 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2778003cb0 con 0x7f2790106400 2024-07-27T03:16:11.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.623+0000 7f27964cd640 1 -- 172.21.15.67:0/1112148703 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279019b6a0 con 0x7f2790106400 2024-07-27T03:16:11.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.623+0000 7f278dffb640 1 -- 172.21.15.67:0/1112148703 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778035070 con 0x7f2790106400 2024-07-27T03:16:11.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.623+0000 7f27964cd640 1 -- 172.21.15.67:0/1112148703 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27901a1cc0 con 0x7f2790106400 2024-07-27T03:16:11.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.624+0000 7f27964cd640 1 -- 172.21.15.67:0/1112148703 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27900ff540 con 0x7f2790106400 2024-07-27T03:16:11.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.626+0000 7f278dffb640 1 -- 172.21.15.67:0/1112148703 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f2778042030 con 0x7f2790106400 2024-07-27T03:16:11.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.627+0000 7f278dffb640 1 --2- 172.21.15.67:0/1112148703 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f277c0b3bd0 0x7f277c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:11.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.627+0000 7f278ffff640 1 --2- 172.21.15.67:0/1112148703 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f277c0b3bd0 0x7f277c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:11.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.627+0000 7f278dffb640 1 -- 172.21.15.67:0/1112148703 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f2778104dc0 con 0x7f2790106400 2024-07-27T03:16:11.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.628+0000 7f278ffff640 1 --2- 172.21.15.67:0/1112148703 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f277c0b3bd0 0x7f277c0b60e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f27800058c0 tx=0x7f278000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:11.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.630+0000 7f278dffb640 1 -- 172.21.15.67:0/1112148703 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f27780cc060 con 0x7f2790106400 2024-07-27T03:16:11.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.815+0000 7f27964cd640 1 -- 172.21.15.67:0/1112148703 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f279002e100 con 0x7f2790106400 2024-07-27T03:16:11.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.816+0000 7f278dffb640 1 -- 172.21.15.67:0/1112148703 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) ==== 56+0+5689 (secure 0 0 0) 0x7f27780cb960 con 0x7f2790106400 2024-07-27T03:16:11.819 INFO:teuthology.orchestra.run.smithi067.stdout:epoch 91 2024-07-27T03:16:11.819 INFO:teuthology.orchestra.run.smithi067.stdout:fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:16:11.819 INFO:teuthology.orchestra.run.smithi067.stdout:created 2024-07-27T03:04:08.096824+0000 2024-07-27T03:16:11.819 INFO:teuthology.orchestra.run.smithi067.stdout:modified 2024-07-27T03:13:57.294556+0000 2024-07-27T03:16:11.819 INFO:teuthology.orchestra.run.smithi067.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-07-27T03:16:11.819 INFO:teuthology.orchestra.run.smithi067.stdout:crush_version 27 2024-07-27T03:16:11.819 INFO:teuthology.orchestra.run.smithi067.stdout:full_ratio 0.95 2024-07-27T03:16:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:backfillfull_ratio 0.9 2024-07-27T03:16:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:nearfull_ratio 0.85 2024-07-27T03:16:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:require_min_compat_client luminous 2024-07-27T03:16:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:min_compat_client luminous 2024-07-27T03:16:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:require_osd_release squid 2024-07-27T03:16:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:stretch_mode_enabled false 2024-07-27T03:16:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 23 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-07-27T03:16:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-07-27T03:16:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-07-27T03:16:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-07-27T03:16:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:max_osd 12 2024-07-27T03:16:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:osd.0 up in weight 1 up_from 8 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6802/2648129674,v1:172.21.15.67:6803/2648129674] [v2:172.21.15.67:6804/2648129674,v1:172.21.15.67:6805/2648129674] exists,up 9f614272-18e8-4562-812c-4f43e40efd2b 2024-07-27T03:16:11.820 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6800/2561753152,v1:172.21.15.149:6801/2561753152] [v2:172.21.15.149:6802/2561753152,v1:172.21.15.149:6803/2561753152] exists,up f59f5879-163a-4269-bbe7-1200eea65341 2024-07-27T03:16:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6800/3770940078,v1:172.21.15.175:6801/3770940078] [v2:172.21.15.175:6802/3770940078,v1:172.21.15.175:6803/3770940078] exists,up 444e0ebe-7ca3-4be9-8bd5-db859f824445 2024-07-27T03:16:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:osd.3 up in weight 1 up_from 26 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6810/3159144693,v1:172.21.15.67:6811/3159144693] [v2:172.21.15.67:6812/3159144693,v1:172.21.15.67:6813/3159144693] exists,up d554db3d-9604-427c-8736-b1bb379f8d5f 2024-07-27T03:16:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:osd.4 up in weight 1 up_from 32 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6808/3393503532,v1:172.21.15.149:6809/3393503532] [v2:172.21.15.149:6810/3393503532,v1:172.21.15.149:6811/3393503532] exists,up c317dbe5-3b49-47a6-8408-7042a3dac345 2024-07-27T03:16:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:osd.5 up in weight 1 up_from 37 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6808/2291204784,v1:172.21.15.175:6809/2291204784] [v2:172.21.15.175:6810/2291204784,v1:172.21.15.175:6811/2291204784] exists,up 5c85abd6-2b42-4957-a9b7-59a9ee36b841 2024-07-27T03:16:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:osd.6 up in weight 1 up_from 43 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6818/2660182879,v1:172.21.15.67:6819/2660182879] [v2:172.21.15.67:6820/2660182879,v1:172.21.15.67:6821/2660182879] exists,up bf6c277d-ec24-4470-98de-cf6f28876ccb 2024-07-27T03:16:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:osd.7 up in weight 1 up_from 48 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6816/2289624404,v1:172.21.15.149:6817/2289624404] [v2:172.21.15.149:6818/2289624404,v1:172.21.15.149:6819/2289624404] exists,up fea2c7cf-d5cb-45ae-9b29-f6e56c903891 2024-07-27T03:16:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:osd.8 up in weight 1 up_from 54 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6816/2286032466,v1:172.21.15.175:6817/2286032466] [v2:172.21.15.175:6818/2286032466,v1:172.21.15.175:6819/2286032466] exists,up 14d04d96-579a-4dc9-a4d4-50e4850b48e9 2024-07-27T03:16:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:osd.9 up in weight 1 up_from 60 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6826/3411081513,v1:172.21.15.67:6827/3411081513] [v2:172.21.15.67:6828/3411081513,v1:172.21.15.67:6829/3411081513] exists,up b65dfb6b-1efe-4314-a8be-ce6a295e99fd 2024-07-27T03:16:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:osd.10 up in weight 1 up_from 66 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6824/2011495212,v1:172.21.15.149:6825/2011495212] [v2:172.21.15.149:6826/2011495212,v1:172.21.15.149:6827/2011495212] exists,up e3dabae0-0d64-4222-984c-fc1cce883456 2024-07-27T03:16:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:osd.11 up in weight 1 up_from 72 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.175:6824/2565514213,v1:172.21.15.175:6825/2565514213] [v2:172.21.15.175:6826/2565514213,v1:172.21.15.175:6827/2565514213] exists,up 406c082c-9b5f-4781-8cbf-3c04fa49e410 2024-07-27T03:16:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 2.a [2,11] 2024-07-27T03:16:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 2.f [8,11] 2024-07-27T03:16:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.c [3,9] 2024-07-27T03:16:11.821 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.e [4,6] 2024-07-27T03:16:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.11 [5,9] 2024-07-27T03:16:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.14 [4,7] 2024-07-27T03:16:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.17 [4,8] 2024-07-27T03:16:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.28 [3,6] 2024-07-27T03:16:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 3.33 [5,9] 2024-07-27T03:16:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.6 [5,8] 2024-07-27T03:16:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.c [9,10] 2024-07-27T03:16:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.e [9,8] 2024-07-27T03:16:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:pg_upmap_items 4.10 [9,8] 2024-07-27T03:16:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/3477601235 expires 2024-07-28T03:04:54.947985+0000 2024-07-27T03:16:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/4130679770 expires 2024-07-28T03:04:30.600273+0000 2024-07-27T03:16:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6800/1921856974 expires 2024-07-28T03:04:54.947985+0000 2024-07-27T03:16:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/1029332005 expires 2024-07-28T03:04:54.947985+0000 2024-07-27T03:16:11.822 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6801/2346308006 expires 2024-07-28T03:04:30.600273+0000 2024-07-27T03:16:11.823 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6801/1921856974 expires 2024-07-28T03:04:54.947985+0000 2024-07-27T03:16:11.823 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/284078926 expires 2024-07-28T03:04:54.947985+0000 2024-07-27T03:16:11.823 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:6800/2346308006 expires 2024-07-28T03:04:30.600273+0000 2024-07-27T03:16:11.823 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/2559121542 expires 2024-07-28T03:04:30.600273+0000 2024-07-27T03:16:11.823 INFO:teuthology.orchestra.run.smithi067.stdout:blocklist 172.21.15.67:0/925406021 expires 2024-07-28T03:04:30.600273+0000 2024-07-27T03:16:11.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.818+0000 7f27964cd640 1 -- 172.21.15.67:0/1112148703 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f277c0b3bd0 msgr2=0x7f277c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:11.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.818+0000 7f27964cd640 1 --2- 172.21.15.67:0/1112148703 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f277c0b3bd0 0x7f277c0b60e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f27800058c0 tx=0x7f278000a2d0 comp rx=0 tx=0).stop 2024-07-27T03:16:11.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.818+0000 7f27964cd640 1 -- 172.21.15.67:0/1112148703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2790106400 msgr2=0x7f279019a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:11.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.818+0000 7f27964cd640 1 --2- 172.21.15.67:0/1112148703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2790106400 0x7f279019a0e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2778006010 tx=0x7f27780043b0 comp rx=0 tx=0).stop 2024-07-27T03:16:11.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.818+0000 7f27964cd640 1 -- 172.21.15.67:0/1112148703 shutdown_connections 2024-07-27T03:16:11.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.818+0000 7f27964cd640 1 --2- 172.21.15.67:0/1112148703 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f277c0b3bd0 0x7f277c0b60e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:11.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.818+0000 7f27964cd640 1 --2- 172.21.15.67:0/1112148703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2790108ee0 0x7f279019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:11.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.818+0000 7f27964cd640 1 --2- 172.21.15.67:0/1112148703 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2790106400 0x7f279019a0e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:11.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.818+0000 7f27964cd640 1 --2- 172.21.15.67:0/1112148703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f279006b330 0x7f2790199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:11.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.818+0000 7f27964cd640 1 -- 172.21.15.67:0/1112148703 >> 172.21.15.67:0/1112148703 conn(0x7f27900fda20 msgr2=0x7f2790104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:11.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.819+0000 7f27964cd640 1 -- 172.21.15.67:0/1112148703 shutdown_connections 2024-07-27T03:16:11.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:11.819+0000 7f27964cd640 1 -- 172.21.15.67:0/1112148703 wait complete. 2024-07-27T03:16:12.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:12 smithi067 ceph-mon[32814]: pgmap v419: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:16:12.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:12 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1112148703' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-07-27T03:16:12.507 INFO:teuthology.run_tasks:Running task fs.ready... 2024-07-27T03:16:12.522 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:16:12.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:12 smithi175 ceph-mon[37125]: pgmap v419: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:16:12.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:12 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1112148703' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-07-27T03:16:12.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:12 smithi149 ceph-mon[37173]: pgmap v419: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:16:12.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:12 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1112148703' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-07-27T03:16:12.800 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:16:13.347 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:13 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:16:13.347 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:13 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:16:13.347 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:13 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:16:13.347 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:13 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:16:13.347 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:13 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:16:13.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:13 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:16:13.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:13 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:16:13.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:13 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:16:13.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:13 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:16:13.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:13 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:16:13.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:13 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:16:13.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:13 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:16:13.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:13 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:16:13.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:13 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:16:13.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:13 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:16:14.456 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:14 smithi067 ceph-mon[32814]: pgmap v420: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:14.456 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:16:14.456 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:16:14.456 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:16:14.456 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:16:14.456 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:16:14.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.547+0000 7fbfccdba640 1 -- 172.21.15.67:0/4277951551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfc81024e0 msgr2=0x7fbfc81028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:14.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.547+0000 7fbfccdba640 1 --2- 172.21.15.67:0/4277951551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfc81024e0 0x7fbfc81028f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009900 tx=0x7fbfb802a410 comp rx=0 tx=0).stop 2024-07-27T03:16:14.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.547+0000 7fbfccdba640 1 -- 172.21.15.67:0/4277951551 shutdown_connections 2024-07-27T03:16:14.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.547+0000 7fbfccdba640 1 --2- 172.21.15.67:0/4277951551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfc8108770 0x7fbfc810efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:14.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.547+0000 7fbfccdba640 1 --2- 172.21.15.67:0/4277951551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfc8107e20 0x7fbfc8108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:14.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.547+0000 7fbfccdba640 1 --2- 172.21.15.67:0/4277951551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfc81024e0 0x7fbfc81028f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:14.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.547+0000 7fbfccdba640 1 -- 172.21.15.67:0/4277951551 >> 172.21.15.67:0/4277951551 conn(0x7fbfc8079180 msgr2=0x7fbfc8079590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:14.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.547+0000 7fbfccdba640 1 -- 172.21.15.67:0/4277951551 shutdown_connections 2024-07-27T03:16:14.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.547+0000 7fbfccdba640 1 -- 172.21.15.67:0/4277951551 wait complete. 2024-07-27T03:16:14.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.548+0000 7fbfccdba640 1 Processor -- start 2024-07-27T03:16:14.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.548+0000 7fbfccdba640 1 -- start start 2024-07-27T03:16:14.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.548+0000 7fbfccdba640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfc81024e0 0x7fbfc8199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:14.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.549+0000 7fbfccdba640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfc8107e20 0x7fbfc819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:14.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.549+0000 7fbfccdba640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfc8108770 0x7fbfc819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:14.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.549+0000 7fbfccdba640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc8112440 con 0x7fbfc8107e20 2024-07-27T03:16:14.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.549+0000 7fbfccdba640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81122c0 con 0x7fbfc81024e0 2024-07-27T03:16:14.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.549+0000 7fbfccdba640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81125c0 con 0x7fbfc8108770 2024-07-27T03:16:14.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.549+0000 7fbfc5d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfc8107e20 0x7fbfc819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:14.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.549+0000 7fbfc5d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfc8107e20 0x7fbfc819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52426/0 (socket says 172.21.15.67:52426) 2024-07-27T03:16:14.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.549+0000 7fbfc5d9b640 1 -- 172.21.15.67:0/3174637091 learned_addr learned my addr 172.21.15.67:0/3174637091 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.549+0000 7fbfc6d9d640 1 --2- 172.21.15.67:0/3174637091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfc8108770 0x7fbfc819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.549+0000 7fbfc659c640 1 --2- 172.21.15.67:0/3174637091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfc81024e0 0x7fbfc8199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.549+0000 7fbfc5d9b640 1 -- 172.21.15.67:0/3174637091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfc8108770 msgr2=0x7fbfc819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.549+0000 7fbfc5d9b640 1 --2- 172.21.15.67:0/3174637091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfc8108770 0x7fbfc819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.549+0000 7fbfc5d9b640 1 -- 172.21.15.67:0/3174637091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfc81024e0 msgr2=0x7fbfc8199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.549+0000 7fbfc5d9b640 1 --2- 172.21.15.67:0/3174637091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfc81024e0 0x7fbfc8199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.549+0000 7fbfc5d9b640 1 -- 172.21.15.67:0/3174637091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc819feb0 con 0x7fbfc8107e20 2024-07-27T03:16:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.549+0000 7fbfc5d9b640 1 --2- 172.21.15.67:0/3174637091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfc8107e20 0x7fbfc819a4e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00d740 tx=0x7fbfbc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.549+0000 7fbfb77fe640 1 -- 172.21.15.67:0/3174637091 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc017070 con 0x7fbfc8107e20 2024-07-27T03:16:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.549+0000 7fbfb77fe640 1 -- 172.21.15.67:0/3174637091 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfbc00de50 con 0x7fbfc8107e20 2024-07-27T03:16:14.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.550+0000 7fbfb77fe640 1 -- 172.21.15.67:0/3174637091 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc00f620 con 0x7fbfc8107e20 2024-07-27T03:16:14.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.550+0000 7fbfccdba640 1 -- 172.21.15.67:0/3174637091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc81a01a0 con 0x7fbfc8107e20 2024-07-27T03:16:14.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.550+0000 7fbfccdba640 1 -- 172.21.15.67:0/3174637091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc81a6900 con 0x7fbfc8107e20 2024-07-27T03:16:14.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.551+0000 7fbfccdba640 1 -- 172.21.15.67:0/3174637091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfc8103760 con 0x7fbfc8107e20 2024-07-27T03:16:14.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.552+0000 7fbfb77fe640 1 -- 172.21.15.67:0/3174637091 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7fbfbc01a070 con 0x7fbfc8107e20 2024-07-27T03:16:14.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.553+0000 7fbfb77fe640 1 --2- 172.21.15.67:0/3174637091 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbfb00b3bd0 0x7fbfb00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:14.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.553+0000 7fbfc659c640 1 --2- 172.21.15.67:0/3174637091 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbfb00b3bd0 0x7fbfb00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:14.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.553+0000 7fbfb77fe640 1 -- 172.21.15.67:0/3174637091 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fbfbc0e3980 con 0x7fbfc8107e20 2024-07-27T03:16:14.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.554+0000 7fbfc659c640 1 --2- 172.21.15.67:0/3174637091 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbfb00b3bd0 0x7fbfb00b60e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8031440 tx=0x7fbfb8005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:14.560 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.557+0000 7fbfb77fe640 1 -- 172.21.15.67:0/3174637091 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fbfbc0aac20 con 0x7fbfc8107e20 2024-07-27T03:16:14.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:14 smithi175 ceph-mon[37125]: pgmap v420: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:14.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:16:14.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:16:14.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:16:14.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:16:14.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:16:14.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:14 smithi149 ceph-mon[37173]: pgmap v420: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:14.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:16:14.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:16:14.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:16:14.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:16:14.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:16:14.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.778+0000 7fbfccdba640 1 -- 172.21.15.67:0/3174637091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfc81028f0 con 0x7fbfc8107e20 2024-07-27T03:16:14.781 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.778+0000 7fbfb77fe640 1 -- 172.21.15.67:0/3174637091 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11983 (secure 0 0 0) 0x7fbfbc0aa520 con 0x7fbfc8107e20 2024-07-27T03:16:14.781 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:16:14.781 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":29,"btime":"2024-07-27T03:13:42:247804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14679,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/252686369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":252686369},{"type":"v1","addr":"172.21.15.67:6835","nonce":252686369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/3358832762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3358832762},{"type":"v1","addr":"172.21.15.67:6837","nonce":3358832762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-27T03:12:34.320208+0000","modified":"2024-07-27T03:13:42.062021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24620,"mds_1":24587,"mds_2":24583,"mds_3":24581,"mds_4":24563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14736":{"gid":14736,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/2162708317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2162708317},{"type":"v1","addr":"172.21.15.67:6839","nonce":2162708317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14748":{"gid":14748,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/795704244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":795704244},{"type":"v1","addr":"172.21.15.67:6841","nonce":795704244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/3868325215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3868325215},{"type":"v1","addr":"172.21.15.149:6833","nonce":3868325215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2462749027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2462749027},{"type":"v1","addr":"172.21.15.175:6835","nonce":2462749027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.149:6835/3730964806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3730964806},{"type":"v1","addr":"172.21.15.149:6835","nonce":3730964806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.175:6837/1304803502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1304803502},{"type":"v1","addr":"172.21.15.175:6837","nonce":1304803502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.175:6839/1670568853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1670568853},{"type":"v1","addr":"172.21.15.175:6839","nonce":1670568853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24587":{"gid":24587,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.149:6837/2932951845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2932951845},{"type":"v1","addr":"172.21.15.149:6837","nonce":2932951845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.149:6839/281746498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":281746498},{"type":"v1","addr":"172.21.15.149:6839","nonce":281746498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24620,"qdb_cluster":[24563,24581,24620,24583,24587]},"id":1}]} 2024-07-27T03:16:14.782 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-07-27T03:16:14.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.781+0000 7fbfccdba640 1 -- 172.21.15.67:0/3174637091 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbfb00b3bd0 msgr2=0x7fbfb00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:14.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.781+0000 7fbfccdba640 1 --2- 172.21.15.67:0/3174637091 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbfb00b3bd0 0x7fbfb00b60e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8031440 tx=0x7fbfb8005d50 comp rx=0 tx=0).stop 2024-07-27T03:16:14.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.781+0000 7fbfccdba640 1 -- 172.21.15.67:0/3174637091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfc8107e20 msgr2=0x7fbfc819a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:14.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.781+0000 7fbfccdba640 1 --2- 172.21.15.67:0/3174637091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfc8107e20 0x7fbfc819a4e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00d740 tx=0x7fbfbc00dc00 comp rx=0 tx=0).stop 2024-07-27T03:16:14.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.781+0000 7fbfccdba640 1 -- 172.21.15.67:0/3174637091 shutdown_connections 2024-07-27T03:16:14.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.781+0000 7fbfccdba640 1 --2- 172.21.15.67:0/3174637091 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbfb00b3bd0 0x7fbfb00b60e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:14.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.781+0000 7fbfccdba640 1 --2- 172.21.15.67:0/3174637091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfc8108770 0x7fbfc819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:14.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.781+0000 7fbfccdba640 1 --2- 172.21.15.67:0/3174637091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbfc8107e20 0x7fbfc819a4e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:14.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.781+0000 7fbfccdba640 1 --2- 172.21.15.67:0/3174637091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbfc81024e0 0x7fbfc8199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:14.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.781+0000 7fbfccdba640 1 -- 172.21.15.67:0/3174637091 >> 172.21.15.67:0/3174637091 conn(0x7fbfc8079180 msgr2=0x7fbfc81013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:14.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.781+0000 7fbfccdba640 1 -- 172.21.15.67:0/3174637091 shutdown_connections 2024-07-27T03:16:14.784 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:14.781+0000 7fbfccdba640 1 -- 172.21.15.67:0/3174637091 wait complete. 2024-07-27T03:16:15.419 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:15 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3174637091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:16:15.449 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:16:15.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:15 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3174637091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:16:15.716 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:16:15.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:15 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3174637091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:16:16.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:16 smithi067 ceph-mon[32814]: pgmap v421: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:16:16.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:16 smithi175 ceph-mon[37125]: pgmap v421: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:16:16.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:16 smithi149 ceph-mon[37173]: pgmap v421: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:16:17.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.295+0000 7f3fad06c640 1 -- 172.21.15.67:0/2982184057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fa8107e30 msgr2=0x7f3fa8108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:17.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.295+0000 7f3fad06c640 1 --2- 172.21.15.67:0/2982184057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fa8107e30 0x7f3fa8108240 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009900 tx=0x7f3f9c02a410 comp rx=0 tx=0).stop 2024-07-27T03:16:17.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.295+0000 7f3fad06c640 1 -- 172.21.15.67:0/2982184057 shutdown_connections 2024-07-27T03:16:17.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.295+0000 7f3fad06c640 1 --2- 172.21.15.67:0/2982184057 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fa8108780 0x7f3fa810efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:17.299 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.295+0000 7f3fad06c640 1 --2- 172.21.15.67:0/2982184057 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fa8107e30 0x7f3fa8108240 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:17.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.295+0000 7f3fad06c640 1 --2- 172.21.15.67:0/2982184057 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fa81024f0 0x7f3fa8102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:17.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.295+0000 7f3fad06c640 1 -- 172.21.15.67:0/2982184057 >> 172.21.15.67:0/2982184057 conn(0x7f3fa80791b0 msgr2=0x7f3fa80795c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:17.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.295+0000 7f3fad06c640 1 -- 172.21.15.67:0/2982184057 shutdown_connections 2024-07-27T03:16:17.300 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.295+0000 7f3fad06c640 1 -- 172.21.15.67:0/2982184057 wait complete. 2024-07-27T03:16:17.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.296+0000 7f3fad06c640 1 Processor -- start 2024-07-27T03:16:17.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.296+0000 7f3fad06c640 1 -- start start 2024-07-27T03:16:17.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.296+0000 7f3fad06c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fa81024f0 0x7f3fa8198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:17.303 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.296+0000 7f3fad06c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fa8107e30 0x7f3fa8199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:17.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.296+0000 7f3fad06c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fa8108780 0x7f3fa819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:17.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.296+0000 7f3fad06c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa81123b0 con 0x7f3fa81024f0 2024-07-27T03:16:17.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.296+0000 7f3fad06c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa8112230 con 0x7f3fa8108780 2024-07-27T03:16:17.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.296+0000 7f3fad06c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa8112530 con 0x7f3fa8107e30 2024-07-27T03:16:17.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.297+0000 7f3fa6d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fa81024f0 0x7f3fa8198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:17.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.297+0000 7f3fa6d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fa81024f0 0x7f3fa8198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39040/0 (socket says 172.21.15.67:39040) 2024-07-27T03:16:17.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.297+0000 7f3fa6d9d640 1 -- 172.21.15.67:0/310721955 learned_addr learned my addr 172.21.15.67:0/310721955 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:17.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.297+0000 7f3fa759e640 1 --2- 172.21.15.67:0/310721955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fa8108780 0x7f3fa819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:17.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.297+0000 7f3fa659c640 1 --2- 172.21.15.67:0/310721955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fa8107e30 0x7f3fa8199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:17.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.297+0000 7f3fa6d9d640 1 -- 172.21.15.67:0/310721955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fa8107e30 msgr2=0x7f3fa8199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:17.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.297+0000 7f3fa6d9d640 1 --2- 172.21.15.67:0/310721955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fa8107e30 0x7f3fa8199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:17.304 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.297+0000 7f3fa6d9d640 1 -- 172.21.15.67:0/310721955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fa8108780 msgr2=0x7f3fa819e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:17.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.297+0000 7f3fa6d9d640 1 --2- 172.21.15.67:0/310721955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fa8108780 0x7f3fa819e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:17.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.297+0000 7f3fa6d9d640 1 -- 172.21.15.67:0/310721955 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa819ee10 con 0x7f3fa81024f0 2024-07-27T03:16:17.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.297+0000 7f3fa6d9d640 1 --2- 172.21.15.67:0/310721955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fa81024f0 0x7f3fa8198ea0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009900 tx=0x7f3f98009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:17.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.297+0000 7f3f8ffff640 1 -- 172.21.15.67:0/310721955 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9800d030 con 0x7f3fa81024f0 2024-07-27T03:16:17.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.297+0000 7f3fad06c640 1 -- 172.21.15.67:0/310721955 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa819f100 con 0x7f3fa81024f0 2024-07-27T03:16:17.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.297+0000 7f3f8ffff640 1 -- 172.21.15.67:0/310721955 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f9800d1d0 con 0x7f3fa81024f0 2024-07-27T03:16:17.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.297+0000 7f3f8ffff640 1 -- 172.21.15.67:0/310721955 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98005280 con 0x7f3fa81024f0 2024-07-27T03:16:17.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.297+0000 7f3fad06c640 1 -- 172.21.15.67:0/310721955 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa806b5b0 con 0x7f3fa81024f0 2024-07-27T03:16:17.305 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.298+0000 7f3fad06c640 1 -- 172.21.15.67:0/310721955 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fa8103770 con 0x7f3fa81024f0 2024-07-27T03:16:17.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.300+0000 7f3f8ffff640 1 -- 172.21.15.67:0/310721955 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f3f98005420 con 0x7f3fa81024f0 2024-07-27T03:16:17.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.301+0000 7f3f8ffff640 1 --2- 172.21.15.67:0/310721955 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3f900b3bd0 0x7f3f900b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:17.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.301+0000 7f3fa659c640 1 --2- 172.21.15.67:0/310721955 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3f900b3bd0 0x7f3f900b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:17.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.301+0000 7f3f8ffff640 1 -- 172.21.15.67:0/310721955 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f3f980e5700 con 0x7f3fa81024f0 2024-07-27T03:16:17.306 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.302+0000 7f3fa659c640 1 --2- 172.21.15.67:0/310721955 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3f900b3bd0 0x7f3f900b60e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c000c00 tx=0x7f3f9c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:17.307 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.304+0000 7f3f8ffff640 1 -- 172.21.15.67:0/310721955 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f3f980ac9a0 con 0x7f3fa81024f0 2024-07-27T03:16:17.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.527+0000 7f3fad06c640 1 -- 172.21.15.67:0/310721955 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fa8199fd0 con 0x7f3fa81024f0 2024-07-27T03:16:17.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.528+0000 7f3f8ffff640 1 -- 172.21.15.67:0/310721955 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11983 (secure 0 0 0) 0x7f3fa8199fd0 con 0x7f3fa81024f0 2024-07-27T03:16:17.532 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:16:17.532 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":29,"btime":"2024-07-27T03:13:42:247804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14679,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/252686369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":252686369},{"type":"v1","addr":"172.21.15.67:6835","nonce":252686369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/3358832762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3358832762},{"type":"v1","addr":"172.21.15.67:6837","nonce":3358832762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-27T03:12:34.320208+0000","modified":"2024-07-27T03:13:42.062021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24620,"mds_1":24587,"mds_2":24583,"mds_3":24581,"mds_4":24563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14736":{"gid":14736,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/2162708317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2162708317},{"type":"v1","addr":"172.21.15.67:6839","nonce":2162708317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14748":{"gid":14748,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/795704244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":795704244},{"type":"v1","addr":"172.21.15.67:6841","nonce":795704244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/3868325215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3868325215},{"type":"v1","addr":"172.21.15.149:6833","nonce":3868325215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2462749027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2462749027},{"type":"v1","addr":"172.21.15.175:6835","nonce":2462749027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.149:6835/3730964806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3730964806},{"type":"v1","addr":"172.21.15.149:6835","nonce":3730964806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.175:6837/1304803502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1304803502},{"type":"v1","addr":"172.21.15.175:6837","nonce":1304803502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.175:6839/1670568853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1670568853},{"type":"v1","addr":"172.21.15.175:6839","nonce":1670568853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24587":{"gid":24587,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.149:6837/2932951845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2932951845},{"type":"v1","addr":"172.21.15.149:6837","nonce":2932951845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.149:6839/281746498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":281746498},{"type":"v1","addr":"172.21.15.149:6839","nonce":281746498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24620,"qdb_cluster":[24563,24581,24620,24583,24587]},"id":1}]} 2024-07-27T03:16:17.532 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 29 2024-07-27T03:16:17.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.530+0000 7f3fad06c640 1 -- 172.21.15.67:0/310721955 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3f900b3bd0 msgr2=0x7f3f900b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:17.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.530+0000 7f3fad06c640 1 --2- 172.21.15.67:0/310721955 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3f900b3bd0 0x7f3f900b60e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c000c00 tx=0x7f3f9c031400 comp rx=0 tx=0).stop 2024-07-27T03:16:17.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.530+0000 7f3fad06c640 1 -- 172.21.15.67:0/310721955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fa81024f0 msgr2=0x7f3fa8198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:17.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.530+0000 7f3fad06c640 1 --2- 172.21.15.67:0/310721955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fa81024f0 0x7f3fa8198ea0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009900 tx=0x7f3f98009dc0 comp rx=0 tx=0).stop 2024-07-27T03:16:17.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.530+0000 7f3fad06c640 1 -- 172.21.15.67:0/310721955 shutdown_connections 2024-07-27T03:16:17.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.530+0000 7f3fad06c640 1 --2- 172.21.15.67:0/310721955 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3f900b3bd0 0x7f3f900b60e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:17.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.530+0000 7f3fad06c640 1 --2- 172.21.15.67:0/310721955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3fa8108780 0x7f3fa819e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:17.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.530+0000 7f3fad06c640 1 --2- 172.21.15.67:0/310721955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fa8107e30 0x7f3fa8199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:17.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.530+0000 7f3fad06c640 1 --2- 172.21.15.67:0/310721955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3fa81024f0 0x7f3fa8198ea0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:17.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.530+0000 7f3fad06c640 1 -- 172.21.15.67:0/310721955 >> 172.21.15.67:0/310721955 conn(0x7f3fa80791b0 msgr2=0x7f3fa8103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:17.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.530+0000 7f3fad06c640 1 -- 172.21.15.67:0/310721955 shutdown_connections 2024-07-27T03:16:17.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:17.530+0000 7f3fad06c640 1 -- 172.21.15.67:0/310721955 wait complete. 2024-07-27T03:16:18.270 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd dump --format=json 2024-07-27T03:16:18.544 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:16:18.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:18 smithi175 ceph-mon[37125]: pgmap v422: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:18.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:18 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/310721955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:16:18.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:18 smithi067 ceph-mon[32814]: pgmap v422: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:18.667 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:18 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/310721955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:16:18.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:18 smithi149 ceph-mon[37173]: pgmap v422: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:18.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:18 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/310721955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:16:20.155 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.147+0000 7f14e0bd4640 1 -- 172.21.15.67:0/3305951764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14dc108f10 msgr2=0x7f14dc109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:20.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.147+0000 7f14e0bd4640 1 --2- 172.21.15.67:0/3305951764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14dc108f10 0x7f14dc109320 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009900 tx=0x7f14c402a3f0 comp rx=0 tx=0).stop 2024-07-27T03:16:20.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.147+0000 7f14e0bd4640 1 -- 172.21.15.67:0/3305951764 shutdown_connections 2024-07-27T03:16:20.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.147+0000 7f14e0bd4640 1 --2- 172.21.15.67:0/3305951764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14dc109860 0x7f14dc110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:20.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.147+0000 7f14e0bd4640 1 --2- 172.21.15.67:0/3305951764 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14dc108f10 0x7f14dc109320 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:20.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.147+0000 7f14e0bd4640 1 --2- 172.21.15.67:0/3305951764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14dc1035d0 0x7f14dc1039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:20.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.147+0000 7f14e0bd4640 1 -- 172.21.15.67:0/3305951764 >> 172.21.15.67:0/3305951764 conn(0x7f14dc07b2c0 msgr2=0x7f14dc07b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:20.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.147+0000 7f14e0bd4640 1 -- 172.21.15.67:0/3305951764 shutdown_connections 2024-07-27T03:16:20.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.147+0000 7f14e0bd4640 1 -- 172.21.15.67:0/3305951764 wait complete. 2024-07-27T03:16:20.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.148+0000 7f14e0bd4640 1 Processor -- start 2024-07-27T03:16:20.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.148+0000 7f14e0bd4640 1 -- start start 2024-07-27T03:16:20.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.148+0000 7f14e0bd4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14dc1035d0 0x7f14dc075180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:20.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.148+0000 7f14e0bd4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14dc108f10 0x7f14dc0756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:20.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.148+0000 7f14e0bd4640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14dc109860 0x7f14dc075c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:20.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.148+0000 7f14e0bd4640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc1134f0 con 0x7f14dc1035d0 2024-07-27T03:16:20.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.148+0000 7f14e0bd4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc113370 con 0x7f14dc108f10 2024-07-27T03:16:20.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.148+0000 7f14e0bd4640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc113670 con 0x7f14dc109860 2024-07-27T03:16:20.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.148+0000 7f14da59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14dc1035d0 0x7f14dc075180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:20.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.148+0000 7f14da59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14dc1035d0 0x7f14dc075180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39054/0 (socket says 172.21.15.67:39054) 2024-07-27T03:16:20.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.148+0000 7f14da59c640 1 -- 172.21.15.67:0/1141693548 learned_addr learned my addr 172.21.15.67:0/1141693548 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:20.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.148+0000 7f14d9d9b640 1 --2- 172.21.15.67:0/1141693548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14dc108f10 0x7f14dc0756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:20.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.148+0000 7f14dad9d640 1 --2- 172.21.15.67:0/1141693548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14dc109860 0x7f14dc075c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:20.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.149+0000 7f14da59c640 1 -- 172.21.15.67:0/1141693548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14dc109860 msgr2=0x7f14dc075c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:20.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.149+0000 7f14da59c640 1 --2- 172.21.15.67:0/1141693548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14dc109860 0x7f14dc075c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:20.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.149+0000 7f14da59c640 1 -- 172.21.15.67:0/1141693548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14dc108f10 msgr2=0x7f14dc0756c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:20.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.149+0000 7f14da59c640 1 --2- 172.21.15.67:0/1141693548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14dc108f10 0x7f14dc0756c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:20.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.149+0000 7f14da59c640 1 -- 172.21.15.67:0/1141693548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14dc1a81e0 con 0x7f14dc1035d0 2024-07-27T03:16:20.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.149+0000 7f14da59c640 1 --2- 172.21.15.67:0/1141693548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14dc1035d0 0x7f14dc075180 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00c890 tx=0x7f14cc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:20.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.149+0000 7f14d37fe640 1 -- 172.21.15.67:0/1141693548 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc004220 con 0x7f14dc1035d0 2024-07-27T03:16:20.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.149+0000 7f14d37fe640 1 -- 172.21.15.67:0/1141693548 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14cc0043c0 con 0x7f14dc1035d0 2024-07-27T03:16:20.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.149+0000 7f14d37fe640 1 -- 172.21.15.67:0/1141693548 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc004a00 con 0x7f14dc1035d0 2024-07-27T03:16:20.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.149+0000 7f14e0bd4640 1 -- 172.21.15.67:0/1141693548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14dc1a84d0 con 0x7f14dc1035d0 2024-07-27T03:16:20.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.149+0000 7f14e0bd4640 1 -- 172.21.15.67:0/1141693548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14dc1a89e0 con 0x7f14dc1035d0 2024-07-27T03:16:20.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.150+0000 7f14e0bd4640 1 -- 172.21.15.67:0/1141693548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14dc104850 con 0x7f14dc1035d0 2024-07-27T03:16:20.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.152+0000 7f14d37fe640 1 -- 172.21.15.67:0/1141693548 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f14cc0079b0 con 0x7f14dc1035d0 2024-07-27T03:16:20.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.153+0000 7f14d37fe640 1 --2- 172.21.15.67:0/1141693548 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f14c80b3bd0 0x7f14c80b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:20.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.153+0000 7f14d9d9b640 1 --2- 172.21.15.67:0/1141693548 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f14c80b3bd0 0x7f14c80b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:20.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.153+0000 7f14d37fe640 1 -- 172.21.15.67:0/1141693548 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f14cc0e40f0 con 0x7f14dc1035d0 2024-07-27T03:16:20.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.154+0000 7f14d9d9b640 1 --2- 172.21.15.67:0/1141693548 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f14c80b3bd0 0x7f14c80b60e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009230 tx=0x7f14c4031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:20.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.156+0000 7f14d37fe640 1 -- 172.21.15.67:0/1141693548 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f14cc014030 con 0x7f14dc1035d0 2024-07-27T03:16:20.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.341+0000 7f14e0bd4640 1 -- 172.21.15.67:0/1141693548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f14dc076480 con 0x7f14dc1035d0 2024-07-27T03:16:20.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.343+0000 7f14d37fe640 1 -- 172.21.15.67:0/1141693548 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24829 (secure 0 0 0) 0x7f14cc0ab390 con 0x7f14dc1035d0 2024-07-27T03:16:20.346 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:16:20.346 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":91,"fsid":"c0782a00-4bc4-11ef-bcc7-c7b262605968","created":"2024-07-27T03:04:08.096824+0000","modified":"2024-07-27T03:13:57.294556+0000","last_up_change":"2024-07-27T03:11:33.314800+0000","last_in_change":"2024-07-27T03:11:14.028095+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-27T03:08:01.042479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-27T03:12:27.565948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-27T03:12:31.353726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-27T03:12:40.599486+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9f614272-18e8-4562-812c-4f43e40efd2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6803","nonce":2648129674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6805","nonce":2648129674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6809","nonce":2648129674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6807","nonce":2648129674}]},"public_addr":"172.21.15.67:6803/2648129674","cluster_addr":"172.21.15.67:6805/2648129674","heartbeat_back_addr":"172.21.15.67:6809/2648129674","heartbeat_front_addr":"172.21.15.67:6807/2648129674","state":["exists","up"]},{"osd":1,"uuid":"f59f5879-163a-4269-bbe7-1200eea65341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6801","nonce":2561753152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6803","nonce":2561753152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6807","nonce":2561753152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6805","nonce":2561753152}]},"public_addr":"172.21.15.149:6801/2561753152","cluster_addr":"172.21.15.149:6803/2561753152","heartbeat_back_addr":"172.21.15.149:6807/2561753152","heartbeat_front_addr":"172.21.15.149:6805/2561753152","state":["exists","up"]},{"osd":2,"uuid":"444e0ebe-7ca3-4be9-8bd5-db859f824445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6801","nonce":3770940078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6803","nonce":3770940078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6807","nonce":3770940078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6805","nonce":3770940078}]},"public_addr":"172.21.15.175:6801/3770940078","cluster_addr":"172.21.15.175:6803/3770940078","heartbeat_back_addr":"172.21.15.175:6807/3770940078","heartbeat_front_addr":"172.21.15.175:6805/3770940078","state":["exists","up"]},{"osd":3,"uuid":"d554db3d-9604-427c-8736-b1bb379f8d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6811","nonce":3159144693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6813","nonce":3159144693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6817","nonce":3159144693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6815","nonce":3159144693}]},"public_addr":"172.21.15.67:6811/3159144693","cluster_addr":"172.21.15.67:6813/3159144693","heartbeat_back_addr":"172.21.15.67:6817/3159144693","heartbeat_front_addr":"172.21.15.67:6815/3159144693","state":["exists","up"]},{"osd":4,"uuid":"c317dbe5-3b49-47a6-8408-7042a3dac345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3393503532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3393503532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3393503532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3393503532}]},"public_addr":"172.21.15.149:6809/3393503532","cluster_addr":"172.21.15.149:6811/3393503532","heartbeat_back_addr":"172.21.15.149:6815/3393503532","heartbeat_front_addr":"172.21.15.149:6813/3393503532","state":["exists","up"]},{"osd":5,"uuid":"5c85abd6-2b42-4957-a9b7-59a9ee36b841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6809","nonce":2291204784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6811","nonce":2291204784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6815","nonce":2291204784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6813","nonce":2291204784}]},"public_addr":"172.21.15.175:6809/2291204784","cluster_addr":"172.21.15.175:6811/2291204784","heartbeat_back_addr":"172.21.15.175:6815/2291204784","heartbeat_front_addr":"172.21.15.175:6813/2291204784","state":["exists","up"]},{"osd":6,"uuid":"bf6c277d-ec24-4470-98de-cf6f28876ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6819","nonce":2660182879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6821","nonce":2660182879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6825","nonce":2660182879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6823","nonce":2660182879}]},"public_addr":"172.21.15.67:6819/2660182879","cluster_addr":"172.21.15.67:6821/2660182879","heartbeat_back_addr":"172.21.15.67:6825/2660182879","heartbeat_front_addr":"172.21.15.67:6823/2660182879","state":["exists","up"]},{"osd":7,"uuid":"fea2c7cf-d5cb-45ae-9b29-f6e56c903891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6817","nonce":2289624404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6819","nonce":2289624404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6823","nonce":2289624404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6821","nonce":2289624404}]},"public_addr":"172.21.15.149:6817/2289624404","cluster_addr":"172.21.15.149:6819/2289624404","heartbeat_back_addr":"172.21.15.149:6823/2289624404","heartbeat_front_addr":"172.21.15.149:6821/2289624404","state":["exists","up"]},{"osd":8,"uuid":"14d04d96-579a-4dc9-a4d4-50e4850b48e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6817","nonce":2286032466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6819","nonce":2286032466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6823","nonce":2286032466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6821","nonce":2286032466}]},"public_addr":"172.21.15.175:6817/2286032466","cluster_addr":"172.21.15.175:6819/2286032466","heartbeat_back_addr":"172.21.15.175:6823/2286032466","heartbeat_front_addr":"172.21.15.175:6821/2286032466","state":["exists","up"]},{"osd":9,"uuid":"b65dfb6b-1efe-4314-a8be-ce6a295e99fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6827","nonce":3411081513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6829","nonce":3411081513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6833","nonce":3411081513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6831","nonce":3411081513}]},"public_addr":"172.21.15.67:6827/3411081513","cluster_addr":"172.21.15.67:6829/3411081513","heartbeat_back_addr":"172.21.15.67:6833/3411081513","heartbeat_front_addr":"172.21.15.67:6831/3411081513","state":["exists","up"]},{"osd":10,"uuid":"e3dabae0-0d64-4222-984c-fc1cce883456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6825","nonce":2011495212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6827","nonce":2011495212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6831","nonce":2011495212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6829","nonce":2011495212}]},"public_addr":"172.21.15.149:6825/2011495212","cluster_addr":"172.21.15.149:6827/2011495212","heartbeat_back_addr":"172.21.15.149:6831/2011495212","heartbeat_front_addr":"172.21.15.149:6829/2011495212","state":["exists","up"]},{"osd":11,"uuid":"406c082c-9b5f-4781-8cbf-3c04fa49e410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6825","nonce":2565514213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6827","nonce":2565514213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6831","nonce":2565514213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6829","nonce":2565514213}]},"public_addr":"172.21.15.175:6825/2565514213","cluster_addr":"172.21.15.175:6827/2565514213","heartbeat_back_addr":"172.21.15.175:6831/2565514213","heartbeat_front_addr":"172.21.15.175:6829/2565514213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:11.399380+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:33.728518+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:56.411181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:19.871731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:42.994576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:06.712815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:31.321967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:55.070898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:18.686108+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:43.187795+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:07.318538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:31.374843+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/3477601235":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/4130679770":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6800/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/1029332005":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6801/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6801/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/284078926":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6800/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/2559121542":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/925406021":"2024-07-28T03:04:30.600273+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-27T03:16:20.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.345+0000 7f14e0bd4640 1 -- 172.21.15.67:0/1141693548 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f14c80b3bd0 msgr2=0x7f14c80b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.345+0000 7f14e0bd4640 1 --2- 172.21.15.67:0/1141693548 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f14c80b3bd0 0x7f14c80b60e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009230 tx=0x7f14c4031400 comp rx=0 tx=0).stop 2024-07-27T03:16:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.345+0000 7f14e0bd4640 1 -- 172.21.15.67:0/1141693548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14dc1035d0 msgr2=0x7f14dc075180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.345+0000 7f14e0bd4640 1 --2- 172.21.15.67:0/1141693548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14dc1035d0 0x7f14dc075180 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00c890 tx=0x7f14cc00cd50 comp rx=0 tx=0).stop 2024-07-27T03:16:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.345+0000 7f14e0bd4640 1 -- 172.21.15.67:0/1141693548 shutdown_connections 2024-07-27T03:16:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.345+0000 7f14e0bd4640 1 --2- 172.21.15.67:0/1141693548 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f14c80b3bd0 0x7f14c80b60e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.345+0000 7f14e0bd4640 1 --2- 172.21.15.67:0/1141693548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14dc109860 0x7f14dc075c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.345+0000 7f14e0bd4640 1 --2- 172.21.15.67:0/1141693548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14dc108f10 0x7f14dc0756c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.345+0000 7f14e0bd4640 1 --2- 172.21.15.67:0/1141693548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f14dc1035d0 0x7f14dc075180 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.345+0000 7f14e0bd4640 1 -- 172.21.15.67:0/1141693548 >> 172.21.15.67:0/1141693548 conn(0x7f14dc07b2c0 msgr2=0x7f14dc107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.345+0000 7f14e0bd4640 1 -- 172.21.15.67:0/1141693548 shutdown_connections 2024-07-27T03:16:20.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:20.345+0000 7f14e0bd4640 1 -- 172.21.15.67:0/1141693548 wait complete. 2024-07-27T03:16:20.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:20 smithi175 ceph-mon[37125]: pgmap v423: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:16:20.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:20 smithi067 ceph-mon[32814]: pgmap v423: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:16:20.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:20 smithi149 ceph-mon[37173]: pgmap v423: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:16:21.063 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-07-27T03:12:34.320208+0000', 'modified': '2024-07-27T03:13:42.062021+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24620, 'mds_1': 24587, 'mds_2': 24583, 'mds_3': 24581, 'mds_4': 24563}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14736': {'gid': 14736, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.67:6839/2162708317', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6838', 'nonce': 2162708317}, {'type': 'v1', 'addr': '172.21.15.67:6839', 'nonce': 2162708317}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14748': {'gid': 14748, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.67:6841/795704244', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6840', 'nonce': 795704244}, {'type': 'v1', 'addr': '172.21.15.67:6841', 'nonce': 795704244}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24526': {'gid': 24526, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6833/3804638701', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6832', 'nonce': 3804638701}, {'type': 'v1', 'addr': '172.21.15.175:6833', 'nonce': 3804638701}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24533': {'gid': 24533, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6833/3868325215', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6832', 'nonce': 3868325215}, {'type': 'v1', 'addr': '172.21.15.149:6833', 'nonce': 3868325215}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24562': {'gid': 24562, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.175:6835/2462749027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6834', 'nonce': 2462749027}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 2462749027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24563': {'gid': 24563, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.149:6835/3730964806', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6834', 'nonce': 3730964806}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 3730964806}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24581': {'gid': 24581, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.175:6837/1304803502', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6836', 'nonce': 1304803502}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 1304803502}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24583': {'gid': 24583, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.175:6839/1670568853', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6838', 'nonce': 1670568853}, {'type': 'v1', 'addr': '172.21.15.175:6839', 'nonce': 1670568853}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24587': {'gid': 24587, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.149:6837/2932951845', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 2932951845}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 2932951845}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24620': {'gid': 24620, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.149:6839/281746498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 281746498}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 281746498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24620, 'qdb_cluster': [24563, 24581, 24620, 24583, 24587]} 2024-07-27T03:16:21.063 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-07-27T03:16:21.064 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.e status 2024-07-27T03:16:21.335 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:16:21.594 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:21 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1141693548' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:16:21.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:21 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1141693548' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:16:21.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:21 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1141693548' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:16:22.430 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:22 smithi067 ceph-mon[32814]: pgmap v424: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:16:22.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:22 smithi175 ceph-mon[37125]: pgmap v424: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:16:22.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:22 smithi149 ceph-mon[37173]: pgmap v424: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:16:22.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.952+0000 7efc25ee1640 1 -- 172.21.15.67:0/3526019538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc201024d0 msgr2=0x7efc201028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:22.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.952+0000 7efc25ee1640 1 --2- 172.21.15.67:0/3526019538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc201024d0 0x7efc201028e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efc10009900 tx=0x7efc1002a3f0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.952+0000 7efc25ee1640 1 -- 172.21.15.67:0/3526019538 shutdown_connections 2024-07-27T03:16:22.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.952+0000 7efc25ee1640 1 --2- 172.21.15.67:0/3526019538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc20108760 0x7efc2010ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.952+0000 7efc25ee1640 1 --2- 172.21.15.67:0/3526019538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc20107e10 0x7efc20108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.952+0000 7efc25ee1640 1 --2- 172.21.15.67:0/3526019538 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc201024d0 0x7efc201028e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.952+0000 7efc25ee1640 1 -- 172.21.15.67:0/3526019538 >> 172.21.15.67:0/3526019538 conn(0x7efc200791a0 msgr2=0x7efc200795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:22.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.952+0000 7efc25ee1640 1 -- 172.21.15.67:0/3526019538 shutdown_connections 2024-07-27T03:16:22.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.952+0000 7efc25ee1640 1 -- 172.21.15.67:0/3526019538 wait complete. 2024-07-27T03:16:22.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.953+0000 7efc25ee1640 1 Processor -- start 2024-07-27T03:16:22.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.953+0000 7efc25ee1640 1 -- start start 2024-07-27T03:16:22.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.953+0000 7efc25ee1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc201024d0 0x7efc20199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:22.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.953+0000 7efc25ee1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc20107e10 0x7efc2019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:22.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.953+0000 7efc25ee1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc20108760 0x7efc2019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:22.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.953+0000 7efc25ee1640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efc201134f0 con 0x7efc20107e10 2024-07-27T03:16:22.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.953+0000 7efc25ee1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efc20113370 con 0x7efc20108760 2024-07-27T03:16:22.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.953+0000 7efc25ee1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efc20113670 con 0x7efc201024d0 2024-07-27T03:16:22.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.953+0000 7efc1effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc20107e10 0x7efc2019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:22.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.953+0000 7efc1effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc20107e10 0x7efc2019a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39070/0 (socket says 172.21.15.67:39070) 2024-07-27T03:16:22.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.953+0000 7efc1effd640 1 -- 172.21.15.67:0/1594275486 learned_addr learned my addr 172.21.15.67:0/1594275486 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:22.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.954+0000 7efc1ffff640 1 --2- 172.21.15.67:0/1594275486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc20108760 0x7efc2019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:22.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.954+0000 7efc1f7fe640 1 --2- 172.21.15.67:0/1594275486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc201024d0 0x7efc20199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:22.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.954+0000 7efc1effd640 1 -- 172.21.15.67:0/1594275486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc201024d0 msgr2=0x7efc20199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:22.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.954+0000 7efc1effd640 1 --2- 172.21.15.67:0/1594275486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc201024d0 0x7efc20199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.954+0000 7efc1effd640 1 -- 172.21.15.67:0/1594275486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc20108760 msgr2=0x7efc2019f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:22.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.954+0000 7efc1effd640 1 --2- 172.21.15.67:0/1594275486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc20108760 0x7efc2019f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.954+0000 7efc1effd640 1 -- 172.21.15.67:0/1594275486 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2019fcf0 con 0x7efc20107e10 2024-07-27T03:16:22.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.954+0000 7efc1effd640 1 --2- 172.21.15.67:0/1594275486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc20107e10 0x7efc2019a3e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7efc1400c970 tx=0x7efc1400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:22.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.954+0000 7efc1cff9640 1 -- 172.21.15.67:0/1594275486 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14014070 con 0x7efc20107e10 2024-07-27T03:16:22.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.954+0000 7efc1cff9640 1 -- 172.21.15.67:0/1594275486 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc14004440 con 0x7efc20107e10 2024-07-27T03:16:22.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.954+0000 7efc1cff9640 1 -- 172.21.15.67:0/1594275486 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14003e50 con 0x7efc20107e10 2024-07-27T03:16:22.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.954+0000 7efc25ee1640 1 -- 172.21.15.67:0/1594275486 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc2019ffe0 con 0x7efc20107e10 2024-07-27T03:16:22.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.954+0000 7efc25ee1640 1 -- 172.21.15.67:0/1594275486 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc201a6800 con 0x7efc20107e10 2024-07-27T03:16:22.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.957+0000 7efc2c466740 1 Processor -- start 2024-07-27T03:16:22.967 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.957+0000 7efc2c466740 1 -- start start 2024-07-27T03:16:22.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.957+0000 7efc2c466740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ec3306f60 0x558ec36ac200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:22.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.957+0000 7efc2c466740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec32de0a0 0x558ec36ac740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:22.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.957+0000 7efc2c466740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ec32e6280 0x558ec36acc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:22.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.957+0000 7efc2c466740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558ec35942b0 con 0x558ec32de0a0 2024-07-27T03:16:22.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.957+0000 7efc2c466740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558ec3622890 con 0x558ec32e6280 2024-07-27T03:16:22.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.957+0000 7efc2c466740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558ec3611d90 con 0x558ec3306f60 2024-07-27T03:16:22.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.957+0000 7efc1f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec32de0a0 0x558ec36ac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:22.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.957+0000 7efc1cff9640 1 -- 172.21.15.67:0/1594275486 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7efc14017070 con 0x7efc20107e10 2024-07-27T03:16:22.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.957+0000 7efc1f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec32de0a0 0x558ec36ac740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39080/0 (socket says 172.21.15.67:39080) 2024-07-27T03:16:22.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.957+0000 7efc1f7fe640 1 -- 172.21.15.67:0/723144224 learned_addr learned my addr 172.21.15.67:0/723144224 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:22.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.958+0000 7efc1ffff640 1 --2- 172.21.15.67:0/723144224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ec3306f60 0x558ec36ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:22.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.958+0000 7efc1f7fe640 1 -- 172.21.15.67:0/723144224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ec3306f60 msgr2=0x558ec36ac200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:22.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.958+0000 7efc1f7fe640 1 --2- 172.21.15.67:0/723144224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ec3306f60 0x558ec36ac200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.958+0000 7efc1f7fe640 1 -- 172.21.15.67:0/723144224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ec32e6280 msgr2=0x558ec36acc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:22.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.958+0000 7efc1f7fe640 1 --2- 172.21.15.67:0/723144224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ec32e6280 0x558ec36acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.958+0000 7efc1f7fe640 1 -- 172.21.15.67:0/723144224 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ec364baf0 con 0x558ec32de0a0 2024-07-27T03:16:22.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.958+0000 7efc1f7fe640 1 --2- 172.21.15.67:0/723144224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec32de0a0 0x558ec36ac740 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7efc1002a910 tx=0x7efc10031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8293f3db75f38f server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:22.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.958+0000 7efc05ffb640 1 -- 172.21.15.67:0/723144224 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1003f070 con 0x558ec32de0a0 2024-07-27T03:16:22.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.958+0000 7efc05ffb640 1 -- 172.21.15.67:0/723144224 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc100316a0 con 0x558ec32de0a0 2024-07-27T03:16:22.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.958+0000 7efc05ffb640 1 -- 172.21.15.67:0/723144224 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc10035070 con 0x558ec32de0a0 2024-07-27T03:16:22.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.958+0000 7efc1cff9640 1 --2- 172.21.15.67:0/1594275486 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7efc080b3e20 0x7efc080b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:22.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.959+0000 7efc1ffff640 1 --2- 172.21.15.67:0/1594275486 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7efc080b3e20 0x7efc080b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:22.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.959+0000 7efc1cff9640 1 -- 172.21.15.67:0/1594275486 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7efc140e3a20 con 0x7efc20107e10 2024-07-27T03:16:22.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.959+0000 7efc2c466740 1 -- 172.21.15.67:0/723144224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec32de0a0 msgr2=0x558ec36ac740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:22.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.959+0000 7efc1ffff640 1 --2- 172.21.15.67:0/1594275486 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7efc080b3e20 0x7efc080b6330 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efc18007f80 tx=0x7efc18007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:22.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.959+0000 7efc2c466740 1 --2- 172.21.15.67:0/723144224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec32de0a0 0x558ec36ac740 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7efc1002a910 tx=0x7efc10031cd0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.959+0000 7efc2c466740 1 -- 172.21.15.67:0/723144224 shutdown_connections 2024-07-27T03:16:22.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.959+0000 7efc2c466740 1 --2- 172.21.15.67:0/723144224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ec32e6280 0x558ec36acc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.959+0000 7efc2c466740 1 --2- 172.21.15.67:0/723144224 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec32de0a0 0x558ec36ac740 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.959+0000 7efc2c466740 1 --2- 172.21.15.67:0/723144224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ec3306f60 0x558ec36ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.959+0000 7efc2c466740 1 -- 172.21.15.67:0/723144224 >> 172.21.15.67:0/723144224 conn(0x558ec32fb350 msgr2=0x558ec36aa0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:22.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.959+0000 7efc2c466740 1 -- 172.21.15.67:0/723144224 shutdown_connections 2024-07-27T03:16:22.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.960+0000 7efc2c466740 1 -- 172.21.15.67:0/723144224 wait complete. 2024-07-27T03:16:22.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.960+0000 7efc2c466740 1 -- start start 2024-07-27T03:16:22.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.960+0000 7efc2c466740 1 Processor -- start 2024-07-27T03:16:22.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.960+0000 7efc2c466740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ec32de0a0 0x558ec36b7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:22.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.960+0000 7efc2c466740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ec32e6280 0x558ec36b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:22.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.960+0000 7efc2c466740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec3306f60 0x558ec36bdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:22.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.960+0000 7efc2c466740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558ec35b47b0 con 0x558ec3306f60 2024-07-27T03:16:22.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.960+0000 7efc2c466740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558ec3615dd0 con 0x558ec32e6280 2024-07-27T03:16:22.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.960+0000 7efc2c466740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558ec34755f0 con 0x558ec32de0a0 2024-07-27T03:16:22.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.961+0000 7efc2c466740 10 client.0 _ll_register_callbacks cb 0x558ec35bc7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:16:22.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.961+0000 7efc1ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec3306f60 0x558ec36bdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:22.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.961+0000 7efc1ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec3306f60 0x558ec36bdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39094/0 (socket says 172.21.15.67:39094) 2024-07-27T03:16:22.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.961+0000 7efc1ffff640 1 -- 172.21.15.67:0/3464732856 learned_addr learned my addr 172.21.15.67:0/3464732856 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:22.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.961+0000 7efc1effd640 1 --2- 172.21.15.67:0/3464732856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ec32e6280 0x558ec36b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:22.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.961+0000 7efc1f7fe640 1 --2- 172.21.15.67:0/3464732856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ec32de0a0 0x558ec36b7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:22.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.961+0000 7efc1ffff640 1 -- 172.21.15.67:0/3464732856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ec32de0a0 msgr2=0x558ec36b7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:22.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.961+0000 7efc1ffff640 1 --2- 172.21.15.67:0/3464732856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ec32de0a0 0x558ec36b7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.961+0000 7efc1ffff640 1 -- 172.21.15.67:0/3464732856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ec32e6280 msgr2=0x558ec36b84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:22.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.961+0000 7efc1ffff640 1 --2- 172.21.15.67:0/3464732856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ec32e6280 0x558ec36b84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.961+0000 7efc1ffff640 1 -- 172.21.15.67:0/3464732856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ec35a7620 con 0x558ec3306f60 2024-07-27T03:16:22.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.961+0000 7efc1ffff640 1 --2- 172.21.15.67:0/3464732856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec3306f60 0x558ec36bdaf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efc1801ec20 tx=0x7efc18004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2aca2e89d7af44 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:22.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.961+0000 7efbfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:16:22.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.961+0000 7efbfeffd640 1 -- 172.21.15.67:0/3464732856 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1802b060 con 0x558ec3306f60 2024-07-27T03:16:22.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.961+0000 7efbfeffd640 1 -- 172.21.15.67:0/3464732856 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7efc18016030 con 0x558ec3306f60 2024-07-27T03:16:22.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.961+0000 7efc2c466740 1 -- 172.21.15.67:0/3464732856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc100316a0 con 0x558ec3306f60 2024-07-27T03:16:22.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.962+0000 7efbfeffd640 1 -- 172.21.15.67:0/3464732856 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efc18032050 con 0x558ec3306f60 2024-07-27T03:16:22.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.962+0000 7efc2c466740 10 client.15354 fetch_fsmap learned FSMap version 29 2024-07-27T03:16:22.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.962+0000 7efc2c466740 1 -- 172.21.15.67:0/3464732856 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558ec35b8aa0 con 0x558ec3306f60 2024-07-27T03:16:22.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.962+0000 7efbfeffd640 1 -- 172.21.15.67:0/3464732856 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc100316a0 con 0x558ec3306f60 2024-07-27T03:16:22.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.962+0000 7efbfeffd640 1 -- 172.21.15.67:0/3464732856 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7efc1801b3f0 con 0x558ec3306f60 2024-07-27T03:16:22.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.962+0000 7efc2c466740 10 client.15354 fetch_fsmap finished waiting for FSMap version 29 2024-07-27T03:16:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.962+0000 7efc2c466740 10 client.15354 resolve_mds: resolved name 'e' to daemon mds.e 2024-07-27T03:16:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.962+0000 7efc2c466740 20 client.15354 populate_metadata read hostname 'smithi067' 2024-07-27T03:16:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.962+0000 7efc2c466740 1 --2- 172.21.15.67:0/3464732856 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x558ec32c58e0 0x558ec36ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.962+0000 7efc2c466740 4 client.15354 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:16:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.962+0000 7efc2c466740 1 -- 172.21.15.67:0/3464732856 --> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ec36247e0 con 0x558ec32c58e0 2024-07-27T03:16:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.963+0000 7efc1f7fe640 1 --2- 172.21.15.67:0/3464732856 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x558ec32c58e0 0x558ec36ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.963+0000 7efc1f7fe640 1 --2- 172.21.15.67:0/3464732856 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x558ec32c58e0 0x558ec36ca300 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1cd3385aeae53a86 server_cookie=fa8483a7b9b0d9ab in_seq=0 out_seq=0 2024-07-27T03:16:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.963+0000 7efbfeffd640 10 client.15354 ms_handle_connect on v2:172.21.15.149:6834/3730964806 2024-07-27T03:16:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.964+0000 7efbfeffd640 1 -- 172.21.15.67:0/3464732856 <== mds.4 v2:172.21.15.149:6834/3730964806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ec36247e0 con 0x558ec32c58e0 2024-07-27T03:16:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.964+0000 7efbfeffd640 10 client.15354 handle_command_reply: tid=0 2024-07-27T03:16:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.964+0000 7efc2c466740 1 client.15354 shutdown 2024-07-27T03:16:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.964+0000 7efc2c466740 1 -- 172.21.15.67:0/3464732856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec3306f60 msgr2=0x558ec36bdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:22.973 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.964+0000 7efc2c466740 1 --2- 172.21.15.67:0/3464732856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec3306f60 0x558ec36bdaf0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efc1801ec20 tx=0x7efc18004ab0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.964+0000 7efc2c466740 1 -- 172.21.15.67:0/3464732856 shutdown_connections 2024-07-27T03:16:22.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.964+0000 7efc2c466740 1 --2- 172.21.15.67:0/3464732856 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x558ec32c58e0 0x558ec36ca300 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.964+0000 7efc2c466740 1 --2- 172.21.15.67:0/3464732856 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec3306f60 0x558ec36bdaf0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.964+0000 7efc2c466740 1 --2- 172.21.15.67:0/3464732856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ec32e6280 0x558ec36b84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.964+0000 7efc2c466740 1 --2- 172.21.15.67:0/3464732856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ec32de0a0 0x558ec36b7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.964+0000 7efbfeffd640 0 client.15354 ms_handle_reset on v2:172.21.15.149:6834/3730964806 2024-07-27T03:16:22.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.964+0000 7efc2c466740 1 -- 172.21.15.67:0/3464732856 >> 172.21.15.67:0/3464732856 conn(0x558ec32fb350 msgr2=0x558ec36af390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:22.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.965+0000 7efc2c466740 1 -- 172.21.15.67:0/3464732856 shutdown_connections 2024-07-27T03:16:22.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.965+0000 7efc2c466740 1 -- 172.21.15.67:0/3464732856 wait complete. 2024-07-27T03:16:22.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.965+0000 7efc2c466740 20 client.15354 trim_cache size 0 max 16384 2024-07-27T03:16:22.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.983+0000 7efc2c466740 1 Processor -- start 2024-07-27T03:16:22.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.983+0000 7efc2c466740 1 -- start start 2024-07-27T03:16:22.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.983+0000 7efc2c466740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ec32de0a0 0x558ec36bda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:22.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.983+0000 7efc2c466740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec32e6280 0x558ec36c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:22.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.983+0000 7efc2c466740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ec3306f60 0x558ec36b1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:22.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.983+0000 7efc2c466740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558ec36c8640 con 0x558ec32e6280 2024-07-27T03:16:22.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.983+0000 7efc2c466740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558ec36c8d10 con 0x558ec3306f60 2024-07-27T03:16:22.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.983+0000 7efc2c466740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558ec36c93e0 con 0x558ec32de0a0 2024-07-27T03:16:22.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.984+0000 7efc1effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec32e6280 0x558ec36c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:22.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.984+0000 7efc1effd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec32e6280 0x558ec36c48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39104/0 (socket says 172.21.15.67:39104) 2024-07-27T03:16:22.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.984+0000 7efc1effd640 1 -- 172.21.15.67:0/3577577028 learned_addr learned my addr 172.21.15.67:0/3577577028 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:22.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.984+0000 7efc1ffff640 1 --2- 172.21.15.67:0/3577577028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ec3306f60 0x558ec36b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:22.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.984+0000 7efc1f7fe640 1 --2- 172.21.15.67:0/3577577028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ec32de0a0 0x558ec36bda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:22.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.984+0000 7efc1effd640 1 -- 172.21.15.67:0/3577577028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ec32de0a0 msgr2=0x558ec36bda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:22.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.984+0000 7efc1effd640 1 --2- 172.21.15.67:0/3577577028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ec32de0a0 0x558ec36bda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.984+0000 7efc1effd640 1 -- 172.21.15.67:0/3577577028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ec3306f60 msgr2=0x558ec36b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:22.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.984+0000 7efc1effd640 1 --2- 172.21.15.67:0/3577577028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ec3306f60 0x558ec36b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.984+0000 7efc1effd640 1 -- 172.21.15.67:0/3577577028 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ec3372540 con 0x558ec32e6280 2024-07-27T03:16:22.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.984+0000 7efc1effd640 1 --2- 172.21.15.67:0/3577577028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec32e6280 0x558ec36c48c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7efc140a6640 tx=0x7efc1400a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dbd2809046557b server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:22.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.984+0000 7efc057fa640 1 -- 172.21.15.67:0/3577577028 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc140aacc0 con 0x558ec32e6280 2024-07-27T03:16:22.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.984+0000 7efc057fa640 1 -- 172.21.15.67:0/3577577028 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc140aa5c0 con 0x558ec32e6280 2024-07-27T03:16:22.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.984+0000 7efc057fa640 1 -- 172.21.15.67:0/3577577028 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc140a5f30 con 0x558ec32e6280 2024-07-27T03:16:22.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.985+0000 7efc2c466740 1 -- 172.21.15.67:0/3577577028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec32e6280 msgr2=0x558ec36c48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:22.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.985+0000 7efc2c466740 1 --2- 172.21.15.67:0/3577577028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec32e6280 0x558ec36c48c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7efc140a6640 tx=0x7efc1400a560 comp rx=0 tx=0).stop 2024-07-27T03:16:22.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.985+0000 7efc2c466740 1 -- 172.21.15.67:0/3577577028 shutdown_connections 2024-07-27T03:16:22.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.985+0000 7efc2c466740 1 --2- 172.21.15.67:0/3577577028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ec3306f60 0x558ec36b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.985+0000 7efc2c466740 1 --2- 172.21.15.67:0/3577577028 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec32e6280 0x558ec36c48c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.985+0000 7efc2c466740 1 --2- 172.21.15.67:0/3577577028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ec32de0a0 0x558ec36bda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.985+0000 7efc2c466740 1 -- 172.21.15.67:0/3577577028 >> 172.21.15.67:0/3577577028 conn(0x558ec32fb350 msgr2=0x558ec36b7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:22.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.985+0000 7efc2c466740 1 -- 172.21.15.67:0/3577577028 shutdown_connections 2024-07-27T03:16:22.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.985+0000 7efc2c466740 1 -- 172.21.15.67:0/3577577028 wait complete. 2024-07-27T03:16:22.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.986+0000 7efc2c466740 1 -- start start 2024-07-27T03:16:22.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.986+0000 7efc2c466740 1 Processor -- start 2024-07-27T03:16:22.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.986+0000 7efc2c466740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ec32de0a0 0x558ec36b1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:22.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.986+0000 7efc2c466740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ec32e6280 0x558ec36af300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:22.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.986+0000 7efc2c466740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec3306f60 0x558ec36af860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:22.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.987+0000 7efc2c466740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558ec36ca180 con 0x558ec3306f60 2024-07-27T03:16:22.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.987+0000 7efc2c466740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558ec36c9ab0 con 0x558ec32e6280 2024-07-27T03:16:22.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.987+0000 7efc2c466740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558ec36ca850 con 0x558ec32de0a0 2024-07-27T03:16:22.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.987+0000 7efc2c466740 10 client.0 _ll_register_callbacks cb 0x7efc18010f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:16:22.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.987+0000 7efc1ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec3306f60 0x558ec36af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:22.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.987+0000 7efc1ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec3306f60 0x558ec36af860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39120/0 (socket says 172.21.15.67:39120) 2024-07-27T03:16:22.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.987+0000 7efc1ffff640 1 -- 172.21.15.67:0/3962730557 learned_addr learned my addr 172.21.15.67:0/3962730557 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:22.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.987+0000 7efc1f7fe640 1 --2- 172.21.15.67:0/3962730557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ec32de0a0 0x558ec36b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:22.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.987+0000 7efc1effd640 1 --2- 172.21.15.67:0/3962730557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ec32e6280 0x558ec36af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:22.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.987+0000 7efc1ffff640 1 -- 172.21.15.67:0/3962730557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ec32de0a0 msgr2=0x558ec36b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:22.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.987+0000 7efc1ffff640 1 --2- 172.21.15.67:0/3962730557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ec32de0a0 0x558ec36b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.987+0000 7efc1ffff640 1 -- 172.21.15.67:0/3962730557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ec32e6280 msgr2=0x558ec36af300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:22.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.987+0000 7efc1ffff640 1 --2- 172.21.15.67:0/3962730557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ec32e6280 0x558ec36af300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.987+0000 7efc1ffff640 1 -- 172.21.15.67:0/3962730557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ec36afff0 con 0x558ec3306f60 2024-07-27T03:16:22.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.987+0000 7efc1f7fe640 1 --2- 172.21.15.67:0/3962730557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ec32de0a0 0x558ec36b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:16:22.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.987+0000 7efc1ffff640 1 --2- 172.21.15.67:0/3962730557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec3306f60 0x558ec36af860 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efc18017040 tx=0x7efc1801b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d2e1c7d0826600 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:22.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.987+0000 7efbfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:16:22.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.987+0000 7efbfeffd640 1 -- 172.21.15.67:0/3962730557 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1803a020 con 0x558ec3306f60 2024-07-27T03:16:22.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.987+0000 7efbfeffd640 1 -- 172.21.15.67:0/3962730557 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7efc18003970 con 0x558ec3306f60 2024-07-27T03:16:22.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.988+0000 7efc2c466740 1 -- 172.21.15.67:0/3962730557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc140aa5c0 con 0x558ec3306f60 2024-07-27T03:16:22.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.988+0000 7efbfeffd640 1 -- 172.21.15.67:0/3962730557 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efc18045050 con 0x558ec3306f60 2024-07-27T03:16:22.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.988+0000 7efc2c466740 10 client.15360 fetch_fsmap learned FSMap version 29 2024-07-27T03:16:22.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.988+0000 7efc2c466740 1 -- 172.21.15.67:0/3962730557 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558ec36c4820 con 0x558ec3306f60 2024-07-27T03:16:22.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.988+0000 7efbfeffd640 1 -- 172.21.15.67:0/3962730557 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc140aa5c0 con 0x558ec3306f60 2024-07-27T03:16:22.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.988+0000 7efbfeffd640 1 -- 172.21.15.67:0/3962730557 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7efc18004610 con 0x558ec3306f60 2024-07-27T03:16:22.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.988+0000 7efc2c466740 10 client.15360 fetch_fsmap finished waiting for FSMap version 29 2024-07-27T03:16:22.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.988+0000 7efc2c466740 10 client.15360 resolve_mds: resolved name 'e' to daemon mds.e 2024-07-27T03:16:22.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.988+0000 7efc2c466740 20 client.15360 populate_metadata read hostname 'smithi067' 2024-07-27T03:16:22.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.988+0000 7efc2c466740 1 --2- 172.21.15.67:0/3962730557 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x558ec329ac90 0x558ec3736870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:22.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.988+0000 7efc2c466740 4 client.15360 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-27T03:16:22.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.988+0000 7efc2c466740 1 -- 172.21.15.67:0/3962730557 --> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] -- command(tid 0: {"prefix": "status"}) -- 0x558ec3620280 con 0x558ec329ac90 2024-07-27T03:16:22.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.989+0000 7efc1f7fe640 1 --2- 172.21.15.67:0/3962730557 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x558ec329ac90 0x558ec3736870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:22.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.989+0000 7efc1f7fe640 1 --2- 172.21.15.67:0/3962730557 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x558ec329ac90 0x558ec3736870 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3d76b1fa0163229c server_cookie=d11d8e8d92810464 in_seq=0 out_seq=0 2024-07-27T03:16:22.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.989+0000 7efbfeffd640 10 client.15360 ms_handle_connect on v2:172.21.15.149:6834/3730964806 2024-07-27T03:16:22.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.989+0000 7efbfeffd640 1 -- 172.21.15.67:0/3962730557 <== mds.4 v2:172.21.15.149:6834/3730964806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558ec3620280 con 0x558ec329ac90 2024-07-27T03:16:22.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.989+0000 7efbfeffd640 10 client.15360 handle_command_reply: tid=0 2024-07-27T03:16:22.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.989+0000 7efc2c466740 1 client.15360 shutdown 2024-07-27T03:16:22.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.989+0000 7efc2c466740 1 -- 172.21.15.67:0/3962730557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec3306f60 msgr2=0x558ec36af860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:22.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.989+0000 7efc2c466740 1 --2- 172.21.15.67:0/3962730557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec3306f60 0x558ec36af860 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efc18017040 tx=0x7efc1801b6a0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.990+0000 7efc2c466740 1 -- 172.21.15.67:0/3962730557 shutdown_connections 2024-07-27T03:16:22.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.990+0000 7efc2c466740 1 --2- 172.21.15.67:0/3962730557 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x558ec329ac90 0x558ec3736870 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.990+0000 7efc2c466740 1 --2- 172.21.15.67:0/3962730557 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x558ec3306f60 0x558ec36af860 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.990+0000 7efc2c466740 1 --2- 172.21.15.67:0/3962730557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558ec32e6280 0x558ec36af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.990+0000 7efc2c466740 1 --2- 172.21.15.67:0/3962730557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x558ec32de0a0 0x558ec36b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:22.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.990+0000 7efc2c466740 1 -- 172.21.15.67:0/3962730557 >> 172.21.15.67:0/3962730557 conn(0x558ec32fb350 msgr2=0x558ec36b7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:22.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.990+0000 7efbfeffd640 0 client.15360 ms_handle_reset on v2:172.21.15.149:6834/3730964806 2024-07-27T03:16:22.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.990+0000 7efc2c466740 1 -- 172.21.15.67:0/3962730557 shutdown_connections 2024-07-27T03:16:22.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.990+0000 7efc2c466740 1 -- 172.21.15.67:0/3962730557 wait complete. 2024-07-27T03:16:22.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.990+0000 7efc2c466740 20 client.15360 trim_cache size 0 max 16384 2024-07-27T03:16:22.997 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:16:22.998 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "c0782a00-4bc4-11ef-bcc7-c7b262605968", 2024-07-27T03:16:22.998 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 4, 2024-07-27T03:16:22.998 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24563, 2024-07-27T03:16:22.998 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-27T03:16:22.998 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-27T03:16:22.998 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-27T03:16:22.998 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 163.75193246699999, 2024-07-27T03:16:22.998 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 29, 2024-07-27T03:16:22.998 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 91, 2024-07-27T03:16:22.998 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-27T03:16:22.998 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 262.581816635 2024-07-27T03:16:22.999 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:16:22.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.990+0000 7efc25ee1640 1 -- 172.21.15.67:0/1594275486 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7efc080b3e20 msgr2=0x7efc080b6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:22.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.990+0000 7efc25ee1640 1 --2- 172.21.15.67:0/1594275486 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7efc080b3e20 0x7efc080b6330 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efc18007f80 tx=0x7efc18007520 comp rx=0 tx=0).stop 2024-07-27T03:16:22.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.990+0000 7efc25ee1640 1 -- 172.21.15.67:0/1594275486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc20107e10 msgr2=0x7efc2019a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:23.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.990+0000 7efc25ee1640 1 --2- 172.21.15.67:0/1594275486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc20107e10 0x7efc2019a3e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7efc1400c970 tx=0x7efc1400ce30 comp rx=0 tx=0).stop 2024-07-27T03:16:23.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.990+0000 7efc25ee1640 1 -- 172.21.15.67:0/1594275486 shutdown_connections 2024-07-27T03:16:23.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.990+0000 7efc25ee1640 1 --2- 172.21.15.67:0/1594275486 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7efc080b3e20 0x7efc080b6330 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:23.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.990+0000 7efc25ee1640 1 --2- 172.21.15.67:0/1594275486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc20108760 0x7efc2019f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:23.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.990+0000 7efc25ee1640 1 --2- 172.21.15.67:0/1594275486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efc20107e10 0x7efc2019a3e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:23.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.990+0000 7efc25ee1640 1 --2- 172.21.15.67:0/1594275486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc201024d0 0x7efc20199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:23.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.990+0000 7efc25ee1640 1 -- 172.21.15.67:0/1594275486 >> 172.21.15.67:0/1594275486 conn(0x7efc200791a0 msgr2=0x7efc20100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:23.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.991+0000 7efc25ee1640 1 -- 172.21.15.67:0/1594275486 shutdown_connections 2024-07-27T03:16:23.000 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:22.991+0000 7efc25ee1640 1 -- 172.21.15.67:0/1594275486 wait complete. 2024-07-27T03:16:23.696 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.c status 2024-07-27T03:16:23.964 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:16:24.418 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:24 smithi067 ceph-mon[32814]: pgmap v425: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:24.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:24 smithi175 ceph-mon[37125]: pgmap v425: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:24.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:24 smithi149 ceph-mon[37173]: pgmap v425: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:25.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.610+0000 7f74d5f8a640 1 -- 172.21.15.67:0/2810564608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74d0107e30 msgr2=0x7f74d0108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:25.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.610+0000 7f74d5f8a640 1 --2- 172.21.15.67:0/2810564608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74d0107e30 0x7f74d0108240 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009900 tx=0x7f74b802a410 comp rx=0 tx=0).stop 2024-07-27T03:16:25.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.610+0000 7f74d5f8a640 1 -- 172.21.15.67:0/2810564608 shutdown_connections 2024-07-27T03:16:25.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.610+0000 7f74d5f8a640 1 --2- 172.21.15.67:0/2810564608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74d0108780 0x7f74d010efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.610+0000 7f74d5f8a640 1 --2- 172.21.15.67:0/2810564608 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74d0107e30 0x7f74d0108240 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.610+0000 7f74d5f8a640 1 --2- 172.21.15.67:0/2810564608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74d01024f0 0x7f74d0102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.610+0000 7f74d5f8a640 1 -- 172.21.15.67:0/2810564608 >> 172.21.15.67:0/2810564608 conn(0x7f74d00791b0 msgr2=0x7f74d00795c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:25.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.610+0000 7f74d5f8a640 1 -- 172.21.15.67:0/2810564608 shutdown_connections 2024-07-27T03:16:25.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.610+0000 7f74d5f8a640 1 -- 172.21.15.67:0/2810564608 wait complete. 2024-07-27T03:16:25.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.611+0000 7f74d5f8a640 1 Processor -- start 2024-07-27T03:16:25.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.611+0000 7f74d5f8a640 1 -- start start 2024-07-27T03:16:25.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.611+0000 7f74d5f8a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74d01024f0 0x7f74d0104b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:25.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.611+0000 7f74d5f8a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74d0107e30 0x7f74d01050b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:25.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.611+0000 7f74d5f8a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74d0108780 0x7f74d01055f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:25.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.611+0000 7f74d5f8a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01124b0 con 0x7f74d0108780 2024-07-27T03:16:25.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.611+0000 7f74d5f8a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f74d0112330 con 0x7f74d0107e30 2024-07-27T03:16:25.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.611+0000 7f74d5f8a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74d0112630 con 0x7f74d01024f0 2024-07-27T03:16:25.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.611+0000 7f74cffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74d0108780 0x7f74d01055f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:25.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.611+0000 7f74cffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74d0108780 0x7f74d01055f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36284/0 (socket says 172.21.15.67:36284) 2024-07-27T03:16:25.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.611+0000 7f74cffff640 1 -- 172.21.15.67:0/4103801085 learned_addr learned my addr 172.21.15.67:0/4103801085 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:25.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.612+0000 7f74c7fff640 1 --2- 172.21.15.67:0/4103801085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74d0107e30 0x7f74d01050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:25.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.612+0000 7f74cf7fe640 1 --2- 172.21.15.67:0/4103801085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74d01024f0 0x7f74d0104b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:25.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.612+0000 7f74cffff640 1 -- 172.21.15.67:0/4103801085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74d01024f0 msgr2=0x7f74d0104b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:25.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.612+0000 7f74cffff640 1 --2- 172.21.15.67:0/4103801085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74d01024f0 0x7f74d0104b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.612+0000 7f74cffff640 1 -- 172.21.15.67:0/4103801085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74d0107e30 msgr2=0x7f74d01050b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:25.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.612+0000 7f74cffff640 1 --2- 172.21.15.67:0/4103801085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74d0107e30 0x7f74d01050b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.628 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.612+0000 7f74cffff640 1 -- 172.21.15.67:0/4103801085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d01a8320 con 0x7f74d0108780 2024-07-27T03:16:25.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.612+0000 7f74cf7fe640 1 --2- 172.21.15.67:0/4103801085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74d01024f0 0x7f74d0104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:16:25.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.612+0000 7f74cffff640 1 --2- 172.21.15.67:0/4103801085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74d0108780 0x7f74d01055f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f74c800b970 tx=0x7f74c800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:25.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.612+0000 7f74cd7fa640 1 -- 172.21.15.67:0/4103801085 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8019070 con 0x7f74d0108780 2024-07-27T03:16:25.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.612+0000 7f74d5f8a640 1 -- 172.21.15.67:0/4103801085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d01a8610 con 0x7f74d0108780 2024-07-27T03:16:25.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.612+0000 7f74cd7fa640 1 -- 172.21.15.67:0/4103801085 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c8014440 con 0x7f74d0108780 2024-07-27T03:16:25.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.612+0000 7f74cd7fa640 1 -- 172.21.15.67:0/4103801085 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c80075d0 con 0x7f74d0108780 2024-07-27T03:16:25.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.612+0000 7f74d5f8a640 1 -- 172.21.15.67:0/4103801085 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d01a8ba0 con 0x7f74d0108780 2024-07-27T03:16:25.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.615+0000 7f74dc50f740 1 Processor -- start 2024-07-27T03:16:25.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.615+0000 7f74cd7fa640 1 -- 172.21.15.67:0/4103801085 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f74c801c070 con 0x7f74d0108780 2024-07-27T03:16:25.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.615+0000 7f74dc50f740 1 -- start start 2024-07-27T03:16:25.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.615+0000 7f74dc50f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b7cf60 0x55f7f9f22200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:25.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.615+0000 7f74dc50f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7f9b540a0 0x55f7f9f22740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:25.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.615+0000 7f74dc50f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7f9b5c280 0x55f7f9f22c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:25.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.615+0000 7f74dc50f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f7f9e0a2b0 con 0x55f7f9b7cf60 2024-07-27T03:16:25.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.615+0000 7f74dc50f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f7f9e98890 con 0x55f7f9b5c280 2024-07-27T03:16:25.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.615+0000 7f74dc50f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f7f9e87d90 con 0x55f7f9b540a0 2024-07-27T03:16:25.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.616+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b7cf60 0x55f7f9f22200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:25.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.616+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b7cf60 0x55f7f9f22200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36296/0 (socket says 172.21.15.67:36296) 2024-07-27T03:16:25.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.616+0000 7f74c7fff640 1 -- 172.21.15.67:0/665724315 learned_addr learned my addr 172.21.15.67:0/665724315 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:25.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.616+0000 7f74cd7fa640 1 --2- 172.21.15.67:0/4103801085 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f74bc0b3bd0 0x7f74bc0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:25.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.616+0000 7f74cf7fe640 1 --2- 172.21.15.67:0/665724315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7f9b540a0 0x55f7f9f22740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:25.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.616+0000 7f74c7fff640 1 --2- 172.21.15.67:0/665724315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7f9b5c280 0x55f7f9f22c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:25.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.616+0000 7f74c7fff640 1 -- 172.21.15.67:0/665724315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7f9b540a0 msgr2=0x55f7f9f22740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:25.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.616+0000 7f74c7fff640 1 --2- 172.21.15.67:0/665724315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7f9b540a0 0x55f7f9f22740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.616+0000 7f74c7fff640 1 -- 172.21.15.67:0/665724315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7f9b5c280 msgr2=0x55f7f9f22c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:25.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.616+0000 7f74c7fff640 1 --2- 172.21.15.67:0/665724315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7f9b5c280 0x55f7f9f22c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.616+0000 7f74cffff640 1 --2- 172.21.15.67:0/4103801085 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f74bc0b3bd0 0x7f74bc0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:25.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.616+0000 7f74c7fff640 1 -- 172.21.15.67:0/665724315 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7f9ec1af0 con 0x55f7f9b7cf60 2024-07-27T03:16:25.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.616+0000 7f74cd7fa640 1 -- 172.21.15.67:0/4103801085 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f74c80e35b0 con 0x7f74d0108780 2024-07-27T03:16:25.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.616+0000 7f74c7fff640 1 --2- 172.21.15.67:0/665724315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b7cf60 0x55f7f9f22200 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009900 tx=0x7f74b8031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba6e6db66637927 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:25.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.616+0000 7f74c5ffb640 1 -- 172.21.15.67:0/665724315 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8044030 con 0x55f7f9b7cf60 2024-07-27T03:16:25.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.617+0000 7f74c5ffb640 1 -- 172.21.15.67:0/665724315 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74b8031700 con 0x55f7f9b7cf60 2024-07-27T03:16:25.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.617+0000 7f74cffff640 1 --2- 172.21.15.67:0/4103801085 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f74bc0b3bd0 0x7f74bc0b60e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f74c80a9f00 tx=0x7f74c80745a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:25.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.617+0000 7f74c5ffb640 1 -- 172.21.15.67:0/665724315 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8035040 con 0x55f7f9b7cf60 2024-07-27T03:16:25.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.617+0000 7f74dc50f740 1 -- 172.21.15.67:0/665724315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b7cf60 msgr2=0x55f7f9f22200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:25.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.617+0000 7f74dc50f740 1 --2- 172.21.15.67:0/665724315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b7cf60 0x55f7f9f22200 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009900 tx=0x7f74b8031b10 comp rx=0 tx=0).stop 2024-07-27T03:16:25.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.617+0000 7f74dc50f740 1 -- 172.21.15.67:0/665724315 shutdown_connections 2024-07-27T03:16:25.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.617+0000 7f74dc50f740 1 --2- 172.21.15.67:0/665724315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7f9b5c280 0x55f7f9f22c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.617+0000 7f74dc50f740 1 --2- 172.21.15.67:0/665724315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7f9b540a0 0x55f7f9f22740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.617+0000 7f74dc50f740 1 --2- 172.21.15.67:0/665724315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b7cf60 0x55f7f9f22200 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.631 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.617+0000 7f74dc50f740 1 -- 172.21.15.67:0/665724315 >> 172.21.15.67:0/665724315 conn(0x55f7f9b71350 msgr2=0x55f7f9f200f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:25.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.617+0000 7f74dc50f740 1 -- 172.21.15.67:0/665724315 shutdown_connections 2024-07-27T03:16:25.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.617+0000 7f74dc50f740 1 -- 172.21.15.67:0/665724315 wait complete. 2024-07-27T03:16:25.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.618+0000 7f74dc50f740 1 -- start start 2024-07-27T03:16:25.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.618+0000 7f74dc50f740 1 Processor -- start 2024-07-27T03:16:25.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.618+0000 7f74dc50f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b540a0 0x55f7f9f2de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:25.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.618+0000 7f74dc50f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7f9b5c280 0x55f7f9f2e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:25.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.618+0000 7f74dc50f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7f9b7cf60 0x55f7f9f39af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:25.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.618+0000 7f74dc50f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f7f9e2a7b0 con 0x55f7f9b540a0 2024-07-27T03:16:25.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.618+0000 7f74dc50f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f7f9e8bdd0 con 0x55f7f9b5c280 2024-07-27T03:16:25.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.618+0000 7f74dc50f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f7f9ceb5f0 con 0x55f7f9b7cf60 2024-07-27T03:16:25.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.618+0000 7f74dc50f740 10 client.0 _ll_register_callbacks cb 0x55f7f9e327e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:16:25.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.619+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b540a0 0x55f7f9f2de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:25.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.619+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b540a0 0x55f7f9f2de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36302/0 (socket says 172.21.15.67:36302) 2024-07-27T03:16:25.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.619+0000 7f74c7fff640 1 -- 172.21.15.67:0/2661603131 learned_addr learned my addr 172.21.15.67:0/2661603131 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:25.632 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.619+0000 7f74cf7fe640 1 --2- 172.21.15.67:0/2661603131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7f9b5c280 0x55f7f9f2e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:25.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.619+0000 7f74c7fff640 1 -- 172.21.15.67:0/2661603131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7f9b7cf60 msgr2=0x55f7f9f39af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:25.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.619+0000 7f74c7fff640 1 --2- 172.21.15.67:0/2661603131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7f9b7cf60 0x55f7f9f39af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.619+0000 7f74c7fff640 1 -- 172.21.15.67:0/2661603131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7f9b5c280 msgr2=0x55f7f9f2e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:25.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.619+0000 7f74c7fff640 1 --2- 172.21.15.67:0/2661603131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7f9b5c280 0x55f7f9f2e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.619+0000 7f74c7fff640 1 -- 172.21.15.67:0/2661603131 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7f9e1d620 con 0x55f7f9b540a0 2024-07-27T03:16:25.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.619+0000 7f74c7fff640 1 --2- 172.21.15.67:0/2661603131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b540a0 0x55f7f9f2de40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f74b802abf0 tx=0x7f74b8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4d1c942febb410 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:25.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.619+0000 7f74aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:16:25.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.619+0000 7f74aeffd640 1 -- 172.21.15.67:0/2661603131 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b804d020 con 0x55f7f9b540a0 2024-07-27T03:16:25.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.619+0000 7f74aeffd640 1 -- 172.21.15.67:0/2661603131 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f74b8004bb0 con 0x55f7f9b540a0 2024-07-27T03:16:25.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.619+0000 7f74dc50f740 1 -- 172.21.15.67:0/2661603131 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74b8031700 con 0x55f7f9b540a0 2024-07-27T03:16:25.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.619+0000 7f74aeffd640 1 -- 172.21.15.67:0/2661603131 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f74b8058050 con 0x55f7f9b540a0 2024-07-27T03:16:25.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.619+0000 7f74dc50f740 10 client.15372 fetch_fsmap learned FSMap version 29 2024-07-27T03:16:25.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.619+0000 7f74dc50f740 1 -- 172.21.15.67:0/2661603131 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f7f9e2eaa0 con 0x55f7f9b540a0 2024-07-27T03:16:25.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.620+0000 7f74aeffd640 1 -- 172.21.15.67:0/2661603131 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74b8031700 con 0x55f7f9b540a0 2024-07-27T03:16:25.633 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.620+0000 7f74aeffd640 1 -- 172.21.15.67:0/2661603131 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f74b803a3f0 con 0x55f7f9b540a0 2024-07-27T03:16:25.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.620+0000 7f74dc50f740 10 client.15372 fetch_fsmap finished waiting for FSMap version 29 2024-07-27T03:16:25.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.620+0000 7f74dc50f740 10 client.15372 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-27T03:16:25.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.620+0000 7f74dc50f740 20 client.15372 populate_metadata read hostname 'smithi067' 2024-07-27T03:16:25.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.620+0000 7f74dc50f740 1 --2- 172.21.15.67:0/2661603131 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55f7f9b3b8e0 0x55f7f9f3fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:25.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.620+0000 7f74dc50f740 4 client.15372 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:16:25.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.620+0000 7f74dc50f740 1 -- 172.21.15.67:0/2661603131 --> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7f9e9a7e0 con 0x55f7f9b3b8e0 2024-07-27T03:16:25.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.620+0000 7f74cf7fe640 1 --2- 172.21.15.67:0/2661603131 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55f7f9b3b8e0 0x55f7f9f3fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:25.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.621+0000 7f74aeffd640 10 client.15372 ms_handle_connect on v2:172.21.15.175:6836/1304803502 2024-07-27T03:16:25.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.621+0000 7f74cf7fe640 1 --2- 172.21.15.67:0/2661603131 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55f7f9b3b8e0 0x55f7f9f3fc70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=63bcc1866b98efb server_cookie=b2a7d0a18447b7a1 in_seq=0 out_seq=0 2024-07-27T03:16:25.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.622+0000 7f74aeffd640 1 -- 172.21.15.67:0/2661603131 <== mds.3 v2:172.21.15.175:6836/1304803502 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7f9e9a7e0 con 0x55f7f9b3b8e0 2024-07-27T03:16:25.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.622+0000 7f74aeffd640 10 client.15372 handle_command_reply: tid=0 2024-07-27T03:16:25.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.622+0000 7f74dc50f740 1 client.15372 shutdown 2024-07-27T03:16:25.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.622+0000 7f74dc50f740 1 -- 172.21.15.67:0/2661603131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b540a0 msgr2=0x55f7f9f2de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:25.634 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.622+0000 7f74dc50f740 1 --2- 172.21.15.67:0/2661603131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b540a0 0x55f7f9f2de40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f74b802abf0 tx=0x7f74b8031400 comp rx=0 tx=0).stop 2024-07-27T03:16:25.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.622+0000 7f74dc50f740 1 -- 172.21.15.67:0/2661603131 shutdown_connections 2024-07-27T03:16:25.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.622+0000 7f74dc50f740 1 --2- 172.21.15.67:0/2661603131 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55f7f9b3b8e0 0x55f7f9f3fc70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.622+0000 7f74dc50f740 1 --2- 172.21.15.67:0/2661603131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7f9b7cf60 0x55f7f9f39af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.622+0000 7f74dc50f740 1 --2- 172.21.15.67:0/2661603131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7f9b5c280 0x55f7f9f2e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.622+0000 7f74dc50f740 1 --2- 172.21.15.67:0/2661603131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b540a0 0x55f7f9f2de40 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.622+0000 7f74aeffd640 0 client.15372 ms_handle_reset on v2:172.21.15.175:6836/1304803502 2024-07-27T03:16:25.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.622+0000 7f74dc50f740 1 -- 172.21.15.67:0/2661603131 >> 172.21.15.67:0/2661603131 conn(0x55f7f9b71350 msgr2=0x55f7f9f253e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:25.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.622+0000 7f74dc50f740 1 -- 172.21.15.67:0/2661603131 shutdown_connections 2024-07-27T03:16:25.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.622+0000 7f74dc50f740 1 -- 172.21.15.67:0/2661603131 wait complete. 2024-07-27T03:16:25.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.622+0000 7f74dc50f740 20 client.15372 trim_cache size 0 max 16384 2024-07-27T03:16:25.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.640+0000 7f74dc50f740 1 Processor -- start 2024-07-27T03:16:25.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.640+0000 7f74dc50f740 1 -- start start 2024-07-27T03:16:25.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.640+0000 7f74dc50f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b540a0 0x55f7f9f3f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.640+0000 7f74dc50f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7f9b5c280 0x55f7f9f27a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.640+0000 7f74dc50f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7f9b7cf60 0x55f7f9f39a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.640+0000 7f74dc50f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f7f9f3dfb0 con 0x55f7f9b540a0 2024-07-27T03:16:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.640+0000 7f74dc50f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f7f9f3e680 con 0x55f7f9b5c280 2024-07-27T03:16:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.640+0000 7f74dc50f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f7f9f3ed50 con 0x55f7f9b7cf60 2024-07-27T03:16:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.641+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b540a0 0x55f7f9f3f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.641+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b540a0 0x55f7f9f3f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36312/0 (socket says 172.21.15.67:36312) 2024-07-27T03:16:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.641+0000 7f74c7fff640 1 -- 172.21.15.67:0/3862272983 learned_addr learned my addr 172.21.15.67:0/3862272983 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.641+0000 7f74cf7fe640 1 --2- 172.21.15.67:0/3862272983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7f9b5c280 0x55f7f9f27a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.641+0000 7f74c7fff640 1 --2- 172.21.15.67:0/3862272983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7f9b7cf60 0x55f7f9f39a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.641+0000 7f74c7fff640 1 -- 172.21.15.67:0/3862272983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7f9b7cf60 msgr2=0x55f7f9f39a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.641+0000 7f74c7fff640 1 --2- 172.21.15.67:0/3862272983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7f9b7cf60 0x55f7f9f39a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.641+0000 7f74c7fff640 1 -- 172.21.15.67:0/3862272983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7f9b5c280 msgr2=0x55f7f9f27a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.641+0000 7f74c7fff640 1 --2- 172.21.15.67:0/3862272983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7f9b5c280 0x55f7f9f27a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.641+0000 7f74c7fff640 1 -- 172.21.15.67:0/3862272983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7f9be8540 con 0x55f7f9b540a0 2024-07-27T03:16:25.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.641+0000 7f74c7fff640 1 --2- 172.21.15.67:0/3862272983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b540a0 0x55f7f9f3f5a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f74b8005d00 tx=0x7f74b8005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f640e2266287162 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.641+0000 7f74c57fa640 1 -- 172.21.15.67:0/3862272983 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8061070 con 0x55f7f9b540a0 2024-07-27T03:16:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.641+0000 7f74c57fa640 1 -- 172.21.15.67:0/3862272983 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74b80054d0 con 0x55f7f9b540a0 2024-07-27T03:16:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.641+0000 7f74c57fa640 1 -- 172.21.15.67:0/3862272983 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8044040 con 0x55f7f9b540a0 2024-07-27T03:16:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.642+0000 7f74dc50f740 1 -- 172.21.15.67:0/3862272983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b540a0 msgr2=0x55f7f9f3f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.642+0000 7f74dc50f740 1 --2- 172.21.15.67:0/3862272983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b540a0 0x55f7f9f3f5a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f74b8005d00 tx=0x7f74b8005c90 comp rx=0 tx=0).stop 2024-07-27T03:16:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.642+0000 7f74dc50f740 1 -- 172.21.15.67:0/3862272983 shutdown_connections 2024-07-27T03:16:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.642+0000 7f74dc50f740 1 --2- 172.21.15.67:0/3862272983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7f9b7cf60 0x55f7f9f39a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.642+0000 7f74dc50f740 1 --2- 172.21.15.67:0/3862272983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7f9b5c280 0x55f7f9f27a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.642+0000 7f74dc50f740 1 --2- 172.21.15.67:0/3862272983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b540a0 0x55f7f9f3f5a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.642+0000 7f74dc50f740 1 -- 172.21.15.67:0/3862272983 >> 172.21.15.67:0/3862272983 conn(0x55f7f9b71350 msgr2=0x55f7f9f27310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.642+0000 7f74dc50f740 1 -- 172.21.15.67:0/3862272983 shutdown_connections 2024-07-27T03:16:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.642+0000 7f74dc50f740 1 -- 172.21.15.67:0/3862272983 wait complete. 2024-07-27T03:16:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.643+0000 7f74dc50f740 1 -- start start 2024-07-27T03:16:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.643+0000 7f74dc50f740 1 Processor -- start 2024-07-27T03:16:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.643+0000 7f74dc50f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b540a0 0x55f7f9f38c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.643+0000 7f74dc50f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7f9b5c280 0x55f7f9f39170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.643+0000 7f74dc50f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7f9b7cf60 0x55f7f9fa6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.643+0000 7f74dc50f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55f7f9f3faf0 con 0x55f7f9b540a0 2024-07-27T03:16:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.643+0000 7f74dc50f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f7f9f3f420 con 0x55f7f9b7cf60 2024-07-27T03:16:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.643+0000 7f74dc50f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55f7f9f401c0 con 0x55f7f9b5c280 2024-07-27T03:16:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.643+0000 7f74dc50f740 10 client.0 _ll_register_callbacks cb 0x7f74b8031530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:16:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.644+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b540a0 0x55f7f9f38c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.644+0000 7f74c7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b540a0 0x55f7f9f38c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36318/0 (socket says 172.21.15.67:36318) 2024-07-27T03:16:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.644+0000 7f74c7fff640 1 -- 172.21.15.67:0/1508533827 learned_addr learned my addr 172.21.15.67:0/1508533827 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.644+0000 7f74cf7fe640 1 --2- 172.21.15.67:0/1508533827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7f9b5c280 0x55f7f9f39170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:25.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.644+0000 7f74c7fff640 1 --2- 172.21.15.67:0/1508533827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7f9b7cf60 0x55f7f9fa6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:25.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.644+0000 7f74c7fff640 1 -- 172.21.15.67:0/1508533827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7f9b5c280 msgr2=0x55f7f9f39170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:25.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.644+0000 7f74c7fff640 1 --2- 172.21.15.67:0/1508533827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7f9b5c280 0x55f7f9f39170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.644+0000 7f74c7fff640 1 -- 172.21.15.67:0/1508533827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7f9b7cf60 msgr2=0x55f7f9fa6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:25.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.644+0000 7f74c7fff640 1 --2- 172.21.15.67:0/1508533827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7f9b7cf60 0x55f7f9fa6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.644+0000 7f74c7fff640 1 -- 172.21.15.67:0/1508533827 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7f9f399f0 con 0x55f7f9b540a0 2024-07-27T03:16:25.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.644+0000 7f74c7fff640 1 --2- 172.21.15.67:0/1508533827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b540a0 0x55f7f9f38c30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f74b803a930 tx=0x7f74b8005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6ab38146b4eec6 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:25.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.644+0000 7f74aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:16:25.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.644+0000 7f74aeffd640 1 -- 172.21.15.67:0/1508533827 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8044070 con 0x55f7f9b540a0 2024-07-27T03:16:25.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.644+0000 7f74aeffd640 1 -- 172.21.15.67:0/1508533827 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f74b80049b0 con 0x55f7f9b540a0 2024-07-27T03:16:25.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.644+0000 7f74dc50f740 1 -- 172.21.15.67:0/1508533827 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74b80054d0 con 0x55f7f9b540a0 2024-07-27T03:16:25.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.645+0000 7f74aeffd640 1 -- 172.21.15.67:0/1508533827 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f74b8061040 con 0x55f7f9b540a0 2024-07-27T03:16:25.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.645+0000 7f74dc50f740 10 client.15378 fetch_fsmap learned FSMap version 29 2024-07-27T03:16:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.645+0000 7f74dc50f740 1 -- 172.21.15.67:0/1508533827 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f7f9fa8480 con 0x55f7f9b540a0 2024-07-27T03:16:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.645+0000 7f74aeffd640 1 -- 172.21.15.67:0/1508533827 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74b80054d0 con 0x55f7f9b540a0 2024-07-27T03:16:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.645+0000 7f74aeffd640 1 -- 172.21.15.67:0/1508533827 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f74b8049470 con 0x55f7f9b540a0 2024-07-27T03:16:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.645+0000 7f74dc50f740 10 client.15378 fetch_fsmap finished waiting for FSMap version 29 2024-07-27T03:16:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.645+0000 7f74dc50f740 10 client.15378 resolve_mds: resolved name 'c' to daemon mds.c 2024-07-27T03:16:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.645+0000 7f74dc50f740 20 client.15378 populate_metadata read hostname 'smithi067' 2024-07-27T03:16:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.645+0000 7f74dc50f740 1 --2- 172.21.15.67:0/1508533827 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55f7f9b10c90 0x55f7f9fab960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.645+0000 7f74dc50f740 4 client.15378 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-27T03:16:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.645+0000 7f74dc50f740 1 -- 172.21.15.67:0/1508533827 --> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] -- command(tid 0: {"prefix": "status"}) -- 0x55f7f9e96280 con 0x55f7f9b10c90 2024-07-27T03:16:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.645+0000 7f74cf7fe640 1 --2- 172.21.15.67:0/1508533827 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55f7f9b10c90 0x55f7f9fab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.646+0000 7f74cf7fe640 1 --2- 172.21.15.67:0/1508533827 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55f7f9b10c90 0x55f7f9fab960 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2c21e3c299590e4a server_cookie=6dbbc7efcf99542e in_seq=0 out_seq=0 2024-07-27T03:16:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.646+0000 7f74aeffd640 10 client.15378 ms_handle_connect on v2:172.21.15.175:6836/1304803502 2024-07-27T03:16:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.646+0000 7f74aeffd640 1 -- 172.21.15.67:0/1508533827 <== mds.3 v2:172.21.15.175:6836/1304803502 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f7f9e96280 con 0x55f7f9b10c90 2024-07-27T03:16:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.646+0000 7f74aeffd640 10 client.15378 handle_command_reply: tid=0 2024-07-27T03:16:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.646+0000 7f74dc50f740 1 client.15378 shutdown 2024-07-27T03:16:25.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.646+0000 7f74dc50f740 1 -- 172.21.15.67:0/1508533827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b540a0 msgr2=0x55f7f9f38c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.646+0000 7f74dc50f740 1 --2- 172.21.15.67:0/1508533827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b540a0 0x55f7f9f38c30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f74b803a930 tx=0x7f74b8005dd0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.646+0000 7f74dc50f740 1 -- 172.21.15.67:0/1508533827 shutdown_connections 2024-07-27T03:16:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.646+0000 7f74dc50f740 1 --2- 172.21.15.67:0/1508533827 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55f7f9b10c90 0x55f7f9fab960 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.646+0000 7f74dc50f740 1 --2- 172.21.15.67:0/1508533827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7f9b7cf60 0x55f7f9fa6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.646+0000 7f74dc50f740 1 --2- 172.21.15.67:0/1508533827 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55f7f9b5c280 0x55f7f9f39170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.646+0000 7f74dc50f740 1 --2- 172.21.15.67:0/1508533827 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55f7f9b540a0 0x55f7f9f38c30 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.646+0000 7f74aeffd640 0 client.15378 ms_handle_reset on v2:172.21.15.175:6836/1304803502 2024-07-27T03:16:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.646+0000 7f74dc50f740 1 -- 172.21.15.67:0/1508533827 >> 172.21.15.67:0/1508533827 conn(0x55f7f9b71350 msgr2=0x55f7f9f1fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.647+0000 7f74dc50f740 1 -- 172.21.15.67:0/1508533827 shutdown_connections 2024-07-27T03:16:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.647+0000 7f74dc50f740 1 -- 172.21.15.67:0/1508533827 wait complete. 2024-07-27T03:16:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.647+0000 7f74dc50f740 20 client.15378 trim_cache size 0 max 16384 2024-07-27T03:16:25.653 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:16:25.654 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "c0782a00-4bc4-11ef-bcc7-c7b262605968", 2024-07-27T03:16:25.654 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 3, 2024-07-27T03:16:25.654 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24581, 2024-07-27T03:16:25.654 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-27T03:16:25.654 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-27T03:16:25.654 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-27T03:16:25.654 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 167.425438082, 2024-07-27T03:16:25.654 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 29, 2024-07-27T03:16:25.654 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 91, 2024-07-27T03:16:25.654 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-27T03:16:25.654 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 260.09800122299998 2024-07-27T03:16:25.654 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:16:25.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.647+0000 7f74d5f8a640 1 -- 172.21.15.67:0/4103801085 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f74bc0b3bd0 msgr2=0x7f74bc0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:25.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.647+0000 7f74d5f8a640 1 --2- 172.21.15.67:0/4103801085 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f74bc0b3bd0 0x7f74bc0b60e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f74c80a9f00 tx=0x7f74c80745a0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.647+0000 7f74d5f8a640 1 -- 172.21.15.67:0/4103801085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74d0108780 msgr2=0x7f74d01055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:25.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.647+0000 7f74d5f8a640 1 --2- 172.21.15.67:0/4103801085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74d0108780 0x7f74d01055f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f74c800b970 tx=0x7f74c800be30 comp rx=0 tx=0).stop 2024-07-27T03:16:25.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.647+0000 7f74d5f8a640 1 -- 172.21.15.67:0/4103801085 shutdown_connections 2024-07-27T03:16:25.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.647+0000 7f74d5f8a640 1 --2- 172.21.15.67:0/4103801085 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f74bc0b3bd0 0x7f74bc0b60e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.647+0000 7f74d5f8a640 1 --2- 172.21.15.67:0/4103801085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f74d0108780 0x7f74d01055f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.647+0000 7f74d5f8a640 1 --2- 172.21.15.67:0/4103801085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74d0107e30 0x7f74d01050b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.647+0000 7f74d5f8a640 1 --2- 172.21.15.67:0/4103801085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74d01024f0 0x7f74d0104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:25.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.647+0000 7f74d5f8a640 1 -- 172.21.15.67:0/4103801085 >> 172.21.15.67:0/4103801085 conn(0x7f74d00791b0 msgr2=0x7f74d0100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:25.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.647+0000 7f74d5f8a640 1 -- 172.21.15.67:0/4103801085 shutdown_connections 2024-07-27T03:16:25.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:25.648+0000 7f74d5f8a640 1 -- 172.21.15.67:0/4103801085 wait complete. 2024-07-27T03:16:26.362 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.i status 2024-07-27T03:16:26.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:26 smithi175 ceph-mon[37125]: pgmap v426: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:16:26.633 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:16:26.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:26 smithi067 ceph-mon[32814]: pgmap v426: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:16:26.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:26 smithi149 ceph-mon[37173]: pgmap v426: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:16:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.301+0000 7f01cbaf1640 1 -- 172.21.15.67:0/1284970988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c41035d0 msgr2=0x7f01c41039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.301+0000 7f01cbaf1640 1 --2- 172.21.15.67:0/1284970988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c41035d0 0x7f01c41039e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f01c0009900 tx=0x7f01c002a410 comp rx=0 tx=0).stop 2024-07-27T03:16:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.301+0000 7f01cbaf1640 1 -- 172.21.15.67:0/1284970988 shutdown_connections 2024-07-27T03:16:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.301+0000 7f01cbaf1640 1 --2- 172.21.15.67:0/1284970988 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01c4109860 0x7f01c4110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.301+0000 7f01cbaf1640 1 --2- 172.21.15.67:0/1284970988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c4108f10 0x7f01c4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.301+0000 7f01cbaf1640 1 --2- 172.21.15.67:0/1284970988 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c41035d0 0x7f01c41039e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.301+0000 7f01cbaf1640 1 -- 172.21.15.67:0/1284970988 >> 172.21.15.67:0/1284970988 conn(0x7f01c407b2c0 msgr2=0x7f01c407b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.301+0000 7f01cbaf1640 1 -- 172.21.15.67:0/1284970988 shutdown_connections 2024-07-27T03:16:28.311 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.301+0000 7f01cbaf1640 1 -- 172.21.15.67:0/1284970988 wait complete. 2024-07-27T03:16:28.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.301+0000 7f01cbaf1640 1 Processor -- start 2024-07-27T03:16:28.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.302+0000 7f01cbaf1640 1 -- start start 2024-07-27T03:16:28.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.302+0000 7f01cbaf1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c41035d0 0x7f01c4199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:28.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.302+0000 7f01cbaf1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01c4108f10 0x7f01c419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:28.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.302+0000 7f01cbaf1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c4109860 0x7f01c419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:28.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.302+0000 7f01cbaf1640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f01c4113580 con 0x7f01c4109860 2024-07-27T03:16:28.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.302+0000 7f01cbaf1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f01c4113400 con 0x7f01c4108f10 2024-07-27T03:16:28.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.302+0000 7f01cbaf1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f01c4113700 con 0x7f01c41035d0 2024-07-27T03:16:28.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.302+0000 7f01ca08e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c4109860 0x7f01c419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:28.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.302+0000 7f01c988d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c41035d0 0x7f01c4199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:28.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.302+0000 7f01ca08e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c4109860 0x7f01c419f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36330/0 (socket says 172.21.15.67:36330) 2024-07-27T03:16:28.312 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.302+0000 7f01ca08e640 1 -- 172.21.15.67:0/1906534983 learned_addr learned my addr 172.21.15.67:0/1906534983 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:28.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.302+0000 7f01c908c640 1 --2- 172.21.15.67:0/1906534983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01c4108f10 0x7f01c419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:28.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.302+0000 7f01ca08e640 1 -- 172.21.15.67:0/1906534983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c41035d0 msgr2=0x7f01c4199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:28.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.302+0000 7f01ca08e640 1 --2- 172.21.15.67:0/1906534983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c41035d0 0x7f01c4199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.302+0000 7f01ca08e640 1 -- 172.21.15.67:0/1906534983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01c4108f10 msgr2=0x7f01c419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:28.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.302+0000 7f01ca08e640 1 --2- 172.21.15.67:0/1906534983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01c4108f10 0x7f01c419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.302+0000 7f01ca08e640 1 -- 172.21.15.67:0/1906534983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c419ff00 con 0x7f01c4109860 2024-07-27T03:16:28.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.303+0000 7f01ca08e640 1 --2- 172.21.15.67:0/1906534983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c4109860 0x7f01c419f7d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00b970 tx=0x7f01bc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:28.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.303+0000 7f01baffd640 1 -- 172.21.15.67:0/1906534983 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc019070 con 0x7f01c4109860 2024-07-27T03:16:28.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.303+0000 7f01baffd640 1 -- 172.21.15.67:0/1906534983 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01bc014440 con 0x7f01c4109860 2024-07-27T03:16:28.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.303+0000 7f01cbaf1640 1 -- 172.21.15.67:0/1906534983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c41a01f0 con 0x7f01c4109860 2024-07-27T03:16:28.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.303+0000 7f01baffd640 1 -- 172.21.15.67:0/1906534983 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc0075d0 con 0x7f01c4109860 2024-07-27T03:16:28.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.303+0000 7f01cbaf1640 1 -- 172.21.15.67:0/1906534983 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c41a6950 con 0x7f01c4109860 2024-07-27T03:16:28.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.305+0000 7f01d2076740 1 Processor -- start 2024-07-27T03:16:28.313 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.305+0000 7f01baffd640 1 -- 172.21.15.67:0/1906534983 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f01bc01c070 con 0x7f01c4109860 2024-07-27T03:16:28.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.306+0000 7f01d2076740 1 -- start start 2024-07-27T03:16:28.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.306+0000 7f01d2076740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dfac6df60 0x556dfb013220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:28.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.306+0000 7f01d2076740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556dfac450a0 0x556dfb013760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:28.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.306+0000 7f01d2076740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac4d280 0x556dfb013ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:28.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.306+0000 7f01d2076740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556dfaefb2d0 con 0x556dfac4d280 2024-07-27T03:16:28.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.306+0000 7f01d2076740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556dfaf898b0 con 0x556dfac6df60 2024-07-27T03:16:28.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.306+0000 7f01d2076740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556dfaf78db0 con 0x556dfac450a0 2024-07-27T03:16:28.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.306+0000 7f01c908c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac4d280 0x556dfb013ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:28.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.306+0000 7f01c908c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac4d280 0x556dfb013ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36332/0 (socket says 172.21.15.67:36332) 2024-07-27T03:16:28.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.306+0000 7f01c908c640 1 -- 172.21.15.67:0/740660387 learned_addr learned my addr 172.21.15.67:0/740660387 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:28.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.306+0000 7f01c988d640 1 --2- 172.21.15.67:0/740660387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556dfac450a0 0x556dfb013760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:28.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.306+0000 7f01c908c640 1 --2- 172.21.15.67:0/740660387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dfac6df60 0x556dfb013220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:28.314 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.307+0000 7f01baffd640 1 --2- 172.21.15.67:0/1906534983 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f01b00b3c50 0x7f01b00b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:28.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.307+0000 7f01c908c640 1 -- 172.21.15.67:0/740660387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556dfac450a0 msgr2=0x556dfb013760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:28.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.307+0000 7f01c908c640 1 --2- 172.21.15.67:0/740660387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556dfac450a0 0x556dfb013760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.307+0000 7f01c908c640 1 -- 172.21.15.67:0/740660387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dfac6df60 msgr2=0x556dfb013220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:28.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.307+0000 7f01c908c640 1 --2- 172.21.15.67:0/740660387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dfac6df60 0x556dfb013220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.307+0000 7f01c908c640 1 -- 172.21.15.67:0/740660387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dfafb2b10 con 0x556dfac4d280 2024-07-27T03:16:28.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.307+0000 7f01c988d640 1 --2- 172.21.15.67:0/740660387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556dfac450a0 0x556dfb013760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:16:28.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.307+0000 7f01ca08e640 1 --2- 172.21.15.67:0/1906534983 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f01b00b3c50 0x7f01b00b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:28.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.307+0000 7f01baffd640 1 -- 172.21.15.67:0/1906534983 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f01bc0e3520 con 0x7f01c4109860 2024-07-27T03:16:28.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.307+0000 7f01c908c640 1 --2- 172.21.15.67:0/740660387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac4d280 0x556dfb013ca0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f01b4014960 tx=0x7f01b400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8211170257f6cad server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:28.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.307+0000 7f01affff640 1 -- 172.21.15.67:0/740660387 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b40142d0 con 0x556dfac4d280 2024-07-27T03:16:28.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.307+0000 7f01affff640 1 -- 172.21.15.67:0/740660387 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01b400fc10 con 0x556dfac4d280 2024-07-27T03:16:28.315 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.307+0000 7f01affff640 1 -- 172.21.15.67:0/740660387 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4012980 con 0x556dfac4d280 2024-07-27T03:16:28.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.309+0000 7f01d2076740 1 -- 172.21.15.67:0/740660387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac4d280 msgr2=0x556dfb013ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:28.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.309+0000 7f01ca08e640 1 --2- 172.21.15.67:0/1906534983 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f01b00b3c50 0x7f01b00b6160 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f01bc0a73e0 tx=0x7f01bc0d9da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:28.319 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.309+0000 7f01d2076740 1 --2- 172.21.15.67:0/740660387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac4d280 0x556dfb013ca0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f01b4014960 tx=0x7f01b400c490 comp rx=0 tx=0).stop 2024-07-27T03:16:28.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.309+0000 7f01d2076740 1 -- 172.21.15.67:0/740660387 shutdown_connections 2024-07-27T03:16:28.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.309+0000 7f01d2076740 1 --2- 172.21.15.67:0/740660387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac4d280 0x556dfb013ca0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.309+0000 7f01d2076740 1 --2- 172.21.15.67:0/740660387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556dfac450a0 0x556dfb013760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.309+0000 7f01d2076740 1 --2- 172.21.15.67:0/740660387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dfac6df60 0x556dfb013220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.309+0000 7f01d2076740 1 -- 172.21.15.67:0/740660387 >> 172.21.15.67:0/740660387 conn(0x556dfac62350 msgr2=0x556dfb011110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:28.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.309+0000 7f01d2076740 1 -- 172.21.15.67:0/740660387 shutdown_connections 2024-07-27T03:16:28.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.309+0000 7f01d2076740 1 -- 172.21.15.67:0/740660387 wait complete. 2024-07-27T03:16:28.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.310+0000 7f01d2076740 1 -- start start 2024-07-27T03:16:28.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.310+0000 7f01d2076740 1 Processor -- start 2024-07-27T03:16:28.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.311+0000 7f01d2076740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dfac450a0 0x556dfb01ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:28.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.311+0000 7f01d2076740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556dfac4d280 0x556dfb01f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:28.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.311+0000 7f01d2076740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac6df60 0x556dfb024b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:28.320 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.311+0000 7f01d2076740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556dfaf1b7d0 con 0x556dfac6df60 2024-07-27T03:16:28.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.311+0000 7f01d2076740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556dfaf7cdf0 con 0x556dfac450a0 2024-07-27T03:16:28.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.311+0000 7f01d2076740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556dfaddc5f0 con 0x556dfac4d280 2024-07-27T03:16:28.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.311+0000 7f01d2076740 10 client.0 _ll_register_callbacks cb 0x556dfaf23800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:16:28.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.311+0000 7f01c908c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac6df60 0x556dfb024b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:28.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.311+0000 7f01c908c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac6df60 0x556dfb024b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36344/0 (socket says 172.21.15.67:36344) 2024-07-27T03:16:28.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.311+0000 7f01c908c640 1 -- 172.21.15.67:0/1670223651 learned_addr learned my addr 172.21.15.67:0/1670223651 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:28.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.311+0000 7f01c908c640 1 --2- 172.21.15.67:0/1670223651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dfac450a0 0x556dfb01ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:28.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.311+0000 7f01c988d640 1 --2- 172.21.15.67:0/1670223651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556dfac4d280 0x556dfb01f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:28.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.311+0000 7f01c908c640 1 -- 172.21.15.67:0/1670223651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556dfac4d280 msgr2=0x556dfb01f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:28.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.311+0000 7f01c908c640 1 --2- 172.21.15.67:0/1670223651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556dfac4d280 0x556dfb01f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.311+0000 7f01c908c640 1 -- 172.21.15.67:0/1670223651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dfac450a0 msgr2=0x556dfb01ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:28.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.311+0000 7f01c908c640 1 --2- 172.21.15.67:0/1670223651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dfac450a0 0x556dfb01ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.311+0000 7f01c908c640 1 -- 172.21.15.67:0/1670223651 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dfaf0e640 con 0x556dfac6df60 2024-07-27T03:16:28.321 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.312+0000 7f01c908c640 1 --2- 172.21.15.67:0/1670223651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac6df60 0x556dfb024b40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f01b4012660 tx=0x7f01b4012560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226bde02a5ca1ad4 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:28.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.312+0000 7f01ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:16:28.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.312+0000 7f01ad7fa640 1 -- 172.21.15.67:0/1670223651 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b40140d0 con 0x556dfac6df60 2024-07-27T03:16:28.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.312+0000 7f01ad7fa640 1 -- 172.21.15.67:0/1670223651 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f01b4014270 con 0x556dfac6df60 2024-07-27T03:16:28.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.312+0000 7f01d2076740 1 -- 172.21.15.67:0/1670223651 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01b400fc10 con 0x556dfac6df60 2024-07-27T03:16:28.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.312+0000 7f01ad7fa640 1 -- 172.21.15.67:0/1670223651 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f01b4033050 con 0x556dfac6df60 2024-07-27T03:16:28.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.312+0000 7f01d2076740 10 client.15390 fetch_fsmap learned FSMap version 29 2024-07-27T03:16:28.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.312+0000 7f01d2076740 1 -- 172.21.15.67:0/1670223651 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556dfaf1fac0 con 0x556dfac6df60 2024-07-27T03:16:28.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.312+0000 7f01ad7fa640 1 -- 172.21.15.67:0/1670223651 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01b400fc10 con 0x556dfac6df60 2024-07-27T03:16:28.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.313+0000 7f01ad7fa640 1 -- 172.21.15.67:0/1670223651 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f01b4012790 con 0x556dfac6df60 2024-07-27T03:16:28.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.313+0000 7f01d2076740 10 client.15390 fetch_fsmap finished waiting for FSMap version 29 2024-07-27T03:16:28.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.313+0000 7f01d2076740 10 client.15390 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-27T03:16:28.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.313+0000 7f01d2076740 20 client.15390 populate_metadata read hostname 'smithi067' 2024-07-27T03:16:28.322 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.313+0000 7f01d2076740 1 --2- 172.21.15.67:0/1670223651 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x556dfac2c8e0 0x556dfb030f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:28.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.313+0000 7f01d2076740 4 client.15390 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:16:28.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.313+0000 7f01d2076740 1 -- 172.21.15.67:0/1670223651 --> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556dfaf8b800 con 0x556dfac2c8e0 2024-07-27T03:16:28.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.313+0000 7f01c988d640 1 --2- 172.21.15.67:0/1670223651 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x556dfac2c8e0 0x556dfb030f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:28.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.314+0000 7f01c988d640 1 --2- 172.21.15.67:0/1670223651 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x556dfac2c8e0 0x556dfb030f50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f3e17e5562d81183 server_cookie=6a21f61a0ec226a0 in_seq=0 out_seq=0 2024-07-27T03:16:28.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.314+0000 7f01ad7fa640 10 client.15390 ms_handle_connect on v2:172.21.15.175:6838/1670568853 2024-07-27T03:16:28.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.314+0000 7f01ad7fa640 1 -- 172.21.15.67:0/1670223651 <== mds.2 v2:172.21.15.175:6838/1670568853 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556dfaf8b800 con 0x556dfac2c8e0 2024-07-27T03:16:28.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.314+0000 7f01ad7fa640 10 client.15390 handle_command_reply: tid=0 2024-07-27T03:16:28.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.314+0000 7f01d2076740 1 client.15390 shutdown 2024-07-27T03:16:28.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.315+0000 7f01d2076740 1 -- 172.21.15.67:0/1670223651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac6df60 msgr2=0x556dfb024b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:28.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.315+0000 7f01d2076740 1 --2- 172.21.15.67:0/1670223651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac6df60 0x556dfb024b40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f01b4012660 tx=0x7f01b4012560 comp rx=0 tx=0).stop 2024-07-27T03:16:28.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.315+0000 7f01d2076740 1 -- 172.21.15.67:0/1670223651 shutdown_connections 2024-07-27T03:16:28.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.315+0000 7f01d2076740 1 --2- 172.21.15.67:0/1670223651 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x556dfac2c8e0 0x556dfb030f50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.315+0000 7f01d2076740 1 --2- 172.21.15.67:0/1670223651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac6df60 0x556dfb024b40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.315+0000 7f01d2076740 1 --2- 172.21.15.67:0/1670223651 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556dfac4d280 0x556dfb01f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.315+0000 7f01d2076740 1 --2- 172.21.15.67:0/1670223651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dfac450a0 0x556dfb01ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.315+0000 7f01ad7fa640 0 client.15390 ms_handle_reset on v2:172.21.15.175:6838/1670568853 2024-07-27T03:16:28.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.315+0000 7f01d2076740 1 -- 172.21.15.67:0/1670223651 >> 172.21.15.67:0/1670223651 conn(0x556dfac62350 msgr2=0x556dfb016400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:28.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.315+0000 7f01d2076740 1 -- 172.21.15.67:0/1670223651 shutdown_connections 2024-07-27T03:16:28.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.315+0000 7f01d2076740 1 -- 172.21.15.67:0/1670223651 wait complete. 2024-07-27T03:16:28.324 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.315+0000 7f01d2076740 20 client.15390 trim_cache size 0 max 16384 2024-07-27T03:16:28.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.333+0000 7f01d2076740 1 Processor -- start 2024-07-27T03:16:28.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.333+0000 7f01d2076740 1 -- start start 2024-07-27T03:16:28.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.334+0000 7f01d2076740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556dfac450a0 0x556dfb024ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:28.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.334+0000 7f01d2076740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac4d280 0x556dfb01eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:28.336 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.334+0000 7f01d2076740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dfac6df60 0x556dfb01f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:28.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.334+0000 7f01d2076740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556dfb02f290 con 0x556dfac4d280 2024-07-27T03:16:28.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.334+0000 7f01d2076740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556dfb02f960 con 0x556dfac6df60 2024-07-27T03:16:28.337 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.334+0000 7f01d2076740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556dfb030030 con 0x556dfac450a0 2024-07-27T03:16:28.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.334+0000 7f01c988d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac4d280 0x556dfb01eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:28.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.334+0000 7f01c988d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac4d280 0x556dfb01eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36356/0 (socket says 172.21.15.67:36356) 2024-07-27T03:16:28.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.334+0000 7f01c988d640 1 -- 172.21.15.67:0/3473640486 learned_addr learned my addr 172.21.15.67:0/3473640486 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:28.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.334+0000 7f01c908c640 1 --2- 172.21.15.67:0/3473640486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dfac6df60 0x556dfb01f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:28.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.334+0000 7f01c908c640 1 --2- 172.21.15.67:0/3473640486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556dfac450a0 0x556dfb024ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:28.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.334+0000 7f01c988d640 1 -- 172.21.15.67:0/3473640486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556dfac450a0 msgr2=0x556dfb024ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:28.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.334+0000 7f01c988d640 1 --2- 172.21.15.67:0/3473640486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556dfac450a0 0x556dfb024ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.334+0000 7f01c988d640 1 -- 172.21.15.67:0/3473640486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dfac6df60 msgr2=0x556dfb01f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:28.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.334+0000 7f01c988d640 1 --2- 172.21.15.67:0/3473640486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dfac6df60 0x556dfb01f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.334+0000 7f01c988d640 1 -- 172.21.15.67:0/3473640486 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dfb02bb20 con 0x556dfac4d280 2024-07-27T03:16:28.338 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.335+0000 7f01c988d640 1 --2- 172.21.15.67:0/3473640486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac4d280 0x556dfb01eee0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f01c0031440 tx=0x7f01c00092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4c98204a001ecb server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:28.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.335+0000 7f01af7fe640 1 -- 172.21.15.67:0/3473640486 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c004b070 con 0x556dfac4d280 2024-07-27T03:16:28.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.335+0000 7f01af7fe640 1 -- 172.21.15.67:0/3473640486 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01c00054f0 con 0x556dfac4d280 2024-07-27T03:16:28.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.335+0000 7f01af7fe640 1 -- 172.21.15.67:0/3473640486 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c0035040 con 0x556dfac4d280 2024-07-27T03:16:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.336+0000 7f01d2076740 1 -- 172.21.15.67:0/3473640486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac4d280 msgr2=0x556dfb01eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.336+0000 7f01d2076740 1 --2- 172.21.15.67:0/3473640486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac4d280 0x556dfb01eee0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f01c0031440 tx=0x7f01c00092d0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.336+0000 7f01d2076740 1 -- 172.21.15.67:0/3473640486 shutdown_connections 2024-07-27T03:16:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.336+0000 7f01d2076740 1 --2- 172.21.15.67:0/3473640486 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dfac6df60 0x556dfb01f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.336+0000 7f01d2076740 1 --2- 172.21.15.67:0/3473640486 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac4d280 0x556dfb01eee0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.336+0000 7f01d2076740 1 --2- 172.21.15.67:0/3473640486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556dfac450a0 0x556dfb024ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.336+0000 7f01d2076740 1 -- 172.21.15.67:0/3473640486 >> 172.21.15.67:0/3473640486 conn(0x556dfac62350 msgr2=0x556dfb01d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.336+0000 7f01d2076740 1 -- 172.21.15.67:0/3473640486 shutdown_connections 2024-07-27T03:16:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.336+0000 7f01d2076740 1 -- 172.21.15.67:0/3473640486 wait complete. 2024-07-27T03:16:28.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.336+0000 7f01d2076740 1 -- start start 2024-07-27T03:16:28.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.336+0000 7f01d2076740 1 Processor -- start 2024-07-27T03:16:28.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.337+0000 7f01d2076740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556dfac450a0 0x556dfb023ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:28.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.337+0000 7f01d2076740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dfac4d280 0x556dfb024240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:28.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.337+0000 7f01d2076740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac6df60 0x556dfb0297e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:28.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.337+0000 7f01d2076740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x556dfb030dd0 con 0x556dfac6df60 2024-07-27T03:16:28.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.337+0000 7f01d2076740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x556dfb030700 con 0x556dfac4d280 2024-07-27T03:16:28.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.337+0000 7f01d2076740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x556dfb0314a0 con 0x556dfac450a0 2024-07-27T03:16:28.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.337+0000 7f01d2076740 10 client.0 _ll_register_callbacks cb 0x7f01b400ced0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:16:28.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.337+0000 7f01c908c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac6df60 0x556dfb0297e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:28.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.337+0000 7f01c908c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac6df60 0x556dfb0297e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36358/0 (socket says 172.21.15.67:36358) 2024-07-27T03:16:28.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.337+0000 7f01c908c640 1 -- 172.21.15.67:0/696868221 learned_addr learned my addr 172.21.15.67:0/696868221 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:28.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.337+0000 7f01c988d640 1 --2- 172.21.15.67:0/696868221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dfac4d280 0x556dfb024240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:28.343 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.337+0000 7f01c908c640 1 --2- 172.21.15.67:0/696868221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556dfac450a0 0x556dfb023ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:28.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.338+0000 7f01c908c640 1 -- 172.21.15.67:0/696868221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556dfac450a0 msgr2=0x556dfb023ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:28.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.338+0000 7f01c908c640 1 --2- 172.21.15.67:0/696868221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556dfac450a0 0x556dfb023ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.338+0000 7f01c908c640 1 -- 172.21.15.67:0/696868221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dfac4d280 msgr2=0x556dfb024240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:28.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.338+0000 7f01c908c640 1 --2- 172.21.15.67:0/696868221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dfac4d280 0x556dfb024240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.338+0000 7f01c908c640 1 -- 172.21.15.67:0/696868221 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dfacd9540 con 0x556dfac6df60 2024-07-27T03:16:28.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.338+0000 7f01c988d640 1 --2- 172.21.15.67:0/696868221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dfac4d280 0x556dfb024240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:16:28.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.338+0000 7f01c908c640 1 --2- 172.21.15.67:0/696868221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac6df60 0x556dfb0297e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f01b4006010 tx=0x7f01b4016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386c905a5d40c6d6 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:28.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.338+0000 7f01ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:16:28.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.338+0000 7f01ad7fa640 1 -- 172.21.15.67:0/696868221 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4014560 con 0x556dfac6df60 2024-07-27T03:16:28.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.338+0000 7f01ad7fa640 1 -- 172.21.15.67:0/696868221 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f01b4040070 con 0x556dfac6df60 2024-07-27T03:16:28.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.338+0000 7f01d2076740 1 -- 172.21.15.67:0/696868221 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01c00054f0 con 0x556dfac6df60 2024-07-27T03:16:28.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.338+0000 7f01ad7fa640 1 -- 172.21.15.67:0/696868221 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f01b4049050 con 0x556dfac6df60 2024-07-27T03:16:28.344 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.338+0000 7f01d2076740 10 client.15396 fetch_fsmap learned FSMap version 29 2024-07-27T03:16:28.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.338+0000 7f01d2076740 1 -- 172.21.15.67:0/696868221 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556dfb02af60 con 0x556dfac6df60 2024-07-27T03:16:28.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.338+0000 7f01ad7fa640 1 -- 172.21.15.67:0/696868221 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01c00054f0 con 0x556dfac6df60 2024-07-27T03:16:28.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.339+0000 7f01ad7fa640 1 -- 172.21.15.67:0/696868221 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f01b400f9a0 con 0x556dfac6df60 2024-07-27T03:16:28.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.339+0000 7f01d2076740 10 client.15396 fetch_fsmap finished waiting for FSMap version 29 2024-07-27T03:16:28.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.339+0000 7f01d2076740 10 client.15396 resolve_mds: resolved name 'i' to daemon mds.i 2024-07-27T03:16:28.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.339+0000 7f01d2076740 20 client.15396 populate_metadata read hostname 'smithi067' 2024-07-27T03:16:28.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.339+0000 7f01d2076740 1 --2- 172.21.15.67:0/696868221 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x556dfac01c90 0x556dfb0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:28.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.339+0000 7f01d2076740 4 client.15396 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-27T03:16:28.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.339+0000 7f01d2076740 1 -- 172.21.15.67:0/696868221 --> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] -- command(tid 0: {"prefix": "status"}) -- 0x556dfaf872a0 con 0x556dfac01c90 2024-07-27T03:16:28.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.339+0000 7f01c988d640 1 --2- 172.21.15.67:0/696868221 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x556dfac01c90 0x556dfb0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:28.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.339+0000 7f01c988d640 1 --2- 172.21.15.67:0/696868221 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x556dfac01c90 0x556dfb0a5730 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7c3d42c142e0181e server_cookie=f6b6089677074e34 in_seq=0 out_seq=0 2024-07-27T03:16:28.345 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.339+0000 7f01ad7fa640 10 client.15396 ms_handle_connect on v2:172.21.15.175:6838/1670568853 2024-07-27T03:16:28.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.340+0000 7f01ad7fa640 1 -- 172.21.15.67:0/696868221 <== mds.2 v2:172.21.15.175:6838/1670568853 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556dfaf872a0 con 0x556dfac01c90 2024-07-27T03:16:28.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.340+0000 7f01ad7fa640 10 client.15396 handle_command_reply: tid=0 2024-07-27T03:16:28.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.340+0000 7f01d2076740 1 client.15396 shutdown 2024-07-27T03:16:28.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.340+0000 7f01d2076740 1 -- 172.21.15.67:0/696868221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac6df60 msgr2=0x556dfb0297e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:28.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.340+0000 7f01d2076740 1 --2- 172.21.15.67:0/696868221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac6df60 0x556dfb0297e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f01b4006010 tx=0x7f01b4016040 comp rx=0 tx=0).stop 2024-07-27T03:16:28.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.340+0000 7f01d2076740 1 -- 172.21.15.67:0/696868221 shutdown_connections 2024-07-27T03:16:28.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.340+0000 7f01d2076740 1 --2- 172.21.15.67:0/696868221 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x556dfac01c90 0x556dfb0a5730 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.340+0000 7f01d2076740 1 --2- 172.21.15.67:0/696868221 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x556dfac6df60 0x556dfb0297e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.340+0000 7f01d2076740 1 --2- 172.21.15.67:0/696868221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556dfac4d280 0x556dfb024240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.340+0000 7f01ad7fa640 0 client.15396 ms_handle_reset on v2:172.21.15.175:6838/1670568853 2024-07-27T03:16:28.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.340+0000 7f01d2076740 1 --2- 172.21.15.67:0/696868221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x556dfac450a0 0x556dfb023ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.347 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.340+0000 7f01d2076740 1 -- 172.21.15.67:0/696868221 >> 172.21.15.67:0/696868221 conn(0x556dfac62350 msgr2=0x556dfb01eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:28.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.340+0000 7f01d2076740 1 -- 172.21.15.67:0/696868221 shutdown_connections 2024-07-27T03:16:28.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.340+0000 7f01d2076740 1 -- 172.21.15.67:0/696868221 wait complete. 2024-07-27T03:16:28.348 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.340+0000 7f01d2076740 20 client.15396 trim_cache size 0 max 16384 2024-07-27T03:16:28.348 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:16:28.348 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "c0782a00-4bc4-11ef-bcc7-c7b262605968", 2024-07-27T03:16:28.348 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 2, 2024-07-27T03:16:28.348 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24583, 2024-07-27T03:16:28.348 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-27T03:16:28.348 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-27T03:16:28.348 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-27T03:16:28.349 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 171.12469214000001, 2024-07-27T03:16:28.349 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 29, 2024-07-27T03:16:28.349 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 91, 2024-07-27T03:16:28.349 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-27T03:16:28.349 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 257.57065256300001 2024-07-27T03:16:28.349 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:16:28.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.341+0000 7f01cbaf1640 1 -- 172.21.15.67:0/1906534983 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f01b00b3c50 msgr2=0x7f01b00b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:28.349 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.341+0000 7f01cbaf1640 1 --2- 172.21.15.67:0/1906534983 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f01b00b3c50 0x7f01b00b6160 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f01bc0a73e0 tx=0x7f01bc0d9da0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.341+0000 7f01cbaf1640 1 -- 172.21.15.67:0/1906534983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c4109860 msgr2=0x7f01c419f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:28.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.341+0000 7f01cbaf1640 1 --2- 172.21.15.67:0/1906534983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c4109860 0x7f01c419f7d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00b970 tx=0x7f01bc00be30 comp rx=0 tx=0).stop 2024-07-27T03:16:28.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.341+0000 7f01cbaf1640 1 -- 172.21.15.67:0/1906534983 shutdown_connections 2024-07-27T03:16:28.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.341+0000 7f01cbaf1640 1 --2- 172.21.15.67:0/1906534983 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f01b00b3c50 0x7f01b00b6160 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.341+0000 7f01cbaf1640 1 --2- 172.21.15.67:0/1906534983 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f01c4109860 0x7f01c419f7d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.341+0000 7f01cbaf1640 1 --2- 172.21.15.67:0/1906534983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01c4108f10 0x7f01c419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.341+0000 7f01cbaf1640 1 --2- 172.21.15.67:0/1906534983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c41035d0 0x7f01c4199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:28.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.341+0000 7f01cbaf1640 1 -- 172.21.15.67:0/1906534983 >> 172.21.15.67:0/1906534983 conn(0x7f01c407b2c0 msgr2=0x7f01c4107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:28.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.341+0000 7f01cbaf1640 1 -- 172.21.15.67:0/1906534983 shutdown_connections 2024-07-27T03:16:28.350 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:28.341+0000 7f01cbaf1640 1 -- 172.21.15.67:0/1906534983 wait complete. 2024-07-27T03:16:28.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:28 smithi175 ceph-mon[37125]: pgmap v427: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:28.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:28 smithi067 ceph-mon[32814]: pgmap v427: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:28.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:28 smithi149 ceph-mon[37173]: pgmap v427: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:28.935 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.h status 2024-07-27T03:16:29.200 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:16:30.601 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:30 smithi067 ceph-mon[32814]: pgmap v428: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:16:30.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:30 smithi175 ceph-mon[37125]: pgmap v428: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:16:30.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:30 smithi149 ceph-mon[37173]: pgmap v428: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:16:30.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.826+0000 7f1dc9876640 1 -- 172.21.15.67:0/2546344215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dc4108760 msgr2=0x7f1dc410ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:30.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.826+0000 7f1dc9876640 1 --2- 172.21.15.67:0/2546344215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dc4108760 0x7f1dc410ef90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009ca0 tx=0x7f1db402a8c0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.826+0000 7f1dc9876640 1 -- 172.21.15.67:0/2546344215 shutdown_connections 2024-07-27T03:16:30.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.826+0000 7f1dc9876640 1 --2- 172.21.15.67:0/2546344215 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dc4108760 0x7f1dc410ef90 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.826+0000 7f1dc9876640 1 --2- 172.21.15.67:0/2546344215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dc4107e10 0x7f1dc4108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.826+0000 7f1dc9876640 1 --2- 172.21.15.67:0/2546344215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dc41024d0 0x7f1dc41028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.826+0000 7f1dc9876640 1 -- 172.21.15.67:0/2546344215 >> 172.21.15.67:0/2546344215 conn(0x7f1dc40791a0 msgr2=0x7f1dc40795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:30.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.826+0000 7f1dc9876640 1 -- 172.21.15.67:0/2546344215 shutdown_connections 2024-07-27T03:16:30.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.826+0000 7f1dc9876640 1 -- 172.21.15.67:0/2546344215 wait complete. 2024-07-27T03:16:30.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.827+0000 7f1dc9876640 1 Processor -- start 2024-07-27T03:16:30.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.827+0000 7f1dc9876640 1 -- start start 2024-07-27T03:16:30.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.828+0000 7f1dc9876640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dc41024d0 0x7f1dc4199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:30.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.828+0000 7f1dc9876640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dc4107e10 0x7f1dc419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:30.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.828+0000 7f1dc9876640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dc4108760 0x7f1dc419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:30.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.828+0000 7f1dc9876640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc41134f0 con 0x7f1dc4108760 2024-07-27T03:16:30.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.828+0000 7f1dc9876640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4113370 con 0x7f1dc4107e10 2024-07-27T03:16:30.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.828+0000 7f1dc9876640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4113670 con 0x7f1dc41024d0 2024-07-27T03:16:30.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.828+0000 7f1dc37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dc4108760 0x7f1dc419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:30.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.828+0000 7f1dc37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dc4108760 0x7f1dc419f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36390/0 (socket says 172.21.15.67:36390) 2024-07-27T03:16:30.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.828+0000 7f1dc37fe640 1 -- 172.21.15.67:0/2087596079 learned_addr learned my addr 172.21.15.67:0/2087596079 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:30.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.828+0000 7f1dc2ffd640 1 --2- 172.21.15.67:0/2087596079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dc41024d0 0x7f1dc4199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:30.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.828+0000 7f1dc27fc640 1 --2- 172.21.15.67:0/2087596079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dc4107e10 0x7f1dc419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:30.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.828+0000 7f1dc37fe640 1 -- 172.21.15.67:0/2087596079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dc41024d0 msgr2=0x7f1dc4199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:30.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.828+0000 7f1dc37fe640 1 --2- 172.21.15.67:0/2087596079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dc41024d0 0x7f1dc4199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.828+0000 7f1dc37fe640 1 -- 172.21.15.67:0/2087596079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dc4107e10 msgr2=0x7f1dc419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:30.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.828+0000 7f1dc37fe640 1 --2- 172.21.15.67:0/2087596079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dc4107e10 0x7f1dc419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.828+0000 7f1dc37fe640 1 -- 172.21.15.67:0/2087596079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc419fdf0 con 0x7f1dc4108760 2024-07-27T03:16:30.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.828+0000 7f1dc37fe640 1 --2- 172.21.15.67:0/2087596079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dc4108760 0x7f1dc419f6c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1db4005cc0 tx=0x7f1db402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:30.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.828+0000 7f1dc8874640 1 -- 172.21.15.67:0/2087596079 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4003bf0 con 0x7f1dc4108760 2024-07-27T03:16:30.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.829+0000 7f1dc8874640 1 -- 172.21.15.67:0/2087596079 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db4003d90 con 0x7f1dc4108760 2024-07-27T03:16:30.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.829+0000 7f1dc9876640 1 -- 172.21.15.67:0/2087596079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc41a0080 con 0x7f1dc4108760 2024-07-27T03:16:30.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.829+0000 7f1dc8874640 1 -- 172.21.15.67:0/2087596079 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db402fbc0 con 0x7f1dc4108760 2024-07-27T03:16:30.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.829+0000 7f1dc9876640 1 -- 172.21.15.67:0/2087596079 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc41a6840 con 0x7f1dc4108760 2024-07-27T03:16:30.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.832+0000 7f1dcfdfb740 1 Processor -- start 2024-07-27T03:16:30.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.832+0000 7f1dc8874640 1 -- 172.21.15.67:0/2087596079 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f1db402fd60 con 0x7f1dc4108760 2024-07-27T03:16:30.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.832+0000 7f1dcfdfb740 1 -- start start 2024-07-27T03:16:30.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.832+0000 7f1dcfdfb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f96991f60 0x561f96d37200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:30.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.832+0000 7f1dcfdfb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f969690a0 0x561f96d37740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:30.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.832+0000 7f1dcfdfb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96971280 0x561f96d37c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.832+0000 7f1dcfdfb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561f96c1f2b0 con 0x561f96971280 2024-07-27T03:16:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.832+0000 7f1dcfdfb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561f96cad890 con 0x561f969690a0 2024-07-27T03:16:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.832+0000 7f1dcfdfb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561f96c9cd90 con 0x561f96991f60 2024-07-27T03:16:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.833+0000 7f1dc27fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96971280 0x561f96d37c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.833+0000 7f1dc27fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96971280 0x561f96d37c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36406/0 (socket says 172.21.15.67:36406) 2024-07-27T03:16:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.833+0000 7f1dc27fc640 1 -- 172.21.15.67:0/1608386241 learned_addr learned my addr 172.21.15.67:0/1608386241 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.833+0000 7f1dc27fc640 1 --2- 172.21.15.67:0/1608386241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f96991f60 0x561f96d37200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.833+0000 7f1dc2ffd640 1 --2- 172.21.15.67:0/1608386241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f969690a0 0x561f96d37740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.833+0000 7f1dc27fc640 1 -- 172.21.15.67:0/1608386241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f96991f60 msgr2=0x561f96d37200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.833+0000 7f1dc27fc640 1 --2- 172.21.15.67:0/1608386241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f96991f60 0x561f96d37200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.833+0000 7f1dc27fc640 1 -- 172.21.15.67:0/1608386241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f969690a0 msgr2=0x561f96d37740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.833+0000 7f1dc27fc640 1 --2- 172.21.15.67:0/1608386241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f969690a0 0x561f96d37740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.833+0000 7f1dc8874640 1 --2- 172.21.15.67:0/2087596079 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f1db00b3bd0 0x7f1db00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:30.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.833+0000 7f1dc27fc640 1 -- 172.21.15.67:0/1608386241 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f96cd6af0 con 0x561f96971280 2024-07-27T03:16:30.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.833+0000 7f1dc2ffd640 1 --2- 172.21.15.67:0/2087596079 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f1db00b3bd0 0x7f1db00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:30.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.833+0000 7f1dc27fc640 1 --2- 172.21.15.67:0/1608386241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96971280 0x561f96d37c80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1dac012700 tx=0x7f1dac00b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a82223e21355c8 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:30.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.833+0000 7f1dc8874640 1 -- 172.21.15.67:0/2087596079 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f1db4105660 con 0x7f1dc4108760 2024-07-27T03:16:30.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.834+0000 7f1db9ffb640 1 -- 172.21.15.67:0/1608386241 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac012c80 con 0x561f96971280 2024-07-27T03:16:30.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.834+0000 7f1db9ffb640 1 -- 172.21.15.67:0/1608386241 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dac012e20 con 0x561f96971280 2024-07-27T03:16:30.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.834+0000 7f1db9ffb640 1 -- 172.21.15.67:0/1608386241 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac0035f0 con 0x561f96971280 2024-07-27T03:16:30.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.834+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/1608386241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96971280 msgr2=0x561f96d37c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:30.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.834+0000 7f1dc2ffd640 1 --2- 172.21.15.67:0/2087596079 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f1db00b3bd0 0x7f1db00b60e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x561f96d35e80 tx=0x7f1dbc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:30.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.834+0000 7f1dcfdfb740 1 --2- 172.21.15.67:0/1608386241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96971280 0x561f96d37c80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1dac012700 tx=0x7f1dac00b6e0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.834+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/1608386241 shutdown_connections 2024-07-27T03:16:30.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.835+0000 7f1dcfdfb740 1 --2- 172.21.15.67:0/1608386241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96971280 0x561f96d37c80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.835+0000 7f1dcfdfb740 1 --2- 172.21.15.67:0/1608386241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f969690a0 0x561f96d37740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.835+0000 7f1dcfdfb740 1 --2- 172.21.15.67:0/1608386241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f96991f60 0x561f96d37200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.835+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/1608386241 >> 172.21.15.67:0/1608386241 conn(0x561f96986350 msgr2=0x561f96d350f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:30.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.835+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/1608386241 shutdown_connections 2024-07-27T03:16:30.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.835+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/1608386241 wait complete. 2024-07-27T03:16:30.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.835+0000 7f1dcfdfb740 1 -- start start 2024-07-27T03:16:30.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.835+0000 7f1dcfdfb740 1 Processor -- start 2024-07-27T03:16:30.843 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.836+0000 7f1dcfdfb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f969690a0 0x561f96d472a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:30.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.836+0000 7f1dcfdfb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f96971280 0x561f96d478a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:30.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.836+0000 7f1dcfdfb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96991f60 0x561f96d52ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:30.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.836+0000 7f1dcfdfb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561f96c3f7b0 con 0x561f96991f60 2024-07-27T03:16:30.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.836+0000 7f1dcfdfb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561f96ca0dd0 con 0x561f96971280 2024-07-27T03:16:30.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.836+0000 7f1dcfdfb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561f96b005f0 con 0x561f969690a0 2024-07-27T03:16:30.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.836+0000 7f1dcfdfb740 10 client.0 _ll_register_callbacks cb 0x561f96c477e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:16:30.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.836+0000 7f1dc37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96991f60 0x561f96d52ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:30.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.836+0000 7f1dc37fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96991f60 0x561f96d52ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36420/0 (socket says 172.21.15.67:36420) 2024-07-27T03:16:30.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.836+0000 7f1dc27fc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f96971280 0x561f96d478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:30.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.836+0000 7f1dc37fe640 1 -- 172.21.15.67:0/2863997406 learned_addr learned my addr 172.21.15.67:0/2863997406 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:30.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.836+0000 7f1dc2ffd640 1 --2- 172.21.15.67:0/2863997406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f969690a0 0x561f96d472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:30.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.836+0000 7f1dc37fe640 1 -- 172.21.15.67:0/2863997406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f969690a0 msgr2=0x561f96d472a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:30.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.836+0000 7f1dc37fe640 1 --2- 172.21.15.67:0/2863997406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f969690a0 0x561f96d472a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.836+0000 7f1dc37fe640 1 -- 172.21.15.67:0/2863997406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f96971280 msgr2=0x561f96d478a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:30.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.836+0000 7f1dc37fe640 1 --2- 172.21.15.67:0/2863997406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f96971280 0x561f96d478a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.836+0000 7f1dc37fe640 1 -- 172.21.15.67:0/2863997406 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f96c32620 con 0x561f96991f60 2024-07-27T03:16:30.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.836+0000 7f1dc37fe640 1 --2- 172.21.15.67:0/2863997406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96991f60 0x561f96d52ec0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1db40cbf00 tx=0x7f1db40cc9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7bcecf809ea86f server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:30.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.837+0000 7f1da2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:16:30.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.837+0000 7f1da2ffd640 1 -- 172.21.15.67:0/2863997406 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db40d0800 con 0x561f96991f60 2024-07-27T03:16:30.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.837+0000 7f1da2ffd640 1 -- 172.21.15.67:0/2863997406 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f1db40c8490 con 0x561f96991f60 2024-07-27T03:16:30.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.837+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/2863997406 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dac012e20 con 0x561f96991f60 2024-07-27T03:16:30.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.837+0000 7f1da2ffd640 1 -- 172.21.15.67:0/2863997406 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1db40cbfd0 con 0x561f96991f60 2024-07-27T03:16:30.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.837+0000 7f1dcfdfb740 10 client.15408 fetch_fsmap learned FSMap version 29 2024-07-27T03:16:30.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.837+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/2863997406 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561f96c43aa0 con 0x561f96991f60 2024-07-27T03:16:30.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.837+0000 7f1da2ffd640 1 -- 172.21.15.67:0/2863997406 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dac012e20 con 0x561f96991f60 2024-07-27T03:16:30.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.837+0000 7f1da2ffd640 1 -- 172.21.15.67:0/2863997406 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1db40b3070 con 0x561f96991f60 2024-07-27T03:16:30.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.837+0000 7f1dcfdfb740 10 client.15408 fetch_fsmap finished waiting for FSMap version 29 2024-07-27T03:16:30.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.837+0000 7f1dcfdfb740 10 client.15408 resolve_mds: resolved name 'h' to daemon mds.h 2024-07-27T03:16:30.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.837+0000 7f1dcfdfb740 20 client.15408 populate_metadata read hostname 'smithi067' 2024-07-27T03:16:30.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.837+0000 7f1dcfdfb740 1 --2- 172.21.15.67:0/2863997406 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x561f969508e0 0x561f96d59570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.837+0000 7f1dcfdfb740 4 client.15408 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:16:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.837+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/2863997406 --> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f96caf7e0 con 0x561f969508e0 2024-07-27T03:16:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.838+0000 7f1dc2ffd640 1 --2- 172.21.15.67:0/2863997406 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x561f969508e0 0x561f96d59570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.838+0000 7f1da2ffd640 10 client.15408 ms_handle_connect on v2:172.21.15.149:6836/2932951845 2024-07-27T03:16:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.838+0000 7f1dc2ffd640 1 --2- 172.21.15.67:0/2863997406 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x561f969508e0 0x561f96d59570 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ece2ba87ff01486f server_cookie=1757004fc44f60b8 in_seq=0 out_seq=0 2024-07-27T03:16:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.839+0000 7f1da2ffd640 1 -- 172.21.15.67:0/2863997406 <== mds.1 v2:172.21.15.149:6836/2932951845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f96caf7e0 con 0x561f969508e0 2024-07-27T03:16:30.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.839+0000 7f1da2ffd640 10 client.15408 handle_command_reply: tid=0 2024-07-27T03:16:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.839+0000 7f1dcfdfb740 1 client.15408 shutdown 2024-07-27T03:16:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.839+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/2863997406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96991f60 msgr2=0x561f96d52ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.839+0000 7f1dcfdfb740 1 --2- 172.21.15.67:0/2863997406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96991f60 0x561f96d52ec0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1db40cbf00 tx=0x7f1db40cc9c0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.839+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/2863997406 shutdown_connections 2024-07-27T03:16:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.839+0000 7f1dcfdfb740 1 --2- 172.21.15.67:0/2863997406 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x561f969508e0 0x561f96d59570 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.839+0000 7f1dcfdfb740 1 --2- 172.21.15.67:0/2863997406 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96991f60 0x561f96d52ec0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.839+0000 7f1dcfdfb740 1 --2- 172.21.15.67:0/2863997406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f96971280 0x561f96d478a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.839+0000 7f1dcfdfb740 1 --2- 172.21.15.67:0/2863997406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f969690a0 0x561f96d472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.839+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/2863997406 >> 172.21.15.67:0/2863997406 conn(0x561f96986350 msgr2=0x561f96d38bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:30.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.839+0000 7f1da2ffd640 0 client.15408 ms_handle_reset on v2:172.21.15.149:6836/2932951845 2024-07-27T03:16:30.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.840+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/2863997406 shutdown_connections 2024-07-27T03:16:30.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.840+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/2863997406 wait complete. 2024-07-27T03:16:30.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.840+0000 7f1dcfdfb740 20 client.15408 trim_cache size 0 max 16384 2024-07-27T03:16:30.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.858+0000 7f1dcfdfb740 1 Processor -- start 2024-07-27T03:16:30.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.858+0000 7f1dcfdfb740 1 -- start start 2024-07-27T03:16:30.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.858+0000 7f1dcfdfb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f969690a0 0x561f96d587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:30.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.858+0000 7f1dcfdfb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96971280 0x561f96d52e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:30.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.858+0000 7f1dcfdfb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f96991f60 0x561f96d49150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:30.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.858+0000 7f1dcfdfb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561f96d578b0 con 0x561f96971280 2024-07-27T03:16:30.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.858+0000 7f1dcfdfb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561f96d57f80 con 0x561f969690a0 2024-07-27T03:16:30.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.858+0000 7f1dcfdfb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561f96d58650 con 0x561f96991f60 2024-07-27T03:16:30.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.858+0000 7f1dc27fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96971280 0x561f96d52e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:30.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.858+0000 7f1dc27fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96971280 0x561f96d52e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36426/0 (socket says 172.21.15.67:36426) 2024-07-27T03:16:30.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.858+0000 7f1dc27fc640 1 -- 172.21.15.67:0/413900548 learned_addr learned my addr 172.21.15.67:0/413900548 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:30.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.858+0000 7f1dc37fe640 1 --2- 172.21.15.67:0/413900548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f96991f60 0x561f96d49150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:30.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.858+0000 7f1dc2ffd640 1 --2- 172.21.15.67:0/413900548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f969690a0 0x561f96d587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:30.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.859+0000 7f1dc27fc640 1 -- 172.21.15.67:0/413900548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f96991f60 msgr2=0x561f96d49150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:30.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.859+0000 7f1dc27fc640 1 --2- 172.21.15.67:0/413900548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f96991f60 0x561f96d49150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.859+0000 7f1dc27fc640 1 -- 172.21.15.67:0/413900548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f969690a0 msgr2=0x561f96d587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:30.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.859+0000 7f1dc27fc640 1 --2- 172.21.15.67:0/413900548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f969690a0 0x561f96d587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.870 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.859+0000 7f1dc27fc640 1 -- 172.21.15.67:0/413900548 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f969fd540 con 0x561f96971280 2024-07-27T03:16:30.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.859+0000 7f1dc37fe640 1 --2- 172.21.15.67:0/413900548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f96991f60 0x561f96d49150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:16:30.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.859+0000 7f1dc27fc640 1 --2- 172.21.15.67:0/413900548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96971280 0x561f96d52e30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0126a0 tx=0x7f1dac010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8781b9c80d93625a server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:30.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.859+0000 7f1db97fa640 1 -- 172.21.15.67:0/413900548 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac010940 con 0x561f96971280 2024-07-27T03:16:30.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.859+0000 7f1db97fa640 1 -- 172.21.15.67:0/413900548 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dac016040 con 0x561f96971280 2024-07-27T03:16:30.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.859+0000 7f1db97fa640 1 -- 172.21.15.67:0/413900548 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac004ca0 con 0x561f96971280 2024-07-27T03:16:30.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.860+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/413900548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96971280 msgr2=0x561f96d52e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:30.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.860+0000 7f1dcfdfb740 1 --2- 172.21.15.67:0/413900548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96971280 0x561f96d52e30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0126a0 tx=0x7f1dac010c00 comp rx=0 tx=0).stop 2024-07-27T03:16:30.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.860+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/413900548 shutdown_connections 2024-07-27T03:16:30.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.860+0000 7f1dcfdfb740 1 --2- 172.21.15.67:0/413900548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f96991f60 0x561f96d49150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.860+0000 7f1dcfdfb740 1 --2- 172.21.15.67:0/413900548 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96971280 0x561f96d52e30 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.860+0000 7f1dcfdfb740 1 --2- 172.21.15.67:0/413900548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f969690a0 0x561f96d587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.860+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/413900548 >> 172.21.15.67:0/413900548 conn(0x561f96986350 msgr2=0x561f96d56e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:30.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.860+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/413900548 shutdown_connections 2024-07-27T03:16:30.871 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.860+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/413900548 wait complete. 2024-07-27T03:16:30.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.860+0000 7f1dcfdfb740 1 -- start start 2024-07-27T03:16:30.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.861+0000 7f1dcfdfb740 1 Processor -- start 2024-07-27T03:16:30.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.861+0000 7f1dcfdfb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f969690a0 0x561f96d5a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:30.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.861+0000 7f1dcfdfb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96971280 0x561f96d5ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:30.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.861+0000 7f1dcfdfb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f96991f60 0x561f96d5fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:30.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.861+0000 7f1dcfdfb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x561f96d593f0 con 0x561f96971280 2024-07-27T03:16:30.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.861+0000 7f1dcfdfb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x561f96d58d20 con 0x561f96991f60 2024-07-27T03:16:30.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.861+0000 7f1dcfdfb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561f96d59ac0 con 0x561f969690a0 2024-07-27T03:16:30.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.861+0000 7f1dcfdfb740 10 client.0 _ll_register_callbacks cb 0x7f1db40c80c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:16:30.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.861+0000 7f1dc27fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96971280 0x561f96d5ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:30.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.861+0000 7f1dc2ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f969690a0 0x561f96d5a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:30.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.861+0000 7f1dc2ffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f969690a0 0x561f96d5a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.67:45220/0 (socket says 172.21.15.67:45220) 2024-07-27T03:16:30.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.861+0000 7f1dc2ffd640 1 -- 172.21.15.67:0/2898601074 learned_addr learned my addr 172.21.15.67:0/2898601074 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:30.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.861+0000 7f1dc27fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96971280 0x561f96d5ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36440/0 (socket says 172.21.15.67:36440) 2024-07-27T03:16:30.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.861+0000 7f1dc37fe640 1 --2- 172.21.15.67:0/2898601074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f96991f60 0x561f96d5fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:30.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.862+0000 7f1dc2ffd640 1 -- 172.21.15.67:0/2898601074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f96991f60 msgr2=0x561f96d5fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:30.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.862+0000 7f1dc2ffd640 1 --2- 172.21.15.67:0/2898601074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f96991f60 0x561f96d5fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.862+0000 7f1dc2ffd640 1 -- 172.21.15.67:0/2898601074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96971280 msgr2=0x561f96d5ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:30.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.862+0000 7f1dc2ffd640 1 --2- 172.21.15.67:0/2898601074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96971280 0x561f96d5ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.862+0000 7f1dc2ffd640 1 -- 172.21.15.67:0/2898601074 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f96d60420 con 0x561f969690a0 2024-07-27T03:16:30.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.862+0000 7f1dc2ffd640 1 --2- 172.21.15.67:0/2898601074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f969690a0 0x561f96d5a7d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc01f570 tx=0x7f1dbc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f12d85d902aa8540 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:30.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.862+0000 7f1da2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-27T03:16:30.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.862+0000 7f1da2ffd640 1 -- 172.21.15.67:0/2898601074 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc02c440 con 0x561f969690a0 2024-07-27T03:16:30.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.862+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/2898601074 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dac016040 con 0x561f969690a0 2024-07-27T03:16:30.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.862+0000 7f1da2ffd640 1 -- 172.21.15.67:0/2898601074 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f1dbc017030 con 0x561f969690a0 2024-07-27T03:16:30.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.862+0000 7f1da2ffd640 1 -- 172.21.15.67:0/2898601074 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1dbc005020 con 0x561f969690a0 2024-07-27T03:16:30.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.862+0000 7f1dcfdfb740 10 client.25060 fetch_fsmap learned FSMap version 29 2024-07-27T03:16:30.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.862+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/2898601074 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561f96d61840 con 0x561f969690a0 2024-07-27T03:16:30.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.863+0000 7f1da2ffd640 1 -- 172.21.15.67:0/2898601074 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dac016040 con 0x561f969690a0 2024-07-27T03:16:30.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.863+0000 7f1da2ffd640 1 -- 172.21.15.67:0/2898601074 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1dbc02c5e0 con 0x561f969690a0 2024-07-27T03:16:30.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.863+0000 7f1dcfdfb740 10 client.25060 fetch_fsmap finished waiting for FSMap version 29 2024-07-27T03:16:30.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.863+0000 7f1dcfdfb740 10 client.25060 resolve_mds: resolved name 'h' to daemon mds.h 2024-07-27T03:16:30.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.863+0000 7f1dcfdfb740 20 client.25060 populate_metadata read hostname 'smithi067' 2024-07-27T03:16:30.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.863+0000 7f1dcfdfb740 1 --2- 172.21.15.67:0/2898601074 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x561f96925c90 0x561f96dc52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:30.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.863+0000 7f1dcfdfb740 4 client.25060 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-27T03:16:30.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.863+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/2898601074 --> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] -- command(tid 0: {"prefix": "status"}) -- 0x561f96cab280 con 0x561f96925c90 2024-07-27T03:16:30.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.863+0000 7f1dc27fc640 1 --2- 172.21.15.67:0/2898601074 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x561f96925c90 0x561f96dc52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:30.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.864+0000 7f1dc27fc640 1 --2- 172.21.15.67:0/2898601074 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x561f96925c90 0x561f96dc52b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d001144badb5e62c server_cookie=c93f7d49a97a32aa in_seq=0 out_seq=0 2024-07-27T03:16:30.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.864+0000 7f1da2ffd640 10 client.25060 ms_handle_connect on v2:172.21.15.149:6836/2932951845 2024-07-27T03:16:30.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.864+0000 7f1da2ffd640 1 -- 172.21.15.67:0/2898601074 <== mds.1 v2:172.21.15.149:6836/2932951845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561f96cab280 con 0x561f96925c90 2024-07-27T03:16:30.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.864+0000 7f1da2ffd640 10 client.25060 handle_command_reply: tid=0 2024-07-27T03:16:30.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.864+0000 7f1dcfdfb740 1 client.25060 shutdown 2024-07-27T03:16:30.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.864+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/2898601074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f969690a0 msgr2=0x561f96d5a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:30.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.864+0000 7f1dcfdfb740 1 --2- 172.21.15.67:0/2898601074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f969690a0 0x561f96d5a7d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc01f570 tx=0x7f1dbc004730 comp rx=0 tx=0).stop 2024-07-27T03:16:30.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.864+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/2898601074 shutdown_connections 2024-07-27T03:16:30.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.864+0000 7f1dcfdfb740 1 --2- 172.21.15.67:0/2898601074 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x561f96925c90 0x561f96dc52b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.864+0000 7f1dcfdfb740 1 --2- 172.21.15.67:0/2898601074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f96991f60 0x561f96d5fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.864+0000 7f1dcfdfb740 1 --2- 172.21.15.67:0/2898601074 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x561f96971280 0x561f96d5ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.864+0000 7f1da2ffd640 0 client.25060 ms_handle_reset on v2:172.21.15.149:6836/2932951845 2024-07-27T03:16:30.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.864+0000 7f1dcfdfb740 1 --2- 172.21.15.67:0/2898601074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561f969690a0 0x561f96d5a7d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.864+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/2898601074 >> 172.21.15.67:0/2898601074 conn(0x561f96986350 msgr2=0x561f96d56e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:30.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.865+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/2898601074 shutdown_connections 2024-07-27T03:16:30.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.865+0000 7f1dcfdfb740 1 -- 172.21.15.67:0/2898601074 wait complete. 2024-07-27T03:16:30.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.865+0000 7f1dcfdfb740 20 client.25060 trim_cache size 0 max 16384 2024-07-27T03:16:30.876 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:16:30.876 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "c0782a00-4bc4-11ef-bcc7-c7b262605968", 2024-07-27T03:16:30.876 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 1, 2024-07-27T03:16:30.876 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24587, 2024-07-27T03:16:30.877 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-27T03:16:30.877 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-27T03:16:30.877 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-27T03:16:30.877 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 174.66314087399999, 2024-07-27T03:16:30.877 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 29, 2024-07-27T03:16:30.877 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 91, 2024-07-27T03:16:30.877 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-27T03:16:30.877 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 262.75884971800002 2024-07-27T03:16:30.877 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:16:30.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.865+0000 7f1dc9876640 1 -- 172.21.15.67:0/2087596079 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f1db00b3bd0 msgr2=0x7f1db00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:30.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.865+0000 7f1dc9876640 1 --2- 172.21.15.67:0/2087596079 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f1db00b3bd0 0x7f1db00b60e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x561f96d35e80 tx=0x7f1dbc00b410 comp rx=0 tx=0).stop 2024-07-27T03:16:30.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.865+0000 7f1dc9876640 1 -- 172.21.15.67:0/2087596079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dc4108760 msgr2=0x7f1dc419f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:30.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.865+0000 7f1dc9876640 1 --2- 172.21.15.67:0/2087596079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dc4108760 0x7f1dc419f6c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1db4005cc0 tx=0x7f1db402f380 comp rx=0 tx=0).stop 2024-07-27T03:16:30.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.865+0000 7f1dc9876640 1 -- 172.21.15.67:0/2087596079 shutdown_connections 2024-07-27T03:16:30.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.865+0000 7f1dc9876640 1 --2- 172.21.15.67:0/2087596079 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f1db00b3bd0 0x7f1db00b60e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.865+0000 7f1dc9876640 1 --2- 172.21.15.67:0/2087596079 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1dc4108760 0x7f1dc419f6c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.865+0000 7f1dc9876640 1 --2- 172.21.15.67:0/2087596079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dc4107e10 0x7f1dc419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.865+0000 7f1dc9876640 1 --2- 172.21.15.67:0/2087596079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1dc41024d0 0x7f1dc4199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:30.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.865+0000 7f1dc9876640 1 -- 172.21.15.67:0/2087596079 >> 172.21.15.67:0/2087596079 conn(0x7f1dc40791a0 msgr2=0x7f1dc4104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:30.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.866+0000 7f1dc9876640 1 -- 172.21.15.67:0/2087596079 shutdown_connections 2024-07-27T03:16:30.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:30.866+0000 7f1dc9876640 1 -- 172.21.15.67:0/2087596079 wait complete. 2024-07-27T03:16:31.608 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.b status 2024-07-27T03:16:31.880 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:16:32.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:32 smithi175 ceph-mon[37125]: pgmap v429: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-07-27T03:16:32.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:32 smithi067 ceph-mon[32814]: pgmap v429: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-07-27T03:16:32.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:32 smithi149 ceph-mon[37173]: pgmap v429: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-07-27T03:16:33.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.602+0000 7f23b00e0640 1 -- 172.21.15.67:0/748459582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23a81098b0 msgr2=0x7f23a81100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:33.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.602+0000 7f23b00e0640 1 --2- 172.21.15.67:0/748459582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23a81098b0 0x7f23a81100e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2398009ca0 tx=0x7f239802a8e0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.602+0000 7f23b00e0640 1 -- 172.21.15.67:0/748459582 shutdown_connections 2024-07-27T03:16:33.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.602+0000 7f23b00e0640 1 --2- 172.21.15.67:0/748459582 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23a81098b0 0x7f23a81100e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.602+0000 7f23b00e0640 1 --2- 172.21.15.67:0/748459582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23a8108f60 0x7f23a8109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.602+0000 7f23b00e0640 1 --2- 172.21.15.67:0/748459582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23a8103620 0x7f23a8103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.602+0000 7f23b00e0640 1 -- 172.21.15.67:0/748459582 >> 172.21.15.67:0/748459582 conn(0x7f23a80ff070 msgr2=0x7f23a81014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:33.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.602+0000 7f23b00e0640 1 -- 172.21.15.67:0/748459582 shutdown_connections 2024-07-27T03:16:33.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.602+0000 7f23b00e0640 1 -- 172.21.15.67:0/748459582 wait complete. 2024-07-27T03:16:33.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.603+0000 7f23b00e0640 1 Processor -- start 2024-07-27T03:16:33.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.603+0000 7f23b00e0640 1 -- start start 2024-07-27T03:16:33.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.603+0000 7f23b00e0640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23a8103620 0x7f23a8199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:33.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.603+0000 7f23b00e0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23a8108f60 0x7f23a819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:33.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.603+0000 7f23b00e0640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23a81098b0 0x7f23a819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:33.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.603+0000 7f23b00e0640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f23a81135b0 con 0x7f23a8108f60 2024-07-27T03:16:33.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.603+0000 7f23b00e0640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f23a8113430 con 0x7f23a81098b0 2024-07-27T03:16:33.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.603+0000 7f23b00e0640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23a8113730 con 0x7f23a8103620 2024-07-27T03:16:33.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.604+0000 7f23ad67b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23a8108f60 0x7f23a819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:33.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.604+0000 7f23ad67b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23a8108f60 0x7f23a819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36464/0 (socket says 172.21.15.67:36464) 2024-07-27T03:16:33.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.604+0000 7f23ad67b640 1 -- 172.21.15.67:0/2781874688 learned_addr learned my addr 172.21.15.67:0/2781874688 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:33.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.604+0000 7f23ade7c640 1 --2- 172.21.15.67:0/2781874688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23a8103620 0x7f23a8199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:33.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.604+0000 7f23ae67d640 1 --2- 172.21.15.67:0/2781874688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23a81098b0 0x7f23a819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:33.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.604+0000 7f23ad67b640 1 -- 172.21.15.67:0/2781874688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23a8103620 msgr2=0x7f23a8199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:33.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.604+0000 7f23ad67b640 1 --2- 172.21.15.67:0/2781874688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23a8103620 0x7f23a8199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.604+0000 7f23ad67b640 1 -- 172.21.15.67:0/2781874688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23a81098b0 msgr2=0x7f23a819f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:33.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.604+0000 7f23ad67b640 1 --2- 172.21.15.67:0/2781874688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23a81098b0 0x7f23a819f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.604+0000 7f23ad67b640 1 -- 172.21.15.67:0/2781874688 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a819fe90 con 0x7f23a8108f60 2024-07-27T03:16:33.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.604+0000 7f23ae67d640 1 --2- 172.21.15.67:0/2781874688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23a81098b0 0x7f23a819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:16:33.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.604+0000 7f23ad67b640 1 --2- 172.21.15.67:0/2781874688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23a8108f60 0x7f23a819a4c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f239c006fd0 tx=0x7f239c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:33.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.604+0000 7f23a6ffd640 1 -- 172.21.15.67:0/2781874688 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c011830 con 0x7f23a8108f60 2024-07-27T03:16:33.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.604+0000 7f23a6ffd640 1 -- 172.21.15.67:0/2781874688 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f239c016070 con 0x7f23a8108f60 2024-07-27T03:16:33.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.604+0000 7f23b00e0640 1 -- 172.21.15.67:0/2781874688 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a81a0180 con 0x7f23a8108f60 2024-07-27T03:16:33.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.604+0000 7f23a6ffd640 1 -- 172.21.15.67:0/2781874688 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c010e00 con 0x7f23a8108f60 2024-07-27T03:16:33.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.605+0000 7f23b00e0640 1 -- 172.21.15.67:0/2781874688 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a81a68e0 con 0x7f23a8108f60 2024-07-27T03:16:33.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.607+0000 7f23b6665740 1 Processor -- start 2024-07-27T03:16:33.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.607+0000 7f23b6665740 1 -- start start 2024-07-27T03:16:33.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.607+0000 7f23a6ffd640 1 -- 172.21.15.67:0/2781874688 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f239c01b460 con 0x7f23a8108f60 2024-07-27T03:16:33.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.607+0000 7f23b6665740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b3446cf60 0x559b34812200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:33.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.607+0000 7f23b6665740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b344440a0 0x559b34812740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:33.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.607+0000 7f23b6665740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b3444c280 0x559b34812c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:33.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.607+0000 7f23b6665740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559b346fa2b0 con 0x559b344440a0 2024-07-27T03:16:33.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.607+0000 7f23b6665740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559b34788890 con 0x559b3446cf60 2024-07-27T03:16:33.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.607+0000 7f23b6665740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559b34777d90 con 0x559b3444c280 2024-07-27T03:16:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.608+0000 7f23ade7c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b344440a0 0x559b34812740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.608+0000 7f23ade7c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b344440a0 0x559b34812740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36476/0 (socket says 172.21.15.67:36476) 2024-07-27T03:16:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.608+0000 7f23ade7c640 1 -- 172.21.15.67:0/971358638 learned_addr learned my addr 172.21.15.67:0/971358638 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.608+0000 7f23ae67d640 1 --2- 172.21.15.67:0/971358638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b3446cf60 0x559b34812200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.608+0000 7f23ad67b640 1 --2- 172.21.15.67:0/971358638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b3444c280 0x559b34812c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.608+0000 7f23ade7c640 1 -- 172.21.15.67:0/971358638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b3444c280 msgr2=0x559b34812c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.608+0000 7f23ade7c640 1 --2- 172.21.15.67:0/971358638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b3444c280 0x559b34812c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.608+0000 7f23ade7c640 1 -- 172.21.15.67:0/971358638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b3446cf60 msgr2=0x559b34812200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.608+0000 7f23ade7c640 1 --2- 172.21.15.67:0/971358638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b3446cf60 0x559b34812200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.608+0000 7f23ade7c640 1 -- 172.21.15.67:0/971358638 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b347b1af0 con 0x559b344440a0 2024-07-27T03:16:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.608+0000 7f23ade7c640 1 --2- 172.21.15.67:0/971358638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b344440a0 0x559b34812740 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f23a000bd10 tx=0x7f23a00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ccea586b2705da3 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:33.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.609+0000 7f238ffff640 1 -- 172.21.15.67:0/971358638 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a0004030 con 0x559b344440a0 2024-07-27T03:16:33.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.609+0000 7f238ffff640 1 -- 172.21.15.67:0/971358638 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23a00035f0 con 0x559b344440a0 2024-07-27T03:16:33.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.609+0000 7f238ffff640 1 -- 172.21.15.67:0/971358638 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a0011770 con 0x559b344440a0 2024-07-27T03:16:33.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.609+0000 7f23a6ffd640 1 --2- 172.21.15.67:0/2781874688 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f23900b3bd0 0x7f23900b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:33.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.609+0000 7f23ae67d640 1 --2- 172.21.15.67:0/2781874688 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f23900b3bd0 0x7f23900b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:33.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.609+0000 7f23a6ffd640 1 -- 172.21.15.67:0/2781874688 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f239c0e3600 con 0x7f23a8108f60 2024-07-27T03:16:33.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.610+0000 7f23ae67d640 1 --2- 172.21.15.67:0/2781874688 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f23900b3bd0 0x7f23900b60e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2398009a60 tx=0x7f2398005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:33.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.610+0000 7f23b6665740 1 -- 172.21.15.67:0/971358638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b344440a0 msgr2=0x559b34812740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:33.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.610+0000 7f23b6665740 1 --2- 172.21.15.67:0/971358638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b344440a0 0x559b34812740 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f23a000bd10 tx=0x7f23a00107a0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.610+0000 7f23b6665740 1 -- 172.21.15.67:0/971358638 shutdown_connections 2024-07-27T03:16:33.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.610+0000 7f23b6665740 1 --2- 172.21.15.67:0/971358638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b3444c280 0x559b34812c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.610+0000 7f23b6665740 1 --2- 172.21.15.67:0/971358638 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b344440a0 0x559b34812740 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.610+0000 7f23b6665740 1 --2- 172.21.15.67:0/971358638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b3446cf60 0x559b34812200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.610+0000 7f23b6665740 1 -- 172.21.15.67:0/971358638 >> 172.21.15.67:0/971358638 conn(0x559b34461350 msgr2=0x559b348100f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:33.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.610+0000 7f23b6665740 1 -- 172.21.15.67:0/971358638 shutdown_connections 2024-07-27T03:16:33.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.610+0000 7f23b6665740 1 -- 172.21.15.67:0/971358638 wait complete. 2024-07-27T03:16:33.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.611+0000 7f23b6665740 1 -- start start 2024-07-27T03:16:33.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.611+0000 7f23b6665740 1 Processor -- start 2024-07-27T03:16:33.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.611+0000 7f23b6665740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b344440a0 0x559b348221d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:33.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.611+0000 7f23b6665740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b3444c280 0x559b34822710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:33.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.612+0000 7f23b6665740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b3446cf60 0x559b3482df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:33.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.612+0000 7f23b6665740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559b3471a7b0 con 0x559b3446cf60 2024-07-27T03:16:33.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.612+0000 7f23b6665740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559b3477bdd0 con 0x559b344440a0 2024-07-27T03:16:33.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.612+0000 7f23b6665740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559b345db5f0 con 0x559b3444c280 2024-07-27T03:16:33.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.612+0000 7f23b6665740 10 client.0 _ll_register_callbacks cb 0x559b347227e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:16:33.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.612+0000 7f23ae67d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b3446cf60 0x559b3482df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:33.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.612+0000 7f23ae67d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b3446cf60 0x559b3482df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36486/0 (socket says 172.21.15.67:36486) 2024-07-27T03:16:33.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.612+0000 7f23ae67d640 1 -- 172.21.15.67:0/1835866672 learned_addr learned my addr 172.21.15.67:0/1835866672 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:33.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.612+0000 7f23ad67b640 1 --2- 172.21.15.67:0/1835866672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b3444c280 0x559b34822710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:33.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.612+0000 7f23ade7c640 1 --2- 172.21.15.67:0/1835866672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b344440a0 0x559b348221d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:33.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.612+0000 7f23ae67d640 1 -- 172.21.15.67:0/1835866672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b3444c280 msgr2=0x559b34822710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:33.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.612+0000 7f23ae67d640 1 --2- 172.21.15.67:0/1835866672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b3444c280 0x559b34822710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.612+0000 7f23ae67d640 1 -- 172.21.15.67:0/1835866672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b344440a0 msgr2=0x559b348221d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:33.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.612+0000 7f23ae67d640 1 --2- 172.21.15.67:0/1835866672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b344440a0 0x559b348221d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.612+0000 7f23ae67d640 1 -- 172.21.15.67:0/1835866672 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b3470d620 con 0x559b3446cf60 2024-07-27T03:16:33.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.612+0000 7f23ade7c640 1 --2- 172.21.15.67:0/1835866672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b344440a0 0x559b348221d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:16:33.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.612+0000 7f23ad67b640 1 --2- 172.21.15.67:0/1835866672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b3444c280 0x559b34822710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:16:33.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.613+0000 7f23ae67d640 1 --2- 172.21.15.67:0/1835866672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b3446cf60 0x559b3482df20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f23980315a0 tx=0x7f239802a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4bab1f5070b3a84 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:33.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.613+0000 7f238d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:16:33.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.613+0000 7f238d7fa640 1 -- 172.21.15.67:0/1835866672 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23980047d0 con 0x559b3446cf60 2024-07-27T03:16:33.624 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.613+0000 7f238d7fa640 1 -- 172.21.15.67:0/1835866672 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f2398003e00 con 0x559b3446cf60 2024-07-27T03:16:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.613+0000 7f23b6665740 1 -- 172.21.15.67:0/1835866672 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23a00035f0 con 0x559b3446cf60 2024-07-27T03:16:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.613+0000 7f238d7fa640 1 -- 172.21.15.67:0/1835866672 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f23980305e0 con 0x559b3446cf60 2024-07-27T03:16:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.613+0000 7f23b6665740 10 client.15426 fetch_fsmap learned FSMap version 29 2024-07-27T03:16:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.613+0000 7f23b6665740 1 -- 172.21.15.67:0/1835866672 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b3471eaa0 con 0x559b3446cf60 2024-07-27T03:16:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.613+0000 7f238d7fa640 1 -- 172.21.15.67:0/1835866672 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23a00035f0 con 0x559b3446cf60 2024-07-27T03:16:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.614+0000 7f238d7fa640 1 -- 172.21.15.67:0/1835866672 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2398050070 con 0x559b3446cf60 2024-07-27T03:16:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.614+0000 7f23b6665740 10 client.15426 fetch_fsmap finished waiting for FSMap version 29 2024-07-27T03:16:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.614+0000 7f23b6665740 10 client.15426 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-27T03:16:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.614+0000 7f23b6665740 20 client.15426 populate_metadata read hostname 'smithi067' 2024-07-27T03:16:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.614+0000 7f23b6665740 1 --2- 172.21.15.67:0/1835866672 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x559b3442b8e0 0x559b348345d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.614+0000 7f23b6665740 4 client.15426 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:16:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.614+0000 7f23b6665740 1 -- 172.21.15.67:0/1835866672 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b3478a7e0 con 0x559b3442b8e0 2024-07-27T03:16:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.614+0000 7f23ade7c640 1 --2- 172.21.15.67:0/1835866672 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x559b3442b8e0 0x559b348345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:33.625 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.615+0000 7f23ade7c640 1 --2- 172.21.15.67:0/1835866672 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x559b3442b8e0 0x559b348345d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e7a8458a12544a6 server_cookie=d7ab008158b3d7a5 in_seq=0 out_seq=0 2024-07-27T03:16:33.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.615+0000 7f238d7fa640 10 client.15426 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:16:33.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.615+0000 7f238d7fa640 1 -- 172.21.15.67:0/1835866672 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b3478a7e0 con 0x559b3442b8e0 2024-07-27T03:16:33.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.615+0000 7f238d7fa640 10 client.15426 handle_command_reply: tid=0 2024-07-27T03:16:33.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.615+0000 7f23b6665740 1 client.15426 shutdown 2024-07-27T03:16:33.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.616+0000 7f23b6665740 1 -- 172.21.15.67:0/1835866672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b3446cf60 msgr2=0x559b3482df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:33.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.616+0000 7f23b6665740 1 --2- 172.21.15.67:0/1835866672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b3446cf60 0x559b3482df20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f23980315a0 tx=0x7f239802a8e0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.616+0000 7f23b6665740 1 -- 172.21.15.67:0/1835866672 shutdown_connections 2024-07-27T03:16:33.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.616+0000 7f23b6665740 1 --2- 172.21.15.67:0/1835866672 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x559b3442b8e0 0x559b348345d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.616+0000 7f23b6665740 1 --2- 172.21.15.67:0/1835866672 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b3446cf60 0x559b3482df20 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.616+0000 7f23b6665740 1 --2- 172.21.15.67:0/1835866672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b3444c280 0x559b34822710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.616+0000 7f238d7fa640 0 client.15426 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:16:33.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.616+0000 7f23b6665740 1 --2- 172.21.15.67:0/1835866672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b344440a0 0x559b348221d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.616+0000 7f23b6665740 1 -- 172.21.15.67:0/1835866672 >> 172.21.15.67:0/1835866672 conn(0x559b34461350 msgr2=0x559b3480e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:33.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.616+0000 7f23b6665740 1 -- 172.21.15.67:0/1835866672 shutdown_connections 2024-07-27T03:16:33.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.616+0000 7f23b6665740 1 -- 172.21.15.67:0/1835866672 wait complete. 2024-07-27T03:16:33.627 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.616+0000 7f23b6665740 20 client.15426 trim_cache size 0 max 16384 2024-07-27T03:16:33.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.634+0000 7f23b6665740 1 Processor -- start 2024-07-27T03:16:33.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.634+0000 7f23b6665740 1 -- start start 2024-07-27T03:16:33.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.634+0000 7f23b6665740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b3442b8e0 0x559b34833f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:33.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.634+0000 7f23b6665740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b344440a0 0x559b348221d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:33.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.634+0000 7f23b6665740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b3444c280 0x559b34820850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:33.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.634+0000 7f23b6665740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559b34832910 con 0x559b3442b8e0 2024-07-27T03:16:33.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.634+0000 7f23b6665740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559b34832fe0 con 0x559b344440a0 2024-07-27T03:16:33.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.634+0000 7f23b6665740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559b348336b0 con 0x559b3444c280 2024-07-27T03:16:33.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.634+0000 7f23ade7c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b3442b8e0 0x559b34833f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:33.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.634+0000 7f23ade7c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b3442b8e0 0x559b34833f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36502/0 (socket says 172.21.15.67:36502) 2024-07-27T03:16:33.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.634+0000 7f23ade7c640 1 -- 172.21.15.67:0/4111241774 learned_addr learned my addr 172.21.15.67:0/4111241774 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:33.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.635+0000 7f23ad67b640 1 --2- 172.21.15.67:0/4111241774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b344440a0 0x559b348221d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:33.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.635+0000 7f23ade7c640 1 -- 172.21.15.67:0/4111241774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b3444c280 msgr2=0x559b34820850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:33.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.635+0000 7f23ae67d640 1 --2- 172.21.15.67:0/4111241774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b3444c280 0x559b34820850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:33.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.635+0000 7f23ade7c640 1 --2- 172.21.15.67:0/4111241774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b3444c280 0x559b34820850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.635+0000 7f23ade7c640 1 -- 172.21.15.67:0/4111241774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b344440a0 msgr2=0x559b348221d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:33.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.635+0000 7f23ade7c640 1 --2- 172.21.15.67:0/4111241774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b344440a0 0x559b348221d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.635+0000 7f23ade7c640 1 -- 172.21.15.67:0/4111241774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b344d8540 con 0x559b3442b8e0 2024-07-27T03:16:33.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.635+0000 7f23ae67d640 1 --2- 172.21.15.67:0/4111241774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b3444c280 0x559b34820850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:16:33.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.635+0000 7f23ade7c640 1 --2- 172.21.15.67:0/4111241774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b3442b8e0 0x559b34833f00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f23a0011010 tx=0x7f23a001d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabf9c992888efaf server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:33.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.635+0000 7f238f7fe640 1 -- 172.21.15.67:0/4111241774 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a00054a0 con 0x559b3442b8e0 2024-07-27T03:16:33.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.635+0000 7f238f7fe640 1 -- 172.21.15.67:0/4111241774 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23a0005640 con 0x559b3442b8e0 2024-07-27T03:16:33.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.635+0000 7f238f7fe640 1 -- 172.21.15.67:0/4111241774 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a00116a0 con 0x559b3442b8e0 2024-07-27T03:16:33.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.636+0000 7f23b6665740 1 -- 172.21.15.67:0/4111241774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b3442b8e0 msgr2=0x559b34833f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:33.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.636+0000 7f23b6665740 1 --2- 172.21.15.67:0/4111241774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b3442b8e0 0x559b34833f00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f23a0011010 tx=0x7f23a001d040 comp rx=0 tx=0).stop 2024-07-27T03:16:33.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.636+0000 7f23b6665740 1 -- 172.21.15.67:0/4111241774 shutdown_connections 2024-07-27T03:16:33.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.636+0000 7f23b6665740 1 --2- 172.21.15.67:0/4111241774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b3444c280 0x559b34820850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.636+0000 7f23b6665740 1 --2- 172.21.15.67:0/4111241774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b344440a0 0x559b348221d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.636+0000 7f23b6665740 1 --2- 172.21.15.67:0/4111241774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b3442b8e0 0x559b34833f00 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.636+0000 7f23b6665740 1 -- 172.21.15.67:0/4111241774 >> 172.21.15.67:0/4111241774 conn(0x559b34461350 msgr2=0x559b348172c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.636+0000 7f23b6665740 1 -- 172.21.15.67:0/4111241774 shutdown_connections 2024-07-27T03:16:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.636+0000 7f23b6665740 1 -- 172.21.15.67:0/4111241774 wait complete. 2024-07-27T03:16:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.636+0000 7f23b6665740 1 -- start start 2024-07-27T03:16:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.636+0000 7f23b6665740 1 Processor -- start 2024-07-27T03:16:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.637+0000 7f23b6665740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b3442b8e0 0x559b34837020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.637+0000 7f23b6665740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b344440a0 0x559b348375e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.637+0000 7f23b6665740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b3444c280 0x559b3483b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.637+0000 7f23b6665740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559b34834450 con 0x559b3444c280 2024-07-27T03:16:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.637+0000 7f23b6665740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559b34833d80 con 0x559b3442b8e0 2024-07-27T03:16:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.637+0000 7f23b6665740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559b34834b20 con 0x559b344440a0 2024-07-27T03:16:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.637+0000 7f23b6665740 10 client.0 _ll_register_callbacks cb 0x7f2398031470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:16:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.637+0000 7f23ae67d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b3444c280 0x559b3483b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.637+0000 7f23ae67d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b3444c280 0x559b3483b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36518/0 (socket says 172.21.15.67:36518) 2024-07-27T03:16:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.637+0000 7f23ae67d640 1 -- 172.21.15.67:0/2523862000 learned_addr learned my addr 172.21.15.67:0/2523862000 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.637+0000 7f23ade7c640 1 --2- 172.21.15.67:0/2523862000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b3442b8e0 0x559b34837020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.637+0000 7f23ad67b640 1 --2- 172.21.15.67:0/2523862000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b344440a0 0x559b348375e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.637+0000 7f23ae67d640 1 -- 172.21.15.67:0/2523862000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b344440a0 msgr2=0x559b348375e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.637+0000 7f23ae67d640 1 --2- 172.21.15.67:0/2523862000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b344440a0 0x559b348375e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.637+0000 7f23ae67d640 1 -- 172.21.15.67:0/2523862000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b3442b8e0 msgr2=0x559b34837020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.637+0000 7f23ae67d640 1 --2- 172.21.15.67:0/2523862000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b3442b8e0 0x559b34837020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.637+0000 7f23ae67d640 1 -- 172.21.15.67:0/2523862000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b3483b700 con 0x559b3444c280 2024-07-27T03:16:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.637+0000 7f23ade7c640 1 --2- 172.21.15.67:0/2523862000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b3442b8e0 0x559b34837020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:16:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.638+0000 7f23ae67d640 1 --2- 172.21.15.67:0/2523862000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b3444c280 0x559b3483b090 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f239802ae00 tx=0x7f2398004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fda2a94d8dcfd95 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.638+0000 7f238d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:16:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.638+0000 7f238d7fa640 1 -- 172.21.15.67:0/2523862000 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398004b40 con 0x559b3444c280 2024-07-27T03:16:33.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.638+0000 7f238d7fa640 1 -- 172.21.15.67:0/2523862000 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f2398004ce0 con 0x559b3444c280 2024-07-27T03:16:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.638+0000 7f23b6665740 1 -- 172.21.15.67:0/2523862000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23a0005640 con 0x559b3444c280 2024-07-27T03:16:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.638+0000 7f238d7fa640 1 -- 172.21.15.67:0/2523862000 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2398067050 con 0x559b3444c280 2024-07-27T03:16:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.638+0000 7f23b6665740 10 client.15432 fetch_fsmap learned FSMap version 29 2024-07-27T03:16:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.638+0000 7f23b6665740 1 -- 172.21.15.67:0/2523862000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b3483ca30 con 0x559b3444c280 2024-07-27T03:16:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.638+0000 7f238d7fa640 1 -- 172.21.15.67:0/2523862000 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23a0005640 con 0x559b3444c280 2024-07-27T03:16:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.639+0000 7f238d7fa640 1 -- 172.21.15.67:0/2523862000 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2398050070 con 0x559b3444c280 2024-07-27T03:16:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.639+0000 7f23b6665740 10 client.15432 fetch_fsmap finished waiting for FSMap version 29 2024-07-27T03:16:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.639+0000 7f23b6665740 10 client.15432 resolve_mds: resolved name 'b' to daemon mds.b 2024-07-27T03:16:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.639+0000 7f23b6665740 20 client.15432 populate_metadata read hostname 'smithi067' 2024-07-27T03:16:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.639+0000 7f23b6665740 1 --2- 172.21.15.67:0/2523862000 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x559b3446cf60 0x559b348a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.639+0000 7f23b6665740 4 client.15432 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "status"}] 2024-07-27T03:16:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.639+0000 7f23b6665740 1 -- 172.21.15.67:0/2523862000 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "status"}) -- 0x559b34786280 con 0x559b3446cf60 2024-07-27T03:16:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.639+0000 7f23ade7c640 1 --2- 172.21.15.67:0/2523862000 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x559b3446cf60 0x559b348a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:33.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.639+0000 7f238d7fa640 10 client.15432 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:16:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.639+0000 7f23ade7c640 1 --2- 172.21.15.67:0/2523862000 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x559b3446cf60 0x559b348a04f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7ca8f19a7cb29c4 server_cookie=88db86b86567b2c2 in_seq=0 out_seq=0 2024-07-27T03:16:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.640+0000 7f238d7fa640 1 -- 172.21.15.67:0/2523862000 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559b34786280 con 0x559b3446cf60 2024-07-27T03:16:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.640+0000 7f238d7fa640 10 client.15432 handle_command_reply: tid=0 2024-07-27T03:16:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.640+0000 7f23b6665740 1 client.15432 shutdown 2024-07-27T03:16:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.640+0000 7f23b6665740 1 -- 172.21.15.67:0/2523862000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b3444c280 msgr2=0x559b3483b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.640+0000 7f23b6665740 1 --2- 172.21.15.67:0/2523862000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b3444c280 0x559b3483b090 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f239802ae00 tx=0x7f2398004500 comp rx=0 tx=0).stop 2024-07-27T03:16:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.640+0000 7f23b6665740 1 -- 172.21.15.67:0/2523862000 shutdown_connections 2024-07-27T03:16:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.640+0000 7f23b6665740 1 --2- 172.21.15.67:0/2523862000 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x559b3446cf60 0x559b348a04f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.640+0000 7f23b6665740 1 --2- 172.21.15.67:0/2523862000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559b3444c280 0x559b3483b090 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.640+0000 7f23b6665740 1 --2- 172.21.15.67:0/2523862000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559b344440a0 0x559b348375e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.640+0000 7f238d7fa640 0 client.15432 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:16:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.640+0000 7f23b6665740 1 --2- 172.21.15.67:0/2523862000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559b3442b8e0 0x559b34837020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.640+0000 7f23b6665740 1 -- 172.21.15.67:0/2523862000 >> 172.21.15.67:0/2523862000 conn(0x559b34461350 msgr2=0x559b3480e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:33.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.640+0000 7f23b6665740 1 -- 172.21.15.67:0/2523862000 shutdown_connections 2024-07-27T03:16:33.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.640+0000 7f23b6665740 1 -- 172.21.15.67:0/2523862000 wait complete. 2024-07-27T03:16:33.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.640+0000 7f23b6665740 20 client.15432 trim_cache size 0 max 16384 2024-07-27T03:16:33.648 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:16:33.648 INFO:teuthology.orchestra.run.smithi067.stdout: "cluster_fsid": "c0782a00-4bc4-11ef-bcc7-c7b262605968", 2024-07-27T03:16:33.648 INFO:teuthology.orchestra.run.smithi067.stdout: "whoami": 0, 2024-07-27T03:16:33.648 INFO:teuthology.orchestra.run.smithi067.stdout: "id": 24620, 2024-07-27T03:16:33.648 INFO:teuthology.orchestra.run.smithi067.stdout: "want_state": "up:active", 2024-07-27T03:16:33.648 INFO:teuthology.orchestra.run.smithi067.stdout: "state": "up:active", 2024-07-27T03:16:33.648 INFO:teuthology.orchestra.run.smithi067.stdout: "fs_name": "cephfs", 2024-07-27T03:16:33.649 INFO:teuthology.orchestra.run.smithi067.stdout: "rank_uptime": 239.07710784299999, 2024-07-27T03:16:33.649 INFO:teuthology.orchestra.run.smithi067.stdout: "mdsmap_epoch": 29, 2024-07-27T03:16:33.649 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch": 91, 2024-07-27T03:16:33.649 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_epoch_barrier": 89, 2024-07-27T03:16:33.649 INFO:teuthology.orchestra.run.smithi067.stdout: "uptime": 253.144354862 2024-07-27T03:16:33.649 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:16:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.641+0000 7f23b00e0640 1 -- 172.21.15.67:0/2781874688 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f23900b3bd0 msgr2=0x7f23900b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.641+0000 7f23b00e0640 1 --2- 172.21.15.67:0/2781874688 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f23900b3bd0 0x7f23900b60e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2398009a60 tx=0x7f2398005c50 comp rx=0 tx=0).stop 2024-07-27T03:16:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.641+0000 7f23b00e0640 1 -- 172.21.15.67:0/2781874688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23a8108f60 msgr2=0x7f23a819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.641+0000 7f23b00e0640 1 --2- 172.21.15.67:0/2781874688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23a8108f60 0x7f23a819a4c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f239c006fd0 tx=0x7f239c00b8e0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.641+0000 7f23b00e0640 1 -- 172.21.15.67:0/2781874688 shutdown_connections 2024-07-27T03:16:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.641+0000 7f23b00e0640 1 --2- 172.21.15.67:0/2781874688 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f23900b3bd0 0x7f23900b60e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.641+0000 7f23b00e0640 1 --2- 172.21.15.67:0/2781874688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23a81098b0 0x7f23a819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.641+0000 7f23b00e0640 1 --2- 172.21.15.67:0/2781874688 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f23a8108f60 0x7f23a819a4c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.641+0000 7f23b00e0640 1 --2- 172.21.15.67:0/2781874688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f23a8103620 0x7f23a8199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.641+0000 7f23b00e0640 1 -- 172.21.15.67:0/2781874688 >> 172.21.15.67:0/2781874688 conn(0x7f23a80ff070 msgr2=0x7f23a8101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:33.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.641+0000 7f23b00e0640 1 -- 172.21.15.67:0/2781874688 shutdown_connections 2024-07-27T03:16:33.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:33.641+0000 7f23b00e0640 1 -- 172.21.15.67:0/2781874688 wait complete. 2024-07-27T03:16:34.406 INFO:teuthology.run_tasks:Running task sequential... 2024-07-27T03:16:34.417 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-07-27T03:16:34.417 INFO:teuthology.task.sequential:In sequential, running task print... 2024-07-27T03:16:34.417 INFO:teuthology.task.print:Enabling mgr modules 2024-07-27T03:16:34.417 INFO:teuthology.run_tasks:Running task kclient... 2024-07-27T03:16:34.446 INFO:tasks.kclient:Mounting kernel clients... 2024-07-27T03:16:34.446 INFO:tasks.kclient:config is {'client.0': {}} 2024-07-27T03:16:34.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:16:34.447 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-07-27T03:16:34.465 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-07-27T03:16:34.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:16:34.466 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-07-27T03:16:34.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:16:34.535 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2024-07-27T03:16:34.599 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:34 smithi067 ceph-mon[32814]: pgmap v430: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:34.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:34 smithi175 ceph-mon[37125]: pgmap v430: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:34.644 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-07-27T03:16:34.644 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-07-27T03:16:34.645 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-27T03:16:34.645 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs ls 2024-07-27T03:16:34.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:34 smithi149 ceph-mon[37173]: pgmap v430: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:16:36.409 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:36 smithi067 ceph-mon[32814]: pgmap v431: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:16:36.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:36 smithi175 ceph-mon[37125]: pgmap v431: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:16:36.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:36 smithi149 ceph-mon[37173]: pgmap v431: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:16:36.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.780+0000 7f5a7c4eb640 1 -- 172.21.15.67:0/2002587304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a74102550 msgr2=0x7f5a74102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:36.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.780+0000 7f5a7c4eb640 1 --2- 172.21.15.67:0/2002587304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a74102550 0x7f5a74102960 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009900 tx=0x7f5a7002a410 comp rx=0 tx=0).stop 2024-07-27T03:16:36.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.780+0000 7f5a7c4eb640 1 -- 172.21.15.67:0/2002587304 shutdown_connections 2024-07-27T03:16:36.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.780+0000 7f5a7c4eb640 1 --2- 172.21.15.67:0/2002587304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a74108790 0x7f5a7410efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:36.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.780+0000 7f5a7c4eb640 1 --2- 172.21.15.67:0/2002587304 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a74107e40 0x7f5a74108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:36.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.780+0000 7f5a7c4eb640 1 --2- 172.21.15.67:0/2002587304 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a74102550 0x7f5a74102960 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:36.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.780+0000 7f5a7c4eb640 1 -- 172.21.15.67:0/2002587304 >> 172.21.15.67:0/2002587304 conn(0x7f5a740791d0 msgr2=0x7f5a740795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:36.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.780+0000 7f5a7c4eb640 1 -- 172.21.15.67:0/2002587304 shutdown_connections 2024-07-27T03:16:36.783 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.780+0000 7f5a7c4eb640 1 -- 172.21.15.67:0/2002587304 wait complete. 2024-07-27T03:16:36.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.781+0000 7f5a7c4eb640 1 Processor -- start 2024-07-27T03:16:36.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.781+0000 7f5a7c4eb640 1 -- start start 2024-07-27T03:16:36.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.781+0000 7f5a7c4eb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a74102550 0x7f5a74199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:36.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.781+0000 7f5a7c4eb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a74107e40 0x7f5a7419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:36.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.782+0000 7f5a7c4eb640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a74108790 0x7f5a7419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:36.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.782+0000 7f5a7c4eb640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5a74113560 con 0x7f5a74102550 2024-07-27T03:16:36.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.782+0000 7f5a7c4eb640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5a741133e0 con 0x7f5a74108790 2024-07-27T03:16:36.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.782+0000 7f5a7c4eb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5a741136e0 con 0x7f5a74107e40 2024-07-27T03:16:36.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.782+0000 7f5a7a287640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a74102550 0x7f5a74199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:36.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.782+0000 7f5a7a287640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a74102550 0x7f5a74199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33430/0 (socket says 172.21.15.67:33430) 2024-07-27T03:16:36.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.782+0000 7f5a7a287640 1 -- 172.21.15.67:0/1612640843 learned_addr learned my addr 172.21.15.67:0/1612640843 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:36.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.782+0000 7f5a79a86640 1 --2- 172.21.15.67:0/1612640843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a74107e40 0x7f5a7419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:36.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.782+0000 7f5a7aa88640 1 --2- 172.21.15.67:0/1612640843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a74108790 0x7f5a7419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:36.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.782+0000 7f5a7a287640 1 -- 172.21.15.67:0/1612640843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a74107e40 msgr2=0x7f5a7419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:36.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.782+0000 7f5a7a287640 1 --2- 172.21.15.67:0/1612640843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a74107e40 0x7f5a7419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:36.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.782+0000 7f5a7a287640 1 -- 172.21.15.67:0/1612640843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a74108790 msgr2=0x7f5a7419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:36.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.782+0000 7f5a7a287640 1 --2- 172.21.15.67:0/1612640843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a74108790 0x7f5a7419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:36.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.782+0000 7f5a7a287640 1 -- 172.21.15.67:0/1612640843 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7419fea0 con 0x7f5a74102550 2024-07-27T03:16:36.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.782+0000 7f5a7a287640 1 --2- 172.21.15.67:0/1612640843 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a74102550 0x7f5a74199f30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5a70006010 tx=0x7f5a700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:36.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.782+0000 7f5a6b7fe640 1 -- 172.21.15.67:0/1612640843 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7003f030 con 0x7f5a74102550 2024-07-27T03:16:36.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.782+0000 7f5a6b7fe640 1 -- 172.21.15.67:0/1612640843 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a70003800 con 0x7f5a74102550 2024-07-27T03:16:36.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.782+0000 7f5a7c4eb640 1 -- 172.21.15.67:0/1612640843 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a741a0130 con 0x7f5a74102550 2024-07-27T03:16:36.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.782+0000 7f5a6b7fe640 1 -- 172.21.15.67:0/1612640843 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70035070 con 0x7f5a74102550 2024-07-27T03:16:36.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.783+0000 7f5a7c4eb640 1 -- 172.21.15.67:0/1612640843 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a741a68f0 con 0x7f5a74102550 2024-07-27T03:16:36.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.783+0000 7f5a7c4eb640 1 -- 172.21.15.67:0/1612640843 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a74103780 con 0x7f5a74102550 2024-07-27T03:16:36.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.785+0000 7f5a6b7fe640 1 -- 172.21.15.67:0/1612640843 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f5a70003cb0 con 0x7f5a74102550 2024-07-27T03:16:36.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.786+0000 7f5a6b7fe640 1 --2- 172.21.15.67:0/1612640843 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5a600b3bd0 0x7f5a600b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:36.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.786+0000 7f5a79a86640 1 --2- 172.21.15.67:0/1612640843 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5a600b3bd0 0x7f5a600b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:36.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.786+0000 7f5a6b7fe640 1 -- 172.21.15.67:0/1612640843 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f5a70104b10 con 0x7f5a74102550 2024-07-27T03:16:36.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.786+0000 7f5a79a86640 1 --2- 172.21.15.67:0/1612640843 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5a600b3bd0 0x7f5a600b60e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5a74079860 tx=0x7f5a64009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:36.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:36.789+0000 7f5a6b7fe640 1 -- 172.21.15.67:0/1612640843 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f5a700cbdb0 con 0x7f5a74102550 2024-07-27T03:16:37.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:37.015+0000 7f5a7c4eb640 1 -- 172.21.15.67:0/1612640843 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5a7419b060 con 0x7f5a74102550 2024-07-27T03:16:37.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:37.016+0000 7f5a6b7fe640 1 -- 172.21.15.67:0/1612640843 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+88 (secure 0 0 0) 0x7f5a7419b060 con 0x7f5a74102550 2024-07-27T03:16:37.018 INFO:teuthology.orchestra.run.smithi067.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-07-27T03:16:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:37.018+0000 7f5a7c4eb640 1 -- 172.21.15.67:0/1612640843 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5a600b3bd0 msgr2=0x7f5a600b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:37.018+0000 7f5a7c4eb640 1 --2- 172.21.15.67:0/1612640843 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5a600b3bd0 0x7f5a600b60e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5a74079860 tx=0x7f5a64009110 comp rx=0 tx=0).stop 2024-07-27T03:16:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:37.018+0000 7f5a7c4eb640 1 -- 172.21.15.67:0/1612640843 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a74102550 msgr2=0x7f5a74199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:37.018+0000 7f5a7c4eb640 1 --2- 172.21.15.67:0/1612640843 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a74102550 0x7f5a74199f30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5a70006010 tx=0x7f5a700042f0 comp rx=0 tx=0).stop 2024-07-27T03:16:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:37.018+0000 7f5a7c4eb640 1 -- 172.21.15.67:0/1612640843 shutdown_connections 2024-07-27T03:16:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:37.018+0000 7f5a7c4eb640 1 --2- 172.21.15.67:0/1612640843 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5a600b3bd0 0x7f5a600b60e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:37.018+0000 7f5a7c4eb640 1 --2- 172.21.15.67:0/1612640843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a74108790 0x7f5a7419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:37.018+0000 7f5a7c4eb640 1 --2- 172.21.15.67:0/1612640843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a74107e40 0x7f5a7419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:37.018+0000 7f5a7c4eb640 1 --2- 172.21.15.67:0/1612640843 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5a74102550 0x7f5a74199f30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:37.018+0000 7f5a7c4eb640 1 -- 172.21.15.67:0/1612640843 >> 172.21.15.67:0/1612640843 conn(0x7f5a740791d0 msgr2=0x7f5a74100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:37.018+0000 7f5a7c4eb640 1 -- 172.21.15.67:0/1612640843 shutdown_connections 2024-07-27T03:16:37.021 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:37.018+0000 7f5a7c4eb640 1 -- 172.21.15.67:0/1612640843 wait complete. 2024-07-27T03:16:37.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:37 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1612640843' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-27T03:16:37.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:37 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1612640843' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-27T03:16:37.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:37 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1612640843' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-07-27T03:16:37.860 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-07-27T03:16:37.861 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-07-27T03:16:37.861 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi067.front.sepia.ceph.com 2024-07-27T03:16:37.861 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-07-27T03:16:37.861 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-07-27T03:16:37.861 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-07-27T03:16:37.861 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-07-27T03:16:37.861 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-07-27T03:16:37.861 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-07-27T03:16:37.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:16:37.862 DEBUG:teuthology.orchestra.run.smithi067:> ip addr 2024-07-27T03:16:37.880 INFO:teuthology.orchestra.run.smithi067.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-07-27T03:16:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-07-27T03:16:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: inet 127.0.0.1/8 scope host lo 2024-07-27T03:16:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-07-27T03:16:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 ::1/128 scope host 2024-07-27T03:16:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-07-27T03:16:37.880 INFO:teuthology.orchestra.run.smithi067.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-27T03:16:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7e brd ff:ff:ff:ff:ff:ff 2024-07-27T03:16:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f0 2024-07-27T03:16:37.880 INFO:teuthology.orchestra.run.smithi067.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-27T03:16:37.880 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:a8:b4:7f brd ff:ff:ff:ff:ff:ff 2024-07-27T03:16:37.881 INFO:teuthology.orchestra.run.smithi067.stdout: altname enp7s0f1 2024-07-27T03:16:37.881 INFO:teuthology.orchestra.run.smithi067.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-07-27T03:16:37.881 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0a brd ff:ff:ff:ff:ff:ff 2024-07-27T03:16:37.881 INFO:teuthology.orchestra.run.smithi067.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-07-27T03:16:37.881 INFO:teuthology.orchestra.run.smithi067.stdout: link/ether 0c:c4:7a:b6:fc:0b brd ff:ff:ff:ff:ff:ff 2024-07-27T03:16:37.881 INFO:teuthology.orchestra.run.smithi067.stdout: inet 172.21.15.67/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-07-27T03:16:37.881 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft 41992sec preferred_lft 41992sec 2024-07-27T03:16:37.881 INFO:teuthology.orchestra.run.smithi067.stdout: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-07-27T03:16:37.881 INFO:teuthology.orchestra.run.smithi067.stdout: valid_lft forever preferred_lft forever 2024-07-27T03:16:37.881 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-07-27T03:16:37.882 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-27T03:16:37.882 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-07-27T03:16:37.882 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add name ceph-brx type bridge 2024-07-27T03:16:37.882 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr flush dev ceph-brx 2024-07-27T03:16:37.882 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set ceph-brx up 2024-07-27T03:16:37.882 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-07-27T03:16:37.882 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-07-27T03:16:37.974 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-27T03:16:38.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-27T03:16:38.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:16:38.030 DEBUG:teuthology.orchestra.run.smithi067:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-07-27T03:16:38.054 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-07-27T03:16:38.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:16:38.056 DEBUG:teuthology.orchestra.run.smithi067:> route 2024-07-27T03:16:38.113 INFO:teuthology.orchestra.run.smithi067.stdout:Kernel IP routing table 2024-07-27T03:16:38.113 INFO:teuthology.orchestra.run.smithi067.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-07-27T03:16:38.113 INFO:teuthology.orchestra.run.smithi067.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-07-27T03:16:38.113 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-07-27T03:16:38.113 INFO:teuthology.orchestra.run.smithi067.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-07-27T03:16:38.113 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-27T03:16:38.114 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-07-27T03:16:38.114 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-07-27T03:16:38.114 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-07-27T03:16:38.114 DEBUG:teuthology.orchestra.run.smithi067:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-07-27T03:16:38.114 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-07-27T03:16:38.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-27T03:16:38.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-27T03:16:38.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:16:38.244 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-07-27T03:16:38.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:16:38.261 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list-id 2024-07-27T03:16:38.317 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-27T03:16:38.317 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-07-27T03:16:38.317 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-27T03:16:38.317 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-07-27T03:16:38.317 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-07-27T03:16:38.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-27T03:16:38.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-27T03:16:38.439 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-07-27T03:16:38.439 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-27T03:16:38.439 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-07-27T03:16:38.439 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-07-27T03:16:38.439 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-07-27T03:16:38.439 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-07-27T03:16:38.439 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-07-27T03:16:38.439 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-07-27T03:16:38.439 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-07-27T03:16:38.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:38 smithi067 ceph-mon[32814]: pgmap v432: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:38.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-27T03:16:38.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-27T03:16:38.559 DEBUG:teuthology.orchestra.run.smithi067:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-07-27T03:16:38.559 DEBUG:teuthology.orchestra.run.smithi067:> set -e 2024-07-27T03:16:38.559 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set brx.0 up 2024-07-27T03:16:38.559 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link set dev brx.0 master ceph-brx 2024-07-27T03:16:38.559 DEBUG:teuthology.orchestra.run.smithi067:> ') 2024-07-27T03:16:38.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:38 smithi175 ceph-mon[37125]: pgmap v432: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:38.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-27T03:16:38.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-27T03:16:38.681 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-27T03:16:38.681 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-07-27T03:16:38.697 INFO:teuthology.orchestra.run.smithi067.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-07-27T03:16:38.697 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-07-27T03:16:38.697 DEBUG:teuthology.orchestra.run.smithi067:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-07-27T03:16:38.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:38 smithi149 ceph-mon[37173]: pgmap v432: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:16:38.753 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-07-27T03:16:38.753 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-07-27T03:16:38.753 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/15d1cbe2-ea58-4f14-900a-340b84819fc4 2024-07-27T03:16:38.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:16:38.754 DEBUG:teuthology.orchestra.run.smithi067:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/15d1cbe2-ea58-4f14-900a-340b84819fc4 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-07-27T03:16:38.848 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-07-27T03:16:38.848 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-07-27T03:16:38.848 INFO:teuthology.orchestra.run.smithi067.stdout:invalid new device string format 2024-07-27T03:16:38.985 INFO:teuthology.orchestra.run.smithi067.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-07-27T03:16:38.985 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-07-27T03:16:38.985 INFO:teuthology.orchestra.run.smithi067.stdout:invalid new device string format 2024-07-27T03:16:38.985 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: resolved to: "172.21.15.67:3300,172.21.15.149:3300,172.21.15.175:3300" 2024-07-27T03:16:38.985 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.67:3300,172.21.15.149:3300,172.21.15.175:3300:/volumes/qa/sv_1/15d1cbe2-ea58-4f14-900a-340b84819fc4 2024-07-27T03:16:38.985 INFO:teuthology.orchestra.run.smithi067.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=c0782a00-4bc4-11ef-bcc7-c7b262605968" will pass to kernel 2024-07-27T03:16:38.986 INFO:teuthology.orchestra.run.smithi067.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-07-27T03:16:38.986 INFO:teuthology.orchestra.run.smithi067.stdout: You just mounted a file system that supports labels which does not 2024-07-27T03:16:38.986 INFO:teuthology.orchestra.run.smithi067.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-07-27T03:16:38.986 INFO:teuthology.orchestra.run.smithi067.stdout: applications will generate AVC messages and not be allowed access to 2024-07-27T03:16:38.986 INFO:teuthology.orchestra.run.smithi067.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-07-27T03:16:38.987 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-07-27T03:16:38.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:16:38.988 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-07-27T03:16:39.014 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-27T03:16:39.105 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-27T03:16:39.108 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-27T03:16:39.114 INFO:teuthology.orchestra.run.smithi067.stdout:client25109 2024-07-27T03:16:39.114 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-27T03:16:39.114 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-27T03:16:39.132 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-27T03:16:39.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-27T03:16:39.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-27T03:16:39.231 INFO:teuthology.orchestra.run.smithi067.stdout:client25109 2024-07-27T03:16:39.232 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-07-27T03:16:39.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-07-27T03:16:39.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-27T03:16:39.291 INFO:teuthology.orchestra.run.smithi067.stdout:client25109 2024-07-27T03:16:39.291 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968.client25109/status') 2024-07-27T03:16:39.341 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-07-27T03:16:39.352 INFO:teuthology.orchestra.run.smithi067.stdout:instance: client.25109 (3)172.21.15.67:0/1466976925 2024-07-27T03:16:39.352 INFO:teuthology.orchestra.run.smithi067.stdout:blocklisted: false 2024-07-27T03:16:39.352 INFO:teuthology.orchestra.run.smithi067.stderr:0+1 records in 2024-07-27T03:16:39.352 INFO:teuthology.orchestra.run.smithi067.stderr:0+1 records out 2024-07-27T03:16:39.352 INFO:teuthology.orchestra.run.smithi067.stderr:71 bytes copied, 3.3161e-05 s, 2.1 MB/s 2024-07-27T03:16:39.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-07-27T03:16:39.364 INFO:teuthology.run_tasks:Running task exec... 2024-07-27T03:16:39.374 INFO:teuthology.task.exec:Executing custom commands... 2024-07-27T03:16:39.374 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-07-27T03:16:39.375 DEBUG:teuthology.orchestra.run.smithi067:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-07-27T03:16:39.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.488+0000 7fcbc8540640 1 -- 172.21.15.67:0/1244889850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbc0108e80 msgr2=0x7fcbc010b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:39.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.488+0000 7fcbc8540640 1 --2- 172.21.15.67:0/1244889850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbc0108e80 0x7fcbc010b390 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009ca0 tx=0x7fcbb802a8c0 comp rx=0 tx=0).stop 2024-07-27T03:16:39.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.488+0000 7fcbc8540640 1 -- 172.21.15.67:0/1244889850 shutdown_connections 2024-07-27T03:16:39.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.488+0000 7fcbc8540640 1 --2- 172.21.15.67:0/1244889850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbc0108e80 0x7fcbc010b390 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:39.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.488+0000 7fcbc8540640 1 --2- 172.21.15.67:0/1244889850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbc01062c0 0x7fcbc0108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:39.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.488+0000 7fcbc8540640 1 --2- 172.21.15.67:0/1244889850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbc006bf20 0x7fcbc0105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:39.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.488+0000 7fcbc8540640 1 -- 172.21.15.67:0/1244889850 >> 172.21.15.67:0/1244889850 conn(0x7fcbc00fd9a0 msgr2=0x7fcbc00ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:39.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.488+0000 7fcbc8540640 1 -- 172.21.15.67:0/1244889850 shutdown_connections 2024-07-27T03:16:39.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.488+0000 7fcbc8540640 1 -- 172.21.15.67:0/1244889850 wait complete. 2024-07-27T03:16:39.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.489+0000 7fcbc8540640 1 Processor -- start 2024-07-27T03:16:39.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.489+0000 7fcbc8540640 1 -- start start 2024-07-27T03:16:39.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.489+0000 7fcbc8540640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbc006bf20 0x7fcbc0199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:39.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.489+0000 7fcbc8540640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbc01062c0 0x7fcbc0199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:39.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.489+0000 7fcbc8540640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbc0108e80 0x7fcbc019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:39.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.489+0000 7fcbc8540640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc010edf0 con 0x7fcbc0108e80 2024-07-27T03:16:39.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.489+0000 7fcbc8540640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc010ec70 con 0x7fcbc01062c0 2024-07-27T03:16:39.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.489+0000 7fcbc8540640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc010ef70 con 0x7fcbc006bf20 2024-07-27T03:16:39.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.489+0000 7fcbc6add640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbc0108e80 0x7fcbc019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:39.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.489+0000 7fcbc6add640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbc0108e80 0x7fcbc019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33440/0 (socket says 172.21.15.67:33440) 2024-07-27T03:16:39.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.489+0000 7fcbc6add640 1 -- 172.21.15.67:0/3257496395 learned_addr learned my addr 172.21.15.67:0/3257496395 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:39.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.489+0000 7fcbc62dc640 1 --2- 172.21.15.67:0/3257496395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbc006bf20 0x7fcbc0199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:39.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.489+0000 7fcbc6add640 1 -- 172.21.15.67:0/3257496395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbc006bf20 msgr2=0x7fcbc0199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:39.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.489+0000 7fcbc6add640 1 --2- 172.21.15.67:0/3257496395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbc006bf20 0x7fcbc0199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:39.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.489+0000 7fcbc5adb640 1 --2- 172.21.15.67:0/3257496395 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbc01062c0 0x7fcbc0199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:39.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.489+0000 7fcbc6add640 1 -- 172.21.15.67:0/3257496395 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbc01062c0 msgr2=0x7fcbc0199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:39.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.489+0000 7fcbc6add640 1 --2- 172.21.15.67:0/3257496395 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbc01062c0 0x7fcbc0199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:39.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.489+0000 7fcbc6add640 1 -- 172.21.15.67:0/3257496395 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc019b330 con 0x7fcbc0108e80 2024-07-27T03:16:39.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.489+0000 7fcbc5adb640 1 --2- 172.21.15.67:0/3257496395 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbc01062c0 0x7fcbc0199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:16:39.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.490+0000 7fcbc6add640 1 --2- 172.21.15.67:0/3257496395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbc0108e80 0x7fcbc019aba0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8003ba0 tx=0x7fcbb8003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:39.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.490+0000 7fcbb77fe640 1 -- 172.21.15.67:0/3257496395 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb80317c0 con 0x7fcbc0108e80 2024-07-27T03:16:39.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.490+0000 7fcbb77fe640 1 -- 172.21.15.67:0/3257496395 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbb80044f0 con 0x7fcbc0108e80 2024-07-27T03:16:39.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.490+0000 7fcbb77fe640 1 -- 172.21.15.67:0/3257496395 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb802fd00 con 0x7fcbc0108e80 2024-07-27T03:16:39.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.490+0000 7fcbc8540640 1 -- 172.21.15.67:0/3257496395 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc019b5c0 con 0x7fcbc0108e80 2024-07-27T03:16:39.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.490+0000 7fcbc8540640 1 -- 172.21.15.67:0/3257496395 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc01a1bc0 con 0x7fcbc0108e80 2024-07-27T03:16:39.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.491+0000 7fcbc8540640 1 -- 172.21.15.67:0/3257496395 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc00ff4c0 con 0x7fcbc0108e80 2024-07-27T03:16:39.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.492+0000 7fcbb77fe640 1 -- 172.21.15.67:0/3257496395 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7fcbb8004040 con 0x7fcbc0108e80 2024-07-27T03:16:39.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.493+0000 7fcbb77fe640 1 --2- 172.21.15.67:0/3257496395 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcbac0b3bd0 0x7fcbac0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:39.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.493+0000 7fcbc62dc640 1 --2- 172.21.15.67:0/3257496395 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcbac0b3bd0 0x7fcbac0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:39.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.493+0000 7fcbb77fe640 1 -- 172.21.15.67:0/3257496395 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fcbb81057e0 con 0x7fcbc0108e80 2024-07-27T03:16:39.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.494+0000 7fcbc62dc640 1 --2- 172.21.15.67:0/3257496395 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcbac0b3bd0 0x7fcbac0b60e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc005e10 tx=0x7fcbbc005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:39.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.498+0000 7fcbb77fe640 1 -- 172.21.15.67:0/3257496395 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fcbb80cca80 con 0x7fcbc0108e80 2024-07-27T03:16:39.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:39.739+0000 7fcbc8540640 1 -- 172.21.15.67:0/3257496395 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fcbc002e100 con 0x7fcbc0108e80 2024-07-27T03:16:40.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.358+0000 7fcbb77fe640 1 -- 172.21.15.67:0/3257496395 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v30) ==== 117+0+0 (secure 0 0 0) 0x7fcbb80cc380 con 0x7fcbc0108e80 2024-07-27T03:16:40.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.361+0000 7fcbc8540640 1 -- 172.21.15.67:0/3257496395 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcbac0b3bd0 msgr2=0x7fcbac0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:40.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.361+0000 7fcbc8540640 1 --2- 172.21.15.67:0/3257496395 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcbac0b3bd0 0x7fcbac0b60e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc005e10 tx=0x7fcbbc005da0 comp rx=0 tx=0).stop 2024-07-27T03:16:40.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.361+0000 7fcbc8540640 1 -- 172.21.15.67:0/3257496395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbc0108e80 msgr2=0x7fcbc019aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:40.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.361+0000 7fcbc8540640 1 --2- 172.21.15.67:0/3257496395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbc0108e80 0x7fcbc019aba0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8003ba0 tx=0x7fcbb8003bd0 comp rx=0 tx=0).stop 2024-07-27T03:16:40.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.361+0000 7fcbc8540640 1 -- 172.21.15.67:0/3257496395 shutdown_connections 2024-07-27T03:16:40.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.361+0000 7fcbc8540640 1 --2- 172.21.15.67:0/3257496395 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcbac0b3bd0 0x7fcbac0b60e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:40.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.361+0000 7fcbc8540640 1 --2- 172.21.15.67:0/3257496395 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcbc0108e80 0x7fcbc019aba0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:40.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.361+0000 7fcbc8540640 1 --2- 172.21.15.67:0/3257496395 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbc01062c0 0x7fcbc0199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:40.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.361+0000 7fcbc8540640 1 --2- 172.21.15.67:0/3257496395 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcbc006bf20 0x7fcbc0199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:40.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.361+0000 7fcbc8540640 1 -- 172.21.15.67:0/3257496395 >> 172.21.15.67:0/3257496395 conn(0x7fcbc00fd9a0 msgr2=0x7fcbc0104730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:40.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.361+0000 7fcbc8540640 1 -- 172.21.15.67:0/3257496395 shutdown_connections 2024-07-27T03:16:40.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.361+0000 7fcbc8540640 1 -- 172.21.15.67:0/3257496395 wait complete. 2024-07-27T03:16:40.376 INFO:teuthology.run_tasks:Running task check-counter... 2024-07-27T03:16:40.388 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-07-27T03:16:40.398 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-27T03:16:40.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.514+0000 7ff25926e640 1 -- 172.21.15.67:0/3842791954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff254106290 msgr2=0x7ff254108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:40.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.514+0000 7ff25926e640 1 --2- 172.21.15.67:0/3842791954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff254106290 0x7ff254108710 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009900 tx=0x7ff23c02a410 comp rx=0 tx=0).stop 2024-07-27T03:16:40.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.515+0000 7ff25926e640 1 -- 172.21.15.67:0/3842791954 shutdown_connections 2024-07-27T03:16:40.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.515+0000 7ff25926e640 1 --2- 172.21.15.67:0/3842791954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff254108e50 0x7ff25410b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:40.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.515+0000 7ff25926e640 1 --2- 172.21.15.67:0/3842791954 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff254106290 0x7ff254108710 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:40.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.515+0000 7ff25926e640 1 --2- 172.21.15.67:0/3842791954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff25406beb0 0x7ff254105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:40.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.515+0000 7ff25926e640 1 -- 172.21.15.67:0/3842791954 >> 172.21.15.67:0/3842791954 conn(0x7ff2540fd950 msgr2=0x7ff2540ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:40.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.515+0000 7ff25926e640 1 -- 172.21.15.67:0/3842791954 shutdown_connections 2024-07-27T03:16:40.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.515+0000 7ff25926e640 1 -- 172.21.15.67:0/3842791954 wait complete. 2024-07-27T03:16:40.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.515+0000 7ff25926e640 1 Processor -- start 2024-07-27T03:16:40.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.515+0000 7ff25926e640 1 -- start start 2024-07-27T03:16:40.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.516+0000 7ff25926e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff25406beb0 0x7ff2541999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.516+0000 7ff25926e640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff254106290 0x7ff25419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.516+0000 7ff25926e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff254108e50 0x7ff25419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.516+0000 7ff25926e640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff25410ed70 con 0x7ff254108e50 2024-07-27T03:16:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.516+0000 7ff25926e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff25410ebf0 con 0x7ff25406beb0 2024-07-27T03:16:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.516+0000 7ff25926e640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff25410eef0 con 0x7ff254106290 2024-07-27T03:16:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.516+0000 7ff2537fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff254108e50 0x7ff25419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.516+0000 7ff2537fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff254108e50 0x7ff25419abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33454/0 (socket says 172.21.15.67:33454) 2024-07-27T03:16:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.516+0000 7ff2537fe640 1 -- 172.21.15.67:0/518432732 learned_addr learned my addr 172.21.15.67:0/518432732 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.516+0000 7ff2527fc640 1 --2- 172.21.15.67:0/518432732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff254106290 0x7ff25419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.516+0000 7ff252ffd640 1 --2- 172.21.15.67:0/518432732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff25406beb0 0x7ff2541999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:40.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.516+0000 7ff2537fe640 1 -- 172.21.15.67:0/518432732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff254106290 msgr2=0x7ff25419a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:40.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.516+0000 7ff2537fe640 1 --2- 172.21.15.67:0/518432732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff254106290 0x7ff25419a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:40.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.516+0000 7ff2537fe640 1 -- 172.21.15.67:0/518432732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff25406beb0 msgr2=0x7ff2541999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:40.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.516+0000 7ff2537fe640 1 --2- 172.21.15.67:0/518432732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff25406beb0 0x7ff2541999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:40.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.516+0000 7ff2537fe640 1 -- 172.21.15.67:0/518432732 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25419b3e0 con 0x7ff254108e50 2024-07-27T03:16:40.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.516+0000 7ff2537fe640 1 --2- 172.21.15.67:0/518432732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff254108e50 0x7ff25419abc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00e970 tx=0x7ff24c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:40.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.516+0000 7ff24bfff640 1 -- 172.21.15.67:0/518432732 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c00ccc0 con 0x7ff254108e50 2024-07-27T03:16:40.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.516+0000 7ff24bfff640 1 -- 172.21.15.67:0/518432732 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff24c00ce60 con 0x7ff254108e50 2024-07-27T03:16:40.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.516+0000 7ff25926e640 1 -- 172.21.15.67:0/518432732 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff25419b6d0 con 0x7ff254108e50 2024-07-27T03:16:40.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.516+0000 7ff24bfff640 1 -- 172.21.15.67:0/518432732 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c010430 con 0x7ff254108e50 2024-07-27T03:16:40.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.517+0000 7ff25926e640 1 -- 172.21.15.67:0/518432732 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2541a1dc0 con 0x7ff254108e50 2024-07-27T03:16:40.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.518+0000 7ff25926e640 1 -- 172.21.15.67:0/518432732 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2540ff470 con 0x7ff254108e50 2024-07-27T03:16:40.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.519+0000 7ff24bfff640 1 -- 172.21.15.67:0/518432732 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7ff24c004f70 con 0x7ff254108e50 2024-07-27T03:16:40.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.520+0000 7ff24bfff640 1 --2- 172.21.15.67:0/518432732 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff2400b3c50 0x7ff2400b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:40.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.520+0000 7ff252ffd640 1 --2- 172.21.15.67:0/518432732 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff2400b3c50 0x7ff2400b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:40.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.520+0000 7ff24bfff640 1 -- 172.21.15.67:0/518432732 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7ff24c0e4680 con 0x7ff254108e50 2024-07-27T03:16:40.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.520+0000 7ff252ffd640 1 --2- 172.21.15.67:0/518432732 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff2400b3c50 0x7ff2400b6160 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff244005fd0 tx=0x7ff244009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:40.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.523+0000 7ff24bfff640 1 -- 172.21.15.67:0/518432732 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7ff24c055600 con 0x7ff254108e50 2024-07-27T03:16:40.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:40 smithi175 ceph-mon[37125]: pgmap v433: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:16:40.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:40 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3257496395' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-07-27T03:16:40.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:40 smithi067 ceph-mon[32814]: pgmap v433: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:16:40.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:40 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3257496395' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-07-27T03:16:40.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.722+0000 7ff25926e640 1 -- 172.21.15.67:0/518432732 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff25402e2d0 con 0x7ff254108e50 2024-07-27T03:16:40.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.723+0000 7ff24bfff640 1 -- 172.21.15.67:0/518432732 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24829 (secure 0 0 0) 0x7ff24c0ab920 con 0x7ff254108e50 2024-07-27T03:16:40.726 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:16:40.726 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":91,"fsid":"c0782a00-4bc4-11ef-bcc7-c7b262605968","created":"2024-07-27T03:04:08.096824+0000","modified":"2024-07-27T03:13:57.294556+0000","last_up_change":"2024-07-27T03:11:33.314800+0000","last_in_change":"2024-07-27T03:11:14.028095+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-27T03:08:01.042479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-27T03:12:27.565948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-27T03:12:31.353726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-27T03:12:40.599486+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9f614272-18e8-4562-812c-4f43e40efd2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6803","nonce":2648129674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6805","nonce":2648129674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6809","nonce":2648129674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6807","nonce":2648129674}]},"public_addr":"172.21.15.67:6803/2648129674","cluster_addr":"172.21.15.67:6805/2648129674","heartbeat_back_addr":"172.21.15.67:6809/2648129674","heartbeat_front_addr":"172.21.15.67:6807/2648129674","state":["exists","up"]},{"osd":1,"uuid":"f59f5879-163a-4269-bbe7-1200eea65341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6801","nonce":2561753152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6803","nonce":2561753152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6807","nonce":2561753152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6805","nonce":2561753152}]},"public_addr":"172.21.15.149:6801/2561753152","cluster_addr":"172.21.15.149:6803/2561753152","heartbeat_back_addr":"172.21.15.149:6807/2561753152","heartbeat_front_addr":"172.21.15.149:6805/2561753152","state":["exists","up"]},{"osd":2,"uuid":"444e0ebe-7ca3-4be9-8bd5-db859f824445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6801","nonce":3770940078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6803","nonce":3770940078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6807","nonce":3770940078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6805","nonce":3770940078}]},"public_addr":"172.21.15.175:6801/3770940078","cluster_addr":"172.21.15.175:6803/3770940078","heartbeat_back_addr":"172.21.15.175:6807/3770940078","heartbeat_front_addr":"172.21.15.175:6805/3770940078","state":["exists","up"]},{"osd":3,"uuid":"d554db3d-9604-427c-8736-b1bb379f8d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6811","nonce":3159144693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6813","nonce":3159144693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6817","nonce":3159144693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6815","nonce":3159144693}]},"public_addr":"172.21.15.67:6811/3159144693","cluster_addr":"172.21.15.67:6813/3159144693","heartbeat_back_addr":"172.21.15.67:6817/3159144693","heartbeat_front_addr":"172.21.15.67:6815/3159144693","state":["exists","up"]},{"osd":4,"uuid":"c317dbe5-3b49-47a6-8408-7042a3dac345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3393503532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3393503532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3393503532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3393503532}]},"public_addr":"172.21.15.149:6809/3393503532","cluster_addr":"172.21.15.149:6811/3393503532","heartbeat_back_addr":"172.21.15.149:6815/3393503532","heartbeat_front_addr":"172.21.15.149:6813/3393503532","state":["exists","up"]},{"osd":5,"uuid":"5c85abd6-2b42-4957-a9b7-59a9ee36b841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6809","nonce":2291204784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6811","nonce":2291204784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6815","nonce":2291204784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6813","nonce":2291204784}]},"public_addr":"172.21.15.175:6809/2291204784","cluster_addr":"172.21.15.175:6811/2291204784","heartbeat_back_addr":"172.21.15.175:6815/2291204784","heartbeat_front_addr":"172.21.15.175:6813/2291204784","state":["exists","up"]},{"osd":6,"uuid":"bf6c277d-ec24-4470-98de-cf6f28876ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6819","nonce":2660182879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6821","nonce":2660182879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6825","nonce":2660182879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6823","nonce":2660182879}]},"public_addr":"172.21.15.67:6819/2660182879","cluster_addr":"172.21.15.67:6821/2660182879","heartbeat_back_addr":"172.21.15.67:6825/2660182879","heartbeat_front_addr":"172.21.15.67:6823/2660182879","state":["exists","up"]},{"osd":7,"uuid":"fea2c7cf-d5cb-45ae-9b29-f6e56c903891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6817","nonce":2289624404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6819","nonce":2289624404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6823","nonce":2289624404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6821","nonce":2289624404}]},"public_addr":"172.21.15.149:6817/2289624404","cluster_addr":"172.21.15.149:6819/2289624404","heartbeat_back_addr":"172.21.15.149:6823/2289624404","heartbeat_front_addr":"172.21.15.149:6821/2289624404","state":["exists","up"]},{"osd":8,"uuid":"14d04d96-579a-4dc9-a4d4-50e4850b48e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6817","nonce":2286032466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6819","nonce":2286032466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6823","nonce":2286032466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6821","nonce":2286032466}]},"public_addr":"172.21.15.175:6817/2286032466","cluster_addr":"172.21.15.175:6819/2286032466","heartbeat_back_addr":"172.21.15.175:6823/2286032466","heartbeat_front_addr":"172.21.15.175:6821/2286032466","state":["exists","up"]},{"osd":9,"uuid":"b65dfb6b-1efe-4314-a8be-ce6a295e99fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6827","nonce":3411081513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6829","nonce":3411081513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6833","nonce":3411081513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6831","nonce":3411081513}]},"public_addr":"172.21.15.67:6827/3411081513","cluster_addr":"172.21.15.67:6829/3411081513","heartbeat_back_addr":"172.21.15.67:6833/3411081513","heartbeat_front_addr":"172.21.15.67:6831/3411081513","state":["exists","up"]},{"osd":10,"uuid":"e3dabae0-0d64-4222-984c-fc1cce883456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6825","nonce":2011495212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6827","nonce":2011495212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6831","nonce":2011495212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6829","nonce":2011495212}]},"public_addr":"172.21.15.149:6825/2011495212","cluster_addr":"172.21.15.149:6827/2011495212","heartbeat_back_addr":"172.21.15.149:6831/2011495212","heartbeat_front_addr":"172.21.15.149:6829/2011495212","state":["exists","up"]},{"osd":11,"uuid":"406c082c-9b5f-4781-8cbf-3c04fa49e410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6825","nonce":2565514213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6827","nonce":2565514213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6831","nonce":2565514213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6829","nonce":2565514213}]},"public_addr":"172.21.15.175:6825/2565514213","cluster_addr":"172.21.15.175:6827/2565514213","heartbeat_back_addr":"172.21.15.175:6831/2565514213","heartbeat_front_addr":"172.21.15.175:6829/2565514213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:11.399380+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:33.728518+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:56.411181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:19.871731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:42.994576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:06.712815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:31.321967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:55.070898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:18.686108+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:43.187795+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:07.318538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:31.374843+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/3477601235":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/4130679770":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6800/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/1029332005":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6801/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6801/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/284078926":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6800/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/2559121542":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/925406021":"2024-07-28T03:04:30.600273+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-27T03:16:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.725+0000 7ff25926e640 1 -- 172.21.15.67:0/518432732 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff2400b3c50 msgr2=0x7ff2400b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.725+0000 7ff25926e640 1 --2- 172.21.15.67:0/518432732 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff2400b3c50 0x7ff2400b6160 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff244005fd0 tx=0x7ff244009500 comp rx=0 tx=0).stop 2024-07-27T03:16:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.725+0000 7ff25926e640 1 -- 172.21.15.67:0/518432732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff254108e50 msgr2=0x7ff25419abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.725+0000 7ff25926e640 1 --2- 172.21.15.67:0/518432732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff254108e50 0x7ff25419abc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00e970 tx=0x7ff24c00ed70 comp rx=0 tx=0).stop 2024-07-27T03:16:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.725+0000 7ff25926e640 1 -- 172.21.15.67:0/518432732 shutdown_connections 2024-07-27T03:16:40.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.725+0000 7ff25926e640 1 --2- 172.21.15.67:0/518432732 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff2400b3c50 0x7ff2400b6160 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.725+0000 7ff25926e640 1 --2- 172.21.15.67:0/518432732 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff254108e50 0x7ff25419abc0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.725+0000 7ff25926e640 1 --2- 172.21.15.67:0/518432732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff254106290 0x7ff25419a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.725+0000 7ff25926e640 1 --2- 172.21.15.67:0/518432732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff25406beb0 0x7ff2541999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.725+0000 7ff25926e640 1 -- 172.21.15.67:0/518432732 >> 172.21.15.67:0/518432732 conn(0x7ff2540fd950 msgr2=0x7ff2541047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.726+0000 7ff25926e640 1 -- 172.21.15.67:0/518432732 shutdown_connections 2024-07-27T03:16:40.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.726+0000 7ff25926e640 1 -- 172.21.15.67:0/518432732 wait complete. 2024-07-27T03:16:40.740 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-27T03:08:01.042479+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-07-27T03:12:27.565948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-07-27T03:12:31.353726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-07-27T03:12:40.599486+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-07-27T03:16:40.741 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-27T03:16:40.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:40 smithi149 ceph-mon[37173]: pgmap v433: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:16:40.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:40 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3257496395' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-07-27T03:16:40.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.856+0000 7f4d3cc29640 1 -- 172.21.15.67:0/2781748892 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d38103a30 msgr2=0x7f4d3810a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:40.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.856+0000 7f4d3cc29640 1 --2- 172.21.15.67:0/2781748892 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d38103a30 0x7f4d3810a3d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4d30009ca0 tx=0x7f4d3002a8c0 comp rx=0 tx=0).stop 2024-07-27T03:16:40.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.856+0000 7f4d3cc29640 1 -- 172.21.15.67:0/2781748892 shutdown_connections 2024-07-27T03:16:40.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.856+0000 7f4d3cc29640 1 --2- 172.21.15.67:0/2781748892 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d38103a30 0x7f4d3810a3d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:40.859 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.856+0000 7f4d3cc29640 1 --2- 172.21.15.67:0/2781748892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d38103090 0x7f4d381034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:40.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.856+0000 7f4d3cc29640 1 --2- 172.21.15.67:0/2781748892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d38102010 0x7f4d38102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:40.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.856+0000 7f4d3cc29640 1 -- 172.21.15.67:0/2781748892 >> 172.21.15.67:0/2781748892 conn(0x7f4d380fd950 msgr2=0x7f4d380ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:40.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.856+0000 7f4d3cc29640 1 -- 172.21.15.67:0/2781748892 shutdown_connections 2024-07-27T03:16:40.860 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.856+0000 7f4d3cc29640 1 -- 172.21.15.67:0/2781748892 wait complete. 2024-07-27T03:16:40.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.857+0000 7f4d3cc29640 1 Processor -- start 2024-07-27T03:16:40.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.857+0000 7f4d3cc29640 1 -- start start 2024-07-27T03:16:40.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.857+0000 7f4d3cc29640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d38102010 0x7f4d3806bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:40.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.857+0000 7f4d3cc29640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d38103090 0x7f4d3819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:40.862 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.857+0000 7f4d3cc29640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d38103a30 0x7f4d381a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:40.863 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.857+0000 7f4d3cc29640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3810dc90 con 0x7f4d38103a30 2024-07-27T03:16:40.865 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.857+0000 7f4d3cc29640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3810db10 con 0x7f4d38103090 2024-07-27T03:16:40.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.857+0000 7f4d3cc29640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3810de10 con 0x7f4d38102010 2024-07-27T03:16:40.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.858+0000 7f4d36d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d38103a30 0x7f4d381a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:40.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.858+0000 7f4d36d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d38103a30 0x7f4d381a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33466/0 (socket says 172.21.15.67:33466) 2024-07-27T03:16:40.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.858+0000 7f4d36d9d640 1 -- 172.21.15.67:0/2634666476 learned_addr learned my addr 172.21.15.67:0/2634666476 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:40.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.858+0000 7f4d35d9b640 1 --2- 172.21.15.67:0/2634666476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d38103090 0x7f4d3819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:40.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.858+0000 7f4d3659c640 1 --2- 172.21.15.67:0/2634666476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d38102010 0x7f4d3806bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:40.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.858+0000 7f4d36d9d640 1 -- 172.21.15.67:0/2634666476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d38102010 msgr2=0x7f4d3806bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:40.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.858+0000 7f4d36d9d640 1 --2- 172.21.15.67:0/2634666476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d38102010 0x7f4d3806bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:40.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.858+0000 7f4d36d9d640 1 -- 172.21.15.67:0/2634666476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d38103090 msgr2=0x7f4d3819e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:40.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.858+0000 7f4d36d9d640 1 --2- 172.21.15.67:0/2634666476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d38103090 0x7f4d3819e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:40.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.858+0000 7f4d36d9d640 1 -- 172.21.15.67:0/2634666476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d381a1400 con 0x7f4d38103a30 2024-07-27T03:16:40.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.858+0000 7f4d36d9d640 1 --2- 172.21.15.67:0/2634666476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d38103a30 0x7f4d381a0be0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4d300097b0 tx=0x7f4d30003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:40.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.858+0000 7f4d2f7fe640 1 -- 172.21.15.67:0/2634666476 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30004240 con 0x7f4d38103a30 2024-07-27T03:16:40.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.858+0000 7f4d3cc29640 1 -- 172.21.15.67:0/2634666476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d381a1690 con 0x7f4d38103a30 2024-07-27T03:16:40.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.858+0000 7f4d2f7fe640 1 -- 172.21.15.67:0/2634666476 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d3002f310 con 0x7f4d38103a30 2024-07-27T03:16:40.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.858+0000 7f4d2f7fe640 1 -- 172.21.15.67:0/2634666476 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3002fb50 con 0x7f4d38103a30 2024-07-27T03:16:40.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.858+0000 7f4d3cc29640 1 -- 172.21.15.67:0/2634666476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d381a1be0 con 0x7f4d38103a30 2024-07-27T03:16:40.867 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.859+0000 7f4d3cc29640 1 -- 172.21.15.67:0/2634666476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d381091e0 con 0x7f4d38103a30 2024-07-27T03:16:40.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.861+0000 7f4d2f7fe640 1 -- 172.21.15.67:0/2634666476 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f4d3002fd50 con 0x7f4d38103a30 2024-07-27T03:16:40.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.862+0000 7f4d2f7fe640 1 --2- 172.21.15.67:0/2634666476 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4d240b3bd0 0x7f4d240b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:40.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.862+0000 7f4d3659c640 1 --2- 172.21.15.67:0/2634666476 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4d240b3bd0 0x7f4d240b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:40.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.862+0000 7f4d2f7fe640 1 -- 172.21.15.67:0/2634666476 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f4d30105830 con 0x7f4d38103a30 2024-07-27T03:16:40.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.862+0000 7f4d3659c640 1 --2- 172.21.15.67:0/2634666476 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4d240b3bd0 0x7f4d240b60e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4d28005fd0 tx=0x7f4d28005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:40.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:40.865+0000 7f4d2f7fe640 1 -- 172.21.15.67:0/2634666476 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f4d300ccad0 con 0x7f4d38103a30 2024-07-27T03:16:41.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.058+0000 7f4d3cc29640 1 -- 172.21.15.67:0/2634666476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4d38063a30 con 0x7f4d38103a30 2024-07-27T03:16:41.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.059+0000 7f4d2f7fe640 1 -- 172.21.15.67:0/2634666476 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f4d300cc3d0 con 0x7f4d38103a30 2024-07-27T03:16:41.062 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-07-27T03:16:41.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.061+0000 7f4d3cc29640 1 -- 172.21.15.67:0/2634666476 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4d240b3bd0 msgr2=0x7f4d240b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:41.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.061+0000 7f4d3cc29640 1 --2- 172.21.15.67:0/2634666476 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4d240b3bd0 0x7f4d240b60e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4d28005fd0 tx=0x7f4d28005850 comp rx=0 tx=0).stop 2024-07-27T03:16:41.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.061+0000 7f4d3cc29640 1 -- 172.21.15.67:0/2634666476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d38103a30 msgr2=0x7f4d381a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:41.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.061+0000 7f4d3cc29640 1 --2- 172.21.15.67:0/2634666476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d38103a30 0x7f4d381a0be0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4d300097b0 tx=0x7f4d30003c20 comp rx=0 tx=0).stop 2024-07-27T03:16:41.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.061+0000 7f4d3cc29640 1 -- 172.21.15.67:0/2634666476 shutdown_connections 2024-07-27T03:16:41.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.061+0000 7f4d3cc29640 1 --2- 172.21.15.67:0/2634666476 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4d240b3bd0 0x7f4d240b60e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.061+0000 7f4d3cc29640 1 --2- 172.21.15.67:0/2634666476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d38103a30 0x7f4d381a0be0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.061+0000 7f4d3cc29640 1 --2- 172.21.15.67:0/2634666476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d38103090 0x7f4d3819e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.061+0000 7f4d3cc29640 1 --2- 172.21.15.67:0/2634666476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d38102010 0x7f4d3806bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.061+0000 7f4d3cc29640 1 -- 172.21.15.67:0/2634666476 >> 172.21.15.67:0/2634666476 conn(0x7f4d380fd950 msgr2=0x7f4d381063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:41.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.061+0000 7f4d3cc29640 1 -- 172.21.15.67:0/2634666476 shutdown_connections 2024-07-27T03:16:41.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.061+0000 7f4d3cc29640 1 -- 172.21.15.67:0/2634666476 wait complete. 2024-07-27T03:16:41.075 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-07-27T03:16:41.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.191+0000 7ff5a64f2640 1 -- 172.21.15.67:0/145818524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5a0103a10 msgr2=0x7ff5a010a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:41.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.191+0000 7ff5a64f2640 1 --2- 172.21.15.67:0/145818524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5a0103a10 0x7ff5a010a3b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff598009ca0 tx=0x7ff59802a8c0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.191+0000 7ff5a64f2640 1 -- 172.21.15.67:0/145818524 shutdown_connections 2024-07-27T03:16:41.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.191+0000 7ff5a64f2640 1 --2- 172.21.15.67:0/145818524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5a0103a10 0x7ff5a010a3b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.191+0000 7ff5a64f2640 1 --2- 172.21.15.67:0/145818524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5a0103070 0x7ff5a0103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.191+0000 7ff5a64f2640 1 --2- 172.21.15.67:0/145818524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5a0101ff0 0x7ff5a0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.191+0000 7ff5a64f2640 1 -- 172.21.15.67:0/145818524 >> 172.21.15.67:0/145818524 conn(0x7ff5a00fd950 msgr2=0x7ff5a00ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:41.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.191+0000 7ff5a64f2640 1 -- 172.21.15.67:0/145818524 shutdown_connections 2024-07-27T03:16:41.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.191+0000 7ff5a64f2640 1 -- 172.21.15.67:0/145818524 wait complete. 2024-07-27T03:16:41.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.192+0000 7ff5a64f2640 1 Processor -- start 2024-07-27T03:16:41.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.192+0000 7ff5a64f2640 1 -- start start 2024-07-27T03:16:41.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.192+0000 7ff5a64f2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5a0101ff0 0x7ff5a0198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:41.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.192+0000 7ff5a64f2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5a0103070 0x7ff5a0198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:41.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.192+0000 7ff5a64f2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5a0103a10 0x7ff5a0199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:41.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.192+0000 7ff5a64f2640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a010dc70 con 0x7ff5a0101ff0 2024-07-27T03:16:41.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.192+0000 7ff5a64f2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a010daf0 con 0x7ff5a0103070 2024-07-27T03:16:41.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.192+0000 7ff5a64f2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a010ddf0 con 0x7ff5a0103a10 2024-07-27T03:16:41.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.192+0000 7ff59ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5a0101ff0 0x7ff5a0198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:41.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.192+0000 7ff59ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5a0101ff0 0x7ff5a0198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33482/0 (socket says 172.21.15.67:33482) 2024-07-27T03:16:41.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.192+0000 7ff59ffff640 1 -- 172.21.15.67:0/2117751739 learned_addr learned my addr 172.21.15.67:0/2117751739 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:41.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.192+0000 7ff59f7fe640 1 --2- 172.21.15.67:0/2117751739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5a0103070 0x7ff5a0198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:41.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.193+0000 7ff5a4a8f640 1 --2- 172.21.15.67:0/2117751739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5a0103a10 0x7ff5a0199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:41.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.193+0000 7ff59ffff640 1 -- 172.21.15.67:0/2117751739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5a0103a10 msgr2=0x7ff5a0199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:41.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.193+0000 7ff59ffff640 1 --2- 172.21.15.67:0/2117751739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5a0103a10 0x7ff5a0199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.193+0000 7ff59ffff640 1 -- 172.21.15.67:0/2117751739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5a0103070 msgr2=0x7ff5a0198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:41.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.193+0000 7ff59ffff640 1 --2- 172.21.15.67:0/2117751739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5a0103070 0x7ff5a0198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.193+0000 7ff59ffff640 1 -- 172.21.15.67:0/2117751739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a019a240 con 0x7ff5a0101ff0 2024-07-27T03:16:41.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.193+0000 7ff59f7fe640 1 --2- 172.21.15.67:0/2117751739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5a0103070 0x7ff5a0198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:16:41.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.193+0000 7ff59ffff640 1 --2- 172.21.15.67:0/2117751739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5a0101ff0 0x7ff5a0198800 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff590009900 tx=0x7ff590009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:41.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.193+0000 7ff59d7fa640 1 -- 172.21.15.67:0/2117751739 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59000d030 con 0x7ff5a0101ff0 2024-07-27T03:16:41.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.193+0000 7ff59d7fa640 1 -- 172.21.15.67:0/2117751739 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff59000d1d0 con 0x7ff5a0101ff0 2024-07-27T03:16:41.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.193+0000 7ff5a64f2640 1 -- 172.21.15.67:0/2117751739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a019a530 con 0x7ff5a0101ff0 2024-07-27T03:16:41.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.193+0000 7ff59d7fa640 1 -- 172.21.15.67:0/2117751739 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff590005280 con 0x7ff5a0101ff0 2024-07-27T03:16:41.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.193+0000 7ff5a64f2640 1 -- 172.21.15.67:0/2117751739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a006a370 con 0x7ff5a0101ff0 2024-07-27T03:16:41.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.194+0000 7ff5a64f2640 1 -- 172.21.15.67:0/2117751739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5a0102460 con 0x7ff5a0101ff0 2024-07-27T03:16:41.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.196+0000 7ff59d7fa640 1 -- 172.21.15.67:0/2117751739 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7ff590005420 con 0x7ff5a0101ff0 2024-07-27T03:16:41.199 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.196+0000 7ff59d7fa640 1 --2- 172.21.15.67:0/2117751739 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff5880b3bd0 0x7ff5880b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:41.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.197+0000 7ff59f7fe640 1 --2- 172.21.15.67:0/2117751739 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff5880b3bd0 0x7ff5880b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:41.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.197+0000 7ff59d7fa640 1 -- 172.21.15.67:0/2117751739 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7ff5900e5810 con 0x7ff5a0101ff0 2024-07-27T03:16:41.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.197+0000 7ff59f7fe640 1 --2- 172.21.15.67:0/2117751739 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff5880b3bd0 0x7ff5880b60e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff594009ad0 tx=0x7ff594005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:41.202 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.199+0000 7ff59d7fa640 1 -- 172.21.15.67:0/2117751739 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7ff5900acab0 con 0x7ff5a0101ff0 2024-07-27T03:16:41.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.398+0000 7ff5a64f2640 1 -- 172.21.15.67:0/2117751739 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff5a0108c90 con 0x7ff5a0101ff0 2024-07-27T03:16:41.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.398+0000 7ff59d7fa640 1 -- 172.21.15.67:0/2117751739 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7ff5900ac3b0 con 0x7ff5a0101ff0 2024-07-27T03:16:41.401 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-07-27T03:16:41.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.400+0000 7ff5a64f2640 1 -- 172.21.15.67:0/2117751739 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff5880b3bd0 msgr2=0x7ff5880b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:41.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.400+0000 7ff5a64f2640 1 --2- 172.21.15.67:0/2117751739 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff5880b3bd0 0x7ff5880b60e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff594009ad0 tx=0x7ff594005f90 comp rx=0 tx=0).stop 2024-07-27T03:16:41.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.400+0000 7ff5a64f2640 1 -- 172.21.15.67:0/2117751739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5a0101ff0 msgr2=0x7ff5a0198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:41.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.400+0000 7ff5a64f2640 1 --2- 172.21.15.67:0/2117751739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5a0101ff0 0x7ff5a0198800 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff590009900 tx=0x7ff590009dc0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.400+0000 7ff5a64f2640 1 -- 172.21.15.67:0/2117751739 shutdown_connections 2024-07-27T03:16:41.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.400+0000 7ff5a64f2640 1 --2- 172.21.15.67:0/2117751739 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff5880b3bd0 0x7ff5880b60e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.400+0000 7ff5a64f2640 1 --2- 172.21.15.67:0/2117751739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff5a0103a10 0x7ff5a0199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.400+0000 7ff5a64f2640 1 --2- 172.21.15.67:0/2117751739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5a0103070 0x7ff5a0198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.400+0000 7ff5a64f2640 1 --2- 172.21.15.67:0/2117751739 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5a0101ff0 0x7ff5a0198800 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.400+0000 7ff5a64f2640 1 -- 172.21.15.67:0/2117751739 >> 172.21.15.67:0/2117751739 conn(0x7ff5a00fd950 msgr2=0x7ff5a0106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:41.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.401+0000 7ff5a64f2640 1 -- 172.21.15.67:0/2117751739 shutdown_connections 2024-07-27T03:16:41.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.401+0000 7ff5a64f2640 1 -- 172.21.15.67:0/2117751739 wait complete. 2024-07-27T03:16:41.415 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-07-27T03:16:41.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.530+0000 7fbcf20f9640 1 -- 172.21.15.67:0/2219007288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcec103070 msgr2=0x7fbcec103480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:41.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.530+0000 7fbcf20f9640 1 --2- 172.21.15.67:0/2219007288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcec103070 0x7fbcec103480 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009900 tx=0x7fbce002a410 comp rx=0 tx=0).stop 2024-07-27T03:16:41.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.530+0000 7fbcf20f9640 1 -- 172.21.15.67:0/2219007288 shutdown_connections 2024-07-27T03:16:41.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.530+0000 7fbcf20f9640 1 --2- 172.21.15.67:0/2219007288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcec103a10 0x7fbcec10a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.530+0000 7fbcf20f9640 1 --2- 172.21.15.67:0/2219007288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcec103070 0x7fbcec103480 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.530+0000 7fbcf20f9640 1 --2- 172.21.15.67:0/2219007288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcec101ff0 0x7fbcec102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.530+0000 7fbcf20f9640 1 -- 172.21.15.67:0/2219007288 >> 172.21.15.67:0/2219007288 conn(0x7fbcec0fd950 msgr2=0x7fbcec0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:41.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.530+0000 7fbcf20f9640 1 -- 172.21.15.67:0/2219007288 shutdown_connections 2024-07-27T03:16:41.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.530+0000 7fbcf20f9640 1 -- 172.21.15.67:0/2219007288 wait complete. 2024-07-27T03:16:41.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.531+0000 7fbcf20f9640 1 Processor -- start 2024-07-27T03:16:41.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.531+0000 7fbcf20f9640 1 -- start start 2024-07-27T03:16:41.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.531+0000 7fbcf20f9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcec101ff0 0x7fbcec06aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:41.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.531+0000 7fbcf20f9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcec103070 0x7fbcec19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:41.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.531+0000 7fbcf20f9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcec103a10 0x7fbcec1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:41.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.531+0000 7fbcf20f9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec10dc70 con 0x7fbcec101ff0 2024-07-27T03:16:41.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.531+0000 7fbcf20f9640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec10daf0 con 0x7fbcec103070 2024-07-27T03:16:41.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.531+0000 7fbcf20f9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec10ddf0 con 0x7fbcec103a10 2024-07-27T03:16:41.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.531+0000 7fbceb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcec101ff0 0x7fbcec06aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:41.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.531+0000 7fbceb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcec101ff0 0x7fbcec06aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33498/0 (socket says 172.21.15.67:33498) 2024-07-27T03:16:41.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.532+0000 7fbceb7fe640 1 -- 172.21.15.67:0/1150086861 learned_addr learned my addr 172.21.15.67:0/1150086861 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:41.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.532+0000 7fbceaffd640 1 --2- 172.21.15.67:0/1150086861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcec103070 0x7fbcec19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:41.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.532+0000 7fbcebfff640 1 --2- 172.21.15.67:0/1150086861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcec103a10 0x7fbcec1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:41.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.532+0000 7fbceb7fe640 1 -- 172.21.15.67:0/1150086861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcec103a10 msgr2=0x7fbcec1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:41.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.532+0000 7fbceb7fe640 1 --2- 172.21.15.67:0/1150086861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcec103a10 0x7fbcec1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.532+0000 7fbceb7fe640 1 -- 172.21.15.67:0/1150086861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcec103070 msgr2=0x7fbcec19e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:41.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.532+0000 7fbceb7fe640 1 --2- 172.21.15.67:0/1150086861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcec103070 0x7fbcec19e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.532+0000 7fbceb7fe640 1 -- 172.21.15.67:0/1150086861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcec1a1380 con 0x7fbcec101ff0 2024-07-27T03:16:41.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.532+0000 7fbceaffd640 1 --2- 172.21.15.67:0/1150086861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcec103070 0x7fbcec19e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:16:41.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.532+0000 7fbcebfff640 1 --2- 172.21.15.67:0/1150086861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcec103a10 0x7fbcec1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:16:41.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.532+0000 7fbceb7fe640 1 --2- 172.21.15.67:0/1150086861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcec101ff0 0x7fbcec06aa00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00c920 tx=0x7fbcdc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:41.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.532+0000 7fbce8ff9640 1 -- 172.21.15.67:0/1150086861 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc007c70 con 0x7fbcec101ff0 2024-07-27T03:16:41.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.532+0000 7fbce8ff9640 1 -- 172.21.15.67:0/1150086861 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcdc007e10 con 0x7fbcec101ff0 2024-07-27T03:16:41.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.532+0000 7fbcf20f9640 1 -- 172.21.15.67:0/1150086861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcec1a1670 con 0x7fbcec101ff0 2024-07-27T03:16:41.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.532+0000 7fbce8ff9640 1 -- 172.21.15.67:0/1150086861 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc018820 con 0x7fbcec101ff0 2024-07-27T03:16:41.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.532+0000 7fbcf20f9640 1 -- 172.21.15.67:0/1150086861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcec1a1bc0 con 0x7fbcec101ff0 2024-07-27T03:16:41.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.533+0000 7fbcf20f9640 1 -- 172.21.15.67:0/1150086861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcec1091c0 con 0x7fbcec101ff0 2024-07-27T03:16:41.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.535+0000 7fbce8ff9640 1 -- 172.21.15.67:0/1150086861 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7fbcdc00f450 con 0x7fbcec101ff0 2024-07-27T03:16:41.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.536+0000 7fbce8ff9640 1 --2- 172.21.15.67:0/1150086861 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbcd40b3bd0 0x7fbcd40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:41.538 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.536+0000 7fbceaffd640 1 --2- 172.21.15.67:0/1150086861 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbcd40b3bd0 0x7fbcd40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:41.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.536+0000 7fbce8ff9640 1 -- 172.21.15.67:0/1150086861 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fbcdc0e4380 con 0x7fbcec101ff0 2024-07-27T03:16:41.539 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.536+0000 7fbceaffd640 1 --2- 172.21.15.67:0/1150086861 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbcd40b3bd0 0x7fbcd40b60e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbce0031440 tx=0x7fbce0005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:41.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.539+0000 7fbce8ff9640 1 -- 172.21.15.67:0/1150086861 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fbcdc014030 con 0x7fbcec101ff0 2024-07-27T03:16:41.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:41 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3257496395' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-07-27T03:16:41.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:41 smithi175 ceph-mon[37125]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-27T03:16:41.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:41 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/518432732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:16:41.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:41 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2634666476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-27T03:16:41.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:41 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3257496395' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-07-27T03:16:41.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:41 smithi067 ceph-mon[32814]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-27T03:16:41.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:41 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/518432732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:16:41.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:41 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2634666476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-27T03:16:41.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.733+0000 7fbcf20f9640 1 -- 172.21.15.67:0/1150086861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbcec063a30 con 0x7fbcec101ff0 2024-07-27T03:16:41.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.733+0000 7fbce8ff9640 1 -- 172.21.15.67:0/1150086861 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7fbcdc0ab750 con 0x7fbcec101ff0 2024-07-27T03:16:41.736 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-07-27T03:16:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.735+0000 7fbcf20f9640 1 -- 172.21.15.67:0/1150086861 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbcd40b3bd0 msgr2=0x7fbcd40b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.736+0000 7fbcf20f9640 1 --2- 172.21.15.67:0/1150086861 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbcd40b3bd0 0x7fbcd40b60e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbce0031440 tx=0x7fbce0005d50 comp rx=0 tx=0).stop 2024-07-27T03:16:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.736+0000 7fbcf20f9640 1 -- 172.21.15.67:0/1150086861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcec101ff0 msgr2=0x7fbcec06aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.736+0000 7fbcf20f9640 1 --2- 172.21.15.67:0/1150086861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcec101ff0 0x7fbcec06aa00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00c920 tx=0x7fbcdc00cde0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.736+0000 7fbcf20f9640 1 -- 172.21.15.67:0/1150086861 shutdown_connections 2024-07-27T03:16:41.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.736+0000 7fbcf20f9640 1 --2- 172.21.15.67:0/1150086861 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbcd40b3bd0 0x7fbcd40b60e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.736+0000 7fbcf20f9640 1 --2- 172.21.15.67:0/1150086861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcec103a10 0x7fbcec1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.736+0000 7fbcf20f9640 1 --2- 172.21.15.67:0/1150086861 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcec103070 0x7fbcec19e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.736+0000 7fbcf20f9640 1 --2- 172.21.15.67:0/1150086861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbcec101ff0 0x7fbcec06aa00 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.736+0000 7fbcf20f9640 1 -- 172.21.15.67:0/1150086861 >> 172.21.15.67:0/1150086861 conn(0x7fbcec0fd950 msgr2=0x7fbcec106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:41.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.736+0000 7fbcf20f9640 1 -- 172.21.15.67:0/1150086861 shutdown_connections 2024-07-27T03:16:41.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.736+0000 7fbcf20f9640 1 -- 172.21.15.67:0/1150086861 wait complete. 2024-07-27T03:16:41.750 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-07-27T03:16:41.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:41 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3257496395' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-07-27T03:16:41.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:41 smithi149 ceph-mon[37173]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-07-27T03:16:41.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:41 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/518432732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:16:41.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:41 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2634666476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-27T03:16:41.868 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.866+0000 7f8dfa926640 1 -- 172.21.15.67:0/1982913716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8df4106310 msgr2=0x7f8df4108730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:41.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.866+0000 7f8dfa926640 1 --2- 172.21.15.67:0/1982913716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8df4106310 0x7f8df4108730 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009900 tx=0x7f8de802a410 comp rx=0 tx=0).stop 2024-07-27T03:16:41.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.866+0000 7f8dfa926640 1 -- 172.21.15.67:0/1982913716 shutdown_connections 2024-07-27T03:16:41.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.866+0000 7f8dfa926640 1 --2- 172.21.15.67:0/1982913716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8df4108e50 0x7f8df410b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.866+0000 7f8dfa926640 1 --2- 172.21.15.67:0/1982913716 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8df4106310 0x7f8df4108730 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.866+0000 7f8dfa926640 1 --2- 172.21.15.67:0/1982913716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8df406bdc0 0x7f8df4105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.866+0000 7f8dfa926640 1 -- 172.21.15.67:0/1982913716 >> 172.21.15.67:0/1982913716 conn(0x7f8df40fd950 msgr2=0x7f8df40ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:41.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.866+0000 7f8dfa926640 1 -- 172.21.15.67:0/1982913716 shutdown_connections 2024-07-27T03:16:41.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.866+0000 7f8dfa926640 1 -- 172.21.15.67:0/1982913716 wait complete. 2024-07-27T03:16:41.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.866+0000 7f8dfa926640 1 Processor -- start 2024-07-27T03:16:41.869 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.866+0000 7f8dfa926640 1 -- start start 2024-07-27T03:16:41.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.867+0000 7f8dfa926640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8df406bdc0 0x7f8df4199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:41.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.867+0000 7f8dfa926640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8df4106310 0x7f8df419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:41.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.867+0000 7f8dfa926640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8df4108e50 0x7f8df419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:41.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.867+0000 7f8dfa926640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f8df410ed70 con 0x7f8df4108e50 2024-07-27T03:16:41.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.867+0000 7f8dfa926640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8df410ebf0 con 0x7f8df406bdc0 2024-07-27T03:16:41.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.867+0000 7f8dfa926640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8df410eef0 con 0x7f8df4106310 2024-07-27T03:16:41.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.867+0000 7f8df8ec3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8df4108e50 0x7f8df419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:41.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.867+0000 7f8df8ec3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8df4108e50 0x7f8df419ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33512/0 (socket says 172.21.15.67:33512) 2024-07-27T03:16:41.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.867+0000 7f8df8ec3640 1 -- 172.21.15.67:0/1936422472 learned_addr learned my addr 172.21.15.67:0/1936422472 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:41.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.867+0000 7f8df3fff640 1 --2- 172.21.15.67:0/1936422472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8df406bdc0 0x7f8df4199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:41.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.867+0000 7f8df37fe640 1 --2- 172.21.15.67:0/1936422472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8df4106310 0x7f8df419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:41.872 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.867+0000 7f8df8ec3640 1 -- 172.21.15.67:0/1936422472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8df4106310 msgr2=0x7f8df419a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.867+0000 7f8df8ec3640 1 --2- 172.21.15.67:0/1936422472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8df4106310 0x7f8df419a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.867+0000 7f8df8ec3640 1 -- 172.21.15.67:0/1936422472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8df406bdc0 msgr2=0x7f8df4199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.867+0000 7f8df8ec3640 1 --2- 172.21.15.67:0/1936422472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8df406bdc0 0x7f8df4199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.867+0000 7f8df8ec3640 1 -- 172.21.15.67:0/1936422472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df419b3b0 con 0x7f8df4108e50 2024-07-27T03:16:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.867+0000 7f8df3fff640 1 --2- 172.21.15.67:0/1936422472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8df406bdc0 0x7f8df4199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:16:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.867+0000 7f8df37fe640 1 --2- 172.21.15.67:0/1936422472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8df4106310 0x7f8df419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:16:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.867+0000 7f8df8ec3640 1 --2- 172.21.15.67:0/1936422472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8df4108e50 0x7f8df419ac20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8dec00eaa0 tx=0x7f8dec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.868+0000 7f8df17fa640 1 -- 172.21.15.67:0/1936422472 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec00cdb0 con 0x7f8df4108e50 2024-07-27T03:16:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.868+0000 7f8dfa926640 1 -- 172.21.15.67:0/1936422472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df419b6a0 con 0x7f8df4108e50 2024-07-27T03:16:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.868+0000 7f8df17fa640 1 -- 172.21.15.67:0/1936422472 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dec005020 con 0x7f8df4108e50 2024-07-27T03:16:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.868+0000 7f8df17fa640 1 -- 172.21.15.67:0/1936422472 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec0116d0 con 0x7f8df4108e50 2024-07-27T03:16:41.873 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.868+0000 7f8dfa926640 1 -- 172.21.15.67:0/1936422472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df41a1e20 con 0x7f8df4108e50 2024-07-27T03:16:41.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.869+0000 7f8dfa926640 1 -- 172.21.15.67:0/1936422472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8df40ff470 con 0x7f8df4108e50 2024-07-27T03:16:41.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.870+0000 7f8df17fa640 1 -- 172.21.15.67:0/1936422472 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f8dec0054a0 con 0x7f8df4108e50 2024-07-27T03:16:41.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.871+0000 7f8df17fa640 1 --2- 172.21.15.67:0/1936422472 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f8ddc0b3bd0 0x7f8ddc0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:41.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.871+0000 7f8df3fff640 1 --2- 172.21.15.67:0/1936422472 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f8ddc0b3bd0 0x7f8ddc0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:41.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.871+0000 7f8df17fa640 1 -- 172.21.15.67:0/1936422472 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f8dec0e45d0 con 0x7f8df4108e50 2024-07-27T03:16:41.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.871+0000 7f8df3fff640 1 --2- 172.21.15.67:0/1936422472 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f8ddc0b3bd0 0x7f8ddc0b60e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8de400abe0 tx=0x7f8de400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:41.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:41.874+0000 7f8df17fa640 1 -- 172.21.15.67:0/1936422472 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f8dec016030 con 0x7f8df4108e50 2024-07-27T03:16:42.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:42.066+0000 7f8dfa926640 1 -- 172.21.15.67:0/1936422472 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8df402e100 con 0x7f8df4108e50 2024-07-27T03:16:42.069 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:42.067+0000 7f8df17fa640 1 -- 172.21.15.67:0/1936422472 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7f8dec0ab870 con 0x7f8df4108e50 2024-07-27T03:16:42.070 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-07-27T03:16:42.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:42.069+0000 7f8dfa926640 1 -- 172.21.15.67:0/1936422472 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f8ddc0b3bd0 msgr2=0x7f8ddc0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:42.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:42.069+0000 7f8dfa926640 1 --2- 172.21.15.67:0/1936422472 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f8ddc0b3bd0 0x7f8ddc0b60e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8de400abe0 tx=0x7f8de400a5c0 comp rx=0 tx=0).stop 2024-07-27T03:16:42.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:42.069+0000 7f8dfa926640 1 -- 172.21.15.67:0/1936422472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8df4108e50 msgr2=0x7f8df419ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:42.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:42.069+0000 7f8dfa926640 1 --2- 172.21.15.67:0/1936422472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8df4108e50 0x7f8df419ac20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8dec00eaa0 tx=0x7f8dec00ef60 comp rx=0 tx=0).stop 2024-07-27T03:16:42.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:42.069+0000 7f8dfa926640 1 -- 172.21.15.67:0/1936422472 shutdown_connections 2024-07-27T03:16:42.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:42.069+0000 7f8dfa926640 1 --2- 172.21.15.67:0/1936422472 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f8ddc0b3bd0 0x7f8ddc0b60e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:42.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:42.069+0000 7f8dfa926640 1 --2- 172.21.15.67:0/1936422472 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8df4108e50 0x7f8df419ac20 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:42.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:42.069+0000 7f8dfa926640 1 --2- 172.21.15.67:0/1936422472 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8df4106310 0x7f8df419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:42.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:42.069+0000 7f8dfa926640 1 --2- 172.21.15.67:0/1936422472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8df406bdc0 0x7f8df4199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:42.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:42.069+0000 7f8dfa926640 1 -- 172.21.15.67:0/1936422472 >> 172.21.15.67:0/1936422472 conn(0x7f8df40fd950 msgr2=0x7f8df41048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:42.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:42.069+0000 7f8dfa926640 1 -- 172.21.15.67:0/1936422472 shutdown_connections 2024-07-27T03:16:42.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:42.069+0000 7f8dfa926640 1 -- 172.21.15.67:0/1936422472 wait complete. 2024-07-27T03:16:42.083 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-07-27T03:16:42.084 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:16:42.349 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:16:42.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:42 smithi175 ceph-mon[37125]: pgmap v434: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-07-27T03:16:42.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:42 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2117751739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-27T03:16:42.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:42 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1150086861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-27T03:16:42.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:42 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1936422472' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-07-27T03:16:42.631 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:42 smithi067 ceph-mon[32814]: pgmap v434: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-07-27T03:16:42.631 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:42 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2117751739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-27T03:16:42.631 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:42 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1150086861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-27T03:16:42.631 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:42 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1936422472' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-07-27T03:16:42.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:42 smithi149 ceph-mon[37173]: pgmap v434: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-07-27T03:16:42.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:42 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2117751739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-27T03:16:42.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:42 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1150086861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-27T03:16:42.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:42 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1936422472' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-07-27T03:16:44.109 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.106+0000 7fd0a9f66640 1 -- 172.21.15.67:0/4117506197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0a41021e0 msgr2=0x7fd0a4110010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:44.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.106+0000 7fd0a9f66640 1 --2- 172.21.15.67:0/4117506197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0a41021e0 0x7fd0a4110010 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009930 tx=0x7fd09c02a420 comp rx=0 tx=0).stop 2024-07-27T03:16:44.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.106+0000 7fd0a9f66640 1 -- 172.21.15.67:0/4117506197 shutdown_connections 2024-07-27T03:16:44.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.106+0000 7fd0a9f66640 1 --2- 172.21.15.67:0/4117506197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0a41021e0 0x7fd0a4110010 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:44.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.106+0000 7fd0a9f66640 1 --2- 172.21.15.67:0/4117506197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0a4101890 0x7fd0a4101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:44.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.106+0000 7fd0a9f66640 1 --2- 172.21.15.67:0/4117506197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0a41081a0 0x7fd0a41085b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:44.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.106+0000 7fd0a9f66640 1 -- 172.21.15.67:0/4117506197 >> 172.21.15.67:0/4117506197 conn(0x7fd0a4077730 msgr2=0x7fd0a4077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:44.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.106+0000 7fd0a9f66640 1 -- 172.21.15.67:0/4117506197 shutdown_connections 2024-07-27T03:16:44.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.106+0000 7fd0a9f66640 1 -- 172.21.15.67:0/4117506197 wait complete. 2024-07-27T03:16:44.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.107+0000 7fd0a9f66640 1 Processor -- start 2024-07-27T03:16:44.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.108+0000 7fd0a9f66640 1 -- start start 2024-07-27T03:16:44.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.108+0000 7fd0a9f66640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0a4101890 0x7fd0a419c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:44.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.108+0000 7fd0a9f66640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0a41021e0 0x7fd0a419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:44.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.108+0000 7fd0a9f66640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0a41081a0 0x7fd0a41a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:44.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.108+0000 7fd0a9f66640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a4113520 con 0x7fd0a41021e0 2024-07-27T03:16:44.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.108+0000 7fd0a9f66640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a41133a0 con 0x7fd0a41081a0 2024-07-27T03:16:44.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.108+0000 7fd0a9f66640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a41136a0 con 0x7fd0a4101890 2024-07-27T03:16:44.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.108+0000 7fd0a2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0a41021e0 0x7fd0a419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:44.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.108+0000 7fd0a2ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0a41021e0 0x7fd0a419c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33532/0 (socket says 172.21.15.67:33532) 2024-07-27T03:16:44.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.108+0000 7fd0a2ffd640 1 -- 172.21.15.67:0/1196077730 learned_addr learned my addr 172.21.15.67:0/1196077730 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:44.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.108+0000 7fd0a37fe640 1 --2- 172.21.15.67:0/1196077730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0a4101890 0x7fd0a419c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:44.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.108+0000 7fd0a3fff640 1 --2- 172.21.15.67:0/1196077730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0a41081a0 0x7fd0a41a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:44.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.108+0000 7fd0a2ffd640 1 -- 172.21.15.67:0/1196077730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0a4101890 msgr2=0x7fd0a419c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:44.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.108+0000 7fd0a2ffd640 1 --2- 172.21.15.67:0/1196077730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0a4101890 0x7fd0a419c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:44.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.108+0000 7fd0a2ffd640 1 -- 172.21.15.67:0/1196077730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0a41081a0 msgr2=0x7fd0a41a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:44.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.108+0000 7fd0a2ffd640 1 --2- 172.21.15.67:0/1196077730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0a41081a0 0x7fd0a41a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:44.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.108+0000 7fd0a2ffd640 1 -- 172.21.15.67:0/1196077730 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a41a2040 con 0x7fd0a41021e0 2024-07-27T03:16:44.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.108+0000 7fd0a3fff640 1 --2- 172.21.15.67:0/1196077730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0a41081a0 0x7fd0a41a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:16:44.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.109+0000 7fd0a2ffd640 1 --2- 172.21.15.67:0/1196077730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0a41021e0 0x7fd0a419c670 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd08c005fd0 tx=0x7fd08c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:44.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.109+0000 7fd0a0ff9640 1 -- 172.21.15.67:0/1196077730 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c017070 con 0x7fd0a41021e0 2024-07-27T03:16:44.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.109+0000 7fd0a9f66640 1 -- 172.21.15.67:0/1196077730 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a41a2330 con 0x7fd0a41021e0 2024-07-27T03:16:44.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.109+0000 7fd0a0ff9640 1 -- 172.21.15.67:0/1196077730 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd08c009c30 con 0x7fd0a41021e0 2024-07-27T03:16:44.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.109+0000 7fd0a0ff9640 1 -- 172.21.15.67:0/1196077730 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c003d30 con 0x7fd0a41021e0 2024-07-27T03:16:44.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.109+0000 7fd0a9f66640 1 -- 172.21.15.67:0/1196077730 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a41a8a90 con 0x7fd0a41021e0 2024-07-27T03:16:44.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.110+0000 7fd0a9f66640 1 -- 172.21.15.67:0/1196077730 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a4101ca0 con 0x7fd0a41021e0 2024-07-27T03:16:44.114 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.112+0000 7fd0a0ff9640 1 -- 172.21.15.67:0/1196077730 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7fd08c003ed0 con 0x7fd0a41021e0 2024-07-27T03:16:44.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.112+0000 7fd0a0ff9640 1 --2- 172.21.15.67:0/1196077730 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd0900b3bd0 0x7fd0900b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:44.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.113+0000 7fd0a37fe640 1 --2- 172.21.15.67:0/1196077730 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd0900b3bd0 0x7fd0900b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:44.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.113+0000 7fd0a0ff9640 1 -- 172.21.15.67:0/1196077730 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fd08c0e4040 con 0x7fd0a41021e0 2024-07-27T03:16:44.115 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.113+0000 7fd0a37fe640 1 --2- 172.21.15.67:0/1196077730 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd0900b3bd0 0x7fd0900b60e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd094005fd0 tx=0x7fd094009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:44.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.116+0000 7fd0a0ff9640 1 -- 172.21.15.67:0/1196077730 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fd08c020030 con 0x7fd0a41021e0 2024-07-27T03:16:44.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.336+0000 7fd0a9f66640 1 -- 172.21.15.67:0/1196077730 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0a41a8d70 con 0x7fd0a41021e0 2024-07-27T03:16:44.339 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.337+0000 7fd0a0ff9640 1 -- 172.21.15.67:0/1196077730 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11983 (secure 0 0 0) 0x7fd08c009de0 con 0x7fd0a41021e0 2024-07-27T03:16:44.340 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:16:44.340 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"btime":"2024-07-27T03:16:40:350577+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14679,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/252686369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":252686369},{"type":"v1","addr":"172.21.15.67:6835","nonce":252686369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/3358832762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3358832762},{"type":"v1","addr":"172.21.15.67:6837","nonce":3358832762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-27T03:12:34.320208+0000","modified":"2024-07-27T03:16:39.741149+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24620,"mds_1":24587,"mds_2":24583,"mds_3":24581,"mds_4":24563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14736":{"gid":14736,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/2162708317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2162708317},{"type":"v1","addr":"172.21.15.67:6839","nonce":2162708317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14748":{"gid":14748,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/795704244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":795704244},{"type":"v1","addr":"172.21.15.67:6841","nonce":795704244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/3868325215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3868325215},{"type":"v1","addr":"172.21.15.149:6833","nonce":3868325215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2462749027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2462749027},{"type":"v1","addr":"172.21.15.175:6835","nonce":2462749027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.149:6835/3730964806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3730964806},{"type":"v1","addr":"172.21.15.149:6835","nonce":3730964806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.175:6837/1304803502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1304803502},{"type":"v1","addr":"172.21.15.175:6837","nonce":1304803502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.175:6839/1670568853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1670568853},{"type":"v1","addr":"172.21.15.175:6839","nonce":1670568853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24587":{"gid":24587,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.149:6837/2932951845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2932951845},{"type":"v1","addr":"172.21.15.149:6837","nonce":2932951845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.149:6839/281746498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":281746498},{"type":"v1","addr":"172.21.15.149:6839","nonce":281746498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24620,"qdb_cluster":[24587,24583,24620,24581,24563]},"id":1}]} 2024-07-27T03:16:44.340 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 30 2024-07-27T03:16:44.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.339+0000 7fd0a9f66640 1 -- 172.21.15.67:0/1196077730 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd0900b3bd0 msgr2=0x7fd0900b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:44.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.339+0000 7fd0a9f66640 1 --2- 172.21.15.67:0/1196077730 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd0900b3bd0 0x7fd0900b60e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd094005fd0 tx=0x7fd094009680 comp rx=0 tx=0).stop 2024-07-27T03:16:44.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.339+0000 7fd0a9f66640 1 -- 172.21.15.67:0/1196077730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0a41021e0 msgr2=0x7fd0a419c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:44.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.339+0000 7fd0a9f66640 1 --2- 172.21.15.67:0/1196077730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0a41021e0 0x7fd0a419c670 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd08c005fd0 tx=0x7fd08c00dde0 comp rx=0 tx=0).stop 2024-07-27T03:16:44.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.339+0000 7fd0a9f66640 1 -- 172.21.15.67:0/1196077730 shutdown_connections 2024-07-27T03:16:44.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.339+0000 7fd0a9f66640 1 --2- 172.21.15.67:0/1196077730 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd0900b3bd0 0x7fd0900b60e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:44.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.339+0000 7fd0a9f66640 1 --2- 172.21.15.67:0/1196077730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0a41081a0 0x7fd0a41a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:44.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.339+0000 7fd0a9f66640 1 --2- 172.21.15.67:0/1196077730 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd0a41021e0 0x7fd0a419c670 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:44.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.339+0000 7fd0a9f66640 1 --2- 172.21.15.67:0/1196077730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd0a4101890 0x7fd0a419c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:44.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.339+0000 7fd0a9f66640 1 -- 172.21.15.67:0/1196077730 >> 172.21.15.67:0/1196077730 conn(0x7fd0a4077730 msgr2=0x7fd0a4105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:44.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.339+0000 7fd0a9f66640 1 -- 172.21.15.67:0/1196077730 shutdown_connections 2024-07-27T03:16:44.342 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:44.339+0000 7fd0a9f66640 1 -- 172.21.15.67:0/1196077730 wait complete. 2024-07-27T03:16:44.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:44 smithi175 ceph-mon[37125]: pgmap v435: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-07-27T03:16:44.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:44 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1196077730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:16:44.637 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:44 smithi067 ceph-mon[32814]: pgmap v435: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-07-27T03:16:44.637 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:44 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1196077730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:16:44.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:44 smithi149 ceph-mon[37173]: pgmap v435: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-07-27T03:16:44.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:44 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1196077730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:16:45.039 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-07-27T03:16:45.039 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-07-27T03:16:45.039 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-27T03:16:45.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.157+0000 7fd3f17e7640 1 -- 172.21.15.67:0/4203007353 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ec103a00 msgr2=0x7fd3ec10a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:45.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.157+0000 7fd3f17e7640 1 --2- 172.21.15.67:0/4203007353 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ec103a00 0x7fd3ec10a3a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009ca0 tx=0x7fd3e402a8c0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.157+0000 7fd3f17e7640 1 -- 172.21.15.67:0/4203007353 shutdown_connections 2024-07-27T03:16:45.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.157+0000 7fd3f17e7640 1 --2- 172.21.15.67:0/4203007353 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ec103a00 0x7fd3ec10a3a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.157+0000 7fd3f17e7640 1 --2- 172.21.15.67:0/4203007353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3ec103060 0x7fd3ec103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.157+0000 7fd3f17e7640 1 --2- 172.21.15.67:0/4203007353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3ec101fe0 0x7fd3ec102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.157+0000 7fd3f17e7640 1 -- 172.21.15.67:0/4203007353 >> 172.21.15.67:0/4203007353 conn(0x7fd3ec0fd920 msgr2=0x7fd3ec0ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:45.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.157+0000 7fd3f17e7640 1 -- 172.21.15.67:0/4203007353 shutdown_connections 2024-07-27T03:16:45.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.157+0000 7fd3f17e7640 1 -- 172.21.15.67:0/4203007353 wait complete. 2024-07-27T03:16:45.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.157+0000 7fd3f17e7640 1 Processor -- start 2024-07-27T03:16:45.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.157+0000 7fd3f17e7640 1 -- start start 2024-07-27T03:16:45.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.158+0000 7fd3f17e7640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3ec101fe0 0x7fd3ec199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:45.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.158+0000 7fd3f17e7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ec103060 0x7fd3ec199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:45.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.158+0000 7fd3f17e7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3ec103a00 0x7fd3ec19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:45.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.158+0000 7fd3f17e7640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec10dc60 con 0x7fd3ec103060 2024-07-27T03:16:45.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.158+0000 7fd3f17e7640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec10dae0 con 0x7fd3ec103a00 2024-07-27T03:16:45.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.158+0000 7fd3f17e7640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec10dde0 con 0x7fd3ec101fe0 2024-07-27T03:16:45.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.158+0000 7fd3e3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ec103060 0x7fd3ec199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:45.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.158+0000 7fd3e3fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ec103060 0x7fd3ec199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51388/0 (socket says 172.21.15.67:51388) 2024-07-27T03:16:45.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.158+0000 7fd3e3fff640 1 -- 172.21.15.67:0/3903221024 learned_addr learned my addr 172.21.15.67:0/3903221024 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:45.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.158+0000 7fd3eaffd640 1 --2- 172.21.15.67:0/3903221024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3ec101fe0 0x7fd3ec199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:45.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.158+0000 7fd3eb7fe640 1 --2- 172.21.15.67:0/3903221024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3ec103a00 0x7fd3ec19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:45.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.158+0000 7fd3e3fff640 1 -- 172.21.15.67:0/3903221024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3ec101fe0 msgr2=0x7fd3ec199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:45.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.158+0000 7fd3e3fff640 1 --2- 172.21.15.67:0/3903221024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3ec101fe0 0x7fd3ec199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.158+0000 7fd3e3fff640 1 -- 172.21.15.67:0/3903221024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3ec103a00 msgr2=0x7fd3ec19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:45.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.158+0000 7fd3e3fff640 1 --2- 172.21.15.67:0/3903221024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3ec103a00 0x7fd3ec19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.158+0000 7fd3e3fff640 1 -- 172.21.15.67:0/3903221024 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ec19b350 con 0x7fd3ec103060 2024-07-27T03:16:45.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.158+0000 7fd3e3fff640 1 --2- 172.21.15.67:0/3903221024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ec103060 0x7fd3ec199ff0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400d950 tx=0x7fd3d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:45.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.159+0000 7fd3e8ff9640 1 -- 172.21.15.67:0/3903221024 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4017030 con 0x7fd3ec103060 2024-07-27T03:16:45.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.159+0000 7fd3e8ff9640 1 -- 172.21.15.67:0/3903221024 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3d4004020 con 0x7fd3ec103060 2024-07-27T03:16:45.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.159+0000 7fd3f17e7640 1 -- 172.21.15.67:0/3903221024 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3ec19b640 con 0x7fd3ec103060 2024-07-27T03:16:45.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.159+0000 7fd3e8ff9640 1 -- 172.21.15.67:0/3903221024 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d40047a0 con 0x7fd3ec103060 2024-07-27T03:16:45.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.159+0000 7fd3f17e7640 1 -- 172.21.15.67:0/3903221024 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3ec1a1d30 con 0x7fd3ec103060 2024-07-27T03:16:45.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.160+0000 7fd3f17e7640 1 -- 172.21.15.67:0/3903221024 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3ec1a1fc0 con 0x7fd3ec103060 2024-07-27T03:16:45.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.161+0000 7fd3e8ff9640 1 -- 172.21.15.67:0/3903221024 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7fd3d4009a30 con 0x7fd3ec103060 2024-07-27T03:16:45.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.162+0000 7fd3e8ff9640 1 --2- 172.21.15.67:0/3903221024 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd3d80b3c50 0x7fd3d80b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:45.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.162+0000 7fd3e8ff9640 1 -- 172.21.15.67:0/3903221024 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fd3d40e39a0 con 0x7fd3ec103060 2024-07-27T03:16:45.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.162+0000 7fd3eaffd640 1 --2- 172.21.15.67:0/3903221024 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd3d80b3c50 0x7fd3d80b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:45.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.163+0000 7fd3eaffd640 1 --2- 172.21.15.67:0/3903221024 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd3d80b3c50 0x7fd3d80b6160 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009bd0 tx=0x7fd3dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:45.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.166+0000 7fd3e8ff9640 1 -- 172.21.15.67:0/3903221024 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fd3d40aac40 con 0x7fd3ec103060 2024-07-27T03:16:45.364 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.362+0000 7fd3f17e7640 1 -- 172.21.15.67:0/3903221024 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd3ec0ff860 con 0x7fd3d80b3c50 2024-07-27T03:16:45.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.368+0000 7fd3e8ff9640 1 -- 172.21.15.67:0/3903221024 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340567 (secure 0 0 0) 0x7fd3ec0ff860 con 0x7fd3d80b3c50 2024-07-27T03:16:45.375 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:16:45.377 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-07-27T03:16:45.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.372+0000 7fd3f17e7640 1 -- 172.21.15.67:0/3903221024 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd3d80b3c50 msgr2=0x7fd3d80b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:45.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.372+0000 7fd3f17e7640 1 --2- 172.21.15.67:0/3903221024 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd3d80b3c50 0x7fd3d80b6160 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009bd0 tx=0x7fd3dc005f90 comp rx=0 tx=0).stop 2024-07-27T03:16:45.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.372+0000 7fd3f17e7640 1 -- 172.21.15.67:0/3903221024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ec103060 msgr2=0x7fd3ec199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:45.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.372+0000 7fd3f17e7640 1 --2- 172.21.15.67:0/3903221024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ec103060 0x7fd3ec199ff0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400d950 tx=0x7fd3d400de10 comp rx=0 tx=0).stop 2024-07-27T03:16:45.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.372+0000 7fd3f17e7640 1 -- 172.21.15.67:0/3903221024 shutdown_connections 2024-07-27T03:16:45.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.372+0000 7fd3f17e7640 1 --2- 172.21.15.67:0/3903221024 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd3d80b3c50 0x7fd3d80b6160 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.372+0000 7fd3f17e7640 1 --2- 172.21.15.67:0/3903221024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd3ec103a00 0x7fd3ec19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.372+0000 7fd3f17e7640 1 --2- 172.21.15.67:0/3903221024 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd3ec103060 0x7fd3ec199ff0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.372+0000 7fd3f17e7640 1 --2- 172.21.15.67:0/3903221024 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3ec101fe0 0x7fd3ec199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.372+0000 7fd3f17e7640 1 -- 172.21.15.67:0/3903221024 >> 172.21.15.67:0/3903221024 conn(0x7fd3ec0fd920 msgr2=0x7fd3ec0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:45.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.372+0000 7fd3f17e7640 1 -- 172.21.15.67:0/3903221024 shutdown_connections 2024-07-27T03:16:45.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.372+0000 7fd3f17e7640 1 -- 172.21.15.67:0/3903221024 wait complete. 2024-07-27T03:16:45.386 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-07-27T03:16:45.119670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666090,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1957,"num_read_kb":1091,"num_write":388,"num_write_kb":2310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951979,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":820,"ondisk_log_size":820,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388604,"kb_used_data":14492,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553220,"statfs":{"total":1151940427776,"available":1151542497280,"internally_reserved":0,"allocated":14839808,"data_stored":8273360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5681,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":77,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003487"},"pg_stats":[{"pgid":"2.1f","version":"89'13","reported_seq":241,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:40.530537+0000","last_change":"2024-07-27T03:12:29.541107+0000","last_active":"2024-07-27T03:16:40.530537+0000","last_peered":"2024-07-27T03:16:40.530537+0000","last_clean":"2024-07-27T03:16:40.530537+0000","last_became_active":"2024-07-27T03:12:29.540806+0000","last_became_peered":"2024-07-27T03:12:29.540806+0000","last_unstale":"2024-07-27T03:16:40.530537+0000","last_undegraded":"2024-07-27T03:16:40.530537+0000","last_fullsized":"2024-07-27T03:16:40.530537+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:38:24.349354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":182,"num_read_kb":182,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.057193+0000","last_change":"2024-07-27T03:12:32.565515+0000","last_active":"2024-07-27T03:15:25.057193+0000","last_peered":"2024-07-27T03:15:25.057193+0000","last_clean":"2024-07-27T03:15:25.057193+0000","last_became_active":"2024-07-27T03:12:32.565134+0000","last_became_peered":"2024-07-27T03:12:32.565134+0000","last_unstale":"2024-07-27T03:15:25.057193+0000","last_undegraded":"2024-07-27T03:15:25.057193+0000","last_fullsized":"2024-07-27T03:15:25.057193+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:11:56.793139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302752+0000","last_change":"2024-07-27T03:12:42.613432+0000","last_active":"2024-07-27T03:13:57.302752+0000","last_peered":"2024-07-27T03:13:57.302752+0000","last_clean":"2024-07-27T03:13:57.302752+0000","last_became_active":"2024-07-27T03:12:42.612986+0000","last_became_peered":"2024-07-27T03:12:42.612986+0000","last_unstale":"2024-07-27T03:13:57.302752+0000","last_undegraded":"2024-07-27T03:13:57.302752+0000","last_fullsized":"2024-07-27T03:13:57.302752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:25:37.505117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:32.726614+0000","last_change":"2024-07-27T03:12:29.614229+0000","last_active":"2024-07-27T03:14:32.726614+0000","last_peered":"2024-07-27T03:14:32.726614+0000","last_clean":"2024-07-27T03:14:32.726614+0000","last_became_active":"2024-07-27T03:12:29.613844+0000","last_became_peered":"2024-07-27T03:12:29.613844+0000","last_unstale":"2024-07-27T03:14:32.726614+0000","last_undegraded":"2024-07-27T03:14:32.726614+0000","last_fullsized":"2024-07-27T03:14:32.726614+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:10:28.917080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.070362+0000","last_change":"2024-07-27T03:12:32.563187+0000","last_active":"2024-07-27T03:15:25.070362+0000","last_peered":"2024-07-27T03:15:25.070362+0000","last_clean":"2024-07-27T03:15:25.070362+0000","last_became_active":"2024-07-27T03:12:32.562674+0000","last_became_peered":"2024-07-27T03:12:32.562674+0000","last_unstale":"2024-07-27T03:15:25.070362+0000","last_undegraded":"2024-07-27T03:15:25.070362+0000","last_fullsized":"2024-07-27T03:15:25.070362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:05:39.211514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303151+0000","last_change":"2024-07-27T03:12:42.611516+0000","last_active":"2024-07-27T03:13:57.303151+0000","last_peered":"2024-07-27T03:13:57.303151+0000","last_clean":"2024-07-27T03:13:57.303151+0000","last_became_active":"2024-07-27T03:12:42.610934+0000","last_became_peered":"2024-07-27T03:12:42.610934+0000","last_unstale":"2024-07-27T03:13:57.303151+0000","last_undegraded":"2024-07-27T03:13:57.303151+0000","last_fullsized":"2024-07-27T03:13:57.303151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:13:17.323063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":58,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305740+0000","last_change":"2024-07-27T03:12:29.539573+0000","last_active":"2024-07-27T03:13:57.305740+0000","last_peered":"2024-07-27T03:13:57.305740+0000","last_clean":"2024-07-27T03:13:57.305740+0000","last_became_active":"2024-07-27T03:12:29.539056+0000","last_became_peered":"2024-07-27T03:12:29.539056+0000","last_unstale":"2024-07-27T03:13:57.305740+0000","last_undegraded":"2024-07-27T03:13:57.305740+0000","last_fullsized":"2024-07-27T03:13:57.305740+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:04:37.513644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.068109+0000","last_change":"2024-07-27T03:12:32.567729+0000","last_active":"2024-07-27T03:15:25.068109+0000","last_peered":"2024-07-27T03:15:25.068109+0000","last_clean":"2024-07-27T03:15:25.068109+0000","last_became_active":"2024-07-27T03:12:32.564827+0000","last_became_peered":"2024-07-27T03:12:32.564827+0000","last_unstale":"2024-07-27T03:15:25.068109+0000","last_undegraded":"2024-07-27T03:15:25.068109+0000","last_fullsized":"2024-07-27T03:15:25.068109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:10:55.314988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303339+0000","last_change":"2024-07-27T03:12:42.612776+0000","last_active":"2024-07-27T03:13:57.303339+0000","last_peered":"2024-07-27T03:13:57.303339+0000","last_clean":"2024-07-27T03:13:57.303339+0000","last_became_active":"2024-07-27T03:12:42.612258+0000","last_became_peered":"2024-07-27T03:12:42.612258+0000","last_unstale":"2024-07-27T03:13:57.303339+0000","last_undegraded":"2024-07-27T03:13:57.303339+0000","last_fullsized":"2024-07-27T03:13:57.303339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:48:06.516002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305959+0000","last_change":"2024-07-27T03:12:29.540070+0000","last_active":"2024-07-27T03:13:57.305959+0000","last_peered":"2024-07-27T03:13:57.305959+0000","last_clean":"2024-07-27T03:13:57.305959+0000","last_became_active":"2024-07-27T03:12:29.539657+0000","last_became_peered":"2024-07-27T03:12:29.539657+0000","last_unstale":"2024-07-27T03:13:57.305959+0000","last_undegraded":"2024-07-27T03:13:57.305959+0000","last_fullsized":"2024-07-27T03:13:57.305959+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:42:48.222613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.067431+0000","last_change":"2024-07-27T03:12:32.574418+0000","last_active":"2024-07-27T03:15:25.067431+0000","last_peered":"2024-07-27T03:15:25.067431+0000","last_clean":"2024-07-27T03:15:25.067431+0000","last_became_active":"2024-07-27T03:12:32.573974+0000","last_became_peered":"2024-07-27T03:12:32.573974+0000","last_unstale":"2024-07-27T03:15:25.067431+0000","last_undegraded":"2024-07-27T03:15:25.067431+0000","last_fullsized":"2024-07-27T03:15:25.067431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:34:30.442833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303642+0000","last_change":"2024-07-27T03:12:42.605155+0000","last_active":"2024-07-27T03:13:57.303642+0000","last_peered":"2024-07-27T03:13:57.303642+0000","last_clean":"2024-07-27T03:13:57.303642+0000","last_became_active":"2024-07-27T03:12:42.604685+0000","last_became_peered":"2024-07-27T03:12:42.604685+0000","last_unstale":"2024-07-27T03:13:57.303642+0000","last_undegraded":"2024-07-27T03:13:57.303642+0000","last_fullsized":"2024-07-27T03:13:57.303642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:28:58.931408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.065859+0000","last_change":"2024-07-27T03:12:32.574711+0000","last_active":"2024-07-27T03:15:25.065859+0000","last_peered":"2024-07-27T03:15:25.065859+0000","last_clean":"2024-07-27T03:15:25.065859+0000","last_became_active":"2024-07-27T03:12:32.574365+0000","last_became_peered":"2024-07-27T03:12:32.574365+0000","last_unstale":"2024-07-27T03:15:25.065859+0000","last_undegraded":"2024-07-27T03:15:25.065859+0000","last_fullsized":"2024-07-27T03:15:25.065859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:50:56.621371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305326+0000","last_change":"2024-07-27T03:12:29.540835+0000","last_active":"2024-07-27T03:13:57.305326+0000","last_peered":"2024-07-27T03:13:57.305326+0000","last_clean":"2024-07-27T03:13:57.305326+0000","last_became_active":"2024-07-27T03:12:29.540461+0000","last_became_peered":"2024-07-27T03:12:29.540461+0000","last_unstale":"2024-07-27T03:13:57.305326+0000","last_undegraded":"2024-07-27T03:13:57.305326+0000","last_fullsized":"2024-07-27T03:13:57.305326+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:19:20.817038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303344+0000","last_change":"2024-07-27T03:12:42.718584+0000","last_active":"2024-07-27T03:13:57.303344+0000","last_peered":"2024-07-27T03:13:57.303344+0000","last_clean":"2024-07-27T03:13:57.303344+0000","last_became_active":"2024-07-27T03:12:42.717752+0000","last_became_peered":"2024-07-27T03:12:42.717752+0000","last_unstale":"2024-07-27T03:13:57.303344+0000","last_undegraded":"2024-07-27T03:13:57.303344+0000","last_fullsized":"2024-07-27T03:13:57.303344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:23:52.629794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.042310+0000","last_change":"2024-07-27T03:12:32.560672+0000","last_active":"2024-07-27T03:15:25.042310+0000","last_peered":"2024-07-27T03:15:25.042310+0000","last_clean":"2024-07-27T03:15:25.042310+0000","last_became_active":"2024-07-27T03:12:32.559172+0000","last_became_peered":"2024-07-27T03:12:32.559172+0000","last_unstale":"2024-07-27T03:15:25.042310+0000","last_undegraded":"2024-07-27T03:15:25.042310+0000","last_fullsized":"2024-07-27T03:15:25.042310+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:02:49.346983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405206+0000","last_change":"2024-07-27T03:12:57.434569+0000","last_active":"2024-07-27T03:13:57.405206+0000","last_peered":"2024-07-27T03:13:57.405206+0000","last_clean":"2024-07-27T03:13:57.405206+0000","last_became_active":"2024-07-27T03:12:57.434038+0000","last_became_peered":"2024-07-27T03:12:57.434038+0000","last_unstale":"2024-07-27T03:13:57.405206+0000","last_undegraded":"2024-07-27T03:13:57.405206+0000","last_fullsized":"2024-07-27T03:13:57.405206+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:43:24.462838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.412383+0000","last_change":"2024-07-27T03:13:57.411931+0000","last_active":"2024-07-27T03:13:57.412383+0000","last_peered":"2024-07-27T03:13:57.412383+0000","last_clean":"2024-07-27T03:13:57.412383+0000","last_became_active":"2024-07-27T03:13:57.407362+0000","last_became_peered":"2024-07-27T03:13:57.407362+0000","last_unstale":"2024-07-27T03:13:57.412383+0000","last_undegraded":"2024-07-27T03:13:57.412383+0000","last_fullsized":"2024-07-27T03:13:57.412383+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:52:08.444268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.058449+0000","last_change":"2024-07-27T03:12:32.572129+0000","last_active":"2024-07-27T03:15:25.058449+0000","last_peered":"2024-07-27T03:15:25.058449+0000","last_clean":"2024-07-27T03:15:25.058449+0000","last_became_active":"2024-07-27T03:12:32.571774+0000","last_became_peered":"2024-07-27T03:12:32.571774+0000","last_unstale":"2024-07-27T03:15:25.058449+0000","last_undegraded":"2024-07-27T03:15:25.058449+0000","last_fullsized":"2024-07-27T03:15:25.058449+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:52:23.316957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404446+0000","last_change":"2024-07-27T03:12:29.539768+0000","last_active":"2024-07-27T03:13:57.404446+0000","last_peered":"2024-07-27T03:13:57.404446+0000","last_clean":"2024-07-27T03:13:57.404446+0000","last_became_active":"2024-07-27T03:12:29.539389+0000","last_became_peered":"2024-07-27T03:12:29.539389+0000","last_unstale":"2024-07-27T03:13:57.404446+0000","last_undegraded":"2024-07-27T03:13:57.404446+0000","last_fullsized":"2024-07-27T03:13:57.404446+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:09:35.961328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302623+0000","last_change":"2024-07-27T03:12:42.617569+0000","last_active":"2024-07-27T03:13:57.302623+0000","last_peered":"2024-07-27T03:13:57.302623+0000","last_clean":"2024-07-27T03:13:57.302623+0000","last_became_active":"2024-07-27T03:12:42.616357+0000","last_became_peered":"2024-07-27T03:12:42.616357+0000","last_unstale":"2024-07-27T03:13:57.302623+0000","last_undegraded":"2024-07-27T03:13:57.302623+0000","last_fullsized":"2024-07-27T03:13:57.302623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:35:11.297631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.018136+0000","last_change":"2024-07-27T03:12:32.566272+0000","last_active":"2024-07-27T03:15:25.018136+0000","last_peered":"2024-07-27T03:15:25.018136+0000","last_clean":"2024-07-27T03:15:25.018136+0000","last_became_active":"2024-07-27T03:12:32.565865+0000","last_became_peered":"2024-07-27T03:12:32.565865+0000","last_unstale":"2024-07-27T03:15:25.018136+0000","last_undegraded":"2024-07-27T03:15:25.018136+0000","last_fullsized":"2024-07-27T03:15:25.018136+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:42:09.539586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405359+0000","last_change":"2024-07-27T03:12:29.613733+0000","last_active":"2024-07-27T03:13:57.405359+0000","last_peered":"2024-07-27T03:13:57.405359+0000","last_clean":"2024-07-27T03:13:57.405359+0000","last_became_active":"2024-07-27T03:12:29.613395+0000","last_became_peered":"2024-07-27T03:12:29.613395+0000","last_unstale":"2024-07-27T03:13:57.405359+0000","last_undegraded":"2024-07-27T03:13:57.405359+0000","last_fullsized":"2024-07-27T03:13:57.405359+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:21:34.431900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.408340+0000","last_change":"2024-07-27T03:13:57.407798+0000","last_active":"2024-07-27T03:13:57.408340+0000","last_peered":"2024-07-27T03:13:57.408340+0000","last_clean":"2024-07-27T03:13:57.408340+0000","last_became_active":"2024-07-27T03:13:57.407275+0000","last_became_peered":"2024-07-27T03:13:57.407275+0000","last_unstale":"2024-07-27T03:13:57.408340+0000","last_undegraded":"2024-07-27T03:13:57.408340+0000","last_fullsized":"2024-07-27T03:13:57.408340+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:47:27.817092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,8],"acting":[4,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.056823+0000","last_change":"2024-07-27T03:12:32.567006+0000","last_active":"2024-07-27T03:15:25.056823+0000","last_peered":"2024-07-27T03:15:25.056823+0000","last_clean":"2024-07-27T03:15:25.056823+0000","last_became_active":"2024-07-27T03:12:32.566205+0000","last_became_peered":"2024-07-27T03:12:32.566205+0000","last_unstale":"2024-07-27T03:15:25.056823+0000","last_undegraded":"2024-07-27T03:15:25.056823+0000","last_fullsized":"2024-07-27T03:15:25.056823+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:11:23.171341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303695+0000","last_change":"2024-07-27T03:12:29.533909+0000","last_active":"2024-07-27T03:13:57.303695+0000","last_peered":"2024-07-27T03:13:57.303695+0000","last_clean":"2024-07-27T03:13:57.303695+0000","last_became_active":"2024-07-27T03:12:29.533327+0000","last_became_peered":"2024-07-27T03:12:29.533327+0000","last_unstale":"2024-07-27T03:13:57.303695+0000","last_undegraded":"2024-07-27T03:13:57.303695+0000","last_fullsized":"2024-07-27T03:13:57.303695+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:35:24.705096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302923+0000","last_change":"2024-07-27T03:12:42.609066+0000","last_active":"2024-07-27T03:13:57.302923+0000","last_peered":"2024-07-27T03:13:57.302923+0000","last_clean":"2024-07-27T03:13:57.302923+0000","last_became_active":"2024-07-27T03:12:42.608561+0000","last_became_peered":"2024-07-27T03:12:42.608561+0000","last_unstale":"2024-07-27T03:13:57.302923+0000","last_undegraded":"2024-07-27T03:13:57.302923+0000","last_fullsized":"2024-07-27T03:13:57.302923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:27:53.657498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.053587+0000","last_change":"2024-07-27T03:12:32.560931+0000","last_active":"2024-07-27T03:15:25.053587+0000","last_peered":"2024-07-27T03:15:25.053587+0000","last_clean":"2024-07-27T03:15:25.053587+0000","last_became_active":"2024-07-27T03:12:32.560549+0000","last_became_peered":"2024-07-27T03:12:32.560549+0000","last_unstale":"2024-07-27T03:15:25.053587+0000","last_undegraded":"2024-07-27T03:15:25.053587+0000","last_fullsized":"2024-07-27T03:15:25.053587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T15:11:25.136035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'17","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.595553+0000","last_change":"2024-07-27T03:12:29.535365+0000","last_active":"2024-07-27T03:15:25.595553+0000","last_peered":"2024-07-27T03:15:25.595553+0000","last_clean":"2024-07-27T03:15:25.595553+0000","last_became_active":"2024-07-27T03:12:29.534952+0000","last_became_peered":"2024-07-27T03:12:29.534952+0000","last_unstale":"2024-07-27T03:15:25.595553+0000","last_undegraded":"2024-07-27T03:15:25.595553+0000","last_fullsized":"2024-07-27T03:15:25.595553+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:34:59.889488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:42.727465+0000","last_change":"2024-07-27T03:12:42.718416+0000","last_active":"2024-07-27T03:14:42.727465+0000","last_peered":"2024-07-27T03:14:42.727465+0000","last_clean":"2024-07-27T03:14:42.727465+0000","last_became_active":"2024-07-27T03:12:42.717949+0000","last_became_peered":"2024-07-27T03:12:42.717949+0000","last_unstale":"2024-07-27T03:14:42.727465+0000","last_undegraded":"2024-07-27T03:14:42.727465+0000","last_fullsized":"2024-07-27T03:14:42.727465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:05:05.762577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.057594+0000","last_change":"2024-07-27T03:12:32.559824+0000","last_active":"2024-07-27T03:15:25.057594+0000","last_peered":"2024-07-27T03:15:25.057594+0000","last_clean":"2024-07-27T03:15:25.057594+0000","last_became_active":"2024-07-27T03:12:32.559435+0000","last_became_peered":"2024-07-27T03:12:32.559435+0000","last_unstale":"2024-07-27T03:15:25.057594+0000","last_undegraded":"2024-07-27T03:15:25.057594+0000","last_fullsized":"2024-07-27T03:15:25.057594+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:16:44.514272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:35.285402+0000","last_change":"2024-07-27T03:12:29.614296+0000","last_active":"2024-07-27T03:14:35.285402+0000","last_peered":"2024-07-27T03:14:35.285402+0000","last_clean":"2024-07-27T03:14:35.285402+0000","last_became_active":"2024-07-27T03:12:29.612987+0000","last_became_peered":"2024-07-27T03:12:29.612987+0000","last_unstale":"2024-07-27T03:14:35.285402+0000","last_undegraded":"2024-07-27T03:14:35.285402+0000","last_fullsized":"2024-07-27T03:14:35.285402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:23:09.226523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405200+0000","last_change":"2024-07-27T03:12:42.616575+0000","last_active":"2024-07-27T03:13:57.405200+0000","last_peered":"2024-07-27T03:13:57.405200+0000","last_clean":"2024-07-27T03:13:57.405200+0000","last_became_active":"2024-07-27T03:12:42.615980+0000","last_became_peered":"2024-07-27T03:12:42.615980+0000","last_unstale":"2024-07-27T03:13:57.405200+0000","last_undegraded":"2024-07-27T03:13:57.405200+0000","last_fullsized":"2024-07-27T03:13:57.405200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:00:59.041910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.062700+0000","last_change":"2024-07-27T03:12:32.556109+0000","last_active":"2024-07-27T03:15:25.062700+0000","last_peered":"2024-07-27T03:15:25.062700+0000","last_clean":"2024-07-27T03:15:25.062700+0000","last_became_active":"2024-07-27T03:12:32.555747+0000","last_became_peered":"2024-07-27T03:12:32.555747+0000","last_unstale":"2024-07-27T03:15:25.062700+0000","last_undegraded":"2024-07-27T03:15:25.062700+0000","last_fullsized":"2024-07-27T03:15:25.062700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:27:58.286605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:36.101756+0000","last_change":"2024-07-27T03:12:29.613523+0000","last_active":"2024-07-27T03:14:36.101756+0000","last_peered":"2024-07-27T03:14:36.101756+0000","last_clean":"2024-07-27T03:14:36.101756+0000","last_became_active":"2024-07-27T03:12:29.613172+0000","last_became_peered":"2024-07-27T03:12:29.613172+0000","last_unstale":"2024-07-27T03:14:36.101756+0000","last_undegraded":"2024-07-27T03:14:36.101756+0000","last_fullsized":"2024-07-27T03:14:36.101756+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:49:45.225221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304930+0000","last_change":"2024-07-27T03:12:42.718536+0000","last_active":"2024-07-27T03:13:57.304930+0000","last_peered":"2024-07-27T03:13:57.304930+0000","last_clean":"2024-07-27T03:13:57.304930+0000","last_became_active":"2024-07-27T03:12:42.718068+0000","last_became_peered":"2024-07-27T03:12:42.718068+0000","last_unstale":"2024-07-27T03:13:57.304930+0000","last_undegraded":"2024-07-27T03:13:57.304930+0000","last_fullsized":"2024-07-27T03:13:57.304930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:18:28.478700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.064182+0000","last_change":"2024-07-27T03:12:32.558060+0000","last_active":"2024-07-27T03:15:25.064182+0000","last_peered":"2024-07-27T03:15:25.064182+0000","last_clean":"2024-07-27T03:15:25.064182+0000","last_became_active":"2024-07-27T03:12:32.557260+0000","last_became_peered":"2024-07-27T03:12:32.557260+0000","last_unstale":"2024-07-27T03:15:25.064182+0000","last_undegraded":"2024-07-27T03:15:25.064182+0000","last_fullsized":"2024-07-27T03:15:25.064182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:36:54.945761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":234,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:41.532322+0000","last_change":"2024-07-27T03:12:29.541080+0000","last_active":"2024-07-27T03:16:41.532322+0000","last_peered":"2024-07-27T03:16:41.532322+0000","last_clean":"2024-07-27T03:16:41.532322+0000","last_became_active":"2024-07-27T03:12:29.540628+0000","last_became_peered":"2024-07-27T03:12:29.540628+0000","last_unstale":"2024-07-27T03:16:41.532322+0000","last_undegraded":"2024-07-27T03:16:41.532322+0000","last_fullsized":"2024-07-27T03:16:41.532322+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:45:17.045124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":183,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348865+0000","last_change":"2024-07-27T03:12:42.611561+0000","last_active":"2024-07-27T03:13:57.348865+0000","last_peered":"2024-07-27T03:13:57.348865+0000","last_clean":"2024-07-27T03:13:57.348865+0000","last_became_active":"2024-07-27T03:12:42.610507+0000","last_became_peered":"2024-07-27T03:12:42.610507+0000","last_unstale":"2024-07-27T03:13:57.348865+0000","last_undegraded":"2024-07-27T03:13:57.348865+0000","last_fullsized":"2024-07-27T03:13:57.348865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:15:08.972807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.058339+0000","last_change":"2024-07-27T03:12:32.562316+0000","last_active":"2024-07-27T03:15:25.058339+0000","last_peered":"2024-07-27T03:15:25.058339+0000","last_clean":"2024-07-27T03:15:25.058339+0000","last_became_active":"2024-07-27T03:12:32.561729+0000","last_became_peered":"2024-07-27T03:12:32.561729+0000","last_unstale":"2024-07-27T03:15:25.058339+0000","last_undegraded":"2024-07-27T03:15:25.058339+0000","last_fullsized":"2024-07-27T03:15:25.058339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:01:19.960269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":238,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:42.533767+0000","last_change":"2024-07-27T03:12:29.613459+0000","last_active":"2024-07-27T03:16:42.533767+0000","last_peered":"2024-07-27T03:16:42.533767+0000","last_clean":"2024-07-27T03:16:42.533767+0000","last_became_active":"2024-07-27T03:12:29.613009+0000","last_became_peered":"2024-07-27T03:12:29.613009+0000","last_unstale":"2024-07-27T03:16:42.533767+0000","last_undegraded":"2024-07-27T03:16:42.533767+0000","last_fullsized":"2024-07-27T03:16:42.533767+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:16:37.230471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":185,"num_read_kb":185,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348695+0000","last_change":"2024-07-27T03:12:42.718516+0000","last_active":"2024-07-27T03:13:57.348695+0000","last_peered":"2024-07-27T03:13:57.348695+0000","last_clean":"2024-07-27T03:13:57.348695+0000","last_became_active":"2024-07-27T03:12:42.718041+0000","last_became_peered":"2024-07-27T03:12:42.718041+0000","last_unstale":"2024-07-27T03:13:57.348695+0000","last_undegraded":"2024-07-27T03:13:57.348695+0000","last_fullsized":"2024-07-27T03:13:57.348695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:35:39.633977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.063195+0000","last_change":"2024-07-27T03:12:32.560752+0000","last_active":"2024-07-27T03:15:25.063195+0000","last_peered":"2024-07-27T03:15:25.063195+0000","last_clean":"2024-07-27T03:15:25.063195+0000","last_became_active":"2024-07-27T03:12:32.560248+0000","last_became_peered":"2024-07-27T03:12:32.560248+0000","last_unstale":"2024-07-27T03:15:25.063195+0000","last_undegraded":"2024-07-27T03:15:25.063195+0000","last_fullsized":"2024-07-27T03:15:25.063195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:59:48.056867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:31.988172+0000","last_change":"2024-07-27T03:12:29.539591+0000","last_active":"2024-07-27T03:14:31.988172+0000","last_peered":"2024-07-27T03:14:31.988172+0000","last_clean":"2024-07-27T03:14:31.988172+0000","last_became_active":"2024-07-27T03:12:29.539165+0000","last_became_peered":"2024-07-27T03:12:29.539165+0000","last_unstale":"2024-07-27T03:14:31.988172+0000","last_undegraded":"2024-07-27T03:14:31.988172+0000","last_fullsized":"2024-07-27T03:14:31.988172+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:04:21.940511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:00.281510+0000","last_change":"2024-07-27T03:13:57.407921+0000","last_active":"2024-07-27T03:14:00.281510+0000","last_peered":"2024-07-27T03:14:00.281510+0000","last_clean":"2024-07-27T03:14:00.281510+0000","last_became_active":"2024-07-27T03:13:57.407382+0000","last_became_peered":"2024-07-27T03:13:57.407382+0000","last_unstale":"2024-07-27T03:14:00.281510+0000","last_undegraded":"2024-07-27T03:14:00.281510+0000","last_fullsized":"2024-07-27T03:14:00.281510+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:48:27.362759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.069802+0000","last_change":"2024-07-27T03:12:32.570653+0000","last_active":"2024-07-27T03:15:25.069802+0000","last_peered":"2024-07-27T03:15:25.069802+0000","last_clean":"2024-07-27T03:15:25.069802+0000","last_became_active":"2024-07-27T03:12:32.569088+0000","last_became_peered":"2024-07-27T03:12:32.569088+0000","last_unstale":"2024-07-27T03:15:25.069802+0000","last_undegraded":"2024-07-27T03:15:25.069802+0000","last_fullsized":"2024-07-27T03:15:25.069802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:50:24.669431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:35.895703+0000","last_change":"2024-07-27T03:12:29.541654+0000","last_active":"2024-07-27T03:14:35.895703+0000","last_peered":"2024-07-27T03:14:35.895703+0000","last_clean":"2024-07-27T03:14:35.895703+0000","last_became_active":"2024-07-27T03:12:29.541264+0000","last_became_peered":"2024-07-27T03:12:29.541264+0000","last_unstale":"2024-07-27T03:14:35.895703+0000","last_undegraded":"2024-07-27T03:14:35.895703+0000","last_fullsized":"2024-07-27T03:14:35.895703+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:48:05.179016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":123,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:01.984277+0000","last_change":"2024-07-27T03:11:11.343199+0000","last_active":"2024-07-27T03:14:01.984277+0000","last_peered":"2024-07-27T03:14:01.984277+0000","last_clean":"2024-07-27T03:14:01.984277+0000","last_became_active":"2024-07-27T03:11:11.337635+0000","last_became_peered":"2024-07-27T03:11:11.337635+0000","last_unstale":"2024-07-27T03:14:01.984277+0000","last_undegraded":"2024-07-27T03:14:01.984277+0000","last_fullsized":"2024-07-27T03:14:01.984277+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:08:01.694830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:08:01.694830+0000","last_clean_scrub_stamp":"2024-07-27T03:08:01.694830+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:48:49.979610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305836+0000","last_change":"2024-07-27T03:12:42.614346+0000","last_active":"2024-07-27T03:13:57.305836+0000","last_peered":"2024-07-27T03:13:57.305836+0000","last_clean":"2024-07-27T03:13:57.305836+0000","last_became_active":"2024-07-27T03:12:42.613881+0000","last_became_peered":"2024-07-27T03:12:42.613881+0000","last_unstale":"2024-07-27T03:13:57.305836+0000","last_undegraded":"2024-07-27T03:13:57.305836+0000","last_fullsized":"2024-07-27T03:13:57.305836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:12:38.299348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.058877+0000","last_change":"2024-07-27T03:12:32.574330+0000","last_active":"2024-07-27T03:15:25.058877+0000","last_peered":"2024-07-27T03:15:25.058877+0000","last_clean":"2024-07-27T03:15:25.058877+0000","last_became_active":"2024-07-27T03:12:32.573977+0000","last_became_peered":"2024-07-27T03:12:32.573977+0000","last_unstale":"2024-07-27T03:15:25.058877+0000","last_undegraded":"2024-07-27T03:15:25.058877+0000","last_fullsized":"2024-07-27T03:15:25.058877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:39:09.925794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":237,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:42.534600+0000","last_change":"2024-07-27T03:12:29.613848+0000","last_active":"2024-07-27T03:16:42.534600+0000","last_peered":"2024-07-27T03:16:42.534600+0000","last_clean":"2024-07-27T03:16:42.534600+0000","last_became_active":"2024-07-27T03:12:29.613473+0000","last_became_peered":"2024-07-27T03:12:29.613473+0000","last_unstale":"2024-07-27T03:16:42.534600+0000","last_undegraded":"2024-07-27T03:16:42.534600+0000","last_fullsized":"2024-07-27T03:16:42.534600+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:52:56.894962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":186,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405373+0000","last_change":"2024-07-27T03:12:42.611295+0000","last_active":"2024-07-27T03:13:57.405373+0000","last_peered":"2024-07-27T03:13:57.405373+0000","last_clean":"2024-07-27T03:13:57.405373+0000","last_became_active":"2024-07-27T03:12:42.610782+0000","last_became_peered":"2024-07-27T03:12:42.610782+0000","last_unstale":"2024-07-27T03:13:57.405373+0000","last_undegraded":"2024-07-27T03:13:57.405373+0000","last_fullsized":"2024-07-27T03:13:57.405373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:24:02.849882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.061556+0000","last_change":"2024-07-27T03:12:56.725924+0000","last_active":"2024-07-27T03:15:25.061556+0000","last_peered":"2024-07-27T03:15:25.061556+0000","last_clean":"2024-07-27T03:15:25.061556+0000","last_became_active":"2024-07-27T03:12:56.725567+0000","last_became_peered":"2024-07-27T03:12:56.725567+0000","last_unstale":"2024-07-27T03:15:25.061556+0000","last_undegraded":"2024-07-27T03:15:25.061556+0000","last_fullsized":"2024-07-27T03:15:25.061556+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:32:23.771483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405225+0000","last_change":"2024-07-27T03:12:29.537863+0000","last_active":"2024-07-27T03:13:57.405225+0000","last_peered":"2024-07-27T03:13:57.405225+0000","last_clean":"2024-07-27T03:13:57.405225+0000","last_became_active":"2024-07-27T03:12:29.537444+0000","last_became_peered":"2024-07-27T03:12:29.537444+0000","last_unstale":"2024-07-27T03:13:57.405225+0000","last_undegraded":"2024-07-27T03:13:57.405225+0000","last_fullsized":"2024-07-27T03:13:57.405225+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:52:05.239809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405265+0000","last_change":"2024-07-27T03:12:42.622599+0000","last_active":"2024-07-27T03:13:57.405265+0000","last_peered":"2024-07-27T03:13:57.405265+0000","last_clean":"2024-07-27T03:13:57.405265+0000","last_became_active":"2024-07-27T03:12:42.622151+0000","last_became_peered":"2024-07-27T03:12:42.622151+0000","last_unstale":"2024-07-27T03:13:57.405265+0000","last_undegraded":"2024-07-27T03:13:57.405265+0000","last_fullsized":"2024-07-27T03:13:57.405265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:34:31.622281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.050080+0000","last_change":"2024-07-27T03:12:32.562300+0000","last_active":"2024-07-27T03:15:25.050080+0000","last_peered":"2024-07-27T03:15:25.050080+0000","last_clean":"2024-07-27T03:15:25.050080+0000","last_became_active":"2024-07-27T03:12:32.561343+0000","last_became_peered":"2024-07-27T03:12:32.561343+0000","last_unstale":"2024-07-27T03:15:25.050080+0000","last_undegraded":"2024-07-27T03:15:25.050080+0000","last_fullsized":"2024-07-27T03:15:25.050080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:01:28.626943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348835+0000","last_change":"2024-07-27T03:12:29.535177+0000","last_active":"2024-07-27T03:13:57.348835+0000","last_peered":"2024-07-27T03:13:57.348835+0000","last_clean":"2024-07-27T03:13:57.348835+0000","last_became_active":"2024-07-27T03:12:29.534662+0000","last_became_peered":"2024-07-27T03:12:29.534662+0000","last_unstale":"2024-07-27T03:13:57.348835+0000","last_undegraded":"2024-07-27T03:13:57.348835+0000","last_fullsized":"2024-07-27T03:13:57.348835+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:50:49.661992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404084+0000","last_change":"2024-07-27T03:12:42.609925+0000","last_active":"2024-07-27T03:13:57.404084+0000","last_peered":"2024-07-27T03:13:57.404084+0000","last_clean":"2024-07-27T03:13:57.404084+0000","last_became_active":"2024-07-27T03:12:42.609456+0000","last_became_peered":"2024-07-27T03:12:42.609456+0000","last_unstale":"2024-07-27T03:13:57.404084+0000","last_undegraded":"2024-07-27T03:13:57.404084+0000","last_fullsized":"2024-07-27T03:13:57.404084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:07:52.128381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.052969+0000","last_change":"2024-07-27T03:12:56.726170+0000","last_active":"2024-07-27T03:15:25.052969+0000","last_peered":"2024-07-27T03:15:25.052969+0000","last_clean":"2024-07-27T03:15:25.052969+0000","last_became_active":"2024-07-27T03:12:56.725685+0000","last_became_peered":"2024-07-27T03:12:56.725685+0000","last_unstale":"2024-07-27T03:15:25.052969+0000","last_undegraded":"2024-07-27T03:15:25.052969+0000","last_fullsized":"2024-07-27T03:15:25.052969+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:53:41.143333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405969+0000","last_change":"2024-07-27T03:12:56.726059+0000","last_active":"2024-07-27T03:13:57.405969+0000","last_peered":"2024-07-27T03:13:57.405969+0000","last_clean":"2024-07-27T03:13:57.405969+0000","last_became_active":"2024-07-27T03:12:56.722294+0000","last_became_peered":"2024-07-27T03:12:56.722294+0000","last_unstale":"2024-07-27T03:13:57.405969+0000","last_undegraded":"2024-07-27T03:13:57.405969+0000","last_fullsized":"2024-07-27T03:13:57.405969+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:18:42.627768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404320+0000","last_change":"2024-07-27T03:12:42.611966+0000","last_active":"2024-07-27T03:13:57.404320+0000","last_peered":"2024-07-27T03:13:57.404320+0000","last_clean":"2024-07-27T03:13:57.404320+0000","last_became_active":"2024-07-27T03:12:42.611522+0000","last_became_peered":"2024-07-27T03:12:42.611522+0000","last_unstale":"2024-07-27T03:13:57.404320+0000","last_undegraded":"2024-07-27T03:13:57.404320+0000","last_fullsized":"2024-07-27T03:13:57.404320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:09:54.034597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.069429+0000","last_change":"2024-07-27T03:12:56.719638+0000","last_active":"2024-07-27T03:15:25.069429+0000","last_peered":"2024-07-27T03:15:25.069429+0000","last_clean":"2024-07-27T03:15:25.069429+0000","last_became_active":"2024-07-27T03:12:56.719227+0000","last_became_peered":"2024-07-27T03:12:56.719227+0000","last_unstale":"2024-07-27T03:15:25.069429+0000","last_undegraded":"2024-07-27T03:15:25.069429+0000","last_fullsized":"2024-07-27T03:15:25.069429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:54:16.891518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304967+0000","last_change":"2024-07-27T03:12:29.614346+0000","last_active":"2024-07-27T03:13:57.304967+0000","last_peered":"2024-07-27T03:13:57.304967+0000","last_clean":"2024-07-27T03:13:57.304967+0000","last_became_active":"2024-07-27T03:12:29.613672+0000","last_became_peered":"2024-07-27T03:12:29.613672+0000","last_unstale":"2024-07-27T03:13:57.304967+0000","last_undegraded":"2024-07-27T03:13:57.304967+0000","last_fullsized":"2024-07-27T03:13:57.304967+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:46:35.505735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348712+0000","last_change":"2024-07-27T03:12:42.618364+0000","last_active":"2024-07-27T03:13:57.348712+0000","last_peered":"2024-07-27T03:13:57.348712+0000","last_clean":"2024-07-27T03:13:57.348712+0000","last_became_active":"2024-07-27T03:12:42.617988+0000","last_became_peered":"2024-07-27T03:12:42.617988+0000","last_unstale":"2024-07-27T03:13:57.348712+0000","last_undegraded":"2024-07-27T03:13:57.348712+0000","last_fullsized":"2024-07-27T03:13:57.348712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:40:21.725877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.061231+0000","last_change":"2024-07-27T03:12:32.563809+0000","last_active":"2024-07-27T03:15:25.061231+0000","last_peered":"2024-07-27T03:15:25.061231+0000","last_clean":"2024-07-27T03:15:25.061231+0000","last_became_active":"2024-07-27T03:12:32.563479+0000","last_became_peered":"2024-07-27T03:12:32.563479+0000","last_unstale":"2024-07-27T03:15:25.061231+0000","last_undegraded":"2024-07-27T03:15:25.061231+0000","last_fullsized":"2024-07-27T03:15:25.061231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:45:47.719241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":238,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:39.530182+0000","last_change":"2024-07-27T03:12:29.543786+0000","last_active":"2024-07-27T03:16:39.530182+0000","last_peered":"2024-07-27T03:16:39.530182+0000","last_clean":"2024-07-27T03:16:39.530182+0000","last_became_active":"2024-07-27T03:12:29.543203+0000","last_became_peered":"2024-07-27T03:12:29.543203+0000","last_unstale":"2024-07-27T03:16:39.530182+0000","last_undegraded":"2024-07-27T03:16:39.530182+0000","last_fullsized":"2024-07-27T03:16:39.530182+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:47:06.742171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":186,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302718+0000","last_change":"2024-07-27T03:12:42.622142+0000","last_active":"2024-07-27T03:13:57.302718+0000","last_peered":"2024-07-27T03:13:57.302718+0000","last_clean":"2024-07-27T03:13:57.302718+0000","last_became_active":"2024-07-27T03:12:42.621681+0000","last_became_peered":"2024-07-27T03:12:42.621681+0000","last_unstale":"2024-07-27T03:13:57.302718+0000","last_undegraded":"2024-07-27T03:13:57.302718+0000","last_fullsized":"2024-07-27T03:13:57.302718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:05:08.168424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.066031+0000","last_change":"2024-07-27T03:12:32.567119+0000","last_active":"2024-07-27T03:15:25.066031+0000","last_peered":"2024-07-27T03:15:25.066031+0000","last_clean":"2024-07-27T03:15:25.066031+0000","last_became_active":"2024-07-27T03:12:32.566757+0000","last_became_peered":"2024-07-27T03:12:32.566757+0000","last_unstale":"2024-07-27T03:15:25.066031+0000","last_undegraded":"2024-07-27T03:15:25.066031+0000","last_fullsized":"2024-07-27T03:15:25.066031+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:31:12.972804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404725+0000","last_change":"2024-07-27T03:12:29.540453+0000","last_active":"2024-07-27T03:13:57.404725+0000","last_peered":"2024-07-27T03:13:57.404725+0000","last_clean":"2024-07-27T03:13:57.404725+0000","last_became_active":"2024-07-27T03:12:29.539195+0000","last_became_peered":"2024-07-27T03:12:29.539195+0000","last_unstale":"2024-07-27T03:13:57.404725+0000","last_undegraded":"2024-07-27T03:13:57.404725+0000","last_fullsized":"2024-07-27T03:13:57.404725+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:44:31.144845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405363+0000","last_change":"2024-07-27T03:12:42.616671+0000","last_active":"2024-07-27T03:13:57.405363+0000","last_peered":"2024-07-27T03:13:57.405363+0000","last_clean":"2024-07-27T03:13:57.405363+0000","last_became_active":"2024-07-27T03:12:42.616198+0000","last_became_peered":"2024-07-27T03:12:42.616198+0000","last_unstale":"2024-07-27T03:13:57.405363+0000","last_undegraded":"2024-07-27T03:13:57.405363+0000","last_fullsized":"2024-07-27T03:13:57.405363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:01:31.584686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.060885+0000","last_change":"2024-07-27T03:12:32.562350+0000","last_active":"2024-07-27T03:15:25.060885+0000","last_peered":"2024-07-27T03:15:25.060885+0000","last_clean":"2024-07-27T03:15:25.060885+0000","last_became_active":"2024-07-27T03:12:32.562016+0000","last_became_peered":"2024-07-27T03:12:32.562016+0000","last_unstale":"2024-07-27T03:15:25.060885+0000","last_undegraded":"2024-07-27T03:15:25.060885+0000","last_fullsized":"2024-07-27T03:15:25.060885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:44:35.027329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303241+0000","last_change":"2024-07-27T03:12:29.541387+0000","last_active":"2024-07-27T03:13:57.303241+0000","last_peered":"2024-07-27T03:13:57.303241+0000","last_clean":"2024-07-27T03:13:57.303241+0000","last_became_active":"2024-07-27T03:12:29.541005+0000","last_became_peered":"2024-07-27T03:12:29.541005+0000","last_unstale":"2024-07-27T03:13:57.303241+0000","last_undegraded":"2024-07-27T03:13:57.303241+0000","last_fullsized":"2024-07-27T03:13:57.303241+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:22:16.268348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404849+0000","last_change":"2024-07-27T03:12:42.610029+0000","last_active":"2024-07-27T03:13:57.404849+0000","last_peered":"2024-07-27T03:13:57.404849+0000","last_clean":"2024-07-27T03:13:57.404849+0000","last_became_active":"2024-07-27T03:12:42.609592+0000","last_became_peered":"2024-07-27T03:12:42.609592+0000","last_unstale":"2024-07-27T03:13:57.404849+0000","last_undegraded":"2024-07-27T03:13:57.404849+0000","last_fullsized":"2024-07-27T03:13:57.404849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:47:15.254770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.051282+0000","last_change":"2024-07-27T03:12:32.568941+0000","last_active":"2024-07-27T03:15:25.051282+0000","last_peered":"2024-07-27T03:15:25.051282+0000","last_clean":"2024-07-27T03:15:25.051282+0000","last_became_active":"2024-07-27T03:12:32.568531+0000","last_became_peered":"2024-07-27T03:12:32.568531+0000","last_unstale":"2024-07-27T03:15:25.051282+0000","last_undegraded":"2024-07-27T03:15:25.051282+0000","last_fullsized":"2024-07-27T03:15:25.051282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:06:22.020505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'41","reported_seq":468,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:44.537754+0000","last_change":"2024-07-27T03:12:29.545131+0000","last_active":"2024-07-27T03:16:44.537754+0000","last_peered":"2024-07-27T03:16:44.537754+0000","last_clean":"2024-07-27T03:16:44.537754+0000","last_became_active":"2024-07-27T03:12:29.544768+0000","last_became_peered":"2024-07-27T03:12:29.544768+0000","last_unstale":"2024-07-27T03:16:44.537754+0000","last_undegraded":"2024-07-27T03:16:44.537754+0000","last_fullsized":"2024-07-27T03:16:44.537754+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:20:57.556128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68002,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":382,"num_read_kb":258,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302087+0000","last_change":"2024-07-27T03:12:42.617483+0000","last_active":"2024-07-27T03:13:57.302087+0000","last_peered":"2024-07-27T03:13:57.302087+0000","last_clean":"2024-07-27T03:13:57.302087+0000","last_became_active":"2024-07-27T03:12:42.617065+0000","last_became_peered":"2024-07-27T03:12:42.617065+0000","last_unstale":"2024-07-27T03:13:57.302087+0000","last_undegraded":"2024-07-27T03:13:57.302087+0000","last_fullsized":"2024-07-27T03:13:57.302087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:52:17.448919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.069655+0000","last_change":"2024-07-27T03:12:56.722257+0000","last_active":"2024-07-27T03:15:25.069655+0000","last_peered":"2024-07-27T03:15:25.069655+0000","last_clean":"2024-07-27T03:15:25.069655+0000","last_became_active":"2024-07-27T03:12:56.721773+0000","last_became_peered":"2024-07-27T03:12:56.721773+0000","last_unstale":"2024-07-27T03:15:25.069655+0000","last_undegraded":"2024-07-27T03:15:25.069655+0000","last_fullsized":"2024-07-27T03:15:25.069655+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:30:44.310705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405205+0000","last_change":"2024-07-27T03:12:29.614171+0000","last_active":"2024-07-27T03:13:57.405205+0000","last_peered":"2024-07-27T03:13:57.405205+0000","last_clean":"2024-07-27T03:13:57.405205+0000","last_became_active":"2024-07-27T03:12:29.613270+0000","last_became_peered":"2024-07-27T03:12:29.613270+0000","last_unstale":"2024-07-27T03:13:57.405205+0000","last_undegraded":"2024-07-27T03:13:57.405205+0000","last_fullsized":"2024-07-27T03:13:57.405205+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:39:32.283778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303808+0000","last_change":"2024-07-27T03:12:42.610375+0000","last_active":"2024-07-27T03:13:57.303808+0000","last_peered":"2024-07-27T03:13:57.303808+0000","last_clean":"2024-07-27T03:13:57.303808+0000","last_became_active":"2024-07-27T03:12:42.609957+0000","last_became_peered":"2024-07-27T03:12:42.609957+0000","last_unstale":"2024-07-27T03:13:57.303808+0000","last_undegraded":"2024-07-27T03:13:57.303808+0000","last_fullsized":"2024-07-27T03:13:57.303808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:11:02.707179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.064641+0000","last_change":"2024-07-27T03:12:56.722332+0000","last_active":"2024-07-27T03:15:25.064641+0000","last_peered":"2024-07-27T03:15:25.064641+0000","last_clean":"2024-07-27T03:15:25.064641+0000","last_became_active":"2024-07-27T03:12:56.721972+0000","last_became_peered":"2024-07-27T03:12:56.721972+0000","last_unstale":"2024-07-27T03:15:25.064641+0000","last_undegraded":"2024-07-27T03:15:25.064641+0000","last_fullsized":"2024-07-27T03:15:25.064641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:54:31.880137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306094+0000","last_change":"2024-07-27T03:12:29.541051+0000","last_active":"2024-07-27T03:13:57.306094+0000","last_peered":"2024-07-27T03:13:57.306094+0000","last_clean":"2024-07-27T03:13:57.306094+0000","last_became_active":"2024-07-27T03:12:29.540619+0000","last_became_peered":"2024-07-27T03:12:29.540619+0000","last_unstale":"2024-07-27T03:13:57.306094+0000","last_undegraded":"2024-07-27T03:13:57.306094+0000","last_fullsized":"2024-07-27T03:13:57.306094+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:23:27.587419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304810+0000","last_change":"2024-07-27T03:12:56.720496+0000","last_active":"2024-07-27T03:13:57.304810+0000","last_peered":"2024-07-27T03:13:57.304810+0000","last_clean":"2024-07-27T03:13:57.304810+0000","last_became_active":"2024-07-27T03:12:56.719784+0000","last_became_peered":"2024-07-27T03:12:56.719784+0000","last_unstale":"2024-07-27T03:13:57.304810+0000","last_undegraded":"2024-07-27T03:13:57.304810+0000","last_fullsized":"2024-07-27T03:13:57.304810+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:07:20.518139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.069653+0000","last_change":"2024-07-27T03:12:32.568409+0000","last_active":"2024-07-27T03:15:25.069653+0000","last_peered":"2024-07-27T03:15:25.069653+0000","last_clean":"2024-07-27T03:15:25.069653+0000","last_became_active":"2024-07-27T03:12:32.568029+0000","last_became_peered":"2024-07-27T03:12:32.568029+0000","last_unstale":"2024-07-27T03:15:25.069653+0000","last_undegraded":"2024-07-27T03:15:25.069653+0000","last_fullsized":"2024-07-27T03:15:25.069653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:44:53.857986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":420,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:44.537142+0000","last_change":"2024-07-27T03:12:29.544286+0000","last_active":"2024-07-27T03:16:44.537142+0000","last_peered":"2024-07-27T03:16:44.537142+0000","last_clean":"2024-07-27T03:16:44.537142+0000","last_became_active":"2024-07-27T03:12:29.543879+0000","last_became_peered":"2024-07-27T03:12:29.543879+0000","last_unstale":"2024-07-27T03:16:44.537142+0000","last_undegraded":"2024-07-27T03:16:44.537142+0000","last_fullsized":"2024-07-27T03:16:44.537142+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:20:36.997298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":370,"num_read_kb":370,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404174+0000","last_change":"2024-07-27T03:12:42.609824+0000","last_active":"2024-07-27T03:13:57.404174+0000","last_peered":"2024-07-27T03:13:57.404174+0000","last_clean":"2024-07-27T03:13:57.404174+0000","last_became_active":"2024-07-27T03:12:42.608502+0000","last_became_peered":"2024-07-27T03:12:42.608502+0000","last_unstale":"2024-07-27T03:13:57.404174+0000","last_undegraded":"2024-07-27T03:13:57.404174+0000","last_fullsized":"2024-07-27T03:13:57.404174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:10:23.253921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:43.944093+0000","last_change":"2024-07-27T03:12:42.610744+0000","last_active":"2024-07-27T03:14:43.944093+0000","last_peered":"2024-07-27T03:14:43.944093+0000","last_clean":"2024-07-27T03:14:43.944093+0000","last_became_active":"2024-07-27T03:12:42.610322+0000","last_became_peered":"2024-07-27T03:12:42.610322+0000","last_unstale":"2024-07-27T03:14:43.944093+0000","last_undegraded":"2024-07-27T03:14:43.944093+0000","last_fullsized":"2024-07-27T03:14:43.944093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:01:47.618013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.059061+0000","last_change":"2024-07-27T03:12:32.558372+0000","last_active":"2024-07-27T03:15:25.059061+0000","last_peered":"2024-07-27T03:15:25.059061+0000","last_clean":"2024-07-27T03:15:25.059061+0000","last_became_active":"2024-07-27T03:12:32.558003+0000","last_became_peered":"2024-07-27T03:12:32.558003+0000","last_unstale":"2024-07-27T03:15:25.059061+0000","last_undegraded":"2024-07-27T03:15:25.059061+0000","last_fullsized":"2024-07-27T03:15:25.059061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:41:43.673807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404495+0000","last_change":"2024-07-27T03:12:29.543182+0000","last_active":"2024-07-27T03:13:57.404495+0000","last_peered":"2024-07-27T03:13:57.404495+0000","last_clean":"2024-07-27T03:13:57.404495+0000","last_became_active":"2024-07-27T03:12:29.542744+0000","last_became_peered":"2024-07-27T03:12:29.542744+0000","last_unstale":"2024-07-27T03:13:57.404495+0000","last_undegraded":"2024-07-27T03:13:57.404495+0000","last_fullsized":"2024-07-27T03:13:57.404495+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:01:17.276549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405941+0000","last_change":"2024-07-27T03:12:42.622297+0000","last_active":"2024-07-27T03:13:57.405941+0000","last_peered":"2024-07-27T03:13:57.405941+0000","last_clean":"2024-07-27T03:13:57.405941+0000","last_became_active":"2024-07-27T03:12:42.621947+0000","last_became_peered":"2024-07-27T03:12:42.621947+0000","last_unstale":"2024-07-27T03:13:57.405941+0000","last_undegraded":"2024-07-27T03:13:57.405941+0000","last_fullsized":"2024-07-27T03:13:57.405941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:15:59.480216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.061353+0000","last_change":"2024-07-27T03:12:32.565987+0000","last_active":"2024-07-27T03:15:25.061353+0000","last_peered":"2024-07-27T03:15:25.061353+0000","last_clean":"2024-07-27T03:15:25.061353+0000","last_became_active":"2024-07-27T03:12:32.565603+0000","last_became_peered":"2024-07-27T03:12:32.565603+0000","last_unstale":"2024-07-27T03:15:25.061353+0000","last_undegraded":"2024-07-27T03:15:25.061353+0000","last_fullsized":"2024-07-27T03:15:25.061353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:30:48.954683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348273+0000","last_change":"2024-07-27T03:12:29.532149+0000","last_active":"2024-07-27T03:13:57.348273+0000","last_peered":"2024-07-27T03:13:57.348273+0000","last_clean":"2024-07-27T03:13:57.348273+0000","last_became_active":"2024-07-27T03:12:29.531761+0000","last_became_peered":"2024-07-27T03:12:29.531761+0000","last_unstale":"2024-07-27T03:13:57.348273+0000","last_undegraded":"2024-07-27T03:13:57.348273+0000","last_fullsized":"2024-07-27T03:13:57.348273+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:44:30.902538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406091+0000","last_change":"2024-07-27T03:12:29.543148+0000","last_active":"2024-07-27T03:13:57.406091+0000","last_peered":"2024-07-27T03:13:57.406091+0000","last_clean":"2024-07-27T03:13:57.406091+0000","last_became_active":"2024-07-27T03:12:29.542774+0000","last_became_peered":"2024-07-27T03:12:29.542774+0000","last_unstale":"2024-07-27T03:13:57.406091+0000","last_undegraded":"2024-07-27T03:13:57.406091+0000","last_fullsized":"2024-07-27T03:13:57.406091+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:20:06.786121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.060716+0000","last_change":"2024-07-27T03:12:32.561516+0000","last_active":"2024-07-27T03:15:25.060716+0000","last_peered":"2024-07-27T03:15:25.060716+0000","last_clean":"2024-07-27T03:15:25.060716+0000","last_became_active":"2024-07-27T03:12:32.561145+0000","last_became_peered":"2024-07-27T03:12:32.561145+0000","last_unstale":"2024-07-27T03:15:25.060716+0000","last_undegraded":"2024-07-27T03:15:25.060716+0000","last_fullsized":"2024-07-27T03:15:25.060716+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:49:26.317206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302982+0000","last_change":"2024-07-27T03:12:42.616735+0000","last_active":"2024-07-27T03:13:57.302982+0000","last_peered":"2024-07-27T03:13:57.302982+0000","last_clean":"2024-07-27T03:13:57.302982+0000","last_became_active":"2024-07-27T03:12:42.616171+0000","last_became_peered":"2024-07-27T03:12:42.616171+0000","last_unstale":"2024-07-27T03:13:57.302982+0000","last_undegraded":"2024-07-27T03:13:57.302982+0000","last_fullsized":"2024-07-27T03:13:57.302982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:45:46.236613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":238,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:41.533609+0000","last_change":"2024-07-27T03:12:29.539775+0000","last_active":"2024-07-27T03:16:41.533609+0000","last_peered":"2024-07-27T03:16:41.533609+0000","last_clean":"2024-07-27T03:16:41.533609+0000","last_became_active":"2024-07-27T03:12:29.539276+0000","last_became_peered":"2024-07-27T03:12:29.539276+0000","last_unstale":"2024-07-27T03:16:41.533609+0000","last_undegraded":"2024-07-27T03:16:41.533609+0000","last_fullsized":"2024-07-27T03:16:41.533609+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:09:19.364679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":187,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.045676+0000","last_change":"2024-07-27T03:12:32.562673+0000","last_active":"2024-07-27T03:15:25.045676+0000","last_peered":"2024-07-27T03:15:25.045676+0000","last_clean":"2024-07-27T03:15:25.045676+0000","last_became_active":"2024-07-27T03:12:32.561591+0000","last_became_peered":"2024-07-27T03:12:32.561591+0000","last_unstale":"2024-07-27T03:15:25.045676+0000","last_undegraded":"2024-07-27T03:15:25.045676+0000","last_fullsized":"2024-07-27T03:15:25.045676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:49:38.095004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303410+0000","last_change":"2024-07-27T03:12:42.611756+0000","last_active":"2024-07-27T03:13:57.303410+0000","last_peered":"2024-07-27T03:13:57.303410+0000","last_clean":"2024-07-27T03:13:57.303410+0000","last_became_active":"2024-07-27T03:12:42.611246+0000","last_became_peered":"2024-07-27T03:12:42.611246+0000","last_unstale":"2024-07-27T03:13:57.303410+0000","last_undegraded":"2024-07-27T03:13:57.303410+0000","last_fullsized":"2024-07-27T03:13:57.303410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:13:06.027887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.061677+0000","last_change":"2024-07-27T03:12:32.571339+0000","last_active":"2024-07-27T03:15:25.061677+0000","last_peered":"2024-07-27T03:15:25.061677+0000","last_clean":"2024-07-27T03:15:25.061677+0000","last_became_active":"2024-07-27T03:12:32.570973+0000","last_became_peered":"2024-07-27T03:12:32.570973+0000","last_unstale":"2024-07-27T03:15:25.061677+0000","last_undegraded":"2024-07-27T03:15:25.061677+0000","last_fullsized":"2024-07-27T03:15:25.061677+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:24:25.357833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.060782+0000","last_change":"2024-07-27T03:12:32.561243+0000","last_active":"2024-07-27T03:15:25.060782+0000","last_peered":"2024-07-27T03:15:25.060782+0000","last_clean":"2024-07-27T03:15:25.060782+0000","last_became_active":"2024-07-27T03:12:32.560748+0000","last_became_peered":"2024-07-27T03:12:32.560748+0000","last_unstale":"2024-07-27T03:15:25.060782+0000","last_undegraded":"2024-07-27T03:15:25.060782+0000","last_fullsized":"2024-07-27T03:15:25.060782+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:47:40.430283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.057184+0000","last_change":"2024-07-27T03:12:32.562545+0000","last_active":"2024-07-27T03:15:25.057184+0000","last_peered":"2024-07-27T03:15:25.057184+0000","last_clean":"2024-07-27T03:15:25.057184+0000","last_became_active":"2024-07-27T03:12:32.562202+0000","last_became_peered":"2024-07-27T03:12:32.562202+0000","last_unstale":"2024-07-27T03:15:25.057184+0000","last_undegraded":"2024-07-27T03:15:25.057184+0000","last_fullsized":"2024-07-27T03:15:25.057184+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:05:20.229232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.065174+0000","last_change":"2024-07-27T03:12:32.565162+0000","last_active":"2024-07-27T03:15:25.065174+0000","last_peered":"2024-07-27T03:15:25.065174+0000","last_clean":"2024-07-27T03:15:25.065174+0000","last_became_active":"2024-07-27T03:12:32.564798+0000","last_became_peered":"2024-07-27T03:12:32.564798+0000","last_unstale":"2024-07-27T03:15:25.065174+0000","last_undegraded":"2024-07-27T03:15:25.065174+0000","last_fullsized":"2024-07-27T03:15:25.065174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:54:29.742624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.060883+0000","last_change":"2024-07-27T03:12:32.556874+0000","last_active":"2024-07-27T03:15:25.060883+0000","last_peered":"2024-07-27T03:15:25.060883+0000","last_clean":"2024-07-27T03:15:25.060883+0000","last_became_active":"2024-07-27T03:12:32.556467+0000","last_became_peered":"2024-07-27T03:12:32.556467+0000","last_unstale":"2024-07-27T03:15:25.060883+0000","last_undegraded":"2024-07-27T03:15:25.060883+0000","last_fullsized":"2024-07-27T03:15:25.060883+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:51:20.806785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.053073+0000","last_change":"2024-07-27T03:12:32.569136+0000","last_active":"2024-07-27T03:15:25.053073+0000","last_peered":"2024-07-27T03:15:25.053073+0000","last_clean":"2024-07-27T03:15:25.053073+0000","last_became_active":"2024-07-27T03:12:32.568778+0000","last_became_peered":"2024-07-27T03:12:32.568778+0000","last_unstale":"2024-07-27T03:15:25.053073+0000","last_undegraded":"2024-07-27T03:15:25.053073+0000","last_fullsized":"2024-07-27T03:15:25.053073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:26:22.654974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.069928+0000","last_change":"2024-07-27T03:12:32.569507+0000","last_active":"2024-07-27T03:15:25.069928+0000","last_peered":"2024-07-27T03:15:25.069928+0000","last_clean":"2024-07-27T03:15:25.069928+0000","last_became_active":"2024-07-27T03:12:32.569109+0000","last_became_peered":"2024-07-27T03:12:32.569109+0000","last_unstale":"2024-07-27T03:15:25.069928+0000","last_undegraded":"2024-07-27T03:15:25.069928+0000","last_fullsized":"2024-07-27T03:15:25.069928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:35:12.944933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.053821+0000","last_change":"2024-07-27T03:12:32.566479+0000","last_active":"2024-07-27T03:15:25.053821+0000","last_peered":"2024-07-27T03:15:25.053821+0000","last_clean":"2024-07-27T03:15:25.053821+0000","last_became_active":"2024-07-27T03:12:32.566114+0000","last_became_peered":"2024-07-27T03:12:32.566114+0000","last_unstale":"2024-07-27T03:15:25.053821+0000","last_undegraded":"2024-07-27T03:15:25.053821+0000","last_fullsized":"2024-07-27T03:15:25.053821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:24:19.944031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.051389+0000","last_change":"2024-07-27T03:12:56.726118+0000","last_active":"2024-07-27T03:15:25.051389+0000","last_peered":"2024-07-27T03:15:25.051389+0000","last_clean":"2024-07-27T03:15:25.051389+0000","last_became_active":"2024-07-27T03:12:56.725637+0000","last_became_peered":"2024-07-27T03:12:56.725637+0000","last_unstale":"2024-07-27T03:15:25.051389+0000","last_undegraded":"2024-07-27T03:15:25.051389+0000","last_fullsized":"2024-07-27T03:15:25.051389+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:36:41.725150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.053321+0000","last_change":"2024-07-27T03:12:32.568232+0000","last_active":"2024-07-27T03:15:25.053321+0000","last_peered":"2024-07-27T03:15:25.053321+0000","last_clean":"2024-07-27T03:15:25.053321+0000","last_became_active":"2024-07-27T03:12:32.567834+0000","last_became_peered":"2024-07-27T03:12:32.567834+0000","last_unstale":"2024-07-27T03:15:25.053321+0000","last_undegraded":"2024-07-27T03:15:25.053321+0000","last_fullsized":"2024-07-27T03:15:25.053321+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:40:28.234539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.070056+0000","last_change":"2024-07-27T03:12:32.573451+0000","last_active":"2024-07-27T03:15:25.070056+0000","last_peered":"2024-07-27T03:15:25.070056+0000","last_clean":"2024-07-27T03:15:25.070056+0000","last_became_active":"2024-07-27T03:12:32.573066+0000","last_became_peered":"2024-07-27T03:12:32.573066+0000","last_unstale":"2024-07-27T03:15:25.070056+0000","last_undegraded":"2024-07-27T03:15:25.070056+0000","last_fullsized":"2024-07-27T03:15:25.070056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:13:29.104055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.063555+0000","last_change":"2024-07-27T03:12:32.561067+0000","last_active":"2024-07-27T03:15:25.063555+0000","last_peered":"2024-07-27T03:15:25.063555+0000","last_clean":"2024-07-27T03:15:25.063555+0000","last_became_active":"2024-07-27T03:12:32.560358+0000","last_became_peered":"2024-07-27T03:12:32.560358+0000","last_unstale":"2024-07-27T03:15:25.063555+0000","last_undegraded":"2024-07-27T03:15:25.063555+0000","last_fullsized":"2024-07-27T03:15:25.063555+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:41:41.026804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.053463+0000","last_change":"2024-07-27T03:12:32.572290+0000","last_active":"2024-07-27T03:15:25.053463+0000","last_peered":"2024-07-27T03:15:25.053463+0000","last_clean":"2024-07-27T03:15:25.053463+0000","last_became_active":"2024-07-27T03:12:32.571877+0000","last_became_peered":"2024-07-27T03:12:32.571877+0000","last_unstale":"2024-07-27T03:15:25.053463+0000","last_undegraded":"2024-07-27T03:15:25.053463+0000","last_fullsized":"2024-07-27T03:15:25.053463+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:34:40.559787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.056923+0000","last_change":"2024-07-27T03:12:32.565495+0000","last_active":"2024-07-27T03:15:25.056923+0000","last_peered":"2024-07-27T03:15:25.056923+0000","last_clean":"2024-07-27T03:15:25.056923+0000","last_became_active":"2024-07-27T03:12:32.563136+0000","last_became_peered":"2024-07-27T03:12:32.563136+0000","last_unstale":"2024-07-27T03:15:25.056923+0000","last_undegraded":"2024-07-27T03:15:25.056923+0000","last_fullsized":"2024-07-27T03:15:25.056923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:31:02.032776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.070390+0000","last_change":"2024-07-27T03:12:32.569477+0000","last_active":"2024-07-27T03:15:25.070390+0000","last_peered":"2024-07-27T03:15:25.070390+0000","last_clean":"2024-07-27T03:15:25.070390+0000","last_became_active":"2024-07-27T03:12:32.569098+0000","last_became_peered":"2024-07-27T03:12:32.569098+0000","last_unstale":"2024-07-27T03:15:25.070390+0000","last_undegraded":"2024-07-27T03:15:25.070390+0000","last_fullsized":"2024-07-27T03:15:25.070390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:44:26.082539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.057708+0000","last_change":"2024-07-27T03:12:32.574704+0000","last_active":"2024-07-27T03:15:25.057708+0000","last_peered":"2024-07-27T03:15:25.057708+0000","last_clean":"2024-07-27T03:15:25.057708+0000","last_became_active":"2024-07-27T03:12:32.574261+0000","last_became_peered":"2024-07-27T03:12:32.574261+0000","last_unstale":"2024-07-27T03:15:25.057708+0000","last_undegraded":"2024-07-27T03:15:25.057708+0000","last_fullsized":"2024-07-27T03:15:25.057708+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:53:06.445267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.068628+0000","last_change":"2024-07-27T03:12:32.573835+0000","last_active":"2024-07-27T03:15:25.068628+0000","last_peered":"2024-07-27T03:15:25.068628+0000","last_clean":"2024-07-27T03:15:25.068628+0000","last_became_active":"2024-07-27T03:12:32.573450+0000","last_became_peered":"2024-07-27T03:12:32.573450+0000","last_unstale":"2024-07-27T03:15:25.068628+0000","last_undegraded":"2024-07-27T03:15:25.068628+0000","last_fullsized":"2024-07-27T03:15:25.068628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:40:45.264000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.060743+0000","last_change":"2024-07-27T03:12:32.568542+0000","last_active":"2024-07-27T03:15:25.060743+0000","last_peered":"2024-07-27T03:15:25.060743+0000","last_clean":"2024-07-27T03:15:25.060743+0000","last_became_active":"2024-07-27T03:12:32.568179+0000","last_became_peered":"2024-07-27T03:12:32.568179+0000","last_unstale":"2024-07-27T03:15:25.060743+0000","last_undegraded":"2024-07-27T03:15:25.060743+0000","last_fullsized":"2024-07-27T03:15:25.060743+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:32:10.005601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.061855+0000","last_change":"2024-07-27T03:12:32.564063+0000","last_active":"2024-07-27T03:15:25.061855+0000","last_peered":"2024-07-27T03:15:25.061855+0000","last_clean":"2024-07-27T03:15:25.061855+0000","last_became_active":"2024-07-27T03:12:32.563705+0000","last_became_peered":"2024-07-27T03:12:32.563705+0000","last_unstale":"2024-07-27T03:15:25.061855+0000","last_undegraded":"2024-07-27T03:15:25.061855+0000","last_fullsized":"2024-07-27T03:15:25.061855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:40:19.234194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.061066+0000","last_change":"2024-07-27T03:12:57.178963+0000","last_active":"2024-07-27T03:15:25.061066+0000","last_peered":"2024-07-27T03:15:25.061066+0000","last_clean":"2024-07-27T03:15:25.061066+0000","last_became_active":"2024-07-27T03:12:57.178550+0000","last_became_peered":"2024-07-27T03:12:57.178550+0000","last_unstale":"2024-07-27T03:15:25.061066+0000","last_undegraded":"2024-07-27T03:15:25.061066+0000","last_fullsized":"2024-07-27T03:15:25.061066+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:20:06.586957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.060099+0000","last_change":"2024-07-27T03:12:32.569863+0000","last_active":"2024-07-27T03:15:25.060099+0000","last_peered":"2024-07-27T03:15:25.060099+0000","last_clean":"2024-07-27T03:15:25.060099+0000","last_became_active":"2024-07-27T03:12:32.566457+0000","last_became_peered":"2024-07-27T03:12:32.566457+0000","last_unstale":"2024-07-27T03:15:25.060099+0000","last_undegraded":"2024-07-27T03:15:25.060099+0000","last_fullsized":"2024-07-27T03:15:25.060099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:28:13.141002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.058759+0000","last_change":"2024-07-27T03:12:32.574490+0000","last_active":"2024-07-27T03:15:25.058759+0000","last_peered":"2024-07-27T03:15:25.058759+0000","last_clean":"2024-07-27T03:15:25.058759+0000","last_became_active":"2024-07-27T03:12:32.574130+0000","last_became_peered":"2024-07-27T03:12:32.574130+0000","last_unstale":"2024-07-27T03:15:25.058759+0000","last_undegraded":"2024-07-27T03:15:25.058759+0000","last_fullsized":"2024-07-27T03:15:25.058759+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:44:22.696669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.066579+0000","last_change":"2024-07-27T03:12:32.560106+0000","last_active":"2024-07-27T03:15:25.066579+0000","last_peered":"2024-07-27T03:15:25.066579+0000","last_clean":"2024-07-27T03:15:25.066579+0000","last_became_active":"2024-07-27T03:12:32.559751+0000","last_became_peered":"2024-07-27T03:12:32.559751+0000","last_unstale":"2024-07-27T03:15:25.066579+0000","last_undegraded":"2024-07-27T03:15:25.066579+0000","last_fullsized":"2024-07-27T03:15:25.066579+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:16:37.647462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.062851+0000","last_change":"2024-07-27T03:12:32.569352+0000","last_active":"2024-07-27T03:15:25.062851+0000","last_peered":"2024-07-27T03:15:25.062851+0000","last_clean":"2024-07-27T03:15:25.062851+0000","last_became_active":"2024-07-27T03:12:32.569004+0000","last_became_peered":"2024-07-27T03:12:32.569004+0000","last_unstale":"2024-07-27T03:15:25.062851+0000","last_undegraded":"2024-07-27T03:15:25.062851+0000","last_fullsized":"2024-07-27T03:15:25.062851+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:21:25.999214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.053831+0000","last_change":"2024-07-27T03:12:32.562775+0000","last_active":"2024-07-27T03:15:25.053831+0000","last_peered":"2024-07-27T03:15:25.053831+0000","last_clean":"2024-07-27T03:15:25.053831+0000","last_became_active":"2024-07-27T03:12:32.562403+0000","last_became_peered":"2024-07-27T03:12:32.562403+0000","last_unstale":"2024-07-27T03:15:25.053831+0000","last_undegraded":"2024-07-27T03:15:25.053831+0000","last_fullsized":"2024-07-27T03:15:25.053831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:16:09.474505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.061943+0000","last_change":"2024-07-27T03:12:32.563271+0000","last_active":"2024-07-27T03:15:25.061943+0000","last_peered":"2024-07-27T03:15:25.061943+0000","last_clean":"2024-07-27T03:15:25.061943+0000","last_became_active":"2024-07-27T03:12:32.562850+0000","last_became_peered":"2024-07-27T03:12:32.562850+0000","last_unstale":"2024-07-27T03:15:25.061943+0000","last_undegraded":"2024-07-27T03:15:25.061943+0000","last_fullsized":"2024-07-27T03:15:25.061943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:40:27.688205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.065581+0000","last_change":"2024-07-27T03:12:32.557580+0000","last_active":"2024-07-27T03:15:25.065581+0000","last_peered":"2024-07-27T03:15:25.065581+0000","last_clean":"2024-07-27T03:15:25.065581+0000","last_became_active":"2024-07-27T03:12:32.557190+0000","last_became_peered":"2024-07-27T03:12:32.557190+0000","last_unstale":"2024-07-27T03:15:25.065581+0000","last_undegraded":"2024-07-27T03:15:25.065581+0000","last_fullsized":"2024-07-27T03:15:25.065581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:58:17.609035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.033909+0000","last_change":"2024-07-27T03:12:32.563669+0000","last_active":"2024-07-27T03:15:25.033909+0000","last_peered":"2024-07-27T03:15:25.033909+0000","last_clean":"2024-07-27T03:15:25.033909+0000","last_became_active":"2024-07-27T03:12:32.563200+0000","last_became_peered":"2024-07-27T03:12:32.563200+0000","last_unstale":"2024-07-27T03:15:25.033909+0000","last_undegraded":"2024-07-27T03:15:25.033909+0000","last_fullsized":"2024-07-27T03:15:25.033909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:14:05.624118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.055913+0000","last_change":"2024-07-27T03:12:32.561793+0000","last_active":"2024-07-27T03:15:25.055913+0000","last_peered":"2024-07-27T03:15:25.055913+0000","last_clean":"2024-07-27T03:15:25.055913+0000","last_became_active":"2024-07-27T03:12:32.561452+0000","last_became_peered":"2024-07-27T03:12:32.561452+0000","last_unstale":"2024-07-27T03:15:25.055913+0000","last_undegraded":"2024-07-27T03:15:25.055913+0000","last_fullsized":"2024-07-27T03:15:25.055913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:22:07.204824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.070321+0000","last_change":"2024-07-27T03:12:32.569811+0000","last_active":"2024-07-27T03:15:25.070321+0000","last_peered":"2024-07-27T03:15:25.070321+0000","last_clean":"2024-07-27T03:15:25.070321+0000","last_became_active":"2024-07-27T03:12:32.569474+0000","last_became_peered":"2024-07-27T03:12:32.569474+0000","last_unstale":"2024-07-27T03:15:25.070321+0000","last_undegraded":"2024-07-27T03:15:25.070321+0000","last_fullsized":"2024-07-27T03:15:25.070321+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:38:39.317362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.063814+0000","last_change":"2024-07-27T03:12:32.565159+0000","last_active":"2024-07-27T03:15:25.063814+0000","last_peered":"2024-07-27T03:15:25.063814+0000","last_clean":"2024-07-27T03:15:25.063814+0000","last_became_active":"2024-07-27T03:12:32.564750+0000","last_became_peered":"2024-07-27T03:12:32.564750+0000","last_unstale":"2024-07-27T03:15:25.063814+0000","last_undegraded":"2024-07-27T03:15:25.063814+0000","last_fullsized":"2024-07-27T03:15:25.063814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:51:26.186510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"91'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.057903+0000","last_change":"2024-07-27T03:12:32.574429+0000","last_active":"2024-07-27T03:15:25.057903+0000","last_peered":"2024-07-27T03:15:25.057903+0000","last_clean":"2024-07-27T03:15:25.057903+0000","last_became_active":"2024-07-27T03:12:32.573986+0000","last_became_peered":"2024-07-27T03:12:32.573986+0000","last_unstale":"2024-07-27T03:15:25.057903+0000","last_undegraded":"2024-07-27T03:15:25.057903+0000","last_fullsized":"2024-07-27T03:15:25.057903+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:35:00.345561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75488,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1881,"num_read_kb":1027,"num_write":263,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":226464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645378,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":525562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841607,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37052,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708100,"statfs":{"total":95995035648,"available":95957094400,"internally_reserved":0,"allocated":1757184,"data_stored":1184986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037835,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1077248,"data_stored":525470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234063,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":525826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430292,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1671168,"data_stored":1123018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593818,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":520974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790046,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1679360,"data_stored":1118068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953571,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":525580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149800,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":520682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411437,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1146880,"data_stored":590758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1126400,"data_stored":590006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738486,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1077248,"data_stored":522430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-27T03:16:45.392 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-27T03:16:45.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.540+0000 7f7711a84640 1 -- 172.21.15.67:0/3562207860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f770c101fe0 msgr2=0x7f770c102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:45.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.540+0000 7f7711a84640 1 --2- 172.21.15.67:0/3562207860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f770c101fe0 0x7f770c102450 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009860 tx=0x7f76fc02a340 comp rx=0 tx=0).stop 2024-07-27T03:16:45.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.540+0000 7f7711a84640 1 -- 172.21.15.67:0/3562207860 shutdown_connections 2024-07-27T03:16:45.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.540+0000 7f7711a84640 1 --2- 172.21.15.67:0/3562207860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f770c103a00 0x7f770c10a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.540+0000 7f7711a84640 1 --2- 172.21.15.67:0/3562207860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f770c103060 0x7f770c103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.540+0000 7f7711a84640 1 --2- 172.21.15.67:0/3562207860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f770c101fe0 0x7f770c102450 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.540+0000 7f7711a84640 1 -- 172.21.15.67:0/3562207860 >> 172.21.15.67:0/3562207860 conn(0x7f770c0fd920 msgr2=0x7f770c0ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:45.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.541+0000 7f7711a84640 1 -- 172.21.15.67:0/3562207860 shutdown_connections 2024-07-27T03:16:45.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.541+0000 7f7711a84640 1 -- 172.21.15.67:0/3562207860 wait complete. 2024-07-27T03:16:45.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.541+0000 7f7711a84640 1 Processor -- start 2024-07-27T03:16:45.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.542+0000 7f7711a84640 1 -- start start 2024-07-27T03:16:45.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.542+0000 7f7711a84640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f770c101fe0 0x7f770c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:45.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.542+0000 7f7711a84640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f770c103060 0x7f770c198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:45.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.542+0000 7f7711a84640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f770c103a00 0x7f770c199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:45.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.542+0000 7f7711a84640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f770c10dc60 con 0x7f770c103060 2024-07-27T03:16:45.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.542+0000 7f7711a84640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f770c10dae0 con 0x7f770c103a00 2024-07-27T03:16:45.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.542+0000 7f7711a84640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f770c10dde0 con 0x7f770c101fe0 2024-07-27T03:16:45.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.542+0000 7f770affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f770c103060 0x7f770c198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:45.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.542+0000 7f770affd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f770c103060 0x7f770c198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51412/0 (socket says 172.21.15.67:51412) 2024-07-27T03:16:45.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.542+0000 7f770affd640 1 -- 172.21.15.67:0/1007859769 learned_addr learned my addr 172.21.15.67:0/1007859769 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:45.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.542+0000 7f770bfff640 1 --2- 172.21.15.67:0/1007859769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f770c103a00 0x7f770c199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:45.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.542+0000 7f770b7fe640 1 --2- 172.21.15.67:0/1007859769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f770c101fe0 0x7f770c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:45.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.543+0000 7f770affd640 1 -- 172.21.15.67:0/1007859769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f770c101fe0 msgr2=0x7f770c198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:45.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.543+0000 7f770affd640 1 --2- 172.21.15.67:0/1007859769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f770c101fe0 0x7f770c198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.543+0000 7f770affd640 1 -- 172.21.15.67:0/1007859769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f770c103a00 msgr2=0x7f770c199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:45.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.543+0000 7f770affd640 1 --2- 172.21.15.67:0/1007859769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f770c103a00 0x7f770c199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.543+0000 7f770affd640 1 -- 172.21.15.67:0/1007859769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770c19a290 con 0x7f770c103060 2024-07-27T03:16:45.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.543+0000 7f770affd640 1 --2- 172.21.15.67:0/1007859769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f770c103060 0x7f770c198ec0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f770000ead0 tx=0x7f770000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:45.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.543+0000 7f7708ff9640 1 -- 172.21.15.67:0/1007859769 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770000bb70 con 0x7f770c103060 2024-07-27T03:16:45.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.543+0000 7f7708ff9640 1 -- 172.21.15.67:0/1007859769 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f770000bd10 con 0x7f770c103060 2024-07-27T03:16:45.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.543+0000 7f7708ff9640 1 -- 172.21.15.67:0/1007859769 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700017410 con 0x7f770c103060 2024-07-27T03:16:45.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.543+0000 7f7711a84640 1 -- 172.21.15.67:0/1007859769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f770c19a580 con 0x7f770c103060 2024-07-27T03:16:45.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.543+0000 7f7711a84640 1 -- 172.21.15.67:0/1007859769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f770c06a370 con 0x7f770c103060 2024-07-27T03:16:45.548 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.544+0000 7f7711a84640 1 -- 172.21.15.67:0/1007859769 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f770c102450 con 0x7f770c103060 2024-07-27T03:16:45.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.546+0000 7f7708ff9640 1 -- 172.21.15.67:0/1007859769 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f7700004e70 con 0x7f770c103060 2024-07-27T03:16:45.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.547+0000 7f7708ff9640 1 --2- 172.21.15.67:0/1007859769 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f76f40b3bd0 0x7f76f40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:45.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.548+0000 7f770b7fe640 1 --2- 172.21.15.67:0/1007859769 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f76f40b3bd0 0x7f76f40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:45.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.548+0000 7f7708ff9640 1 -- 172.21.15.67:0/1007859769 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f77000e5fd0 con 0x7f770c103060 2024-07-27T03:16:45.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.548+0000 7f770b7fe640 1 --2- 172.21.15.67:0/1007859769 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f76f40b3bd0 0x7f76f40b60e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f76fc000c00 tx=0x7f76fc030470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.551+0000 7f7708ff9640 1 -- 172.21.15.67:0/1007859769 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f77000ad270 con 0x7f770c103060 2024-07-27T03:16:45.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.743+0000 7f7711a84640 1 -- 172.21.15.67:0/1007859769 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f770c108d30 con 0x7f76f40b3bd0 2024-07-27T03:16:45.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.748+0000 7f7708ff9640 1 -- 172.21.15.67:0/1007859769 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340567 (secure 0 0 0) 0x7f770c108d30 con 0x7f76f40b3bd0 2024-07-27T03:16:45.754 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:16:45.756 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-07-27T03:16:45.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.752+0000 7f7711a84640 1 -- 172.21.15.67:0/1007859769 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f76f40b3bd0 msgr2=0x7f76f40b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:45.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.752+0000 7f7711a84640 1 --2- 172.21.15.67:0/1007859769 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f76f40b3bd0 0x7f76f40b60e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f76fc000c00 tx=0x7f76fc030470 comp rx=0 tx=0).stop 2024-07-27T03:16:45.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.752+0000 7f7711a84640 1 -- 172.21.15.67:0/1007859769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f770c103060 msgr2=0x7f770c198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:45.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.752+0000 7f7711a84640 1 --2- 172.21.15.67:0/1007859769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f770c103060 0x7f770c198ec0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f770000ead0 tx=0x7f770000ef90 comp rx=0 tx=0).stop 2024-07-27T03:16:45.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.752+0000 7f7711a84640 1 -- 172.21.15.67:0/1007859769 shutdown_connections 2024-07-27T03:16:45.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.752+0000 7f7711a84640 1 --2- 172.21.15.67:0/1007859769 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f76f40b3bd0 0x7f76f40b60e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.752+0000 7f7711a84640 1 --2- 172.21.15.67:0/1007859769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f770c103a00 0x7f770c199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.752+0000 7f7711a84640 1 --2- 172.21.15.67:0/1007859769 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f770c103060 0x7f770c198ec0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.752+0000 7f7711a84640 1 --2- 172.21.15.67:0/1007859769 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f770c101fe0 0x7f770c198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.752+0000 7f7711a84640 1 -- 172.21.15.67:0/1007859769 >> 172.21.15.67:0/1007859769 conn(0x7f770c0fd920 msgr2=0x7f770c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:45.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.752+0000 7f7711a84640 1 -- 172.21.15.67:0/1007859769 shutdown_connections 2024-07-27T03:16:45.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.752+0000 7f7711a84640 1 -- 172.21.15.67:0/1007859769 wait complete. 2024-07-27T03:16:45.766 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-07-27T03:16:45.119670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666090,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1957,"num_read_kb":1091,"num_write":388,"num_write_kb":2310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951979,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":820,"ondisk_log_size":820,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388604,"kb_used_data":14492,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553220,"statfs":{"total":1151940427776,"available":1151542497280,"internally_reserved":0,"allocated":14839808,"data_stored":8273360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5681,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":77,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003487"},"pg_stats":[{"pgid":"2.1f","version":"89'13","reported_seq":241,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:40.530537+0000","last_change":"2024-07-27T03:12:29.541107+0000","last_active":"2024-07-27T03:16:40.530537+0000","last_peered":"2024-07-27T03:16:40.530537+0000","last_clean":"2024-07-27T03:16:40.530537+0000","last_became_active":"2024-07-27T03:12:29.540806+0000","last_became_peered":"2024-07-27T03:12:29.540806+0000","last_unstale":"2024-07-27T03:16:40.530537+0000","last_undegraded":"2024-07-27T03:16:40.530537+0000","last_fullsized":"2024-07-27T03:16:40.530537+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:38:24.349354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":182,"num_read_kb":182,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.057193+0000","last_change":"2024-07-27T03:12:32.565515+0000","last_active":"2024-07-27T03:15:25.057193+0000","last_peered":"2024-07-27T03:15:25.057193+0000","last_clean":"2024-07-27T03:15:25.057193+0000","last_became_active":"2024-07-27T03:12:32.565134+0000","last_became_peered":"2024-07-27T03:12:32.565134+0000","last_unstale":"2024-07-27T03:15:25.057193+0000","last_undegraded":"2024-07-27T03:15:25.057193+0000","last_fullsized":"2024-07-27T03:15:25.057193+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:11:56.793139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302752+0000","last_change":"2024-07-27T03:12:42.613432+0000","last_active":"2024-07-27T03:13:57.302752+0000","last_peered":"2024-07-27T03:13:57.302752+0000","last_clean":"2024-07-27T03:13:57.302752+0000","last_became_active":"2024-07-27T03:12:42.612986+0000","last_became_peered":"2024-07-27T03:12:42.612986+0000","last_unstale":"2024-07-27T03:13:57.302752+0000","last_undegraded":"2024-07-27T03:13:57.302752+0000","last_fullsized":"2024-07-27T03:13:57.302752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:25:37.505117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:32.726614+0000","last_change":"2024-07-27T03:12:29.614229+0000","last_active":"2024-07-27T03:14:32.726614+0000","last_peered":"2024-07-27T03:14:32.726614+0000","last_clean":"2024-07-27T03:14:32.726614+0000","last_became_active":"2024-07-27T03:12:29.613844+0000","last_became_peered":"2024-07-27T03:12:29.613844+0000","last_unstale":"2024-07-27T03:14:32.726614+0000","last_undegraded":"2024-07-27T03:14:32.726614+0000","last_fullsized":"2024-07-27T03:14:32.726614+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:10:28.917080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.070362+0000","last_change":"2024-07-27T03:12:32.563187+0000","last_active":"2024-07-27T03:15:25.070362+0000","last_peered":"2024-07-27T03:15:25.070362+0000","last_clean":"2024-07-27T03:15:25.070362+0000","last_became_active":"2024-07-27T03:12:32.562674+0000","last_became_peered":"2024-07-27T03:12:32.562674+0000","last_unstale":"2024-07-27T03:15:25.070362+0000","last_undegraded":"2024-07-27T03:15:25.070362+0000","last_fullsized":"2024-07-27T03:15:25.070362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:05:39.211514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303151+0000","last_change":"2024-07-27T03:12:42.611516+0000","last_active":"2024-07-27T03:13:57.303151+0000","last_peered":"2024-07-27T03:13:57.303151+0000","last_clean":"2024-07-27T03:13:57.303151+0000","last_became_active":"2024-07-27T03:12:42.610934+0000","last_became_peered":"2024-07-27T03:12:42.610934+0000","last_unstale":"2024-07-27T03:13:57.303151+0000","last_undegraded":"2024-07-27T03:13:57.303151+0000","last_fullsized":"2024-07-27T03:13:57.303151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:13:17.323063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":58,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305740+0000","last_change":"2024-07-27T03:12:29.539573+0000","last_active":"2024-07-27T03:13:57.305740+0000","last_peered":"2024-07-27T03:13:57.305740+0000","last_clean":"2024-07-27T03:13:57.305740+0000","last_became_active":"2024-07-27T03:12:29.539056+0000","last_became_peered":"2024-07-27T03:12:29.539056+0000","last_unstale":"2024-07-27T03:13:57.305740+0000","last_undegraded":"2024-07-27T03:13:57.305740+0000","last_fullsized":"2024-07-27T03:13:57.305740+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:04:37.513644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.068109+0000","last_change":"2024-07-27T03:12:32.567729+0000","last_active":"2024-07-27T03:15:25.068109+0000","last_peered":"2024-07-27T03:15:25.068109+0000","last_clean":"2024-07-27T03:15:25.068109+0000","last_became_active":"2024-07-27T03:12:32.564827+0000","last_became_peered":"2024-07-27T03:12:32.564827+0000","last_unstale":"2024-07-27T03:15:25.068109+0000","last_undegraded":"2024-07-27T03:15:25.068109+0000","last_fullsized":"2024-07-27T03:15:25.068109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:10:55.314988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303339+0000","last_change":"2024-07-27T03:12:42.612776+0000","last_active":"2024-07-27T03:13:57.303339+0000","last_peered":"2024-07-27T03:13:57.303339+0000","last_clean":"2024-07-27T03:13:57.303339+0000","last_became_active":"2024-07-27T03:12:42.612258+0000","last_became_peered":"2024-07-27T03:12:42.612258+0000","last_unstale":"2024-07-27T03:13:57.303339+0000","last_undegraded":"2024-07-27T03:13:57.303339+0000","last_fullsized":"2024-07-27T03:13:57.303339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:48:06.516002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305959+0000","last_change":"2024-07-27T03:12:29.540070+0000","last_active":"2024-07-27T03:13:57.305959+0000","last_peered":"2024-07-27T03:13:57.305959+0000","last_clean":"2024-07-27T03:13:57.305959+0000","last_became_active":"2024-07-27T03:12:29.539657+0000","last_became_peered":"2024-07-27T03:12:29.539657+0000","last_unstale":"2024-07-27T03:13:57.305959+0000","last_undegraded":"2024-07-27T03:13:57.305959+0000","last_fullsized":"2024-07-27T03:13:57.305959+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:42:48.222613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.067431+0000","last_change":"2024-07-27T03:12:32.574418+0000","last_active":"2024-07-27T03:15:25.067431+0000","last_peered":"2024-07-27T03:15:25.067431+0000","last_clean":"2024-07-27T03:15:25.067431+0000","last_became_active":"2024-07-27T03:12:32.573974+0000","last_became_peered":"2024-07-27T03:12:32.573974+0000","last_unstale":"2024-07-27T03:15:25.067431+0000","last_undegraded":"2024-07-27T03:15:25.067431+0000","last_fullsized":"2024-07-27T03:15:25.067431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:34:30.442833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303642+0000","last_change":"2024-07-27T03:12:42.605155+0000","last_active":"2024-07-27T03:13:57.303642+0000","last_peered":"2024-07-27T03:13:57.303642+0000","last_clean":"2024-07-27T03:13:57.303642+0000","last_became_active":"2024-07-27T03:12:42.604685+0000","last_became_peered":"2024-07-27T03:12:42.604685+0000","last_unstale":"2024-07-27T03:13:57.303642+0000","last_undegraded":"2024-07-27T03:13:57.303642+0000","last_fullsized":"2024-07-27T03:13:57.303642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:28:58.931408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.065859+0000","last_change":"2024-07-27T03:12:32.574711+0000","last_active":"2024-07-27T03:15:25.065859+0000","last_peered":"2024-07-27T03:15:25.065859+0000","last_clean":"2024-07-27T03:15:25.065859+0000","last_became_active":"2024-07-27T03:12:32.574365+0000","last_became_peered":"2024-07-27T03:12:32.574365+0000","last_unstale":"2024-07-27T03:15:25.065859+0000","last_undegraded":"2024-07-27T03:15:25.065859+0000","last_fullsized":"2024-07-27T03:15:25.065859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:50:56.621371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305326+0000","last_change":"2024-07-27T03:12:29.540835+0000","last_active":"2024-07-27T03:13:57.305326+0000","last_peered":"2024-07-27T03:13:57.305326+0000","last_clean":"2024-07-27T03:13:57.305326+0000","last_became_active":"2024-07-27T03:12:29.540461+0000","last_became_peered":"2024-07-27T03:12:29.540461+0000","last_unstale":"2024-07-27T03:13:57.305326+0000","last_undegraded":"2024-07-27T03:13:57.305326+0000","last_fullsized":"2024-07-27T03:13:57.305326+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:19:20.817038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303344+0000","last_change":"2024-07-27T03:12:42.718584+0000","last_active":"2024-07-27T03:13:57.303344+0000","last_peered":"2024-07-27T03:13:57.303344+0000","last_clean":"2024-07-27T03:13:57.303344+0000","last_became_active":"2024-07-27T03:12:42.717752+0000","last_became_peered":"2024-07-27T03:12:42.717752+0000","last_unstale":"2024-07-27T03:13:57.303344+0000","last_undegraded":"2024-07-27T03:13:57.303344+0000","last_fullsized":"2024-07-27T03:13:57.303344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:23:52.629794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.042310+0000","last_change":"2024-07-27T03:12:32.560672+0000","last_active":"2024-07-27T03:15:25.042310+0000","last_peered":"2024-07-27T03:15:25.042310+0000","last_clean":"2024-07-27T03:15:25.042310+0000","last_became_active":"2024-07-27T03:12:32.559172+0000","last_became_peered":"2024-07-27T03:12:32.559172+0000","last_unstale":"2024-07-27T03:15:25.042310+0000","last_undegraded":"2024-07-27T03:15:25.042310+0000","last_fullsized":"2024-07-27T03:15:25.042310+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:02:49.346983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405206+0000","last_change":"2024-07-27T03:12:57.434569+0000","last_active":"2024-07-27T03:13:57.405206+0000","last_peered":"2024-07-27T03:13:57.405206+0000","last_clean":"2024-07-27T03:13:57.405206+0000","last_became_active":"2024-07-27T03:12:57.434038+0000","last_became_peered":"2024-07-27T03:12:57.434038+0000","last_unstale":"2024-07-27T03:13:57.405206+0000","last_undegraded":"2024-07-27T03:13:57.405206+0000","last_fullsized":"2024-07-27T03:13:57.405206+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:43:24.462838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.412383+0000","last_change":"2024-07-27T03:13:57.411931+0000","last_active":"2024-07-27T03:13:57.412383+0000","last_peered":"2024-07-27T03:13:57.412383+0000","last_clean":"2024-07-27T03:13:57.412383+0000","last_became_active":"2024-07-27T03:13:57.407362+0000","last_became_peered":"2024-07-27T03:13:57.407362+0000","last_unstale":"2024-07-27T03:13:57.412383+0000","last_undegraded":"2024-07-27T03:13:57.412383+0000","last_fullsized":"2024-07-27T03:13:57.412383+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:52:08.444268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.058449+0000","last_change":"2024-07-27T03:12:32.572129+0000","last_active":"2024-07-27T03:15:25.058449+0000","last_peered":"2024-07-27T03:15:25.058449+0000","last_clean":"2024-07-27T03:15:25.058449+0000","last_became_active":"2024-07-27T03:12:32.571774+0000","last_became_peered":"2024-07-27T03:12:32.571774+0000","last_unstale":"2024-07-27T03:15:25.058449+0000","last_undegraded":"2024-07-27T03:15:25.058449+0000","last_fullsized":"2024-07-27T03:15:25.058449+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:52:23.316957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404446+0000","last_change":"2024-07-27T03:12:29.539768+0000","last_active":"2024-07-27T03:13:57.404446+0000","last_peered":"2024-07-27T03:13:57.404446+0000","last_clean":"2024-07-27T03:13:57.404446+0000","last_became_active":"2024-07-27T03:12:29.539389+0000","last_became_peered":"2024-07-27T03:12:29.539389+0000","last_unstale":"2024-07-27T03:13:57.404446+0000","last_undegraded":"2024-07-27T03:13:57.404446+0000","last_fullsized":"2024-07-27T03:13:57.404446+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:09:35.961328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302623+0000","last_change":"2024-07-27T03:12:42.617569+0000","last_active":"2024-07-27T03:13:57.302623+0000","last_peered":"2024-07-27T03:13:57.302623+0000","last_clean":"2024-07-27T03:13:57.302623+0000","last_became_active":"2024-07-27T03:12:42.616357+0000","last_became_peered":"2024-07-27T03:12:42.616357+0000","last_unstale":"2024-07-27T03:13:57.302623+0000","last_undegraded":"2024-07-27T03:13:57.302623+0000","last_fullsized":"2024-07-27T03:13:57.302623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:35:11.297631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.018136+0000","last_change":"2024-07-27T03:12:32.566272+0000","last_active":"2024-07-27T03:15:25.018136+0000","last_peered":"2024-07-27T03:15:25.018136+0000","last_clean":"2024-07-27T03:15:25.018136+0000","last_became_active":"2024-07-27T03:12:32.565865+0000","last_became_peered":"2024-07-27T03:12:32.565865+0000","last_unstale":"2024-07-27T03:15:25.018136+0000","last_undegraded":"2024-07-27T03:15:25.018136+0000","last_fullsized":"2024-07-27T03:15:25.018136+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:42:09.539586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405359+0000","last_change":"2024-07-27T03:12:29.613733+0000","last_active":"2024-07-27T03:13:57.405359+0000","last_peered":"2024-07-27T03:13:57.405359+0000","last_clean":"2024-07-27T03:13:57.405359+0000","last_became_active":"2024-07-27T03:12:29.613395+0000","last_became_peered":"2024-07-27T03:12:29.613395+0000","last_unstale":"2024-07-27T03:13:57.405359+0000","last_undegraded":"2024-07-27T03:13:57.405359+0000","last_fullsized":"2024-07-27T03:13:57.405359+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:21:34.431900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.408340+0000","last_change":"2024-07-27T03:13:57.407798+0000","last_active":"2024-07-27T03:13:57.408340+0000","last_peered":"2024-07-27T03:13:57.408340+0000","last_clean":"2024-07-27T03:13:57.408340+0000","last_became_active":"2024-07-27T03:13:57.407275+0000","last_became_peered":"2024-07-27T03:13:57.407275+0000","last_unstale":"2024-07-27T03:13:57.408340+0000","last_undegraded":"2024-07-27T03:13:57.408340+0000","last_fullsized":"2024-07-27T03:13:57.408340+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:47:27.817092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,8],"acting":[4,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.056823+0000","last_change":"2024-07-27T03:12:32.567006+0000","last_active":"2024-07-27T03:15:25.056823+0000","last_peered":"2024-07-27T03:15:25.056823+0000","last_clean":"2024-07-27T03:15:25.056823+0000","last_became_active":"2024-07-27T03:12:32.566205+0000","last_became_peered":"2024-07-27T03:12:32.566205+0000","last_unstale":"2024-07-27T03:15:25.056823+0000","last_undegraded":"2024-07-27T03:15:25.056823+0000","last_fullsized":"2024-07-27T03:15:25.056823+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:11:23.171341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303695+0000","last_change":"2024-07-27T03:12:29.533909+0000","last_active":"2024-07-27T03:13:57.303695+0000","last_peered":"2024-07-27T03:13:57.303695+0000","last_clean":"2024-07-27T03:13:57.303695+0000","last_became_active":"2024-07-27T03:12:29.533327+0000","last_became_peered":"2024-07-27T03:12:29.533327+0000","last_unstale":"2024-07-27T03:13:57.303695+0000","last_undegraded":"2024-07-27T03:13:57.303695+0000","last_fullsized":"2024-07-27T03:13:57.303695+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:35:24.705096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302923+0000","last_change":"2024-07-27T03:12:42.609066+0000","last_active":"2024-07-27T03:13:57.302923+0000","last_peered":"2024-07-27T03:13:57.302923+0000","last_clean":"2024-07-27T03:13:57.302923+0000","last_became_active":"2024-07-27T03:12:42.608561+0000","last_became_peered":"2024-07-27T03:12:42.608561+0000","last_unstale":"2024-07-27T03:13:57.302923+0000","last_undegraded":"2024-07-27T03:13:57.302923+0000","last_fullsized":"2024-07-27T03:13:57.302923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:27:53.657498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.053587+0000","last_change":"2024-07-27T03:12:32.560931+0000","last_active":"2024-07-27T03:15:25.053587+0000","last_peered":"2024-07-27T03:15:25.053587+0000","last_clean":"2024-07-27T03:15:25.053587+0000","last_became_active":"2024-07-27T03:12:32.560549+0000","last_became_peered":"2024-07-27T03:12:32.560549+0000","last_unstale":"2024-07-27T03:15:25.053587+0000","last_undegraded":"2024-07-27T03:15:25.053587+0000","last_fullsized":"2024-07-27T03:15:25.053587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T15:11:25.136035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'17","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.595553+0000","last_change":"2024-07-27T03:12:29.535365+0000","last_active":"2024-07-27T03:15:25.595553+0000","last_peered":"2024-07-27T03:15:25.595553+0000","last_clean":"2024-07-27T03:15:25.595553+0000","last_became_active":"2024-07-27T03:12:29.534952+0000","last_became_peered":"2024-07-27T03:12:29.534952+0000","last_unstale":"2024-07-27T03:15:25.595553+0000","last_undegraded":"2024-07-27T03:15:25.595553+0000","last_fullsized":"2024-07-27T03:15:25.595553+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:34:59.889488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:42.727465+0000","last_change":"2024-07-27T03:12:42.718416+0000","last_active":"2024-07-27T03:14:42.727465+0000","last_peered":"2024-07-27T03:14:42.727465+0000","last_clean":"2024-07-27T03:14:42.727465+0000","last_became_active":"2024-07-27T03:12:42.717949+0000","last_became_peered":"2024-07-27T03:12:42.717949+0000","last_unstale":"2024-07-27T03:14:42.727465+0000","last_undegraded":"2024-07-27T03:14:42.727465+0000","last_fullsized":"2024-07-27T03:14:42.727465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:05:05.762577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.057594+0000","last_change":"2024-07-27T03:12:32.559824+0000","last_active":"2024-07-27T03:15:25.057594+0000","last_peered":"2024-07-27T03:15:25.057594+0000","last_clean":"2024-07-27T03:15:25.057594+0000","last_became_active":"2024-07-27T03:12:32.559435+0000","last_became_peered":"2024-07-27T03:12:32.559435+0000","last_unstale":"2024-07-27T03:15:25.057594+0000","last_undegraded":"2024-07-27T03:15:25.057594+0000","last_fullsized":"2024-07-27T03:15:25.057594+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:16:44.514272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:35.285402+0000","last_change":"2024-07-27T03:12:29.614296+0000","last_active":"2024-07-27T03:14:35.285402+0000","last_peered":"2024-07-27T03:14:35.285402+0000","last_clean":"2024-07-27T03:14:35.285402+0000","last_became_active":"2024-07-27T03:12:29.612987+0000","last_became_peered":"2024-07-27T03:12:29.612987+0000","last_unstale":"2024-07-27T03:14:35.285402+0000","last_undegraded":"2024-07-27T03:14:35.285402+0000","last_fullsized":"2024-07-27T03:14:35.285402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:23:09.226523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405200+0000","last_change":"2024-07-27T03:12:42.616575+0000","last_active":"2024-07-27T03:13:57.405200+0000","last_peered":"2024-07-27T03:13:57.405200+0000","last_clean":"2024-07-27T03:13:57.405200+0000","last_became_active":"2024-07-27T03:12:42.615980+0000","last_became_peered":"2024-07-27T03:12:42.615980+0000","last_unstale":"2024-07-27T03:13:57.405200+0000","last_undegraded":"2024-07-27T03:13:57.405200+0000","last_fullsized":"2024-07-27T03:13:57.405200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:00:59.041910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.062700+0000","last_change":"2024-07-27T03:12:32.556109+0000","last_active":"2024-07-27T03:15:25.062700+0000","last_peered":"2024-07-27T03:15:25.062700+0000","last_clean":"2024-07-27T03:15:25.062700+0000","last_became_active":"2024-07-27T03:12:32.555747+0000","last_became_peered":"2024-07-27T03:12:32.555747+0000","last_unstale":"2024-07-27T03:15:25.062700+0000","last_undegraded":"2024-07-27T03:15:25.062700+0000","last_fullsized":"2024-07-27T03:15:25.062700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:27:58.286605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:36.101756+0000","last_change":"2024-07-27T03:12:29.613523+0000","last_active":"2024-07-27T03:14:36.101756+0000","last_peered":"2024-07-27T03:14:36.101756+0000","last_clean":"2024-07-27T03:14:36.101756+0000","last_became_active":"2024-07-27T03:12:29.613172+0000","last_became_peered":"2024-07-27T03:12:29.613172+0000","last_unstale":"2024-07-27T03:14:36.101756+0000","last_undegraded":"2024-07-27T03:14:36.101756+0000","last_fullsized":"2024-07-27T03:14:36.101756+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:49:45.225221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304930+0000","last_change":"2024-07-27T03:12:42.718536+0000","last_active":"2024-07-27T03:13:57.304930+0000","last_peered":"2024-07-27T03:13:57.304930+0000","last_clean":"2024-07-27T03:13:57.304930+0000","last_became_active":"2024-07-27T03:12:42.718068+0000","last_became_peered":"2024-07-27T03:12:42.718068+0000","last_unstale":"2024-07-27T03:13:57.304930+0000","last_undegraded":"2024-07-27T03:13:57.304930+0000","last_fullsized":"2024-07-27T03:13:57.304930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:18:28.478700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.064182+0000","last_change":"2024-07-27T03:12:32.558060+0000","last_active":"2024-07-27T03:15:25.064182+0000","last_peered":"2024-07-27T03:15:25.064182+0000","last_clean":"2024-07-27T03:15:25.064182+0000","last_became_active":"2024-07-27T03:12:32.557260+0000","last_became_peered":"2024-07-27T03:12:32.557260+0000","last_unstale":"2024-07-27T03:15:25.064182+0000","last_undegraded":"2024-07-27T03:15:25.064182+0000","last_fullsized":"2024-07-27T03:15:25.064182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:36:54.945761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":234,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:41.532322+0000","last_change":"2024-07-27T03:12:29.541080+0000","last_active":"2024-07-27T03:16:41.532322+0000","last_peered":"2024-07-27T03:16:41.532322+0000","last_clean":"2024-07-27T03:16:41.532322+0000","last_became_active":"2024-07-27T03:12:29.540628+0000","last_became_peered":"2024-07-27T03:12:29.540628+0000","last_unstale":"2024-07-27T03:16:41.532322+0000","last_undegraded":"2024-07-27T03:16:41.532322+0000","last_fullsized":"2024-07-27T03:16:41.532322+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:45:17.045124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":183,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348865+0000","last_change":"2024-07-27T03:12:42.611561+0000","last_active":"2024-07-27T03:13:57.348865+0000","last_peered":"2024-07-27T03:13:57.348865+0000","last_clean":"2024-07-27T03:13:57.348865+0000","last_became_active":"2024-07-27T03:12:42.610507+0000","last_became_peered":"2024-07-27T03:12:42.610507+0000","last_unstale":"2024-07-27T03:13:57.348865+0000","last_undegraded":"2024-07-27T03:13:57.348865+0000","last_fullsized":"2024-07-27T03:13:57.348865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:15:08.972807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.058339+0000","last_change":"2024-07-27T03:12:32.562316+0000","last_active":"2024-07-27T03:15:25.058339+0000","last_peered":"2024-07-27T03:15:25.058339+0000","last_clean":"2024-07-27T03:15:25.058339+0000","last_became_active":"2024-07-27T03:12:32.561729+0000","last_became_peered":"2024-07-27T03:12:32.561729+0000","last_unstale":"2024-07-27T03:15:25.058339+0000","last_undegraded":"2024-07-27T03:15:25.058339+0000","last_fullsized":"2024-07-27T03:15:25.058339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:01:19.960269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":238,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:42.533767+0000","last_change":"2024-07-27T03:12:29.613459+0000","last_active":"2024-07-27T03:16:42.533767+0000","last_peered":"2024-07-27T03:16:42.533767+0000","last_clean":"2024-07-27T03:16:42.533767+0000","last_became_active":"2024-07-27T03:12:29.613009+0000","last_became_peered":"2024-07-27T03:12:29.613009+0000","last_unstale":"2024-07-27T03:16:42.533767+0000","last_undegraded":"2024-07-27T03:16:42.533767+0000","last_fullsized":"2024-07-27T03:16:42.533767+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:16:37.230471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":185,"num_read_kb":185,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348695+0000","last_change":"2024-07-27T03:12:42.718516+0000","last_active":"2024-07-27T03:13:57.348695+0000","last_peered":"2024-07-27T03:13:57.348695+0000","last_clean":"2024-07-27T03:13:57.348695+0000","last_became_active":"2024-07-27T03:12:42.718041+0000","last_became_peered":"2024-07-27T03:12:42.718041+0000","last_unstale":"2024-07-27T03:13:57.348695+0000","last_undegraded":"2024-07-27T03:13:57.348695+0000","last_fullsized":"2024-07-27T03:13:57.348695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:35:39.633977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.063195+0000","last_change":"2024-07-27T03:12:32.560752+0000","last_active":"2024-07-27T03:15:25.063195+0000","last_peered":"2024-07-27T03:15:25.063195+0000","last_clean":"2024-07-27T03:15:25.063195+0000","last_became_active":"2024-07-27T03:12:32.560248+0000","last_became_peered":"2024-07-27T03:12:32.560248+0000","last_unstale":"2024-07-27T03:15:25.063195+0000","last_undegraded":"2024-07-27T03:15:25.063195+0000","last_fullsized":"2024-07-27T03:15:25.063195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:59:48.056867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:31.988172+0000","last_change":"2024-07-27T03:12:29.539591+0000","last_active":"2024-07-27T03:14:31.988172+0000","last_peered":"2024-07-27T03:14:31.988172+0000","last_clean":"2024-07-27T03:14:31.988172+0000","last_became_active":"2024-07-27T03:12:29.539165+0000","last_became_peered":"2024-07-27T03:12:29.539165+0000","last_unstale":"2024-07-27T03:14:31.988172+0000","last_undegraded":"2024-07-27T03:14:31.988172+0000","last_fullsized":"2024-07-27T03:14:31.988172+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:04:21.940511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:00.281510+0000","last_change":"2024-07-27T03:13:57.407921+0000","last_active":"2024-07-27T03:14:00.281510+0000","last_peered":"2024-07-27T03:14:00.281510+0000","last_clean":"2024-07-27T03:14:00.281510+0000","last_became_active":"2024-07-27T03:13:57.407382+0000","last_became_peered":"2024-07-27T03:13:57.407382+0000","last_unstale":"2024-07-27T03:14:00.281510+0000","last_undegraded":"2024-07-27T03:14:00.281510+0000","last_fullsized":"2024-07-27T03:14:00.281510+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:48:27.362759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.069802+0000","last_change":"2024-07-27T03:12:32.570653+0000","last_active":"2024-07-27T03:15:25.069802+0000","last_peered":"2024-07-27T03:15:25.069802+0000","last_clean":"2024-07-27T03:15:25.069802+0000","last_became_active":"2024-07-27T03:12:32.569088+0000","last_became_peered":"2024-07-27T03:12:32.569088+0000","last_unstale":"2024-07-27T03:15:25.069802+0000","last_undegraded":"2024-07-27T03:15:25.069802+0000","last_fullsized":"2024-07-27T03:15:25.069802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:50:24.669431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:35.895703+0000","last_change":"2024-07-27T03:12:29.541654+0000","last_active":"2024-07-27T03:14:35.895703+0000","last_peered":"2024-07-27T03:14:35.895703+0000","last_clean":"2024-07-27T03:14:35.895703+0000","last_became_active":"2024-07-27T03:12:29.541264+0000","last_became_peered":"2024-07-27T03:12:29.541264+0000","last_unstale":"2024-07-27T03:14:35.895703+0000","last_undegraded":"2024-07-27T03:14:35.895703+0000","last_fullsized":"2024-07-27T03:14:35.895703+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:48:05.179016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":123,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:01.984277+0000","last_change":"2024-07-27T03:11:11.343199+0000","last_active":"2024-07-27T03:14:01.984277+0000","last_peered":"2024-07-27T03:14:01.984277+0000","last_clean":"2024-07-27T03:14:01.984277+0000","last_became_active":"2024-07-27T03:11:11.337635+0000","last_became_peered":"2024-07-27T03:11:11.337635+0000","last_unstale":"2024-07-27T03:14:01.984277+0000","last_undegraded":"2024-07-27T03:14:01.984277+0000","last_fullsized":"2024-07-27T03:14:01.984277+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:08:01.694830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:08:01.694830+0000","last_clean_scrub_stamp":"2024-07-27T03:08:01.694830+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:48:49.979610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305836+0000","last_change":"2024-07-27T03:12:42.614346+0000","last_active":"2024-07-27T03:13:57.305836+0000","last_peered":"2024-07-27T03:13:57.305836+0000","last_clean":"2024-07-27T03:13:57.305836+0000","last_became_active":"2024-07-27T03:12:42.613881+0000","last_became_peered":"2024-07-27T03:12:42.613881+0000","last_unstale":"2024-07-27T03:13:57.305836+0000","last_undegraded":"2024-07-27T03:13:57.305836+0000","last_fullsized":"2024-07-27T03:13:57.305836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:12:38.299348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.058877+0000","last_change":"2024-07-27T03:12:32.574330+0000","last_active":"2024-07-27T03:15:25.058877+0000","last_peered":"2024-07-27T03:15:25.058877+0000","last_clean":"2024-07-27T03:15:25.058877+0000","last_became_active":"2024-07-27T03:12:32.573977+0000","last_became_peered":"2024-07-27T03:12:32.573977+0000","last_unstale":"2024-07-27T03:15:25.058877+0000","last_undegraded":"2024-07-27T03:15:25.058877+0000","last_fullsized":"2024-07-27T03:15:25.058877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:39:09.925794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":237,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:42.534600+0000","last_change":"2024-07-27T03:12:29.613848+0000","last_active":"2024-07-27T03:16:42.534600+0000","last_peered":"2024-07-27T03:16:42.534600+0000","last_clean":"2024-07-27T03:16:42.534600+0000","last_became_active":"2024-07-27T03:12:29.613473+0000","last_became_peered":"2024-07-27T03:12:29.613473+0000","last_unstale":"2024-07-27T03:16:42.534600+0000","last_undegraded":"2024-07-27T03:16:42.534600+0000","last_fullsized":"2024-07-27T03:16:42.534600+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:52:56.894962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":186,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405373+0000","last_change":"2024-07-27T03:12:42.611295+0000","last_active":"2024-07-27T03:13:57.405373+0000","last_peered":"2024-07-27T03:13:57.405373+0000","last_clean":"2024-07-27T03:13:57.405373+0000","last_became_active":"2024-07-27T03:12:42.610782+0000","last_became_peered":"2024-07-27T03:12:42.610782+0000","last_unstale":"2024-07-27T03:13:57.405373+0000","last_undegraded":"2024-07-27T03:13:57.405373+0000","last_fullsized":"2024-07-27T03:13:57.405373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:24:02.849882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.061556+0000","last_change":"2024-07-27T03:12:56.725924+0000","last_active":"2024-07-27T03:15:25.061556+0000","last_peered":"2024-07-27T03:15:25.061556+0000","last_clean":"2024-07-27T03:15:25.061556+0000","last_became_active":"2024-07-27T03:12:56.725567+0000","last_became_peered":"2024-07-27T03:12:56.725567+0000","last_unstale":"2024-07-27T03:15:25.061556+0000","last_undegraded":"2024-07-27T03:15:25.061556+0000","last_fullsized":"2024-07-27T03:15:25.061556+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:32:23.771483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405225+0000","last_change":"2024-07-27T03:12:29.537863+0000","last_active":"2024-07-27T03:13:57.405225+0000","last_peered":"2024-07-27T03:13:57.405225+0000","last_clean":"2024-07-27T03:13:57.405225+0000","last_became_active":"2024-07-27T03:12:29.537444+0000","last_became_peered":"2024-07-27T03:12:29.537444+0000","last_unstale":"2024-07-27T03:13:57.405225+0000","last_undegraded":"2024-07-27T03:13:57.405225+0000","last_fullsized":"2024-07-27T03:13:57.405225+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:52:05.239809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405265+0000","last_change":"2024-07-27T03:12:42.622599+0000","last_active":"2024-07-27T03:13:57.405265+0000","last_peered":"2024-07-27T03:13:57.405265+0000","last_clean":"2024-07-27T03:13:57.405265+0000","last_became_active":"2024-07-27T03:12:42.622151+0000","last_became_peered":"2024-07-27T03:12:42.622151+0000","last_unstale":"2024-07-27T03:13:57.405265+0000","last_undegraded":"2024-07-27T03:13:57.405265+0000","last_fullsized":"2024-07-27T03:13:57.405265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:34:31.622281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.050080+0000","last_change":"2024-07-27T03:12:32.562300+0000","last_active":"2024-07-27T03:15:25.050080+0000","last_peered":"2024-07-27T03:15:25.050080+0000","last_clean":"2024-07-27T03:15:25.050080+0000","last_became_active":"2024-07-27T03:12:32.561343+0000","last_became_peered":"2024-07-27T03:12:32.561343+0000","last_unstale":"2024-07-27T03:15:25.050080+0000","last_undegraded":"2024-07-27T03:15:25.050080+0000","last_fullsized":"2024-07-27T03:15:25.050080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:01:28.626943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348835+0000","last_change":"2024-07-27T03:12:29.535177+0000","last_active":"2024-07-27T03:13:57.348835+0000","last_peered":"2024-07-27T03:13:57.348835+0000","last_clean":"2024-07-27T03:13:57.348835+0000","last_became_active":"2024-07-27T03:12:29.534662+0000","last_became_peered":"2024-07-27T03:12:29.534662+0000","last_unstale":"2024-07-27T03:13:57.348835+0000","last_undegraded":"2024-07-27T03:13:57.348835+0000","last_fullsized":"2024-07-27T03:13:57.348835+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:50:49.661992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404084+0000","last_change":"2024-07-27T03:12:42.609925+0000","last_active":"2024-07-27T03:13:57.404084+0000","last_peered":"2024-07-27T03:13:57.404084+0000","last_clean":"2024-07-27T03:13:57.404084+0000","last_became_active":"2024-07-27T03:12:42.609456+0000","last_became_peered":"2024-07-27T03:12:42.609456+0000","last_unstale":"2024-07-27T03:13:57.404084+0000","last_undegraded":"2024-07-27T03:13:57.404084+0000","last_fullsized":"2024-07-27T03:13:57.404084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:07:52.128381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.052969+0000","last_change":"2024-07-27T03:12:56.726170+0000","last_active":"2024-07-27T03:15:25.052969+0000","last_peered":"2024-07-27T03:15:25.052969+0000","last_clean":"2024-07-27T03:15:25.052969+0000","last_became_active":"2024-07-27T03:12:56.725685+0000","last_became_peered":"2024-07-27T03:12:56.725685+0000","last_unstale":"2024-07-27T03:15:25.052969+0000","last_undegraded":"2024-07-27T03:15:25.052969+0000","last_fullsized":"2024-07-27T03:15:25.052969+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:53:41.143333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405969+0000","last_change":"2024-07-27T03:12:56.726059+0000","last_active":"2024-07-27T03:13:57.405969+0000","last_peered":"2024-07-27T03:13:57.405969+0000","last_clean":"2024-07-27T03:13:57.405969+0000","last_became_active":"2024-07-27T03:12:56.722294+0000","last_became_peered":"2024-07-27T03:12:56.722294+0000","last_unstale":"2024-07-27T03:13:57.405969+0000","last_undegraded":"2024-07-27T03:13:57.405969+0000","last_fullsized":"2024-07-27T03:13:57.405969+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:18:42.627768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404320+0000","last_change":"2024-07-27T03:12:42.611966+0000","last_active":"2024-07-27T03:13:57.404320+0000","last_peered":"2024-07-27T03:13:57.404320+0000","last_clean":"2024-07-27T03:13:57.404320+0000","last_became_active":"2024-07-27T03:12:42.611522+0000","last_became_peered":"2024-07-27T03:12:42.611522+0000","last_unstale":"2024-07-27T03:13:57.404320+0000","last_undegraded":"2024-07-27T03:13:57.404320+0000","last_fullsized":"2024-07-27T03:13:57.404320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:09:54.034597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.069429+0000","last_change":"2024-07-27T03:12:56.719638+0000","last_active":"2024-07-27T03:15:25.069429+0000","last_peered":"2024-07-27T03:15:25.069429+0000","last_clean":"2024-07-27T03:15:25.069429+0000","last_became_active":"2024-07-27T03:12:56.719227+0000","last_became_peered":"2024-07-27T03:12:56.719227+0000","last_unstale":"2024-07-27T03:15:25.069429+0000","last_undegraded":"2024-07-27T03:15:25.069429+0000","last_fullsized":"2024-07-27T03:15:25.069429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:54:16.891518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304967+0000","last_change":"2024-07-27T03:12:29.614346+0000","last_active":"2024-07-27T03:13:57.304967+0000","last_peered":"2024-07-27T03:13:57.304967+0000","last_clean":"2024-07-27T03:13:57.304967+0000","last_became_active":"2024-07-27T03:12:29.613672+0000","last_became_peered":"2024-07-27T03:12:29.613672+0000","last_unstale":"2024-07-27T03:13:57.304967+0000","last_undegraded":"2024-07-27T03:13:57.304967+0000","last_fullsized":"2024-07-27T03:13:57.304967+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:46:35.505735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348712+0000","last_change":"2024-07-27T03:12:42.618364+0000","last_active":"2024-07-27T03:13:57.348712+0000","last_peered":"2024-07-27T03:13:57.348712+0000","last_clean":"2024-07-27T03:13:57.348712+0000","last_became_active":"2024-07-27T03:12:42.617988+0000","last_became_peered":"2024-07-27T03:12:42.617988+0000","last_unstale":"2024-07-27T03:13:57.348712+0000","last_undegraded":"2024-07-27T03:13:57.348712+0000","last_fullsized":"2024-07-27T03:13:57.348712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:40:21.725877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.061231+0000","last_change":"2024-07-27T03:12:32.563809+0000","last_active":"2024-07-27T03:15:25.061231+0000","last_peered":"2024-07-27T03:15:25.061231+0000","last_clean":"2024-07-27T03:15:25.061231+0000","last_became_active":"2024-07-27T03:12:32.563479+0000","last_became_peered":"2024-07-27T03:12:32.563479+0000","last_unstale":"2024-07-27T03:15:25.061231+0000","last_undegraded":"2024-07-27T03:15:25.061231+0000","last_fullsized":"2024-07-27T03:15:25.061231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:45:47.719241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":238,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:39.530182+0000","last_change":"2024-07-27T03:12:29.543786+0000","last_active":"2024-07-27T03:16:39.530182+0000","last_peered":"2024-07-27T03:16:39.530182+0000","last_clean":"2024-07-27T03:16:39.530182+0000","last_became_active":"2024-07-27T03:12:29.543203+0000","last_became_peered":"2024-07-27T03:12:29.543203+0000","last_unstale":"2024-07-27T03:16:39.530182+0000","last_undegraded":"2024-07-27T03:16:39.530182+0000","last_fullsized":"2024-07-27T03:16:39.530182+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:47:06.742171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":186,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302718+0000","last_change":"2024-07-27T03:12:42.622142+0000","last_active":"2024-07-27T03:13:57.302718+0000","last_peered":"2024-07-27T03:13:57.302718+0000","last_clean":"2024-07-27T03:13:57.302718+0000","last_became_active":"2024-07-27T03:12:42.621681+0000","last_became_peered":"2024-07-27T03:12:42.621681+0000","last_unstale":"2024-07-27T03:13:57.302718+0000","last_undegraded":"2024-07-27T03:13:57.302718+0000","last_fullsized":"2024-07-27T03:13:57.302718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:05:08.168424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.066031+0000","last_change":"2024-07-27T03:12:32.567119+0000","last_active":"2024-07-27T03:15:25.066031+0000","last_peered":"2024-07-27T03:15:25.066031+0000","last_clean":"2024-07-27T03:15:25.066031+0000","last_became_active":"2024-07-27T03:12:32.566757+0000","last_became_peered":"2024-07-27T03:12:32.566757+0000","last_unstale":"2024-07-27T03:15:25.066031+0000","last_undegraded":"2024-07-27T03:15:25.066031+0000","last_fullsized":"2024-07-27T03:15:25.066031+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:31:12.972804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404725+0000","last_change":"2024-07-27T03:12:29.540453+0000","last_active":"2024-07-27T03:13:57.404725+0000","last_peered":"2024-07-27T03:13:57.404725+0000","last_clean":"2024-07-27T03:13:57.404725+0000","last_became_active":"2024-07-27T03:12:29.539195+0000","last_became_peered":"2024-07-27T03:12:29.539195+0000","last_unstale":"2024-07-27T03:13:57.404725+0000","last_undegraded":"2024-07-27T03:13:57.404725+0000","last_fullsized":"2024-07-27T03:13:57.404725+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:44:31.144845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405363+0000","last_change":"2024-07-27T03:12:42.616671+0000","last_active":"2024-07-27T03:13:57.405363+0000","last_peered":"2024-07-27T03:13:57.405363+0000","last_clean":"2024-07-27T03:13:57.405363+0000","last_became_active":"2024-07-27T03:12:42.616198+0000","last_became_peered":"2024-07-27T03:12:42.616198+0000","last_unstale":"2024-07-27T03:13:57.405363+0000","last_undegraded":"2024-07-27T03:13:57.405363+0000","last_fullsized":"2024-07-27T03:13:57.405363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:01:31.584686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.060885+0000","last_change":"2024-07-27T03:12:32.562350+0000","last_active":"2024-07-27T03:15:25.060885+0000","last_peered":"2024-07-27T03:15:25.060885+0000","last_clean":"2024-07-27T03:15:25.060885+0000","last_became_active":"2024-07-27T03:12:32.562016+0000","last_became_peered":"2024-07-27T03:12:32.562016+0000","last_unstale":"2024-07-27T03:15:25.060885+0000","last_undegraded":"2024-07-27T03:15:25.060885+0000","last_fullsized":"2024-07-27T03:15:25.060885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:44:35.027329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303241+0000","last_change":"2024-07-27T03:12:29.541387+0000","last_active":"2024-07-27T03:13:57.303241+0000","last_peered":"2024-07-27T03:13:57.303241+0000","last_clean":"2024-07-27T03:13:57.303241+0000","last_became_active":"2024-07-27T03:12:29.541005+0000","last_became_peered":"2024-07-27T03:12:29.541005+0000","last_unstale":"2024-07-27T03:13:57.303241+0000","last_undegraded":"2024-07-27T03:13:57.303241+0000","last_fullsized":"2024-07-27T03:13:57.303241+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:22:16.268348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404849+0000","last_change":"2024-07-27T03:12:42.610029+0000","last_active":"2024-07-27T03:13:57.404849+0000","last_peered":"2024-07-27T03:13:57.404849+0000","last_clean":"2024-07-27T03:13:57.404849+0000","last_became_active":"2024-07-27T03:12:42.609592+0000","last_became_peered":"2024-07-27T03:12:42.609592+0000","last_unstale":"2024-07-27T03:13:57.404849+0000","last_undegraded":"2024-07-27T03:13:57.404849+0000","last_fullsized":"2024-07-27T03:13:57.404849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:47:15.254770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.051282+0000","last_change":"2024-07-27T03:12:32.568941+0000","last_active":"2024-07-27T03:15:25.051282+0000","last_peered":"2024-07-27T03:15:25.051282+0000","last_clean":"2024-07-27T03:15:25.051282+0000","last_became_active":"2024-07-27T03:12:32.568531+0000","last_became_peered":"2024-07-27T03:12:32.568531+0000","last_unstale":"2024-07-27T03:15:25.051282+0000","last_undegraded":"2024-07-27T03:15:25.051282+0000","last_fullsized":"2024-07-27T03:15:25.051282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:06:22.020505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'41","reported_seq":468,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:44.537754+0000","last_change":"2024-07-27T03:12:29.545131+0000","last_active":"2024-07-27T03:16:44.537754+0000","last_peered":"2024-07-27T03:16:44.537754+0000","last_clean":"2024-07-27T03:16:44.537754+0000","last_became_active":"2024-07-27T03:12:29.544768+0000","last_became_peered":"2024-07-27T03:12:29.544768+0000","last_unstale":"2024-07-27T03:16:44.537754+0000","last_undegraded":"2024-07-27T03:16:44.537754+0000","last_fullsized":"2024-07-27T03:16:44.537754+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:20:57.556128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68002,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":382,"num_read_kb":258,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302087+0000","last_change":"2024-07-27T03:12:42.617483+0000","last_active":"2024-07-27T03:13:57.302087+0000","last_peered":"2024-07-27T03:13:57.302087+0000","last_clean":"2024-07-27T03:13:57.302087+0000","last_became_active":"2024-07-27T03:12:42.617065+0000","last_became_peered":"2024-07-27T03:12:42.617065+0000","last_unstale":"2024-07-27T03:13:57.302087+0000","last_undegraded":"2024-07-27T03:13:57.302087+0000","last_fullsized":"2024-07-27T03:13:57.302087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:52:17.448919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.069655+0000","last_change":"2024-07-27T03:12:56.722257+0000","last_active":"2024-07-27T03:15:25.069655+0000","last_peered":"2024-07-27T03:15:25.069655+0000","last_clean":"2024-07-27T03:15:25.069655+0000","last_became_active":"2024-07-27T03:12:56.721773+0000","last_became_peered":"2024-07-27T03:12:56.721773+0000","last_unstale":"2024-07-27T03:15:25.069655+0000","last_undegraded":"2024-07-27T03:15:25.069655+0000","last_fullsized":"2024-07-27T03:15:25.069655+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:30:44.310705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405205+0000","last_change":"2024-07-27T03:12:29.614171+0000","last_active":"2024-07-27T03:13:57.405205+0000","last_peered":"2024-07-27T03:13:57.405205+0000","last_clean":"2024-07-27T03:13:57.405205+0000","last_became_active":"2024-07-27T03:12:29.613270+0000","last_became_peered":"2024-07-27T03:12:29.613270+0000","last_unstale":"2024-07-27T03:13:57.405205+0000","last_undegraded":"2024-07-27T03:13:57.405205+0000","last_fullsized":"2024-07-27T03:13:57.405205+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:39:32.283778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303808+0000","last_change":"2024-07-27T03:12:42.610375+0000","last_active":"2024-07-27T03:13:57.303808+0000","last_peered":"2024-07-27T03:13:57.303808+0000","last_clean":"2024-07-27T03:13:57.303808+0000","last_became_active":"2024-07-27T03:12:42.609957+0000","last_became_peered":"2024-07-27T03:12:42.609957+0000","last_unstale":"2024-07-27T03:13:57.303808+0000","last_undegraded":"2024-07-27T03:13:57.303808+0000","last_fullsized":"2024-07-27T03:13:57.303808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:11:02.707179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.064641+0000","last_change":"2024-07-27T03:12:56.722332+0000","last_active":"2024-07-27T03:15:25.064641+0000","last_peered":"2024-07-27T03:15:25.064641+0000","last_clean":"2024-07-27T03:15:25.064641+0000","last_became_active":"2024-07-27T03:12:56.721972+0000","last_became_peered":"2024-07-27T03:12:56.721972+0000","last_unstale":"2024-07-27T03:15:25.064641+0000","last_undegraded":"2024-07-27T03:15:25.064641+0000","last_fullsized":"2024-07-27T03:15:25.064641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:54:31.880137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306094+0000","last_change":"2024-07-27T03:12:29.541051+0000","last_active":"2024-07-27T03:13:57.306094+0000","last_peered":"2024-07-27T03:13:57.306094+0000","last_clean":"2024-07-27T03:13:57.306094+0000","last_became_active":"2024-07-27T03:12:29.540619+0000","last_became_peered":"2024-07-27T03:12:29.540619+0000","last_unstale":"2024-07-27T03:13:57.306094+0000","last_undegraded":"2024-07-27T03:13:57.306094+0000","last_fullsized":"2024-07-27T03:13:57.306094+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:23:27.587419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304810+0000","last_change":"2024-07-27T03:12:56.720496+0000","last_active":"2024-07-27T03:13:57.304810+0000","last_peered":"2024-07-27T03:13:57.304810+0000","last_clean":"2024-07-27T03:13:57.304810+0000","last_became_active":"2024-07-27T03:12:56.719784+0000","last_became_peered":"2024-07-27T03:12:56.719784+0000","last_unstale":"2024-07-27T03:13:57.304810+0000","last_undegraded":"2024-07-27T03:13:57.304810+0000","last_fullsized":"2024-07-27T03:13:57.304810+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:07:20.518139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.069653+0000","last_change":"2024-07-27T03:12:32.568409+0000","last_active":"2024-07-27T03:15:25.069653+0000","last_peered":"2024-07-27T03:15:25.069653+0000","last_clean":"2024-07-27T03:15:25.069653+0000","last_became_active":"2024-07-27T03:12:32.568029+0000","last_became_peered":"2024-07-27T03:12:32.568029+0000","last_unstale":"2024-07-27T03:15:25.069653+0000","last_undegraded":"2024-07-27T03:15:25.069653+0000","last_fullsized":"2024-07-27T03:15:25.069653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:44:53.857986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":420,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:44.537142+0000","last_change":"2024-07-27T03:12:29.544286+0000","last_active":"2024-07-27T03:16:44.537142+0000","last_peered":"2024-07-27T03:16:44.537142+0000","last_clean":"2024-07-27T03:16:44.537142+0000","last_became_active":"2024-07-27T03:12:29.543879+0000","last_became_peered":"2024-07-27T03:12:29.543879+0000","last_unstale":"2024-07-27T03:16:44.537142+0000","last_undegraded":"2024-07-27T03:16:44.537142+0000","last_fullsized":"2024-07-27T03:16:44.537142+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:20:36.997298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":370,"num_read_kb":370,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404174+0000","last_change":"2024-07-27T03:12:42.609824+0000","last_active":"2024-07-27T03:13:57.404174+0000","last_peered":"2024-07-27T03:13:57.404174+0000","last_clean":"2024-07-27T03:13:57.404174+0000","last_became_active":"2024-07-27T03:12:42.608502+0000","last_became_peered":"2024-07-27T03:12:42.608502+0000","last_unstale":"2024-07-27T03:13:57.404174+0000","last_undegraded":"2024-07-27T03:13:57.404174+0000","last_fullsized":"2024-07-27T03:13:57.404174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:10:23.253921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:43.944093+0000","last_change":"2024-07-27T03:12:42.610744+0000","last_active":"2024-07-27T03:14:43.944093+0000","last_peered":"2024-07-27T03:14:43.944093+0000","last_clean":"2024-07-27T03:14:43.944093+0000","last_became_active":"2024-07-27T03:12:42.610322+0000","last_became_peered":"2024-07-27T03:12:42.610322+0000","last_unstale":"2024-07-27T03:14:43.944093+0000","last_undegraded":"2024-07-27T03:14:43.944093+0000","last_fullsized":"2024-07-27T03:14:43.944093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:01:47.618013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.059061+0000","last_change":"2024-07-27T03:12:32.558372+0000","last_active":"2024-07-27T03:15:25.059061+0000","last_peered":"2024-07-27T03:15:25.059061+0000","last_clean":"2024-07-27T03:15:25.059061+0000","last_became_active":"2024-07-27T03:12:32.558003+0000","last_became_peered":"2024-07-27T03:12:32.558003+0000","last_unstale":"2024-07-27T03:15:25.059061+0000","last_undegraded":"2024-07-27T03:15:25.059061+0000","last_fullsized":"2024-07-27T03:15:25.059061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:41:43.673807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404495+0000","last_change":"2024-07-27T03:12:29.543182+0000","last_active":"2024-07-27T03:13:57.404495+0000","last_peered":"2024-07-27T03:13:57.404495+0000","last_clean":"2024-07-27T03:13:57.404495+0000","last_became_active":"2024-07-27T03:12:29.542744+0000","last_became_peered":"2024-07-27T03:12:29.542744+0000","last_unstale":"2024-07-27T03:13:57.404495+0000","last_undegraded":"2024-07-27T03:13:57.404495+0000","last_fullsized":"2024-07-27T03:13:57.404495+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:01:17.276549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405941+0000","last_change":"2024-07-27T03:12:42.622297+0000","last_active":"2024-07-27T03:13:57.405941+0000","last_peered":"2024-07-27T03:13:57.405941+0000","last_clean":"2024-07-27T03:13:57.405941+0000","last_became_active":"2024-07-27T03:12:42.621947+0000","last_became_peered":"2024-07-27T03:12:42.621947+0000","last_unstale":"2024-07-27T03:13:57.405941+0000","last_undegraded":"2024-07-27T03:13:57.405941+0000","last_fullsized":"2024-07-27T03:13:57.405941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:15:59.480216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.061353+0000","last_change":"2024-07-27T03:12:32.565987+0000","last_active":"2024-07-27T03:15:25.061353+0000","last_peered":"2024-07-27T03:15:25.061353+0000","last_clean":"2024-07-27T03:15:25.061353+0000","last_became_active":"2024-07-27T03:12:32.565603+0000","last_became_peered":"2024-07-27T03:12:32.565603+0000","last_unstale":"2024-07-27T03:15:25.061353+0000","last_undegraded":"2024-07-27T03:15:25.061353+0000","last_fullsized":"2024-07-27T03:15:25.061353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:30:48.954683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348273+0000","last_change":"2024-07-27T03:12:29.532149+0000","last_active":"2024-07-27T03:13:57.348273+0000","last_peered":"2024-07-27T03:13:57.348273+0000","last_clean":"2024-07-27T03:13:57.348273+0000","last_became_active":"2024-07-27T03:12:29.531761+0000","last_became_peered":"2024-07-27T03:12:29.531761+0000","last_unstale":"2024-07-27T03:13:57.348273+0000","last_undegraded":"2024-07-27T03:13:57.348273+0000","last_fullsized":"2024-07-27T03:13:57.348273+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:44:30.902538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406091+0000","last_change":"2024-07-27T03:12:29.543148+0000","last_active":"2024-07-27T03:13:57.406091+0000","last_peered":"2024-07-27T03:13:57.406091+0000","last_clean":"2024-07-27T03:13:57.406091+0000","last_became_active":"2024-07-27T03:12:29.542774+0000","last_became_peered":"2024-07-27T03:12:29.542774+0000","last_unstale":"2024-07-27T03:13:57.406091+0000","last_undegraded":"2024-07-27T03:13:57.406091+0000","last_fullsized":"2024-07-27T03:13:57.406091+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:20:06.786121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.060716+0000","last_change":"2024-07-27T03:12:32.561516+0000","last_active":"2024-07-27T03:15:25.060716+0000","last_peered":"2024-07-27T03:15:25.060716+0000","last_clean":"2024-07-27T03:15:25.060716+0000","last_became_active":"2024-07-27T03:12:32.561145+0000","last_became_peered":"2024-07-27T03:12:32.561145+0000","last_unstale":"2024-07-27T03:15:25.060716+0000","last_undegraded":"2024-07-27T03:15:25.060716+0000","last_fullsized":"2024-07-27T03:15:25.060716+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:49:26.317206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302982+0000","last_change":"2024-07-27T03:12:42.616735+0000","last_active":"2024-07-27T03:13:57.302982+0000","last_peered":"2024-07-27T03:13:57.302982+0000","last_clean":"2024-07-27T03:13:57.302982+0000","last_became_active":"2024-07-27T03:12:42.616171+0000","last_became_peered":"2024-07-27T03:12:42.616171+0000","last_unstale":"2024-07-27T03:13:57.302982+0000","last_undegraded":"2024-07-27T03:13:57.302982+0000","last_fullsized":"2024-07-27T03:13:57.302982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:45:46.236613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":238,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:41.533609+0000","last_change":"2024-07-27T03:12:29.539775+0000","last_active":"2024-07-27T03:16:41.533609+0000","last_peered":"2024-07-27T03:16:41.533609+0000","last_clean":"2024-07-27T03:16:41.533609+0000","last_became_active":"2024-07-27T03:12:29.539276+0000","last_became_peered":"2024-07-27T03:12:29.539276+0000","last_unstale":"2024-07-27T03:16:41.533609+0000","last_undegraded":"2024-07-27T03:16:41.533609+0000","last_fullsized":"2024-07-27T03:16:41.533609+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:09:19.364679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":187,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.045676+0000","last_change":"2024-07-27T03:12:32.562673+0000","last_active":"2024-07-27T03:15:25.045676+0000","last_peered":"2024-07-27T03:15:25.045676+0000","last_clean":"2024-07-27T03:15:25.045676+0000","last_became_active":"2024-07-27T03:12:32.561591+0000","last_became_peered":"2024-07-27T03:12:32.561591+0000","last_unstale":"2024-07-27T03:15:25.045676+0000","last_undegraded":"2024-07-27T03:15:25.045676+0000","last_fullsized":"2024-07-27T03:15:25.045676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:49:38.095004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303410+0000","last_change":"2024-07-27T03:12:42.611756+0000","last_active":"2024-07-27T03:13:57.303410+0000","last_peered":"2024-07-27T03:13:57.303410+0000","last_clean":"2024-07-27T03:13:57.303410+0000","last_became_active":"2024-07-27T03:12:42.611246+0000","last_became_peered":"2024-07-27T03:12:42.611246+0000","last_unstale":"2024-07-27T03:13:57.303410+0000","last_undegraded":"2024-07-27T03:13:57.303410+0000","last_fullsized":"2024-07-27T03:13:57.303410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:13:06.027887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.061677+0000","last_change":"2024-07-27T03:12:32.571339+0000","last_active":"2024-07-27T03:15:25.061677+0000","last_peered":"2024-07-27T03:15:25.061677+0000","last_clean":"2024-07-27T03:15:25.061677+0000","last_became_active":"2024-07-27T03:12:32.570973+0000","last_became_peered":"2024-07-27T03:12:32.570973+0000","last_unstale":"2024-07-27T03:15:25.061677+0000","last_undegraded":"2024-07-27T03:15:25.061677+0000","last_fullsized":"2024-07-27T03:15:25.061677+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:24:25.357833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.060782+0000","last_change":"2024-07-27T03:12:32.561243+0000","last_active":"2024-07-27T03:15:25.060782+0000","last_peered":"2024-07-27T03:15:25.060782+0000","last_clean":"2024-07-27T03:15:25.060782+0000","last_became_active":"2024-07-27T03:12:32.560748+0000","last_became_peered":"2024-07-27T03:12:32.560748+0000","last_unstale":"2024-07-27T03:15:25.060782+0000","last_undegraded":"2024-07-27T03:15:25.060782+0000","last_fullsized":"2024-07-27T03:15:25.060782+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:47:40.430283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.057184+0000","last_change":"2024-07-27T03:12:32.562545+0000","last_active":"2024-07-27T03:15:25.057184+0000","last_peered":"2024-07-27T03:15:25.057184+0000","last_clean":"2024-07-27T03:15:25.057184+0000","last_became_active":"2024-07-27T03:12:32.562202+0000","last_became_peered":"2024-07-27T03:12:32.562202+0000","last_unstale":"2024-07-27T03:15:25.057184+0000","last_undegraded":"2024-07-27T03:15:25.057184+0000","last_fullsized":"2024-07-27T03:15:25.057184+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:05:20.229232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.065174+0000","last_change":"2024-07-27T03:12:32.565162+0000","last_active":"2024-07-27T03:15:25.065174+0000","last_peered":"2024-07-27T03:15:25.065174+0000","last_clean":"2024-07-27T03:15:25.065174+0000","last_became_active":"2024-07-27T03:12:32.564798+0000","last_became_peered":"2024-07-27T03:12:32.564798+0000","last_unstale":"2024-07-27T03:15:25.065174+0000","last_undegraded":"2024-07-27T03:15:25.065174+0000","last_fullsized":"2024-07-27T03:15:25.065174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:54:29.742624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.060883+0000","last_change":"2024-07-27T03:12:32.556874+0000","last_active":"2024-07-27T03:15:25.060883+0000","last_peered":"2024-07-27T03:15:25.060883+0000","last_clean":"2024-07-27T03:15:25.060883+0000","last_became_active":"2024-07-27T03:12:32.556467+0000","last_became_peered":"2024-07-27T03:12:32.556467+0000","last_unstale":"2024-07-27T03:15:25.060883+0000","last_undegraded":"2024-07-27T03:15:25.060883+0000","last_fullsized":"2024-07-27T03:15:25.060883+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:51:20.806785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.053073+0000","last_change":"2024-07-27T03:12:32.569136+0000","last_active":"2024-07-27T03:15:25.053073+0000","last_peered":"2024-07-27T03:15:25.053073+0000","last_clean":"2024-07-27T03:15:25.053073+0000","last_became_active":"2024-07-27T03:12:32.568778+0000","last_became_peered":"2024-07-27T03:12:32.568778+0000","last_unstale":"2024-07-27T03:15:25.053073+0000","last_undegraded":"2024-07-27T03:15:25.053073+0000","last_fullsized":"2024-07-27T03:15:25.053073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:26:22.654974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.069928+0000","last_change":"2024-07-27T03:12:32.569507+0000","last_active":"2024-07-27T03:15:25.069928+0000","last_peered":"2024-07-27T03:15:25.069928+0000","last_clean":"2024-07-27T03:15:25.069928+0000","last_became_active":"2024-07-27T03:12:32.569109+0000","last_became_peered":"2024-07-27T03:12:32.569109+0000","last_unstale":"2024-07-27T03:15:25.069928+0000","last_undegraded":"2024-07-27T03:15:25.069928+0000","last_fullsized":"2024-07-27T03:15:25.069928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:35:12.944933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.053821+0000","last_change":"2024-07-27T03:12:32.566479+0000","last_active":"2024-07-27T03:15:25.053821+0000","last_peered":"2024-07-27T03:15:25.053821+0000","last_clean":"2024-07-27T03:15:25.053821+0000","last_became_active":"2024-07-27T03:12:32.566114+0000","last_became_peered":"2024-07-27T03:12:32.566114+0000","last_unstale":"2024-07-27T03:15:25.053821+0000","last_undegraded":"2024-07-27T03:15:25.053821+0000","last_fullsized":"2024-07-27T03:15:25.053821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:24:19.944031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.051389+0000","last_change":"2024-07-27T03:12:56.726118+0000","last_active":"2024-07-27T03:15:25.051389+0000","last_peered":"2024-07-27T03:15:25.051389+0000","last_clean":"2024-07-27T03:15:25.051389+0000","last_became_active":"2024-07-27T03:12:56.725637+0000","last_became_peered":"2024-07-27T03:12:56.725637+0000","last_unstale":"2024-07-27T03:15:25.051389+0000","last_undegraded":"2024-07-27T03:15:25.051389+0000","last_fullsized":"2024-07-27T03:15:25.051389+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:36:41.725150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.053321+0000","last_change":"2024-07-27T03:12:32.568232+0000","last_active":"2024-07-27T03:15:25.053321+0000","last_peered":"2024-07-27T03:15:25.053321+0000","last_clean":"2024-07-27T03:15:25.053321+0000","last_became_active":"2024-07-27T03:12:32.567834+0000","last_became_peered":"2024-07-27T03:12:32.567834+0000","last_unstale":"2024-07-27T03:15:25.053321+0000","last_undegraded":"2024-07-27T03:15:25.053321+0000","last_fullsized":"2024-07-27T03:15:25.053321+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:40:28.234539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.070056+0000","last_change":"2024-07-27T03:12:32.573451+0000","last_active":"2024-07-27T03:15:25.070056+0000","last_peered":"2024-07-27T03:15:25.070056+0000","last_clean":"2024-07-27T03:15:25.070056+0000","last_became_active":"2024-07-27T03:12:32.573066+0000","last_became_peered":"2024-07-27T03:12:32.573066+0000","last_unstale":"2024-07-27T03:15:25.070056+0000","last_undegraded":"2024-07-27T03:15:25.070056+0000","last_fullsized":"2024-07-27T03:15:25.070056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:13:29.104055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.063555+0000","last_change":"2024-07-27T03:12:32.561067+0000","last_active":"2024-07-27T03:15:25.063555+0000","last_peered":"2024-07-27T03:15:25.063555+0000","last_clean":"2024-07-27T03:15:25.063555+0000","last_became_active":"2024-07-27T03:12:32.560358+0000","last_became_peered":"2024-07-27T03:12:32.560358+0000","last_unstale":"2024-07-27T03:15:25.063555+0000","last_undegraded":"2024-07-27T03:15:25.063555+0000","last_fullsized":"2024-07-27T03:15:25.063555+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:41:41.026804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.053463+0000","last_change":"2024-07-27T03:12:32.572290+0000","last_active":"2024-07-27T03:15:25.053463+0000","last_peered":"2024-07-27T03:15:25.053463+0000","last_clean":"2024-07-27T03:15:25.053463+0000","last_became_active":"2024-07-27T03:12:32.571877+0000","last_became_peered":"2024-07-27T03:12:32.571877+0000","last_unstale":"2024-07-27T03:15:25.053463+0000","last_undegraded":"2024-07-27T03:15:25.053463+0000","last_fullsized":"2024-07-27T03:15:25.053463+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:34:40.559787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.056923+0000","last_change":"2024-07-27T03:12:32.565495+0000","last_active":"2024-07-27T03:15:25.056923+0000","last_peered":"2024-07-27T03:15:25.056923+0000","last_clean":"2024-07-27T03:15:25.056923+0000","last_became_active":"2024-07-27T03:12:32.563136+0000","last_became_peered":"2024-07-27T03:12:32.563136+0000","last_unstale":"2024-07-27T03:15:25.056923+0000","last_undegraded":"2024-07-27T03:15:25.056923+0000","last_fullsized":"2024-07-27T03:15:25.056923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:31:02.032776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.070390+0000","last_change":"2024-07-27T03:12:32.569477+0000","last_active":"2024-07-27T03:15:25.070390+0000","last_peered":"2024-07-27T03:15:25.070390+0000","last_clean":"2024-07-27T03:15:25.070390+0000","last_became_active":"2024-07-27T03:12:32.569098+0000","last_became_peered":"2024-07-27T03:12:32.569098+0000","last_unstale":"2024-07-27T03:15:25.070390+0000","last_undegraded":"2024-07-27T03:15:25.070390+0000","last_fullsized":"2024-07-27T03:15:25.070390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:44:26.082539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.057708+0000","last_change":"2024-07-27T03:12:32.574704+0000","last_active":"2024-07-27T03:15:25.057708+0000","last_peered":"2024-07-27T03:15:25.057708+0000","last_clean":"2024-07-27T03:15:25.057708+0000","last_became_active":"2024-07-27T03:12:32.574261+0000","last_became_peered":"2024-07-27T03:12:32.574261+0000","last_unstale":"2024-07-27T03:15:25.057708+0000","last_undegraded":"2024-07-27T03:15:25.057708+0000","last_fullsized":"2024-07-27T03:15:25.057708+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:53:06.445267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.068628+0000","last_change":"2024-07-27T03:12:32.573835+0000","last_active":"2024-07-27T03:15:25.068628+0000","last_peered":"2024-07-27T03:15:25.068628+0000","last_clean":"2024-07-27T03:15:25.068628+0000","last_became_active":"2024-07-27T03:12:32.573450+0000","last_became_peered":"2024-07-27T03:12:32.573450+0000","last_unstale":"2024-07-27T03:15:25.068628+0000","last_undegraded":"2024-07-27T03:15:25.068628+0000","last_fullsized":"2024-07-27T03:15:25.068628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:40:45.264000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.060743+0000","last_change":"2024-07-27T03:12:32.568542+0000","last_active":"2024-07-27T03:15:25.060743+0000","last_peered":"2024-07-27T03:15:25.060743+0000","last_clean":"2024-07-27T03:15:25.060743+0000","last_became_active":"2024-07-27T03:12:32.568179+0000","last_became_peered":"2024-07-27T03:12:32.568179+0000","last_unstale":"2024-07-27T03:15:25.060743+0000","last_undegraded":"2024-07-27T03:15:25.060743+0000","last_fullsized":"2024-07-27T03:15:25.060743+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:32:10.005601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.061855+0000","last_change":"2024-07-27T03:12:32.564063+0000","last_active":"2024-07-27T03:15:25.061855+0000","last_peered":"2024-07-27T03:15:25.061855+0000","last_clean":"2024-07-27T03:15:25.061855+0000","last_became_active":"2024-07-27T03:12:32.563705+0000","last_became_peered":"2024-07-27T03:12:32.563705+0000","last_unstale":"2024-07-27T03:15:25.061855+0000","last_undegraded":"2024-07-27T03:15:25.061855+0000","last_fullsized":"2024-07-27T03:15:25.061855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:40:19.234194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.061066+0000","last_change":"2024-07-27T03:12:57.178963+0000","last_active":"2024-07-27T03:15:25.061066+0000","last_peered":"2024-07-27T03:15:25.061066+0000","last_clean":"2024-07-27T03:15:25.061066+0000","last_became_active":"2024-07-27T03:12:57.178550+0000","last_became_peered":"2024-07-27T03:12:57.178550+0000","last_unstale":"2024-07-27T03:15:25.061066+0000","last_undegraded":"2024-07-27T03:15:25.061066+0000","last_fullsized":"2024-07-27T03:15:25.061066+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:20:06.586957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.060099+0000","last_change":"2024-07-27T03:12:32.569863+0000","last_active":"2024-07-27T03:15:25.060099+0000","last_peered":"2024-07-27T03:15:25.060099+0000","last_clean":"2024-07-27T03:15:25.060099+0000","last_became_active":"2024-07-27T03:12:32.566457+0000","last_became_peered":"2024-07-27T03:12:32.566457+0000","last_unstale":"2024-07-27T03:15:25.060099+0000","last_undegraded":"2024-07-27T03:15:25.060099+0000","last_fullsized":"2024-07-27T03:15:25.060099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:28:13.141002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.058759+0000","last_change":"2024-07-27T03:12:32.574490+0000","last_active":"2024-07-27T03:15:25.058759+0000","last_peered":"2024-07-27T03:15:25.058759+0000","last_clean":"2024-07-27T03:15:25.058759+0000","last_became_active":"2024-07-27T03:12:32.574130+0000","last_became_peered":"2024-07-27T03:12:32.574130+0000","last_unstale":"2024-07-27T03:15:25.058759+0000","last_undegraded":"2024-07-27T03:15:25.058759+0000","last_fullsized":"2024-07-27T03:15:25.058759+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:44:22.696669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.066579+0000","last_change":"2024-07-27T03:12:32.560106+0000","last_active":"2024-07-27T03:15:25.066579+0000","last_peered":"2024-07-27T03:15:25.066579+0000","last_clean":"2024-07-27T03:15:25.066579+0000","last_became_active":"2024-07-27T03:12:32.559751+0000","last_became_peered":"2024-07-27T03:12:32.559751+0000","last_unstale":"2024-07-27T03:15:25.066579+0000","last_undegraded":"2024-07-27T03:15:25.066579+0000","last_fullsized":"2024-07-27T03:15:25.066579+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:16:37.647462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.062851+0000","last_change":"2024-07-27T03:12:32.569352+0000","last_active":"2024-07-27T03:15:25.062851+0000","last_peered":"2024-07-27T03:15:25.062851+0000","last_clean":"2024-07-27T03:15:25.062851+0000","last_became_active":"2024-07-27T03:12:32.569004+0000","last_became_peered":"2024-07-27T03:12:32.569004+0000","last_unstale":"2024-07-27T03:15:25.062851+0000","last_undegraded":"2024-07-27T03:15:25.062851+0000","last_fullsized":"2024-07-27T03:15:25.062851+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:21:25.999214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.053831+0000","last_change":"2024-07-27T03:12:32.562775+0000","last_active":"2024-07-27T03:15:25.053831+0000","last_peered":"2024-07-27T03:15:25.053831+0000","last_clean":"2024-07-27T03:15:25.053831+0000","last_became_active":"2024-07-27T03:12:32.562403+0000","last_became_peered":"2024-07-27T03:12:32.562403+0000","last_unstale":"2024-07-27T03:15:25.053831+0000","last_undegraded":"2024-07-27T03:15:25.053831+0000","last_fullsized":"2024-07-27T03:15:25.053831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:16:09.474505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.061943+0000","last_change":"2024-07-27T03:12:32.563271+0000","last_active":"2024-07-27T03:15:25.061943+0000","last_peered":"2024-07-27T03:15:25.061943+0000","last_clean":"2024-07-27T03:15:25.061943+0000","last_became_active":"2024-07-27T03:12:32.562850+0000","last_became_peered":"2024-07-27T03:12:32.562850+0000","last_unstale":"2024-07-27T03:15:25.061943+0000","last_undegraded":"2024-07-27T03:15:25.061943+0000","last_fullsized":"2024-07-27T03:15:25.061943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:40:27.688205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.065581+0000","last_change":"2024-07-27T03:12:32.557580+0000","last_active":"2024-07-27T03:15:25.065581+0000","last_peered":"2024-07-27T03:15:25.065581+0000","last_clean":"2024-07-27T03:15:25.065581+0000","last_became_active":"2024-07-27T03:12:32.557190+0000","last_became_peered":"2024-07-27T03:12:32.557190+0000","last_unstale":"2024-07-27T03:15:25.065581+0000","last_undegraded":"2024-07-27T03:15:25.065581+0000","last_fullsized":"2024-07-27T03:15:25.065581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:58:17.609035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.033909+0000","last_change":"2024-07-27T03:12:32.563669+0000","last_active":"2024-07-27T03:15:25.033909+0000","last_peered":"2024-07-27T03:15:25.033909+0000","last_clean":"2024-07-27T03:15:25.033909+0000","last_became_active":"2024-07-27T03:12:32.563200+0000","last_became_peered":"2024-07-27T03:12:32.563200+0000","last_unstale":"2024-07-27T03:15:25.033909+0000","last_undegraded":"2024-07-27T03:15:25.033909+0000","last_fullsized":"2024-07-27T03:15:25.033909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:14:05.624118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.055913+0000","last_change":"2024-07-27T03:12:32.561793+0000","last_active":"2024-07-27T03:15:25.055913+0000","last_peered":"2024-07-27T03:15:25.055913+0000","last_clean":"2024-07-27T03:15:25.055913+0000","last_became_active":"2024-07-27T03:12:32.561452+0000","last_became_peered":"2024-07-27T03:12:32.561452+0000","last_unstale":"2024-07-27T03:15:25.055913+0000","last_undegraded":"2024-07-27T03:15:25.055913+0000","last_fullsized":"2024-07-27T03:15:25.055913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:22:07.204824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.070321+0000","last_change":"2024-07-27T03:12:32.569811+0000","last_active":"2024-07-27T03:15:25.070321+0000","last_peered":"2024-07-27T03:15:25.070321+0000","last_clean":"2024-07-27T03:15:25.070321+0000","last_became_active":"2024-07-27T03:12:32.569474+0000","last_became_peered":"2024-07-27T03:12:32.569474+0000","last_unstale":"2024-07-27T03:15:25.070321+0000","last_undegraded":"2024-07-27T03:15:25.070321+0000","last_fullsized":"2024-07-27T03:15:25.070321+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:38:39.317362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.063814+0000","last_change":"2024-07-27T03:12:32.565159+0000","last_active":"2024-07-27T03:15:25.063814+0000","last_peered":"2024-07-27T03:15:25.063814+0000","last_clean":"2024-07-27T03:15:25.063814+0000","last_became_active":"2024-07-27T03:12:32.564750+0000","last_became_peered":"2024-07-27T03:12:32.564750+0000","last_unstale":"2024-07-27T03:15:25.063814+0000","last_undegraded":"2024-07-27T03:15:25.063814+0000","last_fullsized":"2024-07-27T03:15:25.063814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:51:26.186510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"91'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.057903+0000","last_change":"2024-07-27T03:12:32.574429+0000","last_active":"2024-07-27T03:15:25.057903+0000","last_peered":"2024-07-27T03:15:25.057903+0000","last_clean":"2024-07-27T03:15:25.057903+0000","last_became_active":"2024-07-27T03:12:32.573986+0000","last_became_peered":"2024-07-27T03:12:32.573986+0000","last_unstale":"2024-07-27T03:15:25.057903+0000","last_undegraded":"2024-07-27T03:15:25.057903+0000","last_fullsized":"2024-07-27T03:15:25.057903+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:35:00.345561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75488,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1881,"num_read_kb":1027,"num_write":263,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":226464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645378,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":525562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841607,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37052,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708100,"statfs":{"total":95995035648,"available":95957094400,"internally_reserved":0,"allocated":1757184,"data_stored":1184986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037835,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1077248,"data_stored":525470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234063,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":525826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430292,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1671168,"data_stored":1123018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593818,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":520974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790046,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1679360,"data_stored":1118068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953571,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":525580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149800,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":520682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411437,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1146880,"data_stored":590758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1126400,"data_stored":590006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738486,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1077248,"data_stored":522430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-27T03:16:45.771 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-07-27T03:16:45.772 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-07-27T03:16:45.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.919+0000 7f166fb57640 1 -- 172.21.15.67:0/2880219894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1668108e30 msgr2=0x7f166810b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:45.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.919+0000 7f166fb57640 1 --2- 172.21.15.67:0/2880219894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1668108e30 0x7f166810b340 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1660009ca0 tx=0x7f166002a8c0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.919+0000 7f166fb57640 1 -- 172.21.15.67:0/2880219894 shutdown_connections 2024-07-27T03:16:45.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.919+0000 7f166fb57640 1 --2- 172.21.15.67:0/2880219894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1668108e30 0x7f166810b340 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.919+0000 7f166fb57640 1 --2- 172.21.15.67:0/2880219894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1668106270 0x7f16681086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.919+0000 7f166fb57640 1 --2- 172.21.15.67:0/2880219894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f166806bed0 0x7f1668105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.919+0000 7f166fb57640 1 -- 172.21.15.67:0/2880219894 >> 172.21.15.67:0/2880219894 conn(0x7f16680fd950 msgr2=0x7f16680ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:45.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.919+0000 7f166fb57640 1 -- 172.21.15.67:0/2880219894 shutdown_connections 2024-07-27T03:16:45.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.920+0000 7f166fb57640 1 -- 172.21.15.67:0/2880219894 wait complete. 2024-07-27T03:16:45.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.920+0000 7f166fb57640 1 Processor -- start 2024-07-27T03:16:45.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.920+0000 7f166fb57640 1 -- start start 2024-07-27T03:16:45.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.920+0000 7f166fb57640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f166806bed0 0x7f1668199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:45.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.920+0000 7f166fb57640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1668106270 0x7f1668199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:45.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.920+0000 7f166fb57640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1668108e30 0x7f166819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:45.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.920+0000 7f166fb57640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f166810ed50 con 0x7f1668108e30 2024-07-27T03:16:45.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.920+0000 7f166fb57640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f166810ebd0 con 0x7f1668106270 2024-07-27T03:16:45.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.920+0000 7f166fb57640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f166810eed0 con 0x7f166806bed0 2024-07-27T03:16:45.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.920+0000 7f166e0f4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1668108e30 0x7f166819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:45.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.921+0000 7f166e0f4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1668108e30 0x7f166819ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51442/0 (socket says 172.21.15.67:51442) 2024-07-27T03:16:45.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.921+0000 7f166e0f4640 1 -- 172.21.15.67:0/3718579431 learned_addr learned my addr 172.21.15.67:0/3718579431 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:45.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.921+0000 7f166d8f3640 1 --2- 172.21.15.67:0/3718579431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f166806bed0 0x7f1668199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:45.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.921+0000 7f166d0f2640 1 --2- 172.21.15.67:0/3718579431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1668106270 0x7f1668199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:45.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.921+0000 7f166e0f4640 1 -- 172.21.15.67:0/3718579431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f166806bed0 msgr2=0x7f1668199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:45.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.921+0000 7f166e0f4640 1 --2- 172.21.15.67:0/3718579431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f166806bed0 0x7f1668199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.921+0000 7f166e0f4640 1 -- 172.21.15.67:0/3718579431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1668106270 msgr2=0x7f1668199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:45.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.921+0000 7f166e0f4640 1 --2- 172.21.15.67:0/3718579431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1668106270 0x7f1668199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:45.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.921+0000 7f166e0f4640 1 -- 172.21.15.67:0/3718579431 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166819b360 con 0x7f1668108e30 2024-07-27T03:16:45.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.921+0000 7f166d8f3640 1 --2- 172.21.15.67:0/3718579431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f166806bed0 0x7f1668199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:16:45.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.921+0000 7f166d0f2640 1 --2- 172.21.15.67:0/3718579431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1668106270 0x7f1668199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:16:45.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.921+0000 7f166e0f4640 1 --2- 172.21.15.67:0/3718579431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1668108e30 0x7f166819ab40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f16600097b0 tx=0x7f1660003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:45.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.921+0000 7f165effd640 1 -- 172.21.15.67:0/3718579431 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1660004240 con 0x7f1668108e30 2024-07-27T03:16:45.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.921+0000 7f165effd640 1 -- 172.21.15.67:0/3718579431 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f166002f380 con 0x7f1668108e30 2024-07-27T03:16:45.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.921+0000 7f166fb57640 1 -- 172.21.15.67:0/3718579431 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166819b5f0 con 0x7f1668108e30 2024-07-27T03:16:45.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.921+0000 7f165effd640 1 -- 172.21.15.67:0/3718579431 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166002fc30 con 0x7f1668108e30 2024-07-27T03:16:45.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.921+0000 7f166fb57640 1 -- 172.21.15.67:0/3718579431 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16681a1d40 con 0x7f1668108e30 2024-07-27T03:16:45.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.922+0000 7f166fb57640 1 -- 172.21.15.67:0/3718579431 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16681066e0 con 0x7f1668108e30 2024-07-27T03:16:45.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.924+0000 7f165effd640 1 -- 172.21.15.67:0/3718579431 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f16600043e0 con 0x7f1668108e30 2024-07-27T03:16:45.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.925+0000 7f165effd640 1 --2- 172.21.15.67:0/3718579431 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f16540b3bd0 0x7f16540b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:45.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.925+0000 7f166d8f3640 1 --2- 172.21.15.67:0/3718579431 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f16540b3bd0 0x7f16540b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:45.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.925+0000 7f165effd640 1 -- 172.21.15.67:0/3718579431 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f1660105960 con 0x7f1668108e30 2024-07-27T03:16:45.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.925+0000 7f166d8f3640 1 --2- 172.21.15.67:0/3718579431 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f16540b3bd0 0x7f16540b60e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1664005d00 tx=0x7f1664005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:45.930 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:45.928+0000 7f165effd640 1 -- 172.21.15.67:0/3718579431 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f16600ccc00 con 0x7f1668108e30 2024-07-27T03:16:46.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:46.120+0000 7f166fb57640 1 -- 172.21.15.67:0/3718579431 --> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1668063270 con 0x7f16540b3bd0 2024-07-27T03:16:46.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:46.125+0000 7f165effd640 1 -- 172.21.15.67:0/3718579431 <== mgr.14150 v2:172.21.15.67:6800/1534270904 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340567 (secure 0 0 0) 0x7f1668063270 con 0x7f16540b3bd0 2024-07-27T03:16:46.131 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:16:46.133 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-07-27T03:16:46.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:46.129+0000 7f166fb57640 1 -- 172.21.15.67:0/3718579431 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f16540b3bd0 msgr2=0x7f16540b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:46.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:46.129+0000 7f166fb57640 1 --2- 172.21.15.67:0/3718579431 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f16540b3bd0 0x7f16540b60e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1664005d00 tx=0x7f1664005c70 comp rx=0 tx=0).stop 2024-07-27T03:16:46.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:46.129+0000 7f166fb57640 1 -- 172.21.15.67:0/3718579431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1668108e30 msgr2=0x7f166819ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:46.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:46.129+0000 7f166fb57640 1 --2- 172.21.15.67:0/3718579431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1668108e30 0x7f166819ab40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f16600097b0 tx=0x7f1660003c20 comp rx=0 tx=0).stop 2024-07-27T03:16:46.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:46.129+0000 7f166fb57640 1 -- 172.21.15.67:0/3718579431 shutdown_connections 2024-07-27T03:16:46.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:46.129+0000 7f166fb57640 1 --2- 172.21.15.67:0/3718579431 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f16540b3bd0 0x7f16540b60e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:46.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:46.129+0000 7f166fb57640 1 --2- 172.21.15.67:0/3718579431 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1668108e30 0x7f166819ab40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:46.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:46.129+0000 7f166fb57640 1 --2- 172.21.15.67:0/3718579431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1668106270 0x7f1668199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:46.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:46.129+0000 7f166fb57640 1 --2- 172.21.15.67:0/3718579431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f166806bed0 0x7f1668199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:46.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:46.129+0000 7f166fb57640 1 -- 172.21.15.67:0/3718579431 >> 172.21.15.67:0/3718579431 conn(0x7f16680fd950 msgr2=0x7f1668104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:46.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:46.129+0000 7f166fb57640 1 -- 172.21.15.67:0/3718579431 shutdown_connections 2024-07-27T03:16:46.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:46.129+0000 7f166fb57640 1 -- 172.21.15.67:0/3718579431 wait complete. 2024-07-27T03:16:46.143 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-07-27T03:16:45.119670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666090,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1957,"num_read_kb":1091,"num_write":388,"num_write_kb":2310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951979,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":820,"ondisk_log_size":820,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388604,"kb_used_data":14492,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553220,"statfs":{"total":1151940427776,"available":1151542497280,"internally_reserved":0,"allocated":14839808,"data_stored":8273360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5681,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":137,"num_read_kb":77,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003487"},"pg_stats":[{"pgid":"2.1f","version":"89'13","reported_seq":241,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:40.530537+0000","last_change":"2024-07-27T03:12:29.541107+0000","last_active":"2024-07-27T03:16:40.530537+0000","last_peered":"2024-07-27T03:16:40.530537+0000","last_clean":"2024-07-27T03:16:40.530537+0000","last_became_active":"2024-07-27T03:12:29.540806+0000","last_became_peered":"2024-07-27T03:12:29.540806+0000","last_unstale":"2024-07-27T03:16:40.530537+0000","last_undegraded":"2024-07-27T03:16:40.530537+0000","last_fullsized":"2024-07-27T03:16:40.530537+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:38:24.349354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":182,"num_read_kb":182,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.057193+0000","last_change":"2024-07-27T03:12:32.565515+0000","last_active":"2024-07-27T03:15:25.057193+0000","last_peered":"2024-07-27T03:15:25.057193+0000","last_clean":"2024-07-27T03:15:25.057193+0000","last_became_active":"2024-07-27T03:12:32.565134+0000","last_became_peered":"2024-07-27T03:12:32.565134+0000","last_unstale":"2024-07-27T03:15:25.057193+0000","last_undegraded":"2024-07-27T03:15:25.057193+0000","last_fullsized":"2024-07-27T03:15:25.057193+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:11:56.793139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302752+0000","last_change":"2024-07-27T03:12:42.613432+0000","last_active":"2024-07-27T03:13:57.302752+0000","last_peered":"2024-07-27T03:13:57.302752+0000","last_clean":"2024-07-27T03:13:57.302752+0000","last_became_active":"2024-07-27T03:12:42.612986+0000","last_became_peered":"2024-07-27T03:12:42.612986+0000","last_unstale":"2024-07-27T03:13:57.302752+0000","last_undegraded":"2024-07-27T03:13:57.302752+0000","last_fullsized":"2024-07-27T03:13:57.302752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:25:37.505117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:32.726614+0000","last_change":"2024-07-27T03:12:29.614229+0000","last_active":"2024-07-27T03:14:32.726614+0000","last_peered":"2024-07-27T03:14:32.726614+0000","last_clean":"2024-07-27T03:14:32.726614+0000","last_became_active":"2024-07-27T03:12:29.613844+0000","last_became_peered":"2024-07-27T03:12:29.613844+0000","last_unstale":"2024-07-27T03:14:32.726614+0000","last_undegraded":"2024-07-27T03:14:32.726614+0000","last_fullsized":"2024-07-27T03:14:32.726614+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:10:28.917080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.070362+0000","last_change":"2024-07-27T03:12:32.563187+0000","last_active":"2024-07-27T03:15:25.070362+0000","last_peered":"2024-07-27T03:15:25.070362+0000","last_clean":"2024-07-27T03:15:25.070362+0000","last_became_active":"2024-07-27T03:12:32.562674+0000","last_became_peered":"2024-07-27T03:12:32.562674+0000","last_unstale":"2024-07-27T03:15:25.070362+0000","last_undegraded":"2024-07-27T03:15:25.070362+0000","last_fullsized":"2024-07-27T03:15:25.070362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:05:39.211514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303151+0000","last_change":"2024-07-27T03:12:42.611516+0000","last_active":"2024-07-27T03:13:57.303151+0000","last_peered":"2024-07-27T03:13:57.303151+0000","last_clean":"2024-07-27T03:13:57.303151+0000","last_became_active":"2024-07-27T03:12:42.610934+0000","last_became_peered":"2024-07-27T03:12:42.610934+0000","last_unstale":"2024-07-27T03:13:57.303151+0000","last_undegraded":"2024-07-27T03:13:57.303151+0000","last_fullsized":"2024-07-27T03:13:57.303151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:13:17.323063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":58,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305740+0000","last_change":"2024-07-27T03:12:29.539573+0000","last_active":"2024-07-27T03:13:57.305740+0000","last_peered":"2024-07-27T03:13:57.305740+0000","last_clean":"2024-07-27T03:13:57.305740+0000","last_became_active":"2024-07-27T03:12:29.539056+0000","last_became_peered":"2024-07-27T03:12:29.539056+0000","last_unstale":"2024-07-27T03:13:57.305740+0000","last_undegraded":"2024-07-27T03:13:57.305740+0000","last_fullsized":"2024-07-27T03:13:57.305740+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:04:37.513644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.068109+0000","last_change":"2024-07-27T03:12:32.567729+0000","last_active":"2024-07-27T03:15:25.068109+0000","last_peered":"2024-07-27T03:15:25.068109+0000","last_clean":"2024-07-27T03:15:25.068109+0000","last_became_active":"2024-07-27T03:12:32.564827+0000","last_became_peered":"2024-07-27T03:12:32.564827+0000","last_unstale":"2024-07-27T03:15:25.068109+0000","last_undegraded":"2024-07-27T03:15:25.068109+0000","last_fullsized":"2024-07-27T03:15:25.068109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:10:55.314988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303339+0000","last_change":"2024-07-27T03:12:42.612776+0000","last_active":"2024-07-27T03:13:57.303339+0000","last_peered":"2024-07-27T03:13:57.303339+0000","last_clean":"2024-07-27T03:13:57.303339+0000","last_became_active":"2024-07-27T03:12:42.612258+0000","last_became_peered":"2024-07-27T03:12:42.612258+0000","last_unstale":"2024-07-27T03:13:57.303339+0000","last_undegraded":"2024-07-27T03:13:57.303339+0000","last_fullsized":"2024-07-27T03:13:57.303339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:48:06.516002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305959+0000","last_change":"2024-07-27T03:12:29.540070+0000","last_active":"2024-07-27T03:13:57.305959+0000","last_peered":"2024-07-27T03:13:57.305959+0000","last_clean":"2024-07-27T03:13:57.305959+0000","last_became_active":"2024-07-27T03:12:29.539657+0000","last_became_peered":"2024-07-27T03:12:29.539657+0000","last_unstale":"2024-07-27T03:13:57.305959+0000","last_undegraded":"2024-07-27T03:13:57.305959+0000","last_fullsized":"2024-07-27T03:13:57.305959+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:42:48.222613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.067431+0000","last_change":"2024-07-27T03:12:32.574418+0000","last_active":"2024-07-27T03:15:25.067431+0000","last_peered":"2024-07-27T03:15:25.067431+0000","last_clean":"2024-07-27T03:15:25.067431+0000","last_became_active":"2024-07-27T03:12:32.573974+0000","last_became_peered":"2024-07-27T03:12:32.573974+0000","last_unstale":"2024-07-27T03:15:25.067431+0000","last_undegraded":"2024-07-27T03:15:25.067431+0000","last_fullsized":"2024-07-27T03:15:25.067431+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:34:30.442833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303642+0000","last_change":"2024-07-27T03:12:42.605155+0000","last_active":"2024-07-27T03:13:57.303642+0000","last_peered":"2024-07-27T03:13:57.303642+0000","last_clean":"2024-07-27T03:13:57.303642+0000","last_became_active":"2024-07-27T03:12:42.604685+0000","last_became_peered":"2024-07-27T03:12:42.604685+0000","last_unstale":"2024-07-27T03:13:57.303642+0000","last_undegraded":"2024-07-27T03:13:57.303642+0000","last_fullsized":"2024-07-27T03:13:57.303642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:28:58.931408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.065859+0000","last_change":"2024-07-27T03:12:32.574711+0000","last_active":"2024-07-27T03:15:25.065859+0000","last_peered":"2024-07-27T03:15:25.065859+0000","last_clean":"2024-07-27T03:15:25.065859+0000","last_became_active":"2024-07-27T03:12:32.574365+0000","last_became_peered":"2024-07-27T03:12:32.574365+0000","last_unstale":"2024-07-27T03:15:25.065859+0000","last_undegraded":"2024-07-27T03:15:25.065859+0000","last_fullsized":"2024-07-27T03:15:25.065859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:50:56.621371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305326+0000","last_change":"2024-07-27T03:12:29.540835+0000","last_active":"2024-07-27T03:13:57.305326+0000","last_peered":"2024-07-27T03:13:57.305326+0000","last_clean":"2024-07-27T03:13:57.305326+0000","last_became_active":"2024-07-27T03:12:29.540461+0000","last_became_peered":"2024-07-27T03:12:29.540461+0000","last_unstale":"2024-07-27T03:13:57.305326+0000","last_undegraded":"2024-07-27T03:13:57.305326+0000","last_fullsized":"2024-07-27T03:13:57.305326+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:19:20.817038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303344+0000","last_change":"2024-07-27T03:12:42.718584+0000","last_active":"2024-07-27T03:13:57.303344+0000","last_peered":"2024-07-27T03:13:57.303344+0000","last_clean":"2024-07-27T03:13:57.303344+0000","last_became_active":"2024-07-27T03:12:42.717752+0000","last_became_peered":"2024-07-27T03:12:42.717752+0000","last_unstale":"2024-07-27T03:13:57.303344+0000","last_undegraded":"2024-07-27T03:13:57.303344+0000","last_fullsized":"2024-07-27T03:13:57.303344+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:23:52.629794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.042310+0000","last_change":"2024-07-27T03:12:32.560672+0000","last_active":"2024-07-27T03:15:25.042310+0000","last_peered":"2024-07-27T03:15:25.042310+0000","last_clean":"2024-07-27T03:15:25.042310+0000","last_became_active":"2024-07-27T03:12:32.559172+0000","last_became_peered":"2024-07-27T03:12:32.559172+0000","last_unstale":"2024-07-27T03:15:25.042310+0000","last_undegraded":"2024-07-27T03:15:25.042310+0000","last_fullsized":"2024-07-27T03:15:25.042310+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:02:49.346983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405206+0000","last_change":"2024-07-27T03:12:57.434569+0000","last_active":"2024-07-27T03:13:57.405206+0000","last_peered":"2024-07-27T03:13:57.405206+0000","last_clean":"2024-07-27T03:13:57.405206+0000","last_became_active":"2024-07-27T03:12:57.434038+0000","last_became_peered":"2024-07-27T03:12:57.434038+0000","last_unstale":"2024-07-27T03:13:57.405206+0000","last_undegraded":"2024-07-27T03:13:57.405206+0000","last_fullsized":"2024-07-27T03:13:57.405206+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:43:24.462838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.412383+0000","last_change":"2024-07-27T03:13:57.411931+0000","last_active":"2024-07-27T03:13:57.412383+0000","last_peered":"2024-07-27T03:13:57.412383+0000","last_clean":"2024-07-27T03:13:57.412383+0000","last_became_active":"2024-07-27T03:13:57.407362+0000","last_became_peered":"2024-07-27T03:13:57.407362+0000","last_unstale":"2024-07-27T03:13:57.412383+0000","last_undegraded":"2024-07-27T03:13:57.412383+0000","last_fullsized":"2024-07-27T03:13:57.412383+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:52:08.444268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.058449+0000","last_change":"2024-07-27T03:12:32.572129+0000","last_active":"2024-07-27T03:15:25.058449+0000","last_peered":"2024-07-27T03:15:25.058449+0000","last_clean":"2024-07-27T03:15:25.058449+0000","last_became_active":"2024-07-27T03:12:32.571774+0000","last_became_peered":"2024-07-27T03:12:32.571774+0000","last_unstale":"2024-07-27T03:15:25.058449+0000","last_undegraded":"2024-07-27T03:15:25.058449+0000","last_fullsized":"2024-07-27T03:15:25.058449+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:52:23.316957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404446+0000","last_change":"2024-07-27T03:12:29.539768+0000","last_active":"2024-07-27T03:13:57.404446+0000","last_peered":"2024-07-27T03:13:57.404446+0000","last_clean":"2024-07-27T03:13:57.404446+0000","last_became_active":"2024-07-27T03:12:29.539389+0000","last_became_peered":"2024-07-27T03:12:29.539389+0000","last_unstale":"2024-07-27T03:13:57.404446+0000","last_undegraded":"2024-07-27T03:13:57.404446+0000","last_fullsized":"2024-07-27T03:13:57.404446+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:09:35.961328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302623+0000","last_change":"2024-07-27T03:12:42.617569+0000","last_active":"2024-07-27T03:13:57.302623+0000","last_peered":"2024-07-27T03:13:57.302623+0000","last_clean":"2024-07-27T03:13:57.302623+0000","last_became_active":"2024-07-27T03:12:42.616357+0000","last_became_peered":"2024-07-27T03:12:42.616357+0000","last_unstale":"2024-07-27T03:13:57.302623+0000","last_undegraded":"2024-07-27T03:13:57.302623+0000","last_fullsized":"2024-07-27T03:13:57.302623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:35:11.297631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.018136+0000","last_change":"2024-07-27T03:12:32.566272+0000","last_active":"2024-07-27T03:15:25.018136+0000","last_peered":"2024-07-27T03:15:25.018136+0000","last_clean":"2024-07-27T03:15:25.018136+0000","last_became_active":"2024-07-27T03:12:32.565865+0000","last_became_peered":"2024-07-27T03:12:32.565865+0000","last_unstale":"2024-07-27T03:15:25.018136+0000","last_undegraded":"2024-07-27T03:15:25.018136+0000","last_fullsized":"2024-07-27T03:15:25.018136+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:42:09.539586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405359+0000","last_change":"2024-07-27T03:12:29.613733+0000","last_active":"2024-07-27T03:13:57.405359+0000","last_peered":"2024-07-27T03:13:57.405359+0000","last_clean":"2024-07-27T03:13:57.405359+0000","last_became_active":"2024-07-27T03:12:29.613395+0000","last_became_peered":"2024-07-27T03:12:29.613395+0000","last_unstale":"2024-07-27T03:13:57.405359+0000","last_undegraded":"2024-07-27T03:13:57.405359+0000","last_fullsized":"2024-07-27T03:13:57.405359+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:21:34.431900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.408340+0000","last_change":"2024-07-27T03:13:57.407798+0000","last_active":"2024-07-27T03:13:57.408340+0000","last_peered":"2024-07-27T03:13:57.408340+0000","last_clean":"2024-07-27T03:13:57.408340+0000","last_became_active":"2024-07-27T03:13:57.407275+0000","last_became_peered":"2024-07-27T03:13:57.407275+0000","last_unstale":"2024-07-27T03:13:57.408340+0000","last_undegraded":"2024-07-27T03:13:57.408340+0000","last_fullsized":"2024-07-27T03:13:57.408340+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:47:27.817092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,8],"acting":[4,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.056823+0000","last_change":"2024-07-27T03:12:32.567006+0000","last_active":"2024-07-27T03:15:25.056823+0000","last_peered":"2024-07-27T03:15:25.056823+0000","last_clean":"2024-07-27T03:15:25.056823+0000","last_became_active":"2024-07-27T03:12:32.566205+0000","last_became_peered":"2024-07-27T03:12:32.566205+0000","last_unstale":"2024-07-27T03:15:25.056823+0000","last_undegraded":"2024-07-27T03:15:25.056823+0000","last_fullsized":"2024-07-27T03:15:25.056823+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:11:23.171341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303695+0000","last_change":"2024-07-27T03:12:29.533909+0000","last_active":"2024-07-27T03:13:57.303695+0000","last_peered":"2024-07-27T03:13:57.303695+0000","last_clean":"2024-07-27T03:13:57.303695+0000","last_became_active":"2024-07-27T03:12:29.533327+0000","last_became_peered":"2024-07-27T03:12:29.533327+0000","last_unstale":"2024-07-27T03:13:57.303695+0000","last_undegraded":"2024-07-27T03:13:57.303695+0000","last_fullsized":"2024-07-27T03:13:57.303695+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:35:24.705096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302923+0000","last_change":"2024-07-27T03:12:42.609066+0000","last_active":"2024-07-27T03:13:57.302923+0000","last_peered":"2024-07-27T03:13:57.302923+0000","last_clean":"2024-07-27T03:13:57.302923+0000","last_became_active":"2024-07-27T03:12:42.608561+0000","last_became_peered":"2024-07-27T03:12:42.608561+0000","last_unstale":"2024-07-27T03:13:57.302923+0000","last_undegraded":"2024-07-27T03:13:57.302923+0000","last_fullsized":"2024-07-27T03:13:57.302923+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:27:53.657498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.053587+0000","last_change":"2024-07-27T03:12:32.560931+0000","last_active":"2024-07-27T03:15:25.053587+0000","last_peered":"2024-07-27T03:15:25.053587+0000","last_clean":"2024-07-27T03:15:25.053587+0000","last_became_active":"2024-07-27T03:12:32.560549+0000","last_became_peered":"2024-07-27T03:12:32.560549+0000","last_unstale":"2024-07-27T03:15:25.053587+0000","last_undegraded":"2024-07-27T03:15:25.053587+0000","last_fullsized":"2024-07-27T03:15:25.053587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T15:11:25.136035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'17","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.595553+0000","last_change":"2024-07-27T03:12:29.535365+0000","last_active":"2024-07-27T03:15:25.595553+0000","last_peered":"2024-07-27T03:15:25.595553+0000","last_clean":"2024-07-27T03:15:25.595553+0000","last_became_active":"2024-07-27T03:12:29.534952+0000","last_became_peered":"2024-07-27T03:12:29.534952+0000","last_unstale":"2024-07-27T03:15:25.595553+0000","last_undegraded":"2024-07-27T03:15:25.595553+0000","last_fullsized":"2024-07-27T03:15:25.595553+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:34:59.889488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:42.727465+0000","last_change":"2024-07-27T03:12:42.718416+0000","last_active":"2024-07-27T03:14:42.727465+0000","last_peered":"2024-07-27T03:14:42.727465+0000","last_clean":"2024-07-27T03:14:42.727465+0000","last_became_active":"2024-07-27T03:12:42.717949+0000","last_became_peered":"2024-07-27T03:12:42.717949+0000","last_unstale":"2024-07-27T03:14:42.727465+0000","last_undegraded":"2024-07-27T03:14:42.727465+0000","last_fullsized":"2024-07-27T03:14:42.727465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:05:05.762577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.057594+0000","last_change":"2024-07-27T03:12:32.559824+0000","last_active":"2024-07-27T03:15:25.057594+0000","last_peered":"2024-07-27T03:15:25.057594+0000","last_clean":"2024-07-27T03:15:25.057594+0000","last_became_active":"2024-07-27T03:12:32.559435+0000","last_became_peered":"2024-07-27T03:12:32.559435+0000","last_unstale":"2024-07-27T03:15:25.057594+0000","last_undegraded":"2024-07-27T03:15:25.057594+0000","last_fullsized":"2024-07-27T03:15:25.057594+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:16:44.514272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:35.285402+0000","last_change":"2024-07-27T03:12:29.614296+0000","last_active":"2024-07-27T03:14:35.285402+0000","last_peered":"2024-07-27T03:14:35.285402+0000","last_clean":"2024-07-27T03:14:35.285402+0000","last_became_active":"2024-07-27T03:12:29.612987+0000","last_became_peered":"2024-07-27T03:12:29.612987+0000","last_unstale":"2024-07-27T03:14:35.285402+0000","last_undegraded":"2024-07-27T03:14:35.285402+0000","last_fullsized":"2024-07-27T03:14:35.285402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:23:09.226523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405200+0000","last_change":"2024-07-27T03:12:42.616575+0000","last_active":"2024-07-27T03:13:57.405200+0000","last_peered":"2024-07-27T03:13:57.405200+0000","last_clean":"2024-07-27T03:13:57.405200+0000","last_became_active":"2024-07-27T03:12:42.615980+0000","last_became_peered":"2024-07-27T03:12:42.615980+0000","last_unstale":"2024-07-27T03:13:57.405200+0000","last_undegraded":"2024-07-27T03:13:57.405200+0000","last_fullsized":"2024-07-27T03:13:57.405200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:00:59.041910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.062700+0000","last_change":"2024-07-27T03:12:32.556109+0000","last_active":"2024-07-27T03:15:25.062700+0000","last_peered":"2024-07-27T03:15:25.062700+0000","last_clean":"2024-07-27T03:15:25.062700+0000","last_became_active":"2024-07-27T03:12:32.555747+0000","last_became_peered":"2024-07-27T03:12:32.555747+0000","last_unstale":"2024-07-27T03:15:25.062700+0000","last_undegraded":"2024-07-27T03:15:25.062700+0000","last_fullsized":"2024-07-27T03:15:25.062700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:27:58.286605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:36.101756+0000","last_change":"2024-07-27T03:12:29.613523+0000","last_active":"2024-07-27T03:14:36.101756+0000","last_peered":"2024-07-27T03:14:36.101756+0000","last_clean":"2024-07-27T03:14:36.101756+0000","last_became_active":"2024-07-27T03:12:29.613172+0000","last_became_peered":"2024-07-27T03:12:29.613172+0000","last_unstale":"2024-07-27T03:14:36.101756+0000","last_undegraded":"2024-07-27T03:14:36.101756+0000","last_fullsized":"2024-07-27T03:14:36.101756+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:49:45.225221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304930+0000","last_change":"2024-07-27T03:12:42.718536+0000","last_active":"2024-07-27T03:13:57.304930+0000","last_peered":"2024-07-27T03:13:57.304930+0000","last_clean":"2024-07-27T03:13:57.304930+0000","last_became_active":"2024-07-27T03:12:42.718068+0000","last_became_peered":"2024-07-27T03:12:42.718068+0000","last_unstale":"2024-07-27T03:13:57.304930+0000","last_undegraded":"2024-07-27T03:13:57.304930+0000","last_fullsized":"2024-07-27T03:13:57.304930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:18:28.478700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.064182+0000","last_change":"2024-07-27T03:12:32.558060+0000","last_active":"2024-07-27T03:15:25.064182+0000","last_peered":"2024-07-27T03:15:25.064182+0000","last_clean":"2024-07-27T03:15:25.064182+0000","last_became_active":"2024-07-27T03:12:32.557260+0000","last_became_peered":"2024-07-27T03:12:32.557260+0000","last_unstale":"2024-07-27T03:15:25.064182+0000","last_undegraded":"2024-07-27T03:15:25.064182+0000","last_fullsized":"2024-07-27T03:15:25.064182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:36:54.945761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":234,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:41.532322+0000","last_change":"2024-07-27T03:12:29.541080+0000","last_active":"2024-07-27T03:16:41.532322+0000","last_peered":"2024-07-27T03:16:41.532322+0000","last_clean":"2024-07-27T03:16:41.532322+0000","last_became_active":"2024-07-27T03:12:29.540628+0000","last_became_peered":"2024-07-27T03:12:29.540628+0000","last_unstale":"2024-07-27T03:16:41.532322+0000","last_undegraded":"2024-07-27T03:16:41.532322+0000","last_fullsized":"2024-07-27T03:16:41.532322+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:45:17.045124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":183,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348865+0000","last_change":"2024-07-27T03:12:42.611561+0000","last_active":"2024-07-27T03:13:57.348865+0000","last_peered":"2024-07-27T03:13:57.348865+0000","last_clean":"2024-07-27T03:13:57.348865+0000","last_became_active":"2024-07-27T03:12:42.610507+0000","last_became_peered":"2024-07-27T03:12:42.610507+0000","last_unstale":"2024-07-27T03:13:57.348865+0000","last_undegraded":"2024-07-27T03:13:57.348865+0000","last_fullsized":"2024-07-27T03:13:57.348865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:15:08.972807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.058339+0000","last_change":"2024-07-27T03:12:32.562316+0000","last_active":"2024-07-27T03:15:25.058339+0000","last_peered":"2024-07-27T03:15:25.058339+0000","last_clean":"2024-07-27T03:15:25.058339+0000","last_became_active":"2024-07-27T03:12:32.561729+0000","last_became_peered":"2024-07-27T03:12:32.561729+0000","last_unstale":"2024-07-27T03:15:25.058339+0000","last_undegraded":"2024-07-27T03:15:25.058339+0000","last_fullsized":"2024-07-27T03:15:25.058339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:01:19.960269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":238,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:42.533767+0000","last_change":"2024-07-27T03:12:29.613459+0000","last_active":"2024-07-27T03:16:42.533767+0000","last_peered":"2024-07-27T03:16:42.533767+0000","last_clean":"2024-07-27T03:16:42.533767+0000","last_became_active":"2024-07-27T03:12:29.613009+0000","last_became_peered":"2024-07-27T03:12:29.613009+0000","last_unstale":"2024-07-27T03:16:42.533767+0000","last_undegraded":"2024-07-27T03:16:42.533767+0000","last_fullsized":"2024-07-27T03:16:42.533767+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:16:37.230471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":185,"num_read_kb":185,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348695+0000","last_change":"2024-07-27T03:12:42.718516+0000","last_active":"2024-07-27T03:13:57.348695+0000","last_peered":"2024-07-27T03:13:57.348695+0000","last_clean":"2024-07-27T03:13:57.348695+0000","last_became_active":"2024-07-27T03:12:42.718041+0000","last_became_peered":"2024-07-27T03:12:42.718041+0000","last_unstale":"2024-07-27T03:13:57.348695+0000","last_undegraded":"2024-07-27T03:13:57.348695+0000","last_fullsized":"2024-07-27T03:13:57.348695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:35:39.633977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.063195+0000","last_change":"2024-07-27T03:12:32.560752+0000","last_active":"2024-07-27T03:15:25.063195+0000","last_peered":"2024-07-27T03:15:25.063195+0000","last_clean":"2024-07-27T03:15:25.063195+0000","last_became_active":"2024-07-27T03:12:32.560248+0000","last_became_peered":"2024-07-27T03:12:32.560248+0000","last_unstale":"2024-07-27T03:15:25.063195+0000","last_undegraded":"2024-07-27T03:15:25.063195+0000","last_fullsized":"2024-07-27T03:15:25.063195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:59:48.056867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:31.988172+0000","last_change":"2024-07-27T03:12:29.539591+0000","last_active":"2024-07-27T03:14:31.988172+0000","last_peered":"2024-07-27T03:14:31.988172+0000","last_clean":"2024-07-27T03:14:31.988172+0000","last_became_active":"2024-07-27T03:12:29.539165+0000","last_became_peered":"2024-07-27T03:12:29.539165+0000","last_unstale":"2024-07-27T03:14:31.988172+0000","last_undegraded":"2024-07-27T03:14:31.988172+0000","last_fullsized":"2024-07-27T03:14:31.988172+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:04:21.940511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:00.281510+0000","last_change":"2024-07-27T03:13:57.407921+0000","last_active":"2024-07-27T03:14:00.281510+0000","last_peered":"2024-07-27T03:14:00.281510+0000","last_clean":"2024-07-27T03:14:00.281510+0000","last_became_active":"2024-07-27T03:13:57.407382+0000","last_became_peered":"2024-07-27T03:13:57.407382+0000","last_unstale":"2024-07-27T03:14:00.281510+0000","last_undegraded":"2024-07-27T03:14:00.281510+0000","last_fullsized":"2024-07-27T03:14:00.281510+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:48:27.362759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.069802+0000","last_change":"2024-07-27T03:12:32.570653+0000","last_active":"2024-07-27T03:15:25.069802+0000","last_peered":"2024-07-27T03:15:25.069802+0000","last_clean":"2024-07-27T03:15:25.069802+0000","last_became_active":"2024-07-27T03:12:32.569088+0000","last_became_peered":"2024-07-27T03:12:32.569088+0000","last_unstale":"2024-07-27T03:15:25.069802+0000","last_undegraded":"2024-07-27T03:15:25.069802+0000","last_fullsized":"2024-07-27T03:15:25.069802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:50:24.669431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:35.895703+0000","last_change":"2024-07-27T03:12:29.541654+0000","last_active":"2024-07-27T03:14:35.895703+0000","last_peered":"2024-07-27T03:14:35.895703+0000","last_clean":"2024-07-27T03:14:35.895703+0000","last_became_active":"2024-07-27T03:12:29.541264+0000","last_became_peered":"2024-07-27T03:12:29.541264+0000","last_unstale":"2024-07-27T03:14:35.895703+0000","last_undegraded":"2024-07-27T03:14:35.895703+0000","last_fullsized":"2024-07-27T03:14:35.895703+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:48:05.179016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":123,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:01.984277+0000","last_change":"2024-07-27T03:11:11.343199+0000","last_active":"2024-07-27T03:14:01.984277+0000","last_peered":"2024-07-27T03:14:01.984277+0000","last_clean":"2024-07-27T03:14:01.984277+0000","last_became_active":"2024-07-27T03:11:11.337635+0000","last_became_peered":"2024-07-27T03:11:11.337635+0000","last_unstale":"2024-07-27T03:14:01.984277+0000","last_undegraded":"2024-07-27T03:14:01.984277+0000","last_fullsized":"2024-07-27T03:14:01.984277+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:08:01.694830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:08:01.694830+0000","last_clean_scrub_stamp":"2024-07-27T03:08:01.694830+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:48:49.979610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.305836+0000","last_change":"2024-07-27T03:12:42.614346+0000","last_active":"2024-07-27T03:13:57.305836+0000","last_peered":"2024-07-27T03:13:57.305836+0000","last_clean":"2024-07-27T03:13:57.305836+0000","last_became_active":"2024-07-27T03:12:42.613881+0000","last_became_peered":"2024-07-27T03:12:42.613881+0000","last_unstale":"2024-07-27T03:13:57.305836+0000","last_undegraded":"2024-07-27T03:13:57.305836+0000","last_fullsized":"2024-07-27T03:13:57.305836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:12:38.299348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.058877+0000","last_change":"2024-07-27T03:12:32.574330+0000","last_active":"2024-07-27T03:15:25.058877+0000","last_peered":"2024-07-27T03:15:25.058877+0000","last_clean":"2024-07-27T03:15:25.058877+0000","last_became_active":"2024-07-27T03:12:32.573977+0000","last_became_peered":"2024-07-27T03:12:32.573977+0000","last_unstale":"2024-07-27T03:15:25.058877+0000","last_undegraded":"2024-07-27T03:15:25.058877+0000","last_fullsized":"2024-07-27T03:15:25.058877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:39:09.925794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'5","reported_seq":237,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:42.534600+0000","last_change":"2024-07-27T03:12:29.613848+0000","last_active":"2024-07-27T03:16:42.534600+0000","last_peered":"2024-07-27T03:16:42.534600+0000","last_clean":"2024-07-27T03:16:42.534600+0000","last_became_active":"2024-07-27T03:12:29.613473+0000","last_became_peered":"2024-07-27T03:12:29.613473+0000","last_unstale":"2024-07-27T03:16:42.534600+0000","last_undegraded":"2024-07-27T03:16:42.534600+0000","last_fullsized":"2024-07-27T03:16:42.534600+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:52:56.894962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":186,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405373+0000","last_change":"2024-07-27T03:12:42.611295+0000","last_active":"2024-07-27T03:13:57.405373+0000","last_peered":"2024-07-27T03:13:57.405373+0000","last_clean":"2024-07-27T03:13:57.405373+0000","last_became_active":"2024-07-27T03:12:42.610782+0000","last_became_peered":"2024-07-27T03:12:42.610782+0000","last_unstale":"2024-07-27T03:13:57.405373+0000","last_undegraded":"2024-07-27T03:13:57.405373+0000","last_fullsized":"2024-07-27T03:13:57.405373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:24:02.849882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.061556+0000","last_change":"2024-07-27T03:12:56.725924+0000","last_active":"2024-07-27T03:15:25.061556+0000","last_peered":"2024-07-27T03:15:25.061556+0000","last_clean":"2024-07-27T03:15:25.061556+0000","last_became_active":"2024-07-27T03:12:56.725567+0000","last_became_peered":"2024-07-27T03:12:56.725567+0000","last_unstale":"2024-07-27T03:15:25.061556+0000","last_undegraded":"2024-07-27T03:15:25.061556+0000","last_fullsized":"2024-07-27T03:15:25.061556+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:32:23.771483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405225+0000","last_change":"2024-07-27T03:12:29.537863+0000","last_active":"2024-07-27T03:13:57.405225+0000","last_peered":"2024-07-27T03:13:57.405225+0000","last_clean":"2024-07-27T03:13:57.405225+0000","last_became_active":"2024-07-27T03:12:29.537444+0000","last_became_peered":"2024-07-27T03:12:29.537444+0000","last_unstale":"2024-07-27T03:13:57.405225+0000","last_undegraded":"2024-07-27T03:13:57.405225+0000","last_fullsized":"2024-07-27T03:13:57.405225+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:52:05.239809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405265+0000","last_change":"2024-07-27T03:12:42.622599+0000","last_active":"2024-07-27T03:13:57.405265+0000","last_peered":"2024-07-27T03:13:57.405265+0000","last_clean":"2024-07-27T03:13:57.405265+0000","last_became_active":"2024-07-27T03:12:42.622151+0000","last_became_peered":"2024-07-27T03:12:42.622151+0000","last_unstale":"2024-07-27T03:13:57.405265+0000","last_undegraded":"2024-07-27T03:13:57.405265+0000","last_fullsized":"2024-07-27T03:13:57.405265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:34:31.622281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.050080+0000","last_change":"2024-07-27T03:12:32.562300+0000","last_active":"2024-07-27T03:15:25.050080+0000","last_peered":"2024-07-27T03:15:25.050080+0000","last_clean":"2024-07-27T03:15:25.050080+0000","last_became_active":"2024-07-27T03:12:32.561343+0000","last_became_peered":"2024-07-27T03:12:32.561343+0000","last_unstale":"2024-07-27T03:15:25.050080+0000","last_undegraded":"2024-07-27T03:15:25.050080+0000","last_fullsized":"2024-07-27T03:15:25.050080+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:01:28.626943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348835+0000","last_change":"2024-07-27T03:12:29.535177+0000","last_active":"2024-07-27T03:13:57.348835+0000","last_peered":"2024-07-27T03:13:57.348835+0000","last_clean":"2024-07-27T03:13:57.348835+0000","last_became_active":"2024-07-27T03:12:29.534662+0000","last_became_peered":"2024-07-27T03:12:29.534662+0000","last_unstale":"2024-07-27T03:13:57.348835+0000","last_undegraded":"2024-07-27T03:13:57.348835+0000","last_fullsized":"2024-07-27T03:13:57.348835+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:50:49.661992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404084+0000","last_change":"2024-07-27T03:12:42.609925+0000","last_active":"2024-07-27T03:13:57.404084+0000","last_peered":"2024-07-27T03:13:57.404084+0000","last_clean":"2024-07-27T03:13:57.404084+0000","last_became_active":"2024-07-27T03:12:42.609456+0000","last_became_peered":"2024-07-27T03:12:42.609456+0000","last_unstale":"2024-07-27T03:13:57.404084+0000","last_undegraded":"2024-07-27T03:13:57.404084+0000","last_fullsized":"2024-07-27T03:13:57.404084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:07:52.128381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.052969+0000","last_change":"2024-07-27T03:12:56.726170+0000","last_active":"2024-07-27T03:15:25.052969+0000","last_peered":"2024-07-27T03:15:25.052969+0000","last_clean":"2024-07-27T03:15:25.052969+0000","last_became_active":"2024-07-27T03:12:56.725685+0000","last_became_peered":"2024-07-27T03:12:56.725685+0000","last_unstale":"2024-07-27T03:15:25.052969+0000","last_undegraded":"2024-07-27T03:15:25.052969+0000","last_fullsized":"2024-07-27T03:15:25.052969+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:53:41.143333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405969+0000","last_change":"2024-07-27T03:12:56.726059+0000","last_active":"2024-07-27T03:13:57.405969+0000","last_peered":"2024-07-27T03:13:57.405969+0000","last_clean":"2024-07-27T03:13:57.405969+0000","last_became_active":"2024-07-27T03:12:56.722294+0000","last_became_peered":"2024-07-27T03:12:56.722294+0000","last_unstale":"2024-07-27T03:13:57.405969+0000","last_undegraded":"2024-07-27T03:13:57.405969+0000","last_fullsized":"2024-07-27T03:13:57.405969+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:18:42.627768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404320+0000","last_change":"2024-07-27T03:12:42.611966+0000","last_active":"2024-07-27T03:13:57.404320+0000","last_peered":"2024-07-27T03:13:57.404320+0000","last_clean":"2024-07-27T03:13:57.404320+0000","last_became_active":"2024-07-27T03:12:42.611522+0000","last_became_peered":"2024-07-27T03:12:42.611522+0000","last_unstale":"2024-07-27T03:13:57.404320+0000","last_undegraded":"2024-07-27T03:13:57.404320+0000","last_fullsized":"2024-07-27T03:13:57.404320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:09:54.034597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.069429+0000","last_change":"2024-07-27T03:12:56.719638+0000","last_active":"2024-07-27T03:15:25.069429+0000","last_peered":"2024-07-27T03:15:25.069429+0000","last_clean":"2024-07-27T03:15:25.069429+0000","last_became_active":"2024-07-27T03:12:56.719227+0000","last_became_peered":"2024-07-27T03:12:56.719227+0000","last_unstale":"2024-07-27T03:15:25.069429+0000","last_undegraded":"2024-07-27T03:15:25.069429+0000","last_fullsized":"2024-07-27T03:15:25.069429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:54:16.891518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304967+0000","last_change":"2024-07-27T03:12:29.614346+0000","last_active":"2024-07-27T03:13:57.304967+0000","last_peered":"2024-07-27T03:13:57.304967+0000","last_clean":"2024-07-27T03:13:57.304967+0000","last_became_active":"2024-07-27T03:12:29.613672+0000","last_became_peered":"2024-07-27T03:12:29.613672+0000","last_unstale":"2024-07-27T03:13:57.304967+0000","last_undegraded":"2024-07-27T03:13:57.304967+0000","last_fullsized":"2024-07-27T03:13:57.304967+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:46:35.505735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348712+0000","last_change":"2024-07-27T03:12:42.618364+0000","last_active":"2024-07-27T03:13:57.348712+0000","last_peered":"2024-07-27T03:13:57.348712+0000","last_clean":"2024-07-27T03:13:57.348712+0000","last_became_active":"2024-07-27T03:12:42.617988+0000","last_became_peered":"2024-07-27T03:12:42.617988+0000","last_unstale":"2024-07-27T03:13:57.348712+0000","last_undegraded":"2024-07-27T03:13:57.348712+0000","last_fullsized":"2024-07-27T03:13:57.348712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:40:21.725877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.061231+0000","last_change":"2024-07-27T03:12:32.563809+0000","last_active":"2024-07-27T03:15:25.061231+0000","last_peered":"2024-07-27T03:15:25.061231+0000","last_clean":"2024-07-27T03:15:25.061231+0000","last_became_active":"2024-07-27T03:12:32.563479+0000","last_became_peered":"2024-07-27T03:12:32.563479+0000","last_unstale":"2024-07-27T03:15:25.061231+0000","last_undegraded":"2024-07-27T03:15:25.061231+0000","last_fullsized":"2024-07-27T03:15:25.061231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:45:47.719241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":238,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:39.530182+0000","last_change":"2024-07-27T03:12:29.543786+0000","last_active":"2024-07-27T03:16:39.530182+0000","last_peered":"2024-07-27T03:16:39.530182+0000","last_clean":"2024-07-27T03:16:39.530182+0000","last_became_active":"2024-07-27T03:12:29.543203+0000","last_became_peered":"2024-07-27T03:12:29.543203+0000","last_unstale":"2024-07-27T03:16:39.530182+0000","last_undegraded":"2024-07-27T03:16:39.530182+0000","last_fullsized":"2024-07-27T03:16:39.530182+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:47:06.742171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":186,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302718+0000","last_change":"2024-07-27T03:12:42.622142+0000","last_active":"2024-07-27T03:13:57.302718+0000","last_peered":"2024-07-27T03:13:57.302718+0000","last_clean":"2024-07-27T03:13:57.302718+0000","last_became_active":"2024-07-27T03:12:42.621681+0000","last_became_peered":"2024-07-27T03:12:42.621681+0000","last_unstale":"2024-07-27T03:13:57.302718+0000","last_undegraded":"2024-07-27T03:13:57.302718+0000","last_fullsized":"2024-07-27T03:13:57.302718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:05:08.168424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.066031+0000","last_change":"2024-07-27T03:12:32.567119+0000","last_active":"2024-07-27T03:15:25.066031+0000","last_peered":"2024-07-27T03:15:25.066031+0000","last_clean":"2024-07-27T03:15:25.066031+0000","last_became_active":"2024-07-27T03:12:32.566757+0000","last_became_peered":"2024-07-27T03:12:32.566757+0000","last_unstale":"2024-07-27T03:15:25.066031+0000","last_undegraded":"2024-07-27T03:15:25.066031+0000","last_fullsized":"2024-07-27T03:15:25.066031+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:31:12.972804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404725+0000","last_change":"2024-07-27T03:12:29.540453+0000","last_active":"2024-07-27T03:13:57.404725+0000","last_peered":"2024-07-27T03:13:57.404725+0000","last_clean":"2024-07-27T03:13:57.404725+0000","last_became_active":"2024-07-27T03:12:29.539195+0000","last_became_peered":"2024-07-27T03:12:29.539195+0000","last_unstale":"2024-07-27T03:13:57.404725+0000","last_undegraded":"2024-07-27T03:13:57.404725+0000","last_fullsized":"2024-07-27T03:13:57.404725+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:44:31.144845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405363+0000","last_change":"2024-07-27T03:12:42.616671+0000","last_active":"2024-07-27T03:13:57.405363+0000","last_peered":"2024-07-27T03:13:57.405363+0000","last_clean":"2024-07-27T03:13:57.405363+0000","last_became_active":"2024-07-27T03:12:42.616198+0000","last_became_peered":"2024-07-27T03:12:42.616198+0000","last_unstale":"2024-07-27T03:13:57.405363+0000","last_undegraded":"2024-07-27T03:13:57.405363+0000","last_fullsized":"2024-07-27T03:13:57.405363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:01:31.584686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.060885+0000","last_change":"2024-07-27T03:12:32.562350+0000","last_active":"2024-07-27T03:15:25.060885+0000","last_peered":"2024-07-27T03:15:25.060885+0000","last_clean":"2024-07-27T03:15:25.060885+0000","last_became_active":"2024-07-27T03:12:32.562016+0000","last_became_peered":"2024-07-27T03:12:32.562016+0000","last_unstale":"2024-07-27T03:15:25.060885+0000","last_undegraded":"2024-07-27T03:15:25.060885+0000","last_fullsized":"2024-07-27T03:15:25.060885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:44:35.027329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303241+0000","last_change":"2024-07-27T03:12:29.541387+0000","last_active":"2024-07-27T03:13:57.303241+0000","last_peered":"2024-07-27T03:13:57.303241+0000","last_clean":"2024-07-27T03:13:57.303241+0000","last_became_active":"2024-07-27T03:12:29.541005+0000","last_became_peered":"2024-07-27T03:12:29.541005+0000","last_unstale":"2024-07-27T03:13:57.303241+0000","last_undegraded":"2024-07-27T03:13:57.303241+0000","last_fullsized":"2024-07-27T03:13:57.303241+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:22:16.268348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404849+0000","last_change":"2024-07-27T03:12:42.610029+0000","last_active":"2024-07-27T03:13:57.404849+0000","last_peered":"2024-07-27T03:13:57.404849+0000","last_clean":"2024-07-27T03:13:57.404849+0000","last_became_active":"2024-07-27T03:12:42.609592+0000","last_became_peered":"2024-07-27T03:12:42.609592+0000","last_unstale":"2024-07-27T03:13:57.404849+0000","last_undegraded":"2024-07-27T03:13:57.404849+0000","last_fullsized":"2024-07-27T03:13:57.404849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:47:15.254770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.051282+0000","last_change":"2024-07-27T03:12:32.568941+0000","last_active":"2024-07-27T03:15:25.051282+0000","last_peered":"2024-07-27T03:15:25.051282+0000","last_clean":"2024-07-27T03:15:25.051282+0000","last_became_active":"2024-07-27T03:12:32.568531+0000","last_became_peered":"2024-07-27T03:12:32.568531+0000","last_unstale":"2024-07-27T03:15:25.051282+0000","last_undegraded":"2024-07-27T03:15:25.051282+0000","last_fullsized":"2024-07-27T03:15:25.051282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:06:22.020505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"91'41","reported_seq":468,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:44.537754+0000","last_change":"2024-07-27T03:12:29.545131+0000","last_active":"2024-07-27T03:16:44.537754+0000","last_peered":"2024-07-27T03:16:44.537754+0000","last_clean":"2024-07-27T03:16:44.537754+0000","last_became_active":"2024-07-27T03:12:29.544768+0000","last_became_peered":"2024-07-27T03:12:29.544768+0000","last_unstale":"2024-07-27T03:16:44.537754+0000","last_undegraded":"2024-07-27T03:16:44.537754+0000","last_fullsized":"2024-07-27T03:16:44.537754+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:20:57.556128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68002,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":382,"num_read_kb":258,"num_write":43,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302087+0000","last_change":"2024-07-27T03:12:42.617483+0000","last_active":"2024-07-27T03:13:57.302087+0000","last_peered":"2024-07-27T03:13:57.302087+0000","last_clean":"2024-07-27T03:13:57.302087+0000","last_became_active":"2024-07-27T03:12:42.617065+0000","last_became_peered":"2024-07-27T03:12:42.617065+0000","last_unstale":"2024-07-27T03:13:57.302087+0000","last_undegraded":"2024-07-27T03:13:57.302087+0000","last_fullsized":"2024-07-27T03:13:57.302087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:52:17.448919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.069655+0000","last_change":"2024-07-27T03:12:56.722257+0000","last_active":"2024-07-27T03:15:25.069655+0000","last_peered":"2024-07-27T03:15:25.069655+0000","last_clean":"2024-07-27T03:15:25.069655+0000","last_became_active":"2024-07-27T03:12:56.721773+0000","last_became_peered":"2024-07-27T03:12:56.721773+0000","last_unstale":"2024-07-27T03:15:25.069655+0000","last_undegraded":"2024-07-27T03:15:25.069655+0000","last_fullsized":"2024-07-27T03:15:25.069655+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:30:44.310705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405205+0000","last_change":"2024-07-27T03:12:29.614171+0000","last_active":"2024-07-27T03:13:57.405205+0000","last_peered":"2024-07-27T03:13:57.405205+0000","last_clean":"2024-07-27T03:13:57.405205+0000","last_became_active":"2024-07-27T03:12:29.613270+0000","last_became_peered":"2024-07-27T03:12:29.613270+0000","last_unstale":"2024-07-27T03:13:57.405205+0000","last_undegraded":"2024-07-27T03:13:57.405205+0000","last_fullsized":"2024-07-27T03:13:57.405205+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:39:32.283778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303808+0000","last_change":"2024-07-27T03:12:42.610375+0000","last_active":"2024-07-27T03:13:57.303808+0000","last_peered":"2024-07-27T03:13:57.303808+0000","last_clean":"2024-07-27T03:13:57.303808+0000","last_became_active":"2024-07-27T03:12:42.609957+0000","last_became_peered":"2024-07-27T03:12:42.609957+0000","last_unstale":"2024-07-27T03:13:57.303808+0000","last_undegraded":"2024-07-27T03:13:57.303808+0000","last_fullsized":"2024-07-27T03:13:57.303808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:11:02.707179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.064641+0000","last_change":"2024-07-27T03:12:56.722332+0000","last_active":"2024-07-27T03:15:25.064641+0000","last_peered":"2024-07-27T03:15:25.064641+0000","last_clean":"2024-07-27T03:15:25.064641+0000","last_became_active":"2024-07-27T03:12:56.721972+0000","last_became_peered":"2024-07-27T03:12:56.721972+0000","last_unstale":"2024-07-27T03:15:25.064641+0000","last_undegraded":"2024-07-27T03:15:25.064641+0000","last_fullsized":"2024-07-27T03:15:25.064641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:54:31.880137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.306094+0000","last_change":"2024-07-27T03:12:29.541051+0000","last_active":"2024-07-27T03:13:57.306094+0000","last_peered":"2024-07-27T03:13:57.306094+0000","last_clean":"2024-07-27T03:13:57.306094+0000","last_became_active":"2024-07-27T03:12:29.540619+0000","last_became_peered":"2024-07-27T03:12:29.540619+0000","last_unstale":"2024-07-27T03:13:57.306094+0000","last_undegraded":"2024-07-27T03:13:57.306094+0000","last_fullsized":"2024-07-27T03:13:57.306094+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:23:27.587419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.304810+0000","last_change":"2024-07-27T03:12:56.720496+0000","last_active":"2024-07-27T03:13:57.304810+0000","last_peered":"2024-07-27T03:13:57.304810+0000","last_clean":"2024-07-27T03:13:57.304810+0000","last_became_active":"2024-07-27T03:12:56.719784+0000","last_became_peered":"2024-07-27T03:12:56.719784+0000","last_unstale":"2024-07-27T03:13:57.304810+0000","last_undegraded":"2024-07-27T03:13:57.304810+0000","last_fullsized":"2024-07-27T03:13:57.304810+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:07:20.518139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.069653+0000","last_change":"2024-07-27T03:12:32.568409+0000","last_active":"2024-07-27T03:15:25.069653+0000","last_peered":"2024-07-27T03:15:25.069653+0000","last_clean":"2024-07-27T03:15:25.069653+0000","last_became_active":"2024-07-27T03:12:32.568029+0000","last_became_peered":"2024-07-27T03:12:32.568029+0000","last_unstale":"2024-07-27T03:15:25.069653+0000","last_undegraded":"2024-07-27T03:15:25.069653+0000","last_fullsized":"2024-07-27T03:15:25.069653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:44:53.857986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":420,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:44.537142+0000","last_change":"2024-07-27T03:12:29.544286+0000","last_active":"2024-07-27T03:16:44.537142+0000","last_peered":"2024-07-27T03:16:44.537142+0000","last_clean":"2024-07-27T03:16:44.537142+0000","last_became_active":"2024-07-27T03:12:29.543879+0000","last_became_peered":"2024-07-27T03:12:29.543879+0000","last_unstale":"2024-07-27T03:16:44.537142+0000","last_undegraded":"2024-07-27T03:16:44.537142+0000","last_fullsized":"2024-07-27T03:16:44.537142+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:20:36.997298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":370,"num_read_kb":370,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404174+0000","last_change":"2024-07-27T03:12:42.609824+0000","last_active":"2024-07-27T03:13:57.404174+0000","last_peered":"2024-07-27T03:13:57.404174+0000","last_clean":"2024-07-27T03:13:57.404174+0000","last_became_active":"2024-07-27T03:12:42.608502+0000","last_became_peered":"2024-07-27T03:12:42.608502+0000","last_unstale":"2024-07-27T03:13:57.404174+0000","last_undegraded":"2024-07-27T03:13:57.404174+0000","last_fullsized":"2024-07-27T03:13:57.404174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:10:23.253921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:14:43.944093+0000","last_change":"2024-07-27T03:12:42.610744+0000","last_active":"2024-07-27T03:14:43.944093+0000","last_peered":"2024-07-27T03:14:43.944093+0000","last_clean":"2024-07-27T03:14:43.944093+0000","last_became_active":"2024-07-27T03:12:42.610322+0000","last_became_peered":"2024-07-27T03:12:42.610322+0000","last_unstale":"2024-07-27T03:14:43.944093+0000","last_undegraded":"2024-07-27T03:14:43.944093+0000","last_fullsized":"2024-07-27T03:14:43.944093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:01:47.618013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.059061+0000","last_change":"2024-07-27T03:12:32.558372+0000","last_active":"2024-07-27T03:15:25.059061+0000","last_peered":"2024-07-27T03:15:25.059061+0000","last_clean":"2024-07-27T03:15:25.059061+0000","last_became_active":"2024-07-27T03:12:32.558003+0000","last_became_peered":"2024-07-27T03:12:32.558003+0000","last_unstale":"2024-07-27T03:15:25.059061+0000","last_undegraded":"2024-07-27T03:15:25.059061+0000","last_fullsized":"2024-07-27T03:15:25.059061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:41:43.673807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.404495+0000","last_change":"2024-07-27T03:12:29.543182+0000","last_active":"2024-07-27T03:13:57.404495+0000","last_peered":"2024-07-27T03:13:57.404495+0000","last_clean":"2024-07-27T03:13:57.404495+0000","last_became_active":"2024-07-27T03:12:29.542744+0000","last_became_peered":"2024-07-27T03:12:29.542744+0000","last_unstale":"2024-07-27T03:13:57.404495+0000","last_undegraded":"2024-07-27T03:13:57.404495+0000","last_fullsized":"2024-07-27T03:13:57.404495+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:01:17.276549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.405941+0000","last_change":"2024-07-27T03:12:42.622297+0000","last_active":"2024-07-27T03:13:57.405941+0000","last_peered":"2024-07-27T03:13:57.405941+0000","last_clean":"2024-07-27T03:13:57.405941+0000","last_became_active":"2024-07-27T03:12:42.621947+0000","last_became_peered":"2024-07-27T03:12:42.621947+0000","last_unstale":"2024-07-27T03:13:57.405941+0000","last_undegraded":"2024-07-27T03:13:57.405941+0000","last_fullsized":"2024-07-27T03:13:57.405941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:15:59.480216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.061353+0000","last_change":"2024-07-27T03:12:32.565987+0000","last_active":"2024-07-27T03:15:25.061353+0000","last_peered":"2024-07-27T03:15:25.061353+0000","last_clean":"2024-07-27T03:15:25.061353+0000","last_became_active":"2024-07-27T03:12:32.565603+0000","last_became_peered":"2024-07-27T03:12:32.565603+0000","last_unstale":"2024-07-27T03:15:25.061353+0000","last_undegraded":"2024-07-27T03:15:25.061353+0000","last_fullsized":"2024-07-27T03:15:25.061353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:30:48.954683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.348273+0000","last_change":"2024-07-27T03:12:29.532149+0000","last_active":"2024-07-27T03:13:57.348273+0000","last_peered":"2024-07-27T03:13:57.348273+0000","last_clean":"2024-07-27T03:13:57.348273+0000","last_became_active":"2024-07-27T03:12:29.531761+0000","last_became_peered":"2024-07-27T03:12:29.531761+0000","last_unstale":"2024-07-27T03:13:57.348273+0000","last_undegraded":"2024-07-27T03:13:57.348273+0000","last_fullsized":"2024-07-27T03:13:57.348273+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:44:30.902538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.406091+0000","last_change":"2024-07-27T03:12:29.543148+0000","last_active":"2024-07-27T03:13:57.406091+0000","last_peered":"2024-07-27T03:13:57.406091+0000","last_clean":"2024-07-27T03:13:57.406091+0000","last_became_active":"2024-07-27T03:12:29.542774+0000","last_became_peered":"2024-07-27T03:12:29.542774+0000","last_unstale":"2024-07-27T03:13:57.406091+0000","last_undegraded":"2024-07-27T03:13:57.406091+0000","last_fullsized":"2024-07-27T03:13:57.406091+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:20:06.786121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.060716+0000","last_change":"2024-07-27T03:12:32.561516+0000","last_active":"2024-07-27T03:15:25.060716+0000","last_peered":"2024-07-27T03:15:25.060716+0000","last_clean":"2024-07-27T03:15:25.060716+0000","last_became_active":"2024-07-27T03:12:32.561145+0000","last_became_peered":"2024-07-27T03:12:32.561145+0000","last_unstale":"2024-07-27T03:15:25.060716+0000","last_undegraded":"2024-07-27T03:15:25.060716+0000","last_fullsized":"2024-07-27T03:15:25.060716+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:49:26.317206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.302982+0000","last_change":"2024-07-27T03:12:42.616735+0000","last_active":"2024-07-27T03:13:57.302982+0000","last_peered":"2024-07-27T03:13:57.302982+0000","last_clean":"2024-07-27T03:13:57.302982+0000","last_became_active":"2024-07-27T03:12:42.616171+0000","last_became_peered":"2024-07-27T03:12:42.616171+0000","last_unstale":"2024-07-27T03:13:57.302982+0000","last_undegraded":"2024-07-27T03:13:57.302982+0000","last_fullsized":"2024-07-27T03:13:57.302982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:45:46.236613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":238,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:16:41.533609+0000","last_change":"2024-07-27T03:12:29.539775+0000","last_active":"2024-07-27T03:16:41.533609+0000","last_peered":"2024-07-27T03:16:41.533609+0000","last_clean":"2024-07-27T03:16:41.533609+0000","last_became_active":"2024-07-27T03:12:29.539276+0000","last_became_peered":"2024-07-27T03:12:29.539276+0000","last_unstale":"2024-07-27T03:16:41.533609+0000","last_undegraded":"2024-07-27T03:16:41.533609+0000","last_fullsized":"2024-07-27T03:16:41.533609+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:28.513920+0000","last_clean_scrub_stamp":"2024-07-27T03:12:28.513920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:09:19.364679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":187,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.045676+0000","last_change":"2024-07-27T03:12:32.562673+0000","last_active":"2024-07-27T03:15:25.045676+0000","last_peered":"2024-07-27T03:15:25.045676+0000","last_clean":"2024-07-27T03:15:25.045676+0000","last_became_active":"2024-07-27T03:12:32.561591+0000","last_became_peered":"2024-07-27T03:12:32.561591+0000","last_unstale":"2024-07-27T03:15:25.045676+0000","last_undegraded":"2024-07-27T03:15:25.045676+0000","last_fullsized":"2024-07-27T03:15:25.045676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:49:38.095004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:13:57.303410+0000","last_change":"2024-07-27T03:12:42.611756+0000","last_active":"2024-07-27T03:13:57.303410+0000","last_peered":"2024-07-27T03:13:57.303410+0000","last_clean":"2024-07-27T03:13:57.303410+0000","last_became_active":"2024-07-27T03:12:42.611246+0000","last_became_peered":"2024-07-27T03:12:42.611246+0000","last_unstale":"2024-07-27T03:13:57.303410+0000","last_undegraded":"2024-07-27T03:13:57.303410+0000","last_fullsized":"2024-07-27T03:13:57.303410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:41.589968+0000","last_clean_scrub_stamp":"2024-07-27T03:12:41.589968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:13:06.027887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.061677+0000","last_change":"2024-07-27T03:12:32.571339+0000","last_active":"2024-07-27T03:15:25.061677+0000","last_peered":"2024-07-27T03:15:25.061677+0000","last_clean":"2024-07-27T03:15:25.061677+0000","last_became_active":"2024-07-27T03:12:32.570973+0000","last_became_peered":"2024-07-27T03:12:32.570973+0000","last_unstale":"2024-07-27T03:15:25.061677+0000","last_undegraded":"2024-07-27T03:15:25.061677+0000","last_fullsized":"2024-07-27T03:15:25.061677+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:24:25.357833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.060782+0000","last_change":"2024-07-27T03:12:32.561243+0000","last_active":"2024-07-27T03:15:25.060782+0000","last_peered":"2024-07-27T03:15:25.060782+0000","last_clean":"2024-07-27T03:15:25.060782+0000","last_became_active":"2024-07-27T03:12:32.560748+0000","last_became_peered":"2024-07-27T03:12:32.560748+0000","last_unstale":"2024-07-27T03:15:25.060782+0000","last_undegraded":"2024-07-27T03:15:25.060782+0000","last_fullsized":"2024-07-27T03:15:25.060782+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T09:47:40.430283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.057184+0000","last_change":"2024-07-27T03:12:32.562545+0000","last_active":"2024-07-27T03:15:25.057184+0000","last_peered":"2024-07-27T03:15:25.057184+0000","last_clean":"2024-07-27T03:15:25.057184+0000","last_became_active":"2024-07-27T03:12:32.562202+0000","last_became_peered":"2024-07-27T03:12:32.562202+0000","last_unstale":"2024-07-27T03:15:25.057184+0000","last_undegraded":"2024-07-27T03:15:25.057184+0000","last_fullsized":"2024-07-27T03:15:25.057184+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:05:20.229232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.065174+0000","last_change":"2024-07-27T03:12:32.565162+0000","last_active":"2024-07-27T03:15:25.065174+0000","last_peered":"2024-07-27T03:15:25.065174+0000","last_clean":"2024-07-27T03:15:25.065174+0000","last_became_active":"2024-07-27T03:12:32.564798+0000","last_became_peered":"2024-07-27T03:12:32.564798+0000","last_unstale":"2024-07-27T03:15:25.065174+0000","last_undegraded":"2024-07-27T03:15:25.065174+0000","last_fullsized":"2024-07-27T03:15:25.065174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:54:29.742624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.060883+0000","last_change":"2024-07-27T03:12:32.556874+0000","last_active":"2024-07-27T03:15:25.060883+0000","last_peered":"2024-07-27T03:15:25.060883+0000","last_clean":"2024-07-27T03:15:25.060883+0000","last_became_active":"2024-07-27T03:12:32.556467+0000","last_became_peered":"2024-07-27T03:12:32.556467+0000","last_unstale":"2024-07-27T03:15:25.060883+0000","last_undegraded":"2024-07-27T03:15:25.060883+0000","last_fullsized":"2024-07-27T03:15:25.060883+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:51:20.806785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.053073+0000","last_change":"2024-07-27T03:12:32.569136+0000","last_active":"2024-07-27T03:15:25.053073+0000","last_peered":"2024-07-27T03:15:25.053073+0000","last_clean":"2024-07-27T03:15:25.053073+0000","last_became_active":"2024-07-27T03:12:32.568778+0000","last_became_peered":"2024-07-27T03:12:32.568778+0000","last_unstale":"2024-07-27T03:15:25.053073+0000","last_undegraded":"2024-07-27T03:15:25.053073+0000","last_fullsized":"2024-07-27T03:15:25.053073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:26:22.654974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.069928+0000","last_change":"2024-07-27T03:12:32.569507+0000","last_active":"2024-07-27T03:15:25.069928+0000","last_peered":"2024-07-27T03:15:25.069928+0000","last_clean":"2024-07-27T03:15:25.069928+0000","last_became_active":"2024-07-27T03:12:32.569109+0000","last_became_peered":"2024-07-27T03:12:32.569109+0000","last_unstale":"2024-07-27T03:15:25.069928+0000","last_undegraded":"2024-07-27T03:15:25.069928+0000","last_fullsized":"2024-07-27T03:15:25.069928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:35:12.944933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.053821+0000","last_change":"2024-07-27T03:12:32.566479+0000","last_active":"2024-07-27T03:15:25.053821+0000","last_peered":"2024-07-27T03:15:25.053821+0000","last_clean":"2024-07-27T03:15:25.053821+0000","last_became_active":"2024-07-27T03:12:32.566114+0000","last_became_peered":"2024-07-27T03:12:32.566114+0000","last_unstale":"2024-07-27T03:15:25.053821+0000","last_undegraded":"2024-07-27T03:15:25.053821+0000","last_fullsized":"2024-07-27T03:15:25.053821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:24:19.944031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.051389+0000","last_change":"2024-07-27T03:12:56.726118+0000","last_active":"2024-07-27T03:15:25.051389+0000","last_peered":"2024-07-27T03:15:25.051389+0000","last_clean":"2024-07-27T03:15:25.051389+0000","last_became_active":"2024-07-27T03:12:56.725637+0000","last_became_peered":"2024-07-27T03:12:56.725637+0000","last_unstale":"2024-07-27T03:15:25.051389+0000","last_undegraded":"2024-07-27T03:15:25.051389+0000","last_fullsized":"2024-07-27T03:15:25.051389+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:36:41.725150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.053321+0000","last_change":"2024-07-27T03:12:32.568232+0000","last_active":"2024-07-27T03:15:25.053321+0000","last_peered":"2024-07-27T03:15:25.053321+0000","last_clean":"2024-07-27T03:15:25.053321+0000","last_became_active":"2024-07-27T03:12:32.567834+0000","last_became_peered":"2024-07-27T03:12:32.567834+0000","last_unstale":"2024-07-27T03:15:25.053321+0000","last_undegraded":"2024-07-27T03:15:25.053321+0000","last_fullsized":"2024-07-27T03:15:25.053321+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:40:28.234539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.070056+0000","last_change":"2024-07-27T03:12:32.573451+0000","last_active":"2024-07-27T03:15:25.070056+0000","last_peered":"2024-07-27T03:15:25.070056+0000","last_clean":"2024-07-27T03:15:25.070056+0000","last_became_active":"2024-07-27T03:12:32.573066+0000","last_became_peered":"2024-07-27T03:12:32.573066+0000","last_unstale":"2024-07-27T03:15:25.070056+0000","last_undegraded":"2024-07-27T03:15:25.070056+0000","last_fullsized":"2024-07-27T03:15:25.070056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:13:29.104055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.063555+0000","last_change":"2024-07-27T03:12:32.561067+0000","last_active":"2024-07-27T03:15:25.063555+0000","last_peered":"2024-07-27T03:15:25.063555+0000","last_clean":"2024-07-27T03:15:25.063555+0000","last_became_active":"2024-07-27T03:12:32.560358+0000","last_became_peered":"2024-07-27T03:12:32.560358+0000","last_unstale":"2024-07-27T03:15:25.063555+0000","last_undegraded":"2024-07-27T03:15:25.063555+0000","last_fullsized":"2024-07-27T03:15:25.063555+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:41:41.026804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.053463+0000","last_change":"2024-07-27T03:12:32.572290+0000","last_active":"2024-07-27T03:15:25.053463+0000","last_peered":"2024-07-27T03:15:25.053463+0000","last_clean":"2024-07-27T03:15:25.053463+0000","last_became_active":"2024-07-27T03:12:32.571877+0000","last_became_peered":"2024-07-27T03:12:32.571877+0000","last_unstale":"2024-07-27T03:15:25.053463+0000","last_undegraded":"2024-07-27T03:15:25.053463+0000","last_fullsized":"2024-07-27T03:15:25.053463+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:34:40.559787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.056923+0000","last_change":"2024-07-27T03:12:32.565495+0000","last_active":"2024-07-27T03:15:25.056923+0000","last_peered":"2024-07-27T03:15:25.056923+0000","last_clean":"2024-07-27T03:15:25.056923+0000","last_became_active":"2024-07-27T03:12:32.563136+0000","last_became_peered":"2024-07-27T03:12:32.563136+0000","last_unstale":"2024-07-27T03:15:25.056923+0000","last_undegraded":"2024-07-27T03:15:25.056923+0000","last_fullsized":"2024-07-27T03:15:25.056923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:31:02.032776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.070390+0000","last_change":"2024-07-27T03:12:32.569477+0000","last_active":"2024-07-27T03:15:25.070390+0000","last_peered":"2024-07-27T03:15:25.070390+0000","last_clean":"2024-07-27T03:15:25.070390+0000","last_became_active":"2024-07-27T03:12:32.569098+0000","last_became_peered":"2024-07-27T03:12:32.569098+0000","last_unstale":"2024-07-27T03:15:25.070390+0000","last_undegraded":"2024-07-27T03:15:25.070390+0000","last_fullsized":"2024-07-27T03:15:25.070390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:44:26.082539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.057708+0000","last_change":"2024-07-27T03:12:32.574704+0000","last_active":"2024-07-27T03:15:25.057708+0000","last_peered":"2024-07-27T03:15:25.057708+0000","last_clean":"2024-07-27T03:15:25.057708+0000","last_became_active":"2024-07-27T03:12:32.574261+0000","last_became_peered":"2024-07-27T03:12:32.574261+0000","last_unstale":"2024-07-27T03:15:25.057708+0000","last_undegraded":"2024-07-27T03:15:25.057708+0000","last_fullsized":"2024-07-27T03:15:25.057708+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:53:06.445267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.068628+0000","last_change":"2024-07-27T03:12:32.573835+0000","last_active":"2024-07-27T03:15:25.068628+0000","last_peered":"2024-07-27T03:15:25.068628+0000","last_clean":"2024-07-27T03:15:25.068628+0000","last_became_active":"2024-07-27T03:12:32.573450+0000","last_became_peered":"2024-07-27T03:12:32.573450+0000","last_unstale":"2024-07-27T03:15:25.068628+0000","last_undegraded":"2024-07-27T03:15:25.068628+0000","last_fullsized":"2024-07-27T03:15:25.068628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:40:45.264000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.060743+0000","last_change":"2024-07-27T03:12:32.568542+0000","last_active":"2024-07-27T03:15:25.060743+0000","last_peered":"2024-07-27T03:15:25.060743+0000","last_clean":"2024-07-27T03:15:25.060743+0000","last_became_active":"2024-07-27T03:12:32.568179+0000","last_became_peered":"2024-07-27T03:12:32.568179+0000","last_unstale":"2024-07-27T03:15:25.060743+0000","last_undegraded":"2024-07-27T03:15:25.060743+0000","last_fullsized":"2024-07-27T03:15:25.060743+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T12:32:10.005601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.061855+0000","last_change":"2024-07-27T03:12:32.564063+0000","last_active":"2024-07-27T03:15:25.061855+0000","last_peered":"2024-07-27T03:15:25.061855+0000","last_clean":"2024-07-27T03:15:25.061855+0000","last_became_active":"2024-07-27T03:12:32.563705+0000","last_became_peered":"2024-07-27T03:12:32.563705+0000","last_unstale":"2024-07-27T03:15:25.061855+0000","last_undegraded":"2024-07-27T03:15:25.061855+0000","last_fullsized":"2024-07-27T03:15:25.061855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:40:19.234194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.061066+0000","last_change":"2024-07-27T03:12:57.178963+0000","last_active":"2024-07-27T03:15:25.061066+0000","last_peered":"2024-07-27T03:15:25.061066+0000","last_clean":"2024-07-27T03:15:25.061066+0000","last_became_active":"2024-07-27T03:12:57.178550+0000","last_became_peered":"2024-07-27T03:12:57.178550+0000","last_unstale":"2024-07-27T03:15:25.061066+0000","last_undegraded":"2024-07-27T03:15:25.061066+0000","last_fullsized":"2024-07-27T03:15:25.061066+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T10:20:06.586957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.060099+0000","last_change":"2024-07-27T03:12:32.569863+0000","last_active":"2024-07-27T03:15:25.060099+0000","last_peered":"2024-07-27T03:15:25.060099+0000","last_clean":"2024-07-27T03:15:25.060099+0000","last_became_active":"2024-07-27T03:12:32.566457+0000","last_became_peered":"2024-07-27T03:12:32.566457+0000","last_unstale":"2024-07-27T03:15:25.060099+0000","last_undegraded":"2024-07-27T03:15:25.060099+0000","last_fullsized":"2024-07-27T03:15:25.060099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T07:28:13.141002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.058759+0000","last_change":"2024-07-27T03:12:32.574490+0000","last_active":"2024-07-27T03:15:25.058759+0000","last_peered":"2024-07-27T03:15:25.058759+0000","last_clean":"2024-07-27T03:15:25.058759+0000","last_became_active":"2024-07-27T03:12:32.574130+0000","last_became_peered":"2024-07-27T03:12:32.574130+0000","last_unstale":"2024-07-27T03:15:25.058759+0000","last_undegraded":"2024-07-27T03:15:25.058759+0000","last_fullsized":"2024-07-27T03:15:25.058759+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:44:22.696669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.066579+0000","last_change":"2024-07-27T03:12:32.560106+0000","last_active":"2024-07-27T03:15:25.066579+0000","last_peered":"2024-07-27T03:15:25.066579+0000","last_clean":"2024-07-27T03:15:25.066579+0000","last_became_active":"2024-07-27T03:12:32.559751+0000","last_became_peered":"2024-07-27T03:12:32.559751+0000","last_unstale":"2024-07-27T03:15:25.066579+0000","last_undegraded":"2024-07-27T03:15:25.066579+0000","last_fullsized":"2024-07-27T03:15:25.066579+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T06:16:37.647462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.062851+0000","last_change":"2024-07-27T03:12:32.569352+0000","last_active":"2024-07-27T03:15:25.062851+0000","last_peered":"2024-07-27T03:15:25.062851+0000","last_clean":"2024-07-27T03:15:25.062851+0000","last_became_active":"2024-07-27T03:12:32.569004+0000","last_became_peered":"2024-07-27T03:12:32.569004+0000","last_unstale":"2024-07-27T03:15:25.062851+0000","last_undegraded":"2024-07-27T03:15:25.062851+0000","last_fullsized":"2024-07-27T03:15:25.062851+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:21:25.999214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.053831+0000","last_change":"2024-07-27T03:12:32.562775+0000","last_active":"2024-07-27T03:15:25.053831+0000","last_peered":"2024-07-27T03:15:25.053831+0000","last_clean":"2024-07-27T03:15:25.053831+0000","last_became_active":"2024-07-27T03:12:32.562403+0000","last_became_peered":"2024-07-27T03:12:32.562403+0000","last_unstale":"2024-07-27T03:15:25.053831+0000","last_undegraded":"2024-07-27T03:15:25.053831+0000","last_fullsized":"2024-07-27T03:15:25.053831+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T03:16:09.474505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.061943+0000","last_change":"2024-07-27T03:12:32.563271+0000","last_active":"2024-07-27T03:15:25.061943+0000","last_peered":"2024-07-27T03:15:25.061943+0000","last_clean":"2024-07-27T03:15:25.061943+0000","last_became_active":"2024-07-27T03:12:32.562850+0000","last_became_peered":"2024-07-27T03:12:32.562850+0000","last_unstale":"2024-07-27T03:15:25.061943+0000","last_undegraded":"2024-07-27T03:15:25.061943+0000","last_fullsized":"2024-07-27T03:15:25.061943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T13:40:27.688205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.065581+0000","last_change":"2024-07-27T03:12:32.557580+0000","last_active":"2024-07-27T03:15:25.065581+0000","last_peered":"2024-07-27T03:15:25.065581+0000","last_clean":"2024-07-27T03:15:25.065581+0000","last_became_active":"2024-07-27T03:12:32.557190+0000","last_became_peered":"2024-07-27T03:12:32.557190+0000","last_unstale":"2024-07-27T03:15:25.065581+0000","last_undegraded":"2024-07-27T03:15:25.065581+0000","last_fullsized":"2024-07-27T03:15:25.065581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:58:17.609035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.033909+0000","last_change":"2024-07-27T03:12:32.563669+0000","last_active":"2024-07-27T03:15:25.033909+0000","last_peered":"2024-07-27T03:15:25.033909+0000","last_clean":"2024-07-27T03:15:25.033909+0000","last_became_active":"2024-07-27T03:12:32.563200+0000","last_became_peered":"2024-07-27T03:12:32.563200+0000","last_unstale":"2024-07-27T03:15:25.033909+0000","last_undegraded":"2024-07-27T03:15:25.033909+0000","last_fullsized":"2024-07-27T03:15:25.033909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T05:14:05.624118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.055913+0000","last_change":"2024-07-27T03:12:32.561793+0000","last_active":"2024-07-27T03:15:25.055913+0000","last_peered":"2024-07-27T03:15:25.055913+0000","last_clean":"2024-07-27T03:15:25.055913+0000","last_became_active":"2024-07-27T03:12:32.561452+0000","last_became_peered":"2024-07-27T03:12:32.561452+0000","last_unstale":"2024-07-27T03:15:25.055913+0000","last_undegraded":"2024-07-27T03:15:25.055913+0000","last_fullsized":"2024-07-27T03:15:25.055913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T14:22:07.204824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.070321+0000","last_change":"2024-07-27T03:12:32.569811+0000","last_active":"2024-07-27T03:15:25.070321+0000","last_peered":"2024-07-27T03:15:25.070321+0000","last_clean":"2024-07-27T03:15:25.070321+0000","last_became_active":"2024-07-27T03:12:32.569474+0000","last_became_peered":"2024-07-27T03:12:32.569474+0000","last_unstale":"2024-07-27T03:15:25.070321+0000","last_undegraded":"2024-07-27T03:15:25.070321+0000","last_fullsized":"2024-07-27T03:15:25.070321+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T11:38:39.317362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.063814+0000","last_change":"2024-07-27T03:12:32.565159+0000","last_active":"2024-07-27T03:15:25.063814+0000","last_peered":"2024-07-27T03:15:25.063814+0000","last_clean":"2024-07-27T03:15:25.063814+0000","last_became_active":"2024-07-27T03:12:32.564750+0000","last_became_peered":"2024-07-27T03:12:32.564750+0000","last_unstale":"2024-07-27T03:15:25.063814+0000","last_undegraded":"2024-07-27T03:15:25.063814+0000","last_fullsized":"2024-07-27T03:15:25.063814+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T04:51:26.186510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"91'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-07-27T03:15:25.057903+0000","last_change":"2024-07-27T03:12:32.574429+0000","last_active":"2024-07-27T03:15:25.057903+0000","last_peered":"2024-07-27T03:15:25.057903+0000","last_clean":"2024-07-27T03:15:25.057903+0000","last_became_active":"2024-07-27T03:12:32.573986+0000","last_became_peered":"2024-07-27T03:12:32.573986+0000","last_unstale":"2024-07-27T03:15:25.057903+0000","last_undegraded":"2024-07-27T03:15:25.057903+0000","last_fullsized":"2024-07-27T03:15:25.057903+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-07-27T03:12:31.532658+0000","last_clean_scrub_stamp":"2024-07-27T03:12:31.532658+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-07-28T08:35:00.345561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75488,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1881,"num_read_kb":1027,"num_write":263,"num_write_kb":158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":226464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645378,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":525562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841607,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37052,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708100,"statfs":{"total":95995035648,"available":95957094400,"internally_reserved":0,"allocated":1757184,"data_stored":1184986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037835,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1077248,"data_stored":525470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234063,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":525826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430292,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1671168,"data_stored":1123018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593818,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":520974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790046,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1679360,"data_stored":1118068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953571,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":525580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149800,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":520682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411437,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1146880,"data_stored":590758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1126400,"data_stored":590006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738486,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1077248,"data_stored":522430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-07-27T03:16:46.148 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:16:46.449 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:16:46.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:46 smithi067 ceph-mon[32814]: pgmap v436: 129 pgs: 129 active+clean; 650 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 11 op/s 2024-07-27T03:16:46.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:46 smithi067 ceph-mon[32814]: from='client.15486 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:16:46.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:46 smithi067 ceph-mon[32814]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:16:46.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:46 smithi149 ceph-mon[37173]: pgmap v436: 129 pgs: 129 active+clean; 650 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 11 op/s 2024-07-27T03:16:46.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:46 smithi149 ceph-mon[37173]: from='client.15486 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:16:46.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:46 smithi149 ceph-mon[37173]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:16:46.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:46 smithi175 ceph-mon[37125]: pgmap v436: 129 pgs: 129 active+clean; 650 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 767 B/s wr, 11 op/s 2024-07-27T03:16:46.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:46 smithi175 ceph-mon[37125]: from='client.15486 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:16:46.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:46 smithi175 ceph-mon[37125]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:16:47.657 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:47 smithi067 ceph-mon[32814]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:16:47.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:47 smithi149 ceph-mon[37173]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:16:47.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:47 smithi175 ceph-mon[37125]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-07-27T03:16:48.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.210+0000 7f67ab9f2640 1 -- 172.21.15.67:0/2151534202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67a4109860 msgr2=0x7f67a4110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:48.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.210+0000 7f67ab9f2640 1 --2- 172.21.15.67:0/2151534202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67a4109860 0x7f67a4110090 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f679c009ca0 tx=0x7f679c02a8e0 comp rx=0 tx=0).stop 2024-07-27T03:16:48.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.210+0000 7f67ab9f2640 1 -- 172.21.15.67:0/2151534202 shutdown_connections 2024-07-27T03:16:48.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.210+0000 7f67ab9f2640 1 --2- 172.21.15.67:0/2151534202 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67a4109860 0x7f67a4110090 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:48.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.210+0000 7f67ab9f2640 1 --2- 172.21.15.67:0/2151534202 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a4108f10 0x7f67a4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:48.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.210+0000 7f67ab9f2640 1 --2- 172.21.15.67:0/2151534202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67a41035d0 0x7f67a41039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:48.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.210+0000 7f67ab9f2640 1 -- 172.21.15.67:0/2151534202 >> 172.21.15.67:0/2151534202 conn(0x7f67a407b2c0 msgr2=0x7f67a407b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:48.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.210+0000 7f67ab9f2640 1 -- 172.21.15.67:0/2151534202 shutdown_connections 2024-07-27T03:16:48.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.210+0000 7f67ab9f2640 1 -- 172.21.15.67:0/2151534202 wait complete. 2024-07-27T03:16:48.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.210+0000 7f67ab9f2640 1 Processor -- start 2024-07-27T03:16:48.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.211+0000 7f67ab9f2640 1 -- start start 2024-07-27T03:16:48.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.211+0000 7f67ab9f2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a41035d0 0x7f67a4075180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:48.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.211+0000 7f67ab9f2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67a4108f10 0x7f67a40756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:48.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.211+0000 7f67ab9f2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67a4109860 0x7f67a4075c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:48.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.211+0000 7f67ab9f2640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f67a4113560 con 0x7f67a4108f10 2024-07-27T03:16:48.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.211+0000 7f67ab9f2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f67a41133e0 con 0x7f67a4109860 2024-07-27T03:16:48.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.211+0000 7f67ab9f2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f67a41136e0 con 0x7f67a41035d0 2024-07-27T03:16:48.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.211+0000 7f67a8f8d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67a4108f10 0x7f67a40756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:48.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.211+0000 7f67a8f8d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67a4108f10 0x7f67a40756c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51452/0 (socket says 172.21.15.67:51452) 2024-07-27T03:16:48.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.211+0000 7f67a8f8d640 1 -- 172.21.15.67:0/1345627101 learned_addr learned my addr 172.21.15.67:0/1345627101 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:48.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.211+0000 7f67a978e640 1 --2- 172.21.15.67:0/1345627101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a41035d0 0x7f67a4075180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:48.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.211+0000 7f67a9f8f640 1 --2- 172.21.15.67:0/1345627101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67a4109860 0x7f67a4075c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:48.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.211+0000 7f67a8f8d640 1 -- 172.21.15.67:0/1345627101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a41035d0 msgr2=0x7f67a4075180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:48.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.211+0000 7f67a8f8d640 1 --2- 172.21.15.67:0/1345627101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a41035d0 0x7f67a4075180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:48.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.211+0000 7f67a8f8d640 1 -- 172.21.15.67:0/1345627101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67a4109860 msgr2=0x7f67a4075c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:48.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.211+0000 7f67a8f8d640 1 --2- 172.21.15.67:0/1345627101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67a4109860 0x7f67a4075c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:48.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.211+0000 7f67a8f8d640 1 -- 172.21.15.67:0/1345627101 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a41a81d0 con 0x7f67a4108f10 2024-07-27T03:16:48.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.212+0000 7f67a8f8d640 1 --2- 172.21.15.67:0/1345627101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67a4108f10 0x7f67a40756c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6794006fd0 tx=0x7f679400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:48.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.212+0000 7f679a7fc640 1 -- 172.21.15.67:0/1345627101 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6794011830 con 0x7f67a4108f10 2024-07-27T03:16:48.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.212+0000 7f679a7fc640 1 -- 172.21.15.67:0/1345627101 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6794016070 con 0x7f67a4108f10 2024-07-27T03:16:48.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.212+0000 7f67ab9f2640 1 -- 172.21.15.67:0/1345627101 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a41a84c0 con 0x7f67a4108f10 2024-07-27T03:16:48.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.212+0000 7f679a7fc640 1 -- 172.21.15.67:0/1345627101 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6794010e00 con 0x7f67a4108f10 2024-07-27T03:16:48.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.212+0000 7f67ab9f2640 1 -- 172.21.15.67:0/1345627101 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a41a8a80 con 0x7f67a4108f10 2024-07-27T03:16:48.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.213+0000 7f67ab9f2640 1 -- 172.21.15.67:0/1345627101 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67a41039e0 con 0x7f67a4108f10 2024-07-27T03:16:48.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.214+0000 7f679a7fc640 1 -- 172.21.15.67:0/1345627101 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f679401b460 con 0x7f67a4108f10 2024-07-27T03:16:48.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.215+0000 7f679a7fc640 1 --2- 172.21.15.67:0/1345627101 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f67900b3bd0 0x7f67900b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:48.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.215+0000 7f67a978e640 1 --2- 172.21.15.67:0/1345627101 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f67900b3bd0 0x7f67900b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:48.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.215+0000 7f679a7fc640 1 -- 172.21.15.67:0/1345627101 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f67940e3d50 con 0x7f67a4108f10 2024-07-27T03:16:48.218 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.216+0000 7f67a978e640 1 --2- 172.21.15.67:0/1345627101 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f67900b3bd0 0x7f67900b60e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f67a0005fd0 tx=0x7f67a0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:48.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.218+0000 7f679a7fc640 1 -- 172.21.15.67:0/1345627101 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f67940aaff0 con 0x7f67a4108f10 2024-07-27T03:16:48.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.437+0000 7f67ab9f2640 1 -- 172.21.15.67:0/1345627101 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67a4076480 con 0x7f67a4108f10 2024-07-27T03:16:48.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.438+0000 7f679a7fc640 1 -- 172.21.15.67:0/1345627101 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11983 (secure 0 0 0) 0x7f67a4076480 con 0x7f67a4108f10 2024-07-27T03:16:48.441 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:16:48.441 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"btime":"2024-07-27T03:16:40:350577+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14679,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/252686369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":252686369},{"type":"v1","addr":"172.21.15.67:6835","nonce":252686369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/3358832762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3358832762},{"type":"v1","addr":"172.21.15.67:6837","nonce":3358832762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-27T03:12:34.320208+0000","modified":"2024-07-27T03:16:39.741149+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24620,"mds_1":24587,"mds_2":24583,"mds_3":24581,"mds_4":24563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14736":{"gid":14736,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/2162708317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2162708317},{"type":"v1","addr":"172.21.15.67:6839","nonce":2162708317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14748":{"gid":14748,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/795704244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":795704244},{"type":"v1","addr":"172.21.15.67:6841","nonce":795704244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/3868325215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3868325215},{"type":"v1","addr":"172.21.15.149:6833","nonce":3868325215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2462749027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2462749027},{"type":"v1","addr":"172.21.15.175:6835","nonce":2462749027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.149:6835/3730964806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3730964806},{"type":"v1","addr":"172.21.15.149:6835","nonce":3730964806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.175:6837/1304803502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1304803502},{"type":"v1","addr":"172.21.15.175:6837","nonce":1304803502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.175:6839/1670568853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1670568853},{"type":"v1","addr":"172.21.15.175:6839","nonce":1670568853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24587":{"gid":24587,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.149:6837/2932951845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2932951845},{"type":"v1","addr":"172.21.15.149:6837","nonce":2932951845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.149:6839/281746498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":281746498},{"type":"v1","addr":"172.21.15.149:6839","nonce":281746498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24620,"qdb_cluster":[24587,24583,24620,24581,24563]},"id":1}]} 2024-07-27T03:16:48.441 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 30 2024-07-27T03:16:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.440+0000 7f67ab9f2640 1 -- 172.21.15.67:0/1345627101 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f67900b3bd0 msgr2=0x7f67900b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.440+0000 7f67ab9f2640 1 --2- 172.21.15.67:0/1345627101 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f67900b3bd0 0x7f67900b60e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f67a0005fd0 tx=0x7f67a0005850 comp rx=0 tx=0).stop 2024-07-27T03:16:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.440+0000 7f67ab9f2640 1 -- 172.21.15.67:0/1345627101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67a4108f10 msgr2=0x7f67a40756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.440+0000 7f67ab9f2640 1 --2- 172.21.15.67:0/1345627101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67a4108f10 0x7f67a40756c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6794006fd0 tx=0x7f679400b8e0 comp rx=0 tx=0).stop 2024-07-27T03:16:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.440+0000 7f67ab9f2640 1 -- 172.21.15.67:0/1345627101 shutdown_connections 2024-07-27T03:16:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.440+0000 7f67ab9f2640 1 --2- 172.21.15.67:0/1345627101 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f67900b3bd0 0x7f67900b60e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.440+0000 7f67ab9f2640 1 --2- 172.21.15.67:0/1345627101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67a4109860 0x7f67a4075c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.440+0000 7f67ab9f2640 1 --2- 172.21.15.67:0/1345627101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f67a4108f10 0x7f67a40756c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.440+0000 7f67ab9f2640 1 --2- 172.21.15.67:0/1345627101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67a41035d0 0x7f67a4075180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:48.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.440+0000 7f67ab9f2640 1 -- 172.21.15.67:0/1345627101 >> 172.21.15.67:0/1345627101 conn(0x7f67a407b2c0 msgr2=0x7f67a4106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:48.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.440+0000 7f67ab9f2640 1 -- 172.21.15.67:0/1345627101 shutdown_connections 2024-07-27T03:16:48.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:48.440+0000 7f67ab9f2640 1 -- 172.21.15.67:0/1345627101 wait complete. 2024-07-27T03:16:48.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:48 smithi067 ceph-mon[32814]: pgmap v437: 129 pgs: 129 active+clean; 650 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-07-27T03:16:48.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:48 smithi149 ceph-mon[37173]: pgmap v437: 129 pgs: 129 active+clean; 650 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-07-27T03:16:48.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:48 smithi175 ceph-mon[37125]: pgmap v437: 129 pgs: 129 active+clean; 650 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-07-27T03:16:49.347 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd dump --format=json 2024-07-27T03:16:49.617 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:16:49.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:49 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1345627101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:16:49.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:49 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1345627101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:16:49.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:49 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1345627101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:16:50.708 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:50 smithi067 ceph-mon[32814]: pgmap v438: 129 pgs: 129 active+clean; 650 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-07-27T03:16:50.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:50 smithi149 ceph-mon[37173]: pgmap v438: 129 pgs: 129 active+clean; 650 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-07-27T03:16:50.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:50 smithi175 ceph-mon[37125]: pgmap v438: 129 pgs: 129 active+clean; 650 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-07-27T03:16:51.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.204+0000 7f5106ac2640 1 -- 172.21.15.67:0/1376984433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5100108760 msgr2=0x7f510010ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:51.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.204+0000 7f5106ac2640 1 --2- 172.21.15.67:0/1376984433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5100108760 0x7f510010ef90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009ca0 tx=0x7f50f802a8c0 comp rx=0 tx=0).stop 2024-07-27T03:16:51.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.204+0000 7f5106ac2640 1 -- 172.21.15.67:0/1376984433 shutdown_connections 2024-07-27T03:16:51.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.204+0000 7f5106ac2640 1 --2- 172.21.15.67:0/1376984433 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5100108760 0x7f510010ef90 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:51.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.204+0000 7f5106ac2640 1 --2- 172.21.15.67:0/1376984433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5100107e10 0x7f5100108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:51.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.204+0000 7f5106ac2640 1 --2- 172.21.15.67:0/1376984433 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51001024d0 0x7f51001028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:51.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.204+0000 7f5106ac2640 1 -- 172.21.15.67:0/1376984433 >> 172.21.15.67:0/1376984433 conn(0x7f51000791a0 msgr2=0x7f51000795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:51.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.204+0000 7f5106ac2640 1 -- 172.21.15.67:0/1376984433 shutdown_connections 2024-07-27T03:16:51.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.204+0000 7f5106ac2640 1 -- 172.21.15.67:0/1376984433 wait complete. 2024-07-27T03:16:51.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.205+0000 7f5106ac2640 1 Processor -- start 2024-07-27T03:16:51.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.205+0000 7f5106ac2640 1 -- start start 2024-07-27T03:16:51.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.206+0000 7f5106ac2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51001024d0 0x7f5100198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:51.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.206+0000 7f5106ac2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5100107e10 0x7f51001993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:51.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.206+0000 7f5106ac2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5100108760 0x7f510019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:51.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.206+0000 7f5106ac2640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5100112410 con 0x7f5100107e10 2024-07-27T03:16:51.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.206+0000 7f5106ac2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5100112290 con 0x7f51001024d0 2024-07-27T03:16:51.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.206+0000 7f5106ac2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5100112590 con 0x7f5100108760 2024-07-27T03:16:51.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.206+0000 7f50f7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5100107e10 0x7f51001993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:51.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.206+0000 7f50f7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5100107e10 0x7f51001993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51470/0 (socket says 172.21.15.67:51470) 2024-07-27T03:16:51.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.206+0000 7f50f7fff640 1 -- 172.21.15.67:0/4005643040 learned_addr learned my addr 172.21.15.67:0/4005643040 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:51.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.206+0000 7f510485e640 1 --2- 172.21.15.67:0/4005643040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51001024d0 0x7f5100198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:51.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.206+0000 7f510505f640 1 --2- 172.21.15.67:0/4005643040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5100108760 0x7f510019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:51.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.206+0000 7f50f7fff640 1 -- 172.21.15.67:0/4005643040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5100108760 msgr2=0x7f510019e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:51.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.206+0000 7f50f7fff640 1 --2- 172.21.15.67:0/4005643040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5100108760 0x7f510019e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:51.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.206+0000 7f50f7fff640 1 -- 172.21.15.67:0/4005643040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51001024d0 msgr2=0x7f5100198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:51.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.206+0000 7f50f7fff640 1 --2- 172.21.15.67:0/4005643040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51001024d0 0x7f5100198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:51.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.206+0000 7f50f7fff640 1 -- 172.21.15.67:0/4005643040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510019ed90 con 0x7f5100107e10 2024-07-27T03:16:51.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.207+0000 7f50f7fff640 1 --2- 172.21.15.67:0/4005643040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5100107e10 0x7f51001993c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f50f000c970 tx=0x7f50f000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:51.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.207+0000 7f50f5ffb640 1 -- 172.21.15.67:0/4005643040 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0007bb0 con 0x7f5100107e10 2024-07-27T03:16:51.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.207+0000 7f50f5ffb640 1 -- 172.21.15.67:0/4005643040 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50f0007d50 con 0x7f5100107e10 2024-07-27T03:16:51.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.207+0000 7f5106ac2640 1 -- 172.21.15.67:0/4005643040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510019f080 con 0x7f5100107e10 2024-07-27T03:16:51.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.207+0000 7f50f5ffb640 1 -- 172.21.15.67:0/4005643040 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f000f450 con 0x7f5100107e10 2024-07-27T03:16:51.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.207+0000 7f5106ac2640 1 -- 172.21.15.67:0/4005643040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51001a57e0 con 0x7f5100107e10 2024-07-27T03:16:51.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.208+0000 7f5106ac2640 1 -- 172.21.15.67:0/4005643040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51001028e0 con 0x7f5100107e10 2024-07-27T03:16:51.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.210+0000 7f50f5ffb640 1 -- 172.21.15.67:0/4005643040 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f50f0004f20 con 0x7f5100107e10 2024-07-27T03:16:51.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.210+0000 7f50f5ffb640 1 --2- 172.21.15.67:0/4005643040 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f50e80b3bd0 0x7f50e80b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:51.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.211+0000 7f510485e640 1 --2- 172.21.15.67:0/4005643040 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f50e80b3bd0 0x7f50e80b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:51.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.211+0000 7f50f5ffb640 1 -- 172.21.15.67:0/4005643040 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f50f00e3eb0 con 0x7f5100107e10 2024-07-27T03:16:51.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.211+0000 7f510485e640 1 --2- 172.21.15.67:0/4005643040 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f50e80b3bd0 0x7f50e80b60e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00bbe0 tx=0x7f50fc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:51.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.213+0000 7f50f5ffb640 1 -- 172.21.15.67:0/4005643040 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f50f0014030 con 0x7f5100107e10 2024-07-27T03:16:51.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.398+0000 7f5106ac2640 1 -- 172.21.15.67:0/4005643040 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5100199f50 con 0x7f5100107e10 2024-07-27T03:16:51.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.399+0000 7f50f5ffb640 1 -- 172.21.15.67:0/4005643040 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24829 (secure 0 0 0) 0x7f5100199f50 con 0x7f5100107e10 2024-07-27T03:16:51.402 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:16:51.402 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":91,"fsid":"c0782a00-4bc4-11ef-bcc7-c7b262605968","created":"2024-07-27T03:04:08.096824+0000","modified":"2024-07-27T03:13:57.294556+0000","last_up_change":"2024-07-27T03:11:33.314800+0000","last_in_change":"2024-07-27T03:11:14.028095+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-27T03:08:01.042479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-27T03:12:27.565948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-27T03:12:31.353726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-27T03:12:40.599486+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9f614272-18e8-4562-812c-4f43e40efd2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6803","nonce":2648129674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6805","nonce":2648129674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6809","nonce":2648129674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6807","nonce":2648129674}]},"public_addr":"172.21.15.67:6803/2648129674","cluster_addr":"172.21.15.67:6805/2648129674","heartbeat_back_addr":"172.21.15.67:6809/2648129674","heartbeat_front_addr":"172.21.15.67:6807/2648129674","state":["exists","up"]},{"osd":1,"uuid":"f59f5879-163a-4269-bbe7-1200eea65341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6801","nonce":2561753152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6803","nonce":2561753152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6807","nonce":2561753152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6805","nonce":2561753152}]},"public_addr":"172.21.15.149:6801/2561753152","cluster_addr":"172.21.15.149:6803/2561753152","heartbeat_back_addr":"172.21.15.149:6807/2561753152","heartbeat_front_addr":"172.21.15.149:6805/2561753152","state":["exists","up"]},{"osd":2,"uuid":"444e0ebe-7ca3-4be9-8bd5-db859f824445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6801","nonce":3770940078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6803","nonce":3770940078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6807","nonce":3770940078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6805","nonce":3770940078}]},"public_addr":"172.21.15.175:6801/3770940078","cluster_addr":"172.21.15.175:6803/3770940078","heartbeat_back_addr":"172.21.15.175:6807/3770940078","heartbeat_front_addr":"172.21.15.175:6805/3770940078","state":["exists","up"]},{"osd":3,"uuid":"d554db3d-9604-427c-8736-b1bb379f8d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6811","nonce":3159144693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6813","nonce":3159144693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6817","nonce":3159144693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6815","nonce":3159144693}]},"public_addr":"172.21.15.67:6811/3159144693","cluster_addr":"172.21.15.67:6813/3159144693","heartbeat_back_addr":"172.21.15.67:6817/3159144693","heartbeat_front_addr":"172.21.15.67:6815/3159144693","state":["exists","up"]},{"osd":4,"uuid":"c317dbe5-3b49-47a6-8408-7042a3dac345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3393503532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3393503532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3393503532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3393503532}]},"public_addr":"172.21.15.149:6809/3393503532","cluster_addr":"172.21.15.149:6811/3393503532","heartbeat_back_addr":"172.21.15.149:6815/3393503532","heartbeat_front_addr":"172.21.15.149:6813/3393503532","state":["exists","up"]},{"osd":5,"uuid":"5c85abd6-2b42-4957-a9b7-59a9ee36b841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6809","nonce":2291204784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6811","nonce":2291204784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6815","nonce":2291204784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6813","nonce":2291204784}]},"public_addr":"172.21.15.175:6809/2291204784","cluster_addr":"172.21.15.175:6811/2291204784","heartbeat_back_addr":"172.21.15.175:6815/2291204784","heartbeat_front_addr":"172.21.15.175:6813/2291204784","state":["exists","up"]},{"osd":6,"uuid":"bf6c277d-ec24-4470-98de-cf6f28876ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6819","nonce":2660182879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6821","nonce":2660182879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6825","nonce":2660182879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6823","nonce":2660182879}]},"public_addr":"172.21.15.67:6819/2660182879","cluster_addr":"172.21.15.67:6821/2660182879","heartbeat_back_addr":"172.21.15.67:6825/2660182879","heartbeat_front_addr":"172.21.15.67:6823/2660182879","state":["exists","up"]},{"osd":7,"uuid":"fea2c7cf-d5cb-45ae-9b29-f6e56c903891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6817","nonce":2289624404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6819","nonce":2289624404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6823","nonce":2289624404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6821","nonce":2289624404}]},"public_addr":"172.21.15.149:6817/2289624404","cluster_addr":"172.21.15.149:6819/2289624404","heartbeat_back_addr":"172.21.15.149:6823/2289624404","heartbeat_front_addr":"172.21.15.149:6821/2289624404","state":["exists","up"]},{"osd":8,"uuid":"14d04d96-579a-4dc9-a4d4-50e4850b48e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6817","nonce":2286032466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6819","nonce":2286032466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6823","nonce":2286032466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6821","nonce":2286032466}]},"public_addr":"172.21.15.175:6817/2286032466","cluster_addr":"172.21.15.175:6819/2286032466","heartbeat_back_addr":"172.21.15.175:6823/2286032466","heartbeat_front_addr":"172.21.15.175:6821/2286032466","state":["exists","up"]},{"osd":9,"uuid":"b65dfb6b-1efe-4314-a8be-ce6a295e99fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6827","nonce":3411081513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6829","nonce":3411081513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6833","nonce":3411081513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6831","nonce":3411081513}]},"public_addr":"172.21.15.67:6827/3411081513","cluster_addr":"172.21.15.67:6829/3411081513","heartbeat_back_addr":"172.21.15.67:6833/3411081513","heartbeat_front_addr":"172.21.15.67:6831/3411081513","state":["exists","up"]},{"osd":10,"uuid":"e3dabae0-0d64-4222-984c-fc1cce883456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6825","nonce":2011495212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6827","nonce":2011495212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6831","nonce":2011495212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6829","nonce":2011495212}]},"public_addr":"172.21.15.149:6825/2011495212","cluster_addr":"172.21.15.149:6827/2011495212","heartbeat_back_addr":"172.21.15.149:6831/2011495212","heartbeat_front_addr":"172.21.15.149:6829/2011495212","state":["exists","up"]},{"osd":11,"uuid":"406c082c-9b5f-4781-8cbf-3c04fa49e410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6825","nonce":2565514213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6827","nonce":2565514213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6831","nonce":2565514213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6829","nonce":2565514213}]},"public_addr":"172.21.15.175:6825/2565514213","cluster_addr":"172.21.15.175:6827/2565514213","heartbeat_back_addr":"172.21.15.175:6831/2565514213","heartbeat_front_addr":"172.21.15.175:6829/2565514213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:11.399380+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:33.728518+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:56.411181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:19.871731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:42.994576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:06.712815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:31.321967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:55.070898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:18.686108+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:43.187795+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:07.318538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:31.374843+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/3477601235":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/4130679770":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6800/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/1029332005":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6801/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6801/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/284078926":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6800/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/2559121542":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/925406021":"2024-07-28T03:04:30.600273+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-27T03:16:51.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.401+0000 7f5106ac2640 1 -- 172.21.15.67:0/4005643040 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f50e80b3bd0 msgr2=0x7f50e80b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:51.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.401+0000 7f5106ac2640 1 --2- 172.21.15.67:0/4005643040 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f50e80b3bd0 0x7f50e80b60e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00bbe0 tx=0x7f50fc00b390 comp rx=0 tx=0).stop 2024-07-27T03:16:51.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.401+0000 7f5106ac2640 1 -- 172.21.15.67:0/4005643040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5100107e10 msgr2=0x7f51001993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:51.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.401+0000 7f5106ac2640 1 --2- 172.21.15.67:0/4005643040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5100107e10 0x7f51001993c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f50f000c970 tx=0x7f50f000ce30 comp rx=0 tx=0).stop 2024-07-27T03:16:51.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.401+0000 7f5106ac2640 1 -- 172.21.15.67:0/4005643040 shutdown_connections 2024-07-27T03:16:51.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.401+0000 7f5106ac2640 1 --2- 172.21.15.67:0/4005643040 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f50e80b3bd0 0x7f50e80b60e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:51.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.401+0000 7f5106ac2640 1 --2- 172.21.15.67:0/4005643040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5100108760 0x7f510019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:51.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.401+0000 7f5106ac2640 1 --2- 172.21.15.67:0/4005643040 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5100107e10 0x7f51001993c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:51.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.401+0000 7f5106ac2640 1 --2- 172.21.15.67:0/4005643040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51001024d0 0x7f5100198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:51.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.401+0000 7f5106ac2640 1 -- 172.21.15.67:0/4005643040 >> 172.21.15.67:0/4005643040 conn(0x7f51000791a0 msgr2=0x7f5100103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:51.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.401+0000 7f5106ac2640 1 -- 172.21.15.67:0/4005643040 shutdown_connections 2024-07-27T03:16:51.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:51.401+0000 7f5106ac2640 1 -- 172.21.15.67:0/4005643040 wait complete. 2024-07-27T03:16:52.154 DEBUG:tasks.fwd_scrub:Yielding 2024-07-27T03:16:52.155 INFO:teuthology.run_tasks:Running task workunit... 2024-07-27T03:16:52.168 INFO:tasks.workunit:Pulling workunits from ref af8fdbf079811fe158ce62927bd145c35b74d4b8 2024-07-27T03:16:52.169 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-07-27T03:16:52.169 INFO:tasks.workunit:timeout=3h 2024-07-27T03:16:52.169 INFO:tasks.workunit:cleanup=True 2024-07-27T03:16:52.169 DEBUG:teuthology.orchestra.run.smithi067:> stat -- /home/ubuntu/cephtest/mnt.0 2024-07-27T03:16:52.170 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-07-27T03:16:52.170 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-27T03:16:52.170 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-27T03:16:52.190 INFO:teuthology.orchestra.run.smithi067.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-07-27T03:16:52.190 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-07-27T03:16:52.190 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-07-27T03:16:52.190 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-07-27T03:16:52.190 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-07-27T03:16:52.190 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-07-27 03:13:53.228121608 +0000 2024-07-27T03:16:52.190 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-07-27 03:13:53.228121608 +0000 2024-07-27T03:16:52.190 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-07-27 03:16:39.009465543 +0000 2024-07-27T03:16:52.190 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-07-27 03:13:53.228121608 +0000 2024-07-27T03:16:52.190 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-07-27T03:16:52.191 DEBUG:teuthology.orchestra.run.smithi067:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-07-27T03:16:52.265 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout af8fdbf079811fe158ce62927bd145c35b74d4b8 2024-07-27T03:16:52.322 INFO:tasks.workunit.client.0.smithi067.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-07-27T03:16:52.461 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:16:52.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:52 smithi067 ceph-mon[32814]: pgmap v439: 129 pgs: 129 active+clean; 650 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 12 op/s 2024-07-27T03:16:52.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:52 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/4005643040' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:16:52.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:52 smithi149 ceph-mon[37173]: pgmap v439: 129 pgs: 129 active+clean; 650 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 12 op/s 2024-07-27T03:16:52.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:52 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/4005643040' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:16:52.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:52 smithi175 ceph-mon[37125]: pgmap v439: 129 pgs: 129 active+clean; 650 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 767 B/s wr, 12 op/s 2024-07-27T03:16:52.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:52 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/4005643040' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:16:54.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.125+0000 7f466cfeb640 1 -- 172.21.15.67:0/2440849527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4668107e30 msgr2=0x7f4668108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:54.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.125+0000 7f466cfeb640 1 --2- 172.21.15.67:0/2440849527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4668107e30 0x7f4668108240 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f465c009900 tx=0x7f465c02a410 comp rx=0 tx=0).stop 2024-07-27T03:16:54.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.125+0000 7f466cfeb640 1 -- 172.21.15.67:0/2440849527 shutdown_connections 2024-07-27T03:16:54.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.125+0000 7f466cfeb640 1 --2- 172.21.15.67:0/2440849527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4668108780 0x7f466810efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.125+0000 7f466cfeb640 1 --2- 172.21.15.67:0/2440849527 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4668107e30 0x7f4668108240 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.125+0000 7f466cfeb640 1 --2- 172.21.15.67:0/2440849527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f46681024f0 0x7f4668102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.125+0000 7f466cfeb640 1 -- 172.21.15.67:0/2440849527 >> 172.21.15.67:0/2440849527 conn(0x7f46680791b0 msgr2=0x7f46680795c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:54.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.125+0000 7f466cfeb640 1 -- 172.21.15.67:0/2440849527 shutdown_connections 2024-07-27T03:16:54.128 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.125+0000 7f466cfeb640 1 -- 172.21.15.67:0/2440849527 wait complete. 2024-07-27T03:16:54.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.126+0000 7f466cfeb640 1 Processor -- start 2024-07-27T03:16:54.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.126+0000 7f466cfeb640 1 -- start start 2024-07-27T03:16:54.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.126+0000 7f466cfeb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46681024f0 0x7f4668197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:54.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.126+0000 7f466cfeb640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4668107e30 0x7f4668198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:54.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.126+0000 7f466cfeb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4668108780 0x7f466819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:54.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.126+0000 7f466cfeb640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f46681112f0 con 0x7f4668108780 2024-07-27T03:16:54.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.126+0000 7f466cfeb640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4668111170 con 0x7f4668107e30 2024-07-27T03:16:54.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.126+0000 7f466cfeb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4668111470 con 0x7f46681024f0 2024-07-27T03:16:54.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.126+0000 7f4666d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4668108780 0x7f466819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:54.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.126+0000 7f4666d9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4668108780 0x7f466819d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51496/0 (socket says 172.21.15.67:51496) 2024-07-27T03:16:54.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.126+0000 7f4666d9d640 1 -- 172.21.15.67:0/3795073506 learned_addr learned my addr 172.21.15.67:0/3795073506 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:54.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.126+0000 7f4665d9b640 1 --2- 172.21.15.67:0/3795073506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4668107e30 0x7f4668198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:54.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.126+0000 7f466659c640 1 --2- 172.21.15.67:0/3795073506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46681024f0 0x7f4668197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:54.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.127+0000 7f4666d9d640 1 -- 172.21.15.67:0/3795073506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46681024f0 msgr2=0x7f4668197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:54.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.127+0000 7f4666d9d640 1 --2- 172.21.15.67:0/3795073506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46681024f0 0x7f4668197d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.127+0000 7f4666d9d640 1 -- 172.21.15.67:0/3795073506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4668107e30 msgr2=0x7f4668198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:54.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.127+0000 7f4666d9d640 1 --2- 172.21.15.67:0/3795073506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4668107e30 0x7f4668198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.127+0000 7f4666d9d640 1 -- 172.21.15.67:0/3795073506 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466819dcf0 con 0x7f4668108780 2024-07-27T03:16:54.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.127+0000 7f466659c640 1 --2- 172.21.15.67:0/3795073506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46681024f0 0x7f4668197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:16:54.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.127+0000 7f4666d9d640 1 --2- 172.21.15.67:0/3795073506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4668108780 0x7f466819d5c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f466000eaa0 tx=0x7f466000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:54.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.127+0000 7f46577fe640 1 -- 172.21.15.67:0/3795073506 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466000cdb0 con 0x7f4668108780 2024-07-27T03:16:54.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.127+0000 7f46577fe640 1 -- 172.21.15.67:0/3795073506 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4660005020 con 0x7f4668108780 2024-07-27T03:16:54.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.127+0000 7f466cfeb640 1 -- 172.21.15.67:0/3795073506 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466819dfe0 con 0x7f4668108780 2024-07-27T03:16:54.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.127+0000 7f46577fe640 1 -- 172.21.15.67:0/3795073506 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46600116d0 con 0x7f4668108780 2024-07-27T03:16:54.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.127+0000 7f466cfeb640 1 -- 172.21.15.67:0/3795073506 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46681a4740 con 0x7f4668108780 2024-07-27T03:16:54.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.130+0000 7f46577fe640 1 -- 172.21.15.67:0/3795073506 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f46600051d0 con 0x7f4668108780 2024-07-27T03:16:54.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.130+0000 7f4673570740 1 Processor -- start 2024-07-27T03:16:54.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.131+0000 7f4673570740 1 -- start start 2024-07-27T03:16:54.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.131+0000 7f4673570740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af393a9350 0x55af3974de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:54.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.131+0000 7f4673570740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af39380490 0x55af3974e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:54.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.131+0000 7f4673570740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39388670 0x55af3974e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:54.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.131+0000 7f4673570740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55af396c4a60 con 0x55af39388670 2024-07-27T03:16:54.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.131+0000 7f4673570740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55af396b3f60 con 0x55af393a9350 2024-07-27T03:16:54.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.131+0000 7f4673570740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55af396b7fa0 con 0x55af39380490 2024-07-27T03:16:54.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.131+0000 7f46577fe640 1 --2- 172.21.15.67:0/3795073506 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f46500b5de0 0x7f46500b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:54.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.131+0000 7f4665d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39388670 0x55af3974e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:54.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.131+0000 7f4665d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39388670 0x55af3974e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51498/0 (socket says 172.21.15.67:51498) 2024-07-27T03:16:54.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.131+0000 7f4665d9b640 1 -- 172.21.15.67:0/1577433875 learned_addr learned my addr 172.21.15.67:0/1577433875 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:54.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.131+0000 7f4666d9d640 1 --2- 172.21.15.67:0/3795073506 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f46500b5de0 0x7f46500b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:54.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.131+0000 7f4665d9b640 1 --2- 172.21.15.67:0/1577433875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af393a9350 0x55af3974de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:54.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.131+0000 7f466659c640 1 --2- 172.21.15.67:0/1577433875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af39380490 0x55af3974e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:54.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.132+0000 7f46577fe640 1 -- 172.21.15.67:0/3795073506 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f46600e41e0 con 0x7f4668108780 2024-07-27T03:16:54.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.132+0000 7f4665d9b640 1 -- 172.21.15.67:0/1577433875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af39380490 msgr2=0x55af3974e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:54.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.132+0000 7f4665d9b640 1 --2- 172.21.15.67:0/1577433875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af39380490 0x55af3974e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.132+0000 7f4665d9b640 1 -- 172.21.15.67:0/1577433875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af393a9350 msgr2=0x55af3974de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:54.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.132+0000 7f4665d9b640 1 --2- 172.21.15.67:0/1577433875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af393a9350 0x55af3974de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.132+0000 7f4665d9b640 1 -- 172.21.15.67:0/1577433875 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af39414930 con 0x55af39388670 2024-07-27T03:16:54.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.132+0000 7f4665d9b640 1 --2- 172.21.15.67:0/1577433875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39388670 0x55af3974e890 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f466806b130 tx=0x7f465c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea4702800ed44a5 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:54.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.132+0000 7f4666d9d640 1 --2- 172.21.15.67:0/3795073506 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f46500b5de0 0x7f46500b82f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f466004b160 tx=0x7f4660075120 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:54.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.132+0000 7f4654ff9640 1 -- 172.21.15.67:0/1577433875 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c035040 con 0x55af39388670 2024-07-27T03:16:54.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.132+0000 7f4654ff9640 1 -- 172.21.15.67:0/1577433875 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f465c031790 con 0x55af39388670 2024-07-27T03:16:54.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.132+0000 7f4654ff9640 1 -- 172.21.15.67:0/1577433875 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c003e50 con 0x55af39388670 2024-07-27T03:16:54.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.133+0000 7f4673570740 1 -- 172.21.15.67:0/1577433875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39388670 msgr2=0x55af3974e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:54.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.133+0000 7f4673570740 1 --2- 172.21.15.67:0/1577433875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39388670 0x55af3974e890 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f466806b130 tx=0x7f465c0058b0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.133+0000 7f4673570740 1 -- 172.21.15.67:0/1577433875 shutdown_connections 2024-07-27T03:16:54.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.133+0000 7f4673570740 1 --2- 172.21.15.67:0/1577433875 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39388670 0x55af3974e890 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.133+0000 7f4673570740 1 --2- 172.21.15.67:0/1577433875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af39380490 0x55af3974e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.133+0000 7f4673570740 1 --2- 172.21.15.67:0/1577433875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af393a9350 0x55af3974de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.133+0000 7f4673570740 1 -- 172.21.15.67:0/1577433875 >> 172.21.15.67:0/1577433875 conn(0x55af3939d740 msgr2=0x55af3974bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:54.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.133+0000 7f4673570740 1 -- 172.21.15.67:0/1577433875 shutdown_connections 2024-07-27T03:16:54.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.133+0000 7f4673570740 1 -- 172.21.15.67:0/1577433875 wait complete. 2024-07-27T03:16:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.134+0000 7f4673570740 1 -- start start 2024-07-27T03:16:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.134+0000 7f4673570740 1 Processor -- start 2024-07-27T03:16:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.134+0000 7f4673570740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39380490 0x55af39759970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.134+0000 7f4673570740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af39388670 0x55af39759eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.134+0000 7f4673570740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af393a9350 0x55af397658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.134+0000 7f4673570740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55af39517890 con 0x55af39380490 2024-07-27T03:16:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.134+0000 7f4673570740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55af39656580 con 0x55af393a9350 2024-07-27T03:16:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.134+0000 7f4673570740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55af3975c590 con 0x55af39388670 2024-07-27T03:16:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.134+0000 7f4673570740 10 client.0 _ll_register_callbacks cb 0x55af396adf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:16:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.134+0000 7f4665d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39380490 0x55af39759970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.134+0000 7f4665d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39380490 0x55af39759970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51512/0 (socket says 172.21.15.67:51512) 2024-07-27T03:16:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.134+0000 7f4665d9b640 1 -- 172.21.15.67:0/3849610346 learned_addr learned my addr 172.21.15.67:0/3849610346 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.134+0000 7f466659c640 1 --2- 172.21.15.67:0/3849610346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af39388670 0x55af39759eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.134+0000 7f4665d9b640 1 --2- 172.21.15.67:0/3849610346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af393a9350 0x55af397658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:54.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.135+0000 7f4665d9b640 1 -- 172.21.15.67:0/3849610346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af39388670 msgr2=0x55af39759eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:54.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.135+0000 7f4665d9b640 1 --2- 172.21.15.67:0/3849610346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af39388670 0x55af39759eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.135+0000 7f4665d9b640 1 -- 172.21.15.67:0/3849610346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af393a9350 msgr2=0x55af397658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:54.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.135+0000 7f4665d9b640 1 --2- 172.21.15.67:0/3849610346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af393a9350 0x55af397658b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.135+0000 7f4665d9b640 1 -- 172.21.15.67:0/3849610346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af39765e20 con 0x55af39380490 2024-07-27T03:16:54.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.135+0000 7f4665d9b640 1 --2- 172.21.15.67:0/3849610346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39380490 0x55af39759970 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f465c001eb0 tx=0x7f465c003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da2b93db5d6611a server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:54.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.135+0000 7f464dffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:16:54.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.135+0000 7f464dffb640 1 -- 172.21.15.67:0/3849610346 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c053030 con 0x55af39380490 2024-07-27T03:16:54.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.135+0000 7f464dffb640 1 -- 172.21.15.67:0/3849610346 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f465c003de0 con 0x55af39380490 2024-07-27T03:16:54.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.135+0000 7f4673570740 1 -- 172.21.15.67:0/3849610346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f465c031790 con 0x55af39380490 2024-07-27T03:16:54.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.135+0000 7f464dffb640 1 -- 172.21.15.67:0/3849610346 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f465c000e60 con 0x55af39380490 2024-07-27T03:16:54.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.135+0000 7f4673570740 10 client.15522 fetch_fsmap learned FSMap version 30 2024-07-27T03:16:54.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.135+0000 7f4673570740 1 -- 172.21.15.67:0/3849610346 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55af39768750 con 0x55af39380490 2024-07-27T03:16:54.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.135+0000 7f464dffb640 1 -- 172.21.15.67:0/3849610346 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f465c031790 con 0x55af39380490 2024-07-27T03:16:54.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.136+0000 7f464dffb640 1 -- 172.21.15.67:0/3849610346 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f465c048070 con 0x55af39380490 2024-07-27T03:16:54.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.136+0000 7f4673570740 10 client.15522 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:16:54.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.136+0000 7f4673570740 10 client.15522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:16:54.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.136+0000 7f4673570740 20 client.15522 populate_metadata read hostname 'smithi067' 2024-07-27T03:16:54.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.136+0000 7f4673570740 1 --2- 172.21.15.67:0/3849610346 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55af39367cd0 0x55af3976bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:54.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.136+0000 7f4673570740 4 client.15522 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:16:54.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.136+0000 7f4673570740 1 -- 172.21.15.67:0/3849610346 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af396c69b0 con 0x55af39367cd0 2024-07-27T03:16:54.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.136+0000 7f466659c640 1 --2- 172.21.15.67:0/3849610346 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55af39367cd0 0x55af3976bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:54.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.137+0000 7f466659c640 1 --2- 172.21.15.67:0/3849610346 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55af39367cd0 0x55af3976bab0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f163b6ebcd917c1 server_cookie=2d4e89ecc94895a2 in_seq=0 out_seq=0 2024-07-27T03:16:54.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.137+0000 7f464dffb640 10 client.15522 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:16:54.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.137+0000 7f464dffb640 1 -- 172.21.15.67:0/3849610346 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af396c69b0 con 0x55af39367cd0 2024-07-27T03:16:54.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.137+0000 7f464dffb640 10 client.15522 handle_command_reply: tid=0 2024-07-27T03:16:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.137+0000 7f4673570740 1 client.15522 shutdown 2024-07-27T03:16:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.138+0000 7f4673570740 1 -- 172.21.15.67:0/3849610346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39380490 msgr2=0x55af39759970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.138+0000 7f4673570740 1 --2- 172.21.15.67:0/3849610346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39380490 0x55af39759970 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f465c001eb0 tx=0x7f465c003a30 comp rx=0 tx=0).stop 2024-07-27T03:16:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.138+0000 7f4673570740 1 -- 172.21.15.67:0/3849610346 shutdown_connections 2024-07-27T03:16:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.138+0000 7f4673570740 1 --2- 172.21.15.67:0/3849610346 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55af39367cd0 0x55af3976bab0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.138+0000 7f4673570740 1 --2- 172.21.15.67:0/3849610346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af393a9350 0x55af397658b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.138+0000 7f4673570740 1 --2- 172.21.15.67:0/3849610346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af39388670 0x55af39759eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.138+0000 7f4673570740 1 --2- 172.21.15.67:0/3849610346 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39380490 0x55af39759970 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.138+0000 7f4673570740 1 -- 172.21.15.67:0/3849610346 >> 172.21.15.67:0/3849610346 conn(0x55af3939d740 msgr2=0x55af39750fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.138+0000 7f464dffb640 0 client.15522 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:16:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.138+0000 7f4673570740 1 -- 172.21.15.67:0/3849610346 shutdown_connections 2024-07-27T03:16:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.138+0000 7f4673570740 1 -- 172.21.15.67:0/3849610346 wait complete. 2024-07-27T03:16:54.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.138+0000 7f4673570740 20 client.15522 trim_cache size 0 max 16384 2024-07-27T03:16:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.155+0000 7f4673570740 1 Processor -- start 2024-07-27T03:16:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.155+0000 7f4673570740 1 -- start start 2024-07-27T03:16:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.156+0000 7f4673570740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39380490 0x55af39765820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.156+0000 7f4673570740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af39388670 0x55af3974aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.156+0000 7f4673570740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af393a9350 0x55af39758ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.156+0000 7f4673570740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f46681ae4e0 con 0x55af39380490 2024-07-27T03:16:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.156+0000 7f4673570740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f46681b0020 con 0x55af39388670 2024-07-27T03:16:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.156+0000 7f4673570740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46681aebb0 con 0x55af393a9350 2024-07-27T03:16:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.156+0000 7f4665d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39380490 0x55af39765820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:54.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.156+0000 7f4665d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39380490 0x55af39765820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51518/0 (socket says 172.21.15.67:51518) 2024-07-27T03:16:54.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.156+0000 7f4665d9b640 1 -- 172.21.15.67:0/3752511804 learned_addr learned my addr 172.21.15.67:0/3752511804 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:54.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.156+0000 7f466659c640 1 --2- 172.21.15.67:0/3752511804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af39388670 0x55af3974aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:54.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.156+0000 7f4665d9b640 1 --2- 172.21.15.67:0/3752511804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af393a9350 0x55af39758ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:54.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.156+0000 7f4665d9b640 1 -- 172.21.15.67:0/3752511804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af393a9350 msgr2=0x55af39758ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:54.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.156+0000 7f4665d9b640 1 --2- 172.21.15.67:0/3752511804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af393a9350 0x55af39758ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.156+0000 7f4665d9b640 1 -- 172.21.15.67:0/3752511804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af39388670 msgr2=0x55af3974aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:54.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.156+0000 7f4665d9b640 1 --2- 172.21.15.67:0/3752511804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af39388670 0x55af3974aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.156+0000 7f4665d9b640 1 -- 172.21.15.67:0/3752511804 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af39750100 con 0x55af39380490 2024-07-27T03:16:54.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.156+0000 7f4665d9b640 1 --2- 172.21.15.67:0/3752511804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39380490 0x55af39765820 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f465c003a00 tx=0x7f465c009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b4305304d3bcf server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:54.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.157+0000 7f464ffff640 1 -- 172.21.15.67:0/3752511804 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c041070 con 0x55af39380490 2024-07-27T03:16:54.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.157+0000 7f464ffff640 1 -- 172.21.15.67:0/3752511804 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f465c03d910 con 0x55af39380490 2024-07-27T03:16:54.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.157+0000 7f464ffff640 1 -- 172.21.15.67:0/3752511804 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c053040 con 0x55af39380490 2024-07-27T03:16:54.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.157+0000 7f4673570740 1 -- 172.21.15.67:0/3752511804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39380490 msgr2=0x55af39765820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:54.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.157+0000 7f4673570740 1 --2- 172.21.15.67:0/3752511804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39380490 0x55af39765820 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f465c003a00 tx=0x7f465c009520 comp rx=0 tx=0).stop 2024-07-27T03:16:54.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.157+0000 7f4673570740 1 -- 172.21.15.67:0/3752511804 shutdown_connections 2024-07-27T03:16:54.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.157+0000 7f4673570740 1 --2- 172.21.15.67:0/3752511804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af393a9350 0x55af39758ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.157+0000 7f4673570740 1 --2- 172.21.15.67:0/3752511804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af39388670 0x55af3974aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.157+0000 7f4673570740 1 --2- 172.21.15.67:0/3752511804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39380490 0x55af39765820 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.157+0000 7f4673570740 1 -- 172.21.15.67:0/3752511804 >> 172.21.15.67:0/3752511804 conn(0x55af3939d740 msgr2=0x55af3974edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:54.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.157+0000 7f4673570740 1 -- 172.21.15.67:0/3752511804 shutdown_connections 2024-07-27T03:16:54.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.157+0000 7f4673570740 1 -- 172.21.15.67:0/3752511804 wait complete. 2024-07-27T03:16:54.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.158+0000 7f4673570740 1 -- start start 2024-07-27T03:16:54.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.158+0000 7f4673570740 1 Processor -- start 2024-07-27T03:16:54.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.158+0000 7f4673570740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af39380490 0x55af3976c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:54.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.158+0000 7f4673570740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39388670 0x55af3976cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:54.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.158+0000 7f4673570740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af393a9350 0x55af397d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:54.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.158+0000 7f4673570740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f46681af950 con 0x55af39388670 2024-07-27T03:16:54.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.158+0000 7f4673570740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f46681af280 con 0x55af393a9350 2024-07-27T03:16:54.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.158+0000 7f4673570740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46681ac4c0 con 0x55af39380490 2024-07-27T03:16:54.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.158+0000 7f4673570740 10 client.0 _ll_register_callbacks cb 0x7f465c032d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:16:54.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.159+0000 7f466659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39388670 0x55af3976cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:54.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.159+0000 7f4665d9b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af393a9350 0x55af397d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:54.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.159+0000 7f466659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39388670 0x55af3976cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51534/0 (socket says 172.21.15.67:51534) 2024-07-27T03:16:54.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.159+0000 7f4665d9b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af393a9350 0x55af397d05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.67:50578/0 (socket says 172.21.15.67:50578) 2024-07-27T03:16:54.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.159+0000 7f466659c640 1 -- 172.21.15.67:0/3101534950 learned_addr learned my addr 172.21.15.67:0/3101534950 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:54.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.159+0000 7f4665d9b640 1 --2- 172.21.15.67:0/3101534950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af39380490 0x55af3976c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:54.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.159+0000 7f466659c640 1 -- 172.21.15.67:0/3101534950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af39380490 msgr2=0x55af3976c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:54.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.159+0000 7f466659c640 1 --2- 172.21.15.67:0/3101534950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af39380490 0x55af3976c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.159+0000 7f466659c640 1 -- 172.21.15.67:0/3101534950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af393a9350 msgr2=0x55af397d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:54.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.159+0000 7f466659c640 1 --2- 172.21.15.67:0/3101534950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af393a9350 0x55af397d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.159+0000 7f466659c640 1 -- 172.21.15.67:0/3101534950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af397d0c30 con 0x55af39388670 2024-07-27T03:16:54.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.159+0000 7f466659c640 1 --2- 172.21.15.67:0/3101534950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39388670 0x55af3976cb40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x55af396b4090 tx=0x7f465800fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22445cc6e364479 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:54.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.159+0000 7f464dffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:16:54.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.159+0000 7f464dffb640 1 -- 172.21.15.67:0/3101534950 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465800ea60 con 0x55af39388670 2024-07-27T03:16:54.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.159+0000 7f464dffb640 1 -- 172.21.15.67:0/3101534950 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f465800ec00 con 0x55af39388670 2024-07-27T03:16:54.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.159+0000 7f4673570740 1 -- 172.21.15.67:0/3101534950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55af396c2450 con 0x55af39388670 2024-07-27T03:16:54.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.160+0000 7f464dffb640 1 -- 172.21.15.67:0/3101534950 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f465802d050 con 0x55af39388670 2024-07-27T03:16:54.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.160+0000 7f4673570740 10 client.15528 fetch_fsmap learned FSMap version 30 2024-07-27T03:16:54.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.160+0000 7f4673570740 1 -- 172.21.15.67:0/3101534950 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55af397d80f0 con 0x55af39388670 2024-07-27T03:16:54.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.160+0000 7f464dffb640 1 -- 172.21.15.67:0/3101534950 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55af396c2450 con 0x55af39388670 2024-07-27T03:16:54.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.160+0000 7f464dffb640 1 -- 172.21.15.67:0/3101534950 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f46580254e0 con 0x55af39388670 2024-07-27T03:16:54.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.160+0000 7f4673570740 10 client.15528 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:16:54.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.160+0000 7f4673570740 10 client.15528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:16:54.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.160+0000 7f4673570740 20 client.15528 populate_metadata read hostname 'smithi067' 2024-07-27T03:16:54.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.160+0000 7f4673570740 1 --2- 172.21.15.67:0/3101534950 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55af3933d080 0x55af397da8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:54.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.160+0000 7f4673570740 4 client.15528 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-27T03:16:54.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.160+0000 7f4673570740 1 -- 172.21.15.67:0/3101534950 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55af396bfb90 con 0x55af3933d080 2024-07-27T03:16:54.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.161+0000 7f4665d9b640 1 --2- 172.21.15.67:0/3101534950 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55af3933d080 0x55af397da8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:54.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.161+0000 7f464dffb640 10 client.15528 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:16:54.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.161+0000 7f4665d9b640 1 --2- 172.21.15.67:0/3101534950 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55af3933d080 0x55af397da8c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed5acfc99497f259 server_cookie=b740e735af1640c6 in_seq=0 out_seq=0 2024-07-27T03:16:54.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.162+0000 7f464dffb640 1 -- 172.21.15.67:0/3101534950 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55af396bfb90 con 0x55af3933d080 2024-07-27T03:16:54.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.162+0000 7f464dffb640 10 client.15528 handle_command_reply: tid=0 2024-07-27T03:16:54.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.162+0000 7f4673570740 1 client.15528 shutdown 2024-07-27T03:16:54.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.162+0000 7f4673570740 1 -- 172.21.15.67:0/3101534950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39388670 msgr2=0x55af3976cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:54.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.162+0000 7f4673570740 1 --2- 172.21.15.67:0/3101534950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39388670 0x55af3976cb40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x55af396b4090 tx=0x7f465800fb30 comp rx=0 tx=0).stop 2024-07-27T03:16:54.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.162+0000 7f4673570740 1 -- 172.21.15.67:0/3101534950 shutdown_connections 2024-07-27T03:16:54.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.162+0000 7f4673570740 1 --2- 172.21.15.67:0/3101534950 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55af3933d080 0x55af397da8c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.162+0000 7f4673570740 1 --2- 172.21.15.67:0/3101534950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55af393a9350 0x55af397d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.162+0000 7f4673570740 1 --2- 172.21.15.67:0/3101534950 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55af39388670 0x55af3976cb40 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.162+0000 7f464dffb640 0 client.15528 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:16:54.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.162+0000 7f4673570740 1 --2- 172.21.15.67:0/3101534950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55af39380490 0x55af3976c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.162+0000 7f4673570740 1 -- 172.21.15.67:0/3101534950 >> 172.21.15.67:0/3101534950 conn(0x55af3939d740 msgr2=0x55af3975b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:54.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.162+0000 7f4673570740 1 -- 172.21.15.67:0/3101534950 shutdown_connections 2024-07-27T03:16:54.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.163+0000 7f4673570740 1 -- 172.21.15.67:0/3101534950 wait complete. 2024-07-27T03:16:54.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.163+0000 7f4673570740 20 client.15528 trim_cache size 0 max 16384 2024-07-27T03:16:54.169 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:16:54.169 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-27T03:16:54.169 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "0514564b-71b1-453f-86f8-49f921df7979", 2024-07-27T03:16:54.169 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-27T03:16:54.169 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:16:54.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.163+0000 7f466cfeb640 1 -- 172.21.15.67:0/3795073506 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f46500b5de0 msgr2=0x7f46500b82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:54.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.163+0000 7f466cfeb640 1 --2- 172.21.15.67:0/3795073506 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f46500b5de0 0x7f46500b82f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f466004b160 tx=0x7f4660075120 comp rx=0 tx=0).stop 2024-07-27T03:16:54.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.163+0000 7f466cfeb640 1 -- 172.21.15.67:0/3795073506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4668108780 msgr2=0x7f466819d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:54.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.163+0000 7f466cfeb640 1 --2- 172.21.15.67:0/3795073506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4668108780 0x7f466819d5c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f466000eaa0 tx=0x7f466000ef60 comp rx=0 tx=0).stop 2024-07-27T03:16:54.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.163+0000 7f466cfeb640 1 -- 172.21.15.67:0/3795073506 shutdown_connections 2024-07-27T03:16:54.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.163+0000 7f466cfeb640 1 --2- 172.21.15.67:0/3795073506 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f46500b5de0 0x7f46500b82f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.163+0000 7f466cfeb640 1 --2- 172.21.15.67:0/3795073506 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4668108780 0x7f466819d5c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.163+0000 7f466cfeb640 1 --2- 172.21.15.67:0/3795073506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4668107e30 0x7f4668198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.163+0000 7f466cfeb640 1 --2- 172.21.15.67:0/3795073506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46681024f0 0x7f4668197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:54.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.163+0000 7f466cfeb640 1 -- 172.21.15.67:0/3795073506 >> 172.21.15.67:0/3795073506 conn(0x7f46680791b0 msgr2=0x7f4668103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:54.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.163+0000 7f466cfeb640 1 -- 172.21.15.67:0/3795073506 shutdown_connections 2024-07-27T03:16:54.171 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:54.164+0000 7f466cfeb640 1 -- 172.21.15.67:0/3795073506 wait complete. 2024-07-27T03:16:54.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:54 smithi067 ceph-mon[32814]: pgmap v440: 129 pgs: 129 active+clean; 650 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-07-27T03:16:54.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:54 smithi149 ceph-mon[37173]: pgmap v440: 129 pgs: 129 active+clean; 650 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-07-27T03:16:54.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:54 smithi175 ceph-mon[37125]: pgmap v440: 129 pgs: 129 active+clean; 650 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-07-27T03:16:54.912 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-27T03:16:55.228 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:16:55.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:55 smithi067 ceph-mon[32814]: scrub queued for path: / 2024-07-27T03:16:55.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:55 smithi067 ceph-mon[32814]: scrub summary: idle+waiting paths [/] 2024-07-27T03:16:55.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:55 smithi067 ceph-mon[32814]: scrub summary: active paths [/] 2024-07-27T03:16:55.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:55 smithi067 ceph-mon[32814]: scrub summary: idle+waiting paths [/] 2024-07-27T03:16:55.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:55 smithi149 ceph-mon[37173]: scrub queued for path: / 2024-07-27T03:16:55.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:55 smithi149 ceph-mon[37173]: scrub summary: idle+waiting paths [/] 2024-07-27T03:16:55.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:55 smithi149 ceph-mon[37173]: scrub summary: active paths [/] 2024-07-27T03:16:55.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:55 smithi149 ceph-mon[37173]: scrub summary: idle+waiting paths [/] 2024-07-27T03:16:55.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:55 smithi175 ceph-mon[37125]: scrub queued for path: / 2024-07-27T03:16:55.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:55 smithi175 ceph-mon[37125]: scrub summary: idle+waiting paths [/] 2024-07-27T03:16:55.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:55 smithi175 ceph-mon[37125]: scrub summary: active paths [/] 2024-07-27T03:16:55.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:55 smithi175 ceph-mon[37125]: scrub summary: idle+waiting paths [/] 2024-07-27T03:16:56.644 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:56 smithi067 ceph-mon[32814]: pgmap v441: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-07-27T03:16:56.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:56 smithi149 ceph-mon[37173]: pgmap v441: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-07-27T03:16:56.874 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.865+0000 7f382d382640 1 -- 172.21.15.67:0/2721924963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3828107e20 msgr2=0x7f3828108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:56.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.865+0000 7f382d382640 1 --2- 172.21.15.67:0/2721924963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3828107e20 0x7f3828108230 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3810009900 tx=0x7f381002a410 comp rx=0 tx=0).stop 2024-07-27T03:16:56.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.865+0000 7f382d382640 1 -- 172.21.15.67:0/2721924963 shutdown_connections 2024-07-27T03:16:56.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.865+0000 7f382d382640 1 --2- 172.21.15.67:0/2721924963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3828108770 0x7f382810efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.865+0000 7f382d382640 1 --2- 172.21.15.67:0/2721924963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3828107e20 0x7f3828108230 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.865+0000 7f382d382640 1 --2- 172.21.15.67:0/2721924963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38281024e0 0x7f38281028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.865+0000 7f382d382640 1 -- 172.21.15.67:0/2721924963 >> 172.21.15.67:0/2721924963 conn(0x7f38280791a0 msgr2=0x7f38280795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:56.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.865+0000 7f382d382640 1 -- 172.21.15.67:0/2721924963 shutdown_connections 2024-07-27T03:16:56.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.865+0000 7f382d382640 1 -- 172.21.15.67:0/2721924963 wait complete. 2024-07-27T03:16:56.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.866+0000 7f382d382640 1 Processor -- start 2024-07-27T03:16:56.875 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.866+0000 7f382d382640 1 -- start start 2024-07-27T03:16:56.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.866+0000 7f382d382640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38281024e0 0x7f3828199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:56.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.866+0000 7f382d382640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3828107e20 0x7f382819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:56.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.866+0000 7f382d382640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3828108770 0x7f382819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:56.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.866+0000 7f382d382640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f38281123c0 con 0x7f3828107e20 2024-07-27T03:16:56.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.866+0000 7f382d382640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3828112240 con 0x7f38281024e0 2024-07-27T03:16:56.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.866+0000 7f382d382640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3828112540 con 0x7f3828108770 2024-07-27T03:16:56.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.866+0000 7f38267fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3828107e20 0x7f382819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:56.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.866+0000 7f38267fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3828107e20 0x7f382819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48694/0 (socket says 172.21.15.67:48694) 2024-07-27T03:16:56.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.866+0000 7f38267fc640 1 -- 172.21.15.67:0/267763227 learned_addr learned my addr 172.21.15.67:0/267763227 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:56.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.866+0000 7f3826ffd640 1 --2- 172.21.15.67:0/267763227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38281024e0 0x7f3828199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:56.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.867+0000 7f38277fe640 1 --2- 172.21.15.67:0/267763227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3828108770 0x7f382819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:56.876 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.867+0000 7f38267fc640 1 -- 172.21.15.67:0/267763227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3828108770 msgr2=0x7f382819f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:56.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.867+0000 7f38267fc640 1 --2- 172.21.15.67:0/267763227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3828108770 0x7f382819f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.867+0000 7f38267fc640 1 -- 172.21.15.67:0/267763227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38281024e0 msgr2=0x7f3828199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:56.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.867+0000 7f38267fc640 1 --2- 172.21.15.67:0/267763227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38281024e0 0x7f3828199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.867+0000 7f38267fc640 1 -- 172.21.15.67:0/267763227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382819fe90 con 0x7f3828107e20 2024-07-27T03:16:56.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.867+0000 7f38267fc640 1 --2- 172.21.15.67:0/267763227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3828107e20 0x7f382819a4c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3810032fd0 tx=0x7f3810004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:56.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.867+0000 7f381ffff640 1 -- 172.21.15.67:0/267763227 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381003f030 con 0x7f3828107e20 2024-07-27T03:16:56.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.867+0000 7f381ffff640 1 -- 172.21.15.67:0/267763227 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3810004490 con 0x7f3828107e20 2024-07-27T03:16:56.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.867+0000 7f382d382640 1 -- 172.21.15.67:0/267763227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38281a0120 con 0x7f3828107e20 2024-07-27T03:16:56.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.867+0000 7f381ffff640 1 -- 172.21.15.67:0/267763227 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810035070 con 0x7f3828107e20 2024-07-27T03:16:56.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.867+0000 7f382d382640 1 -- 172.21.15.67:0/267763227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38281a68e0 con 0x7f3828107e20 2024-07-27T03:16:56.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.870+0000 7f3833907740 1 Processor -- start 2024-07-27T03:16:56.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.870+0000 7f3833907740 1 -- start start 2024-07-27T03:16:56.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.870+0000 7f381ffff640 1 -- 172.21.15.67:0/267763227 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f3810042030 con 0x7f3828107e20 2024-07-27T03:16:56.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.870+0000 7f3833907740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee69324070 0x55ee696c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:56.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.870+0000 7f3833907740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee692fb1b0 0x55ee696c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:56.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.870+0000 7f3833907740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee69303390 0x55ee696c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:56.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.870+0000 7f3833907740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ee695b13b0 con 0x55ee692fb1b0 2024-07-27T03:16:56.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.870+0000 7f3833907740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ee6963fad0 con 0x55ee69324070 2024-07-27T03:16:56.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.870+0000 7f3833907740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ee6962efd0 con 0x55ee69303390 2024-07-27T03:16:56.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.870+0000 7f3826ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee692fb1b0 0x55ee696c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:56.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.871+0000 7f3826ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee692fb1b0 0x55ee696c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48710/0 (socket says 172.21.15.67:48710) 2024-07-27T03:16:56.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.871+0000 7f3826ffd640 1 -- 172.21.15.67:0/506446059 learned_addr learned my addr 172.21.15.67:0/506446059 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:56.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.871+0000 7f38277fe640 1 --2- 172.21.15.67:0/506446059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee69324070 0x55ee696c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:56.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.871+0000 7f38267fc640 1 --2- 172.21.15.67:0/506446059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee69303390 0x55ee696c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:56.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.871+0000 7f3826ffd640 1 -- 172.21.15.67:0/506446059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee69303390 msgr2=0x55ee696c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:56.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.871+0000 7f3826ffd640 1 --2- 172.21.15.67:0/506446059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee69303390 0x55ee696c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.871+0000 7f3826ffd640 1 -- 172.21.15.67:0/506446059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee69324070 msgr2=0x55ee696c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:56.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.871+0000 7f3826ffd640 1 --2- 172.21.15.67:0/506446059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee69324070 0x55ee696c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.871+0000 7f3826ffd640 1 -- 172.21.15.67:0/506446059 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee695c4720 con 0x55ee692fb1b0 2024-07-27T03:16:56.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.871+0000 7f38277fe640 1 --2- 172.21.15.67:0/506446059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee69324070 0x55ee696c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:16:56.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.871+0000 7f381ffff640 1 --2- 172.21.15.67:0/267763227 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f38140b3bd0 0x7f38140b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:56.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.871+0000 7f3826ffd640 1 --2- 172.21.15.67:0/506446059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee692fb1b0 0x55ee696c9230 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f382000bcb0 tx=0x7f382000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6f117a24101e2 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:56.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.872+0000 7f38277fe640 1 --2- 172.21.15.67:0/267763227 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f38140b3bd0 0x7f38140b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:56.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.872+0000 7f381ffff640 1 -- 172.21.15.67:0/267763227 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f3810104590 con 0x7f3828107e20 2024-07-27T03:16:56.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.872+0000 7f381d7fa640 1 -- 172.21.15.67:0/506446059 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3820015070 con 0x55ee692fb1b0 2024-07-27T03:16:56.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.872+0000 7f381d7fa640 1 -- 172.21.15.67:0/506446059 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f382001a910 con 0x55ee692fb1b0 2024-07-27T03:16:56.879 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.872+0000 7f381d7fa640 1 -- 172.21.15.67:0/506446059 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3820011810 con 0x55ee692fb1b0 2024-07-27T03:16:56.880 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:56 smithi175 ceph-mon[37125]: pgmap v441: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-07-27T03:16:56.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.873+0000 7f3833907740 1 -- 172.21.15.67:0/506446059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee692fb1b0 msgr2=0x55ee696c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:56.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.873+0000 7f3833907740 1 --2- 172.21.15.67:0/506446059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee692fb1b0 0x55ee696c9230 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f382000bcb0 tx=0x7f382000f730 comp rx=0 tx=0).stop 2024-07-27T03:16:56.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.873+0000 7f38277fe640 1 --2- 172.21.15.67:0/267763227 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f38140b3bd0 0x7f38140b60e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3818005fd0 tx=0x7f3818009740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:56.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.873+0000 7f3833907740 1 -- 172.21.15.67:0/506446059 shutdown_connections 2024-07-27T03:16:56.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.873+0000 7f3833907740 1 --2- 172.21.15.67:0/506446059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee69303390 0x55ee696c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.873+0000 7f3833907740 1 --2- 172.21.15.67:0/506446059 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee692fb1b0 0x55ee696c9230 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.873+0000 7f3833907740 1 --2- 172.21.15.67:0/506446059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee69324070 0x55ee696c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.873+0000 7f3833907740 1 -- 172.21.15.67:0/506446059 >> 172.21.15.67:0/506446059 conn(0x55ee69318460 msgr2=0x55ee696c6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:56.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.873+0000 7f3833907740 1 -- 172.21.15.67:0/506446059 shutdown_connections 2024-07-27T03:16:56.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.873+0000 7f3833907740 1 -- 172.21.15.67:0/506446059 wait complete. 2024-07-27T03:16:56.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.873+0000 7f3833907740 1 -- start start 2024-07-27T03:16:56.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.873+0000 7f3833907740 1 Processor -- start 2024-07-27T03:16:56.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.874+0000 7f3833907740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee692fb1b0 0x55ee696d48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:56.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.874+0000 7f3833907740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee69303390 0x55ee696d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:56.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.874+0000 7f3833907740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee69324070 0x55ee696e0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:56.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.874+0000 7f3833907740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ee695d18b0 con 0x55ee69303390 2024-07-27T03:16:56.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.874+0000 7f3833907740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ee69633010 con 0x55ee692fb1b0 2024-07-27T03:16:56.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.874+0000 7f3833907740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ee69492700 con 0x55ee69324070 2024-07-27T03:16:56.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.874+0000 7f3833907740 10 client.0 _ll_register_callbacks cb 0x55ee695d9d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:16:56.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.874+0000 7f38267fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee69303390 0x55ee696d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:56.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.874+0000 7f38267fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee69303390 0x55ee696d4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48718/0 (socket says 172.21.15.67:48718) 2024-07-27T03:16:56.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.874+0000 7f38267fc640 1 -- 172.21.15.67:0/2161711373 learned_addr learned my addr 172.21.15.67:0/2161711373 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:56.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.874+0000 7f3826ffd640 1 --2- 172.21.15.67:0/2161711373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee692fb1b0 0x55ee696d48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:56.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.874+0000 7f38277fe640 1 --2- 172.21.15.67:0/2161711373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee69324070 0x55ee696e0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:56.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.874+0000 7f38267fc640 1 -- 172.21.15.67:0/2161711373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee69324070 msgr2=0x55ee696e0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:56.885 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.874+0000 7f38267fc640 1 --2- 172.21.15.67:0/2161711373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee69324070 0x55ee696e0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.874+0000 7f38267fc640 1 -- 172.21.15.67:0/2161711373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee692fb1b0 msgr2=0x55ee696d48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:56.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.874+0000 7f38267fc640 1 --2- 172.21.15.67:0/2161711373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee692fb1b0 0x55ee696d48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.874+0000 7f38267fc640 1 -- 172.21.15.67:0/2161711373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee695d5ba0 con 0x55ee69303390 2024-07-27T03:16:56.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.874+0000 7f38277fe640 1 --2- 172.21.15.67:0/2161711373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee69324070 0x55ee696e0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:16:56.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.874+0000 7f38267fc640 1 --2- 172.21.15.67:0/2161711373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee69303390 0x55ee696d4e20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f38100c83a0 tx=0x7f3810000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315d7c579ba76dba server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:56.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.874+0000 7f380e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:16:56.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.875+0000 7f380e7fc640 1 -- 172.21.15.67:0/2161711373 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810049040 con 0x55ee69303390 2024-07-27T03:16:56.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.875+0000 7f380e7fc640 1 -- 172.21.15.67:0/2161711373 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f38100cf450 con 0x55ee69303390 2024-07-27T03:16:56.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.875+0000 7f3833907740 1 -- 172.21.15.67:0/2161711373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f382001a910 con 0x55ee69303390 2024-07-27T03:16:56.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.875+0000 7f380e7fc640 1 -- 172.21.15.67:0/2161711373 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f38100caa80 con 0x55ee69303390 2024-07-27T03:16:56.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.875+0000 7f3833907740 10 client.15540 fetch_fsmap learned FSMap version 30 2024-07-27T03:16:56.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.875+0000 7f3833907740 1 -- 172.21.15.67:0/2161711373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ee6938f650 con 0x55ee69303390 2024-07-27T03:16:56.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.875+0000 7f380e7fc640 1 -- 172.21.15.67:0/2161711373 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f382001a910 con 0x55ee69303390 2024-07-27T03:16:56.886 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.875+0000 7f380e7fc640 1 -- 172.21.15.67:0/2161711373 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f38100c73c0 con 0x55ee69303390 2024-07-27T03:16:56.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.875+0000 7f3833907740 10 client.15540 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:16:56.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.875+0000 7f3833907740 10 client.15540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:16:56.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.875+0000 7f3833907740 20 client.15540 populate_metadata read hostname 'smithi067' 2024-07-27T03:16:56.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.875+0000 7f3833907740 1 --2- 172.21.15.67:0/2161711373 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55ee692e29f0 0x55ee696e6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:56.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.875+0000 7f3833907740 4 client.15540 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:16:56.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.875+0000 7f3833907740 1 -- 172.21.15.67:0/2161711373 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee69641a20 con 0x55ee692e29f0 2024-07-27T03:16:56.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.875+0000 7f3826ffd640 1 --2- 172.21.15.67:0/2161711373 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55ee692e29f0 0x55ee696e6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:56.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.876+0000 7f380e7fc640 10 client.15540 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:16:56.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.876+0000 7f3826ffd640 1 --2- 172.21.15.67:0/2161711373 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55ee692e29f0 0x55ee696e6540 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=945f8fc0ca31b487 server_cookie=7ffb89d93641f0e5 in_seq=0 out_seq=0 2024-07-27T03:16:56.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.877+0000 7f380e7fc640 1 -- 172.21.15.67:0/2161711373 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee69641a20 con 0x55ee692e29f0 2024-07-27T03:16:56.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.877+0000 7f380e7fc640 10 client.15540 handle_command_reply: tid=0 2024-07-27T03:16:56.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.877+0000 7f3833907740 1 client.15540 shutdown 2024-07-27T03:16:56.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.877+0000 7f3833907740 1 -- 172.21.15.67:0/2161711373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee69303390 msgr2=0x55ee696d4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:56.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.877+0000 7f3833907740 1 --2- 172.21.15.67:0/2161711373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee69303390 0x55ee696d4e20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f38100c83a0 tx=0x7f3810000bc0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.887 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.877+0000 7f3833907740 1 -- 172.21.15.67:0/2161711373 shutdown_connections 2024-07-27T03:16:56.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.877+0000 7f3833907740 1 --2- 172.21.15.67:0/2161711373 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55ee692e29f0 0x55ee696e6540 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.877+0000 7f3833907740 1 --2- 172.21.15.67:0/2161711373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee69324070 0x55ee696e0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.877+0000 7f3833907740 1 --2- 172.21.15.67:0/2161711373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee69303390 0x55ee696d4e20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.877+0000 7f380e7fc640 0 client.15540 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:16:56.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.877+0000 7f3833907740 1 --2- 172.21.15.67:0/2161711373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee692fb1b0 0x55ee696d48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.877+0000 7f3833907740 1 -- 172.21.15.67:0/2161711373 >> 172.21.15.67:0/2161711373 conn(0x55ee69318460 msgr2=0x55ee696cbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:56.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.877+0000 7f3833907740 1 -- 172.21.15.67:0/2161711373 shutdown_connections 2024-07-27T03:16:56.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.877+0000 7f3833907740 1 -- 172.21.15.67:0/2161711373 wait complete. 2024-07-27T03:16:56.888 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.877+0000 7f3833907740 20 client.15540 trim_cache size 0 max 16384 2024-07-27T03:16:56.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.894+0000 7f3833907740 1 Processor -- start 2024-07-27T03:16:56.897 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.895+0000 7f3833907740 1 -- start start 2024-07-27T03:16:56.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.895+0000 7f3833907740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee692fb1b0 0x55ee696ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:56.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.895+0000 7f3833907740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee69303390 0x55ee696e0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:56.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.895+0000 7f3833907740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee69324070 0x55ee696e09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:56.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.895+0000 7f3833907740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f38281b05f0 con 0x55ee69303390 2024-07-27T03:16:56.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.895+0000 7f3833907740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f38281b2130 con 0x55ee692fb1b0 2024-07-27T03:16:56.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.895+0000 7f3833907740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38281b0cc0 con 0x55ee69324070 2024-07-27T03:16:56.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.895+0000 7f38267fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee69303390 0x55ee696e0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:56.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.895+0000 7f38267fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee69303390 0x55ee696e0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48726/0 (socket says 172.21.15.67:48726) 2024-07-27T03:16:56.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.895+0000 7f38267fc640 1 -- 172.21.15.67:0/4149538484 learned_addr learned my addr 172.21.15.67:0/4149538484 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:56.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.895+0000 7f3826ffd640 1 --2- 172.21.15.67:0/4149538484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee692fb1b0 0x55ee696ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:56.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.895+0000 7f38277fe640 1 --2- 172.21.15.67:0/4149538484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee69324070 0x55ee696e09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:56.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.896+0000 7f38267fc640 1 -- 172.21.15.67:0/4149538484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee69324070 msgr2=0x55ee696e09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:56.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.896+0000 7f38267fc640 1 --2- 172.21.15.67:0/4149538484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee69324070 0x55ee696e09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.896+0000 7f38267fc640 1 -- 172.21.15.67:0/4149538484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee692fb1b0 msgr2=0x55ee696ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:56.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.896+0000 7f38267fc640 1 --2- 172.21.15.67:0/4149538484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee692fb1b0 0x55ee696ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.896+0000 7f38267fc640 1 -- 172.21.15.67:0/4149538484 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee696d3640 con 0x55ee69303390 2024-07-27T03:16:56.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.896+0000 7f3826ffd640 1 --2- 172.21.15.67:0/4149538484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee692fb1b0 0x55ee696ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:16:56.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.896+0000 7f38267fc640 1 --2- 172.21.15.67:0/4149538484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee69303390 0x55ee696e0460 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3810032af0 tx=0x7f3810009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fce9a16ca93a81c server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:56.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.896+0000 7f381cff9640 1 -- 172.21.15.67:0/4149538484 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810049040 con 0x55ee69303390 2024-07-27T03:16:56.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.896+0000 7f381cff9640 1 -- 172.21.15.67:0/4149538484 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38100cb830 con 0x55ee69303390 2024-07-27T03:16:56.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.896+0000 7f381cff9640 1 -- 172.21.15.67:0/4149538484 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38100ca8a0 con 0x55ee69303390 2024-07-27T03:16:56.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.897+0000 7f3833907740 1 -- 172.21.15.67:0/4149538484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee69303390 msgr2=0x55ee696e0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:56.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.897+0000 7f3833907740 1 --2- 172.21.15.67:0/4149538484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee69303390 0x55ee696e0460 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3810032af0 tx=0x7f3810009260 comp rx=0 tx=0).stop 2024-07-27T03:16:56.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.897+0000 7f3833907740 1 -- 172.21.15.67:0/4149538484 shutdown_connections 2024-07-27T03:16:56.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.897+0000 7f3833907740 1 --2- 172.21.15.67:0/4149538484 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee69324070 0x55ee696e09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.897+0000 7f3833907740 1 --2- 172.21.15.67:0/4149538484 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee69303390 0x55ee696e0460 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.897+0000 7f3833907740 1 --2- 172.21.15.67:0/4149538484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee692fb1b0 0x55ee696ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.904 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.897+0000 7f3833907740 1 -- 172.21.15.67:0/4149538484 >> 172.21.15.67:0/4149538484 conn(0x55ee69318460 msgr2=0x55ee696cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:56.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.897+0000 7f3833907740 1 -- 172.21.15.67:0/4149538484 shutdown_connections 2024-07-27T03:16:56.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.897+0000 7f3833907740 1 -- 172.21.15.67:0/4149538484 wait complete. 2024-07-27T03:16:56.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.898+0000 7f3833907740 1 -- start start 2024-07-27T03:16:56.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.898+0000 7f3833907740 1 Processor -- start 2024-07-27T03:16:56.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.898+0000 7f3833907740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee692fb1b0 0x55ee696d8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:56.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.898+0000 7f3833907740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee69303390 0x55ee696d8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:56.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.898+0000 7f3833907740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee69324070 0x55ee696d8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:56.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.898+0000 7f3833907740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f38281b1a60 con 0x55ee69303390 2024-07-27T03:16:56.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.898+0000 7f3833907740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f38281b1390 con 0x55ee692fb1b0 2024-07-27T03:16:56.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.898+0000 7f3833907740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38281ae5d0 con 0x55ee69324070 2024-07-27T03:16:56.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.898+0000 7f3833907740 10 client.0 _ll_register_callbacks cb 0x7f3810005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:16:56.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.898+0000 7f38267fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee69303390 0x55ee696d8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:56.905 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.898+0000 7f38267fc640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee69303390 0x55ee696d8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48740/0 (socket says 172.21.15.67:48740) 2024-07-27T03:16:56.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.898+0000 7f38267fc640 1 -- 172.21.15.67:0/2402319426 learned_addr learned my addr 172.21.15.67:0/2402319426 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:16:56.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.898+0000 7f3826ffd640 1 --2- 172.21.15.67:0/2402319426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee692fb1b0 0x55ee696d8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:56.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.898+0000 7f38277fe640 1 --2- 172.21.15.67:0/2402319426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee69324070 0x55ee696d8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:56.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.899+0000 7f38267fc640 1 -- 172.21.15.67:0/2402319426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee69324070 msgr2=0x55ee696d8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:56.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.899+0000 7f38267fc640 1 --2- 172.21.15.67:0/2402319426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee69324070 0x55ee696d8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.899+0000 7f38267fc640 1 -- 172.21.15.67:0/2402319426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee692fb1b0 msgr2=0x55ee696d8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:16:56.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.899+0000 7f38267fc640 1 --2- 172.21.15.67:0/2402319426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee692fb1b0 0x55ee696d8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.899+0000 7f38267fc640 1 -- 172.21.15.67:0/2402319426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee6974dd70 con 0x55ee69303390 2024-07-27T03:16:56.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.899+0000 7f3826ffd640 1 --2- 172.21.15.67:0/2402319426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee692fb1b0 0x55ee696d8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:16:56.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.899+0000 7f38277fe640 1 --2- 172.21.15.67:0/2402319426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee69324070 0x55ee696d8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:16:56.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.899+0000 7f38267fc640 1 --2- 172.21.15.67:0/2402319426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee69303390 0x55ee696d8aa0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f38100c83a0 tx=0x7f38100cf730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b12553cd4f7d94 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:16:56.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.899+0000 7f380e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:16:56.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.899+0000 7f380e7fc640 1 -- 172.21.15.67:0/2402319426 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381008e020 con 0x55ee69303390 2024-07-27T03:16:56.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.899+0000 7f380e7fc640 1 -- 172.21.15.67:0/2402319426 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f38100d3f60 con 0x55ee69303390 2024-07-27T03:16:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.899+0000 7f3833907740 1 -- 172.21.15.67:0/2402319426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38100cb830 con 0x55ee69303390 2024-07-27T03:16:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.899+0000 7f380e7fc640 1 -- 172.21.15.67:0/2402319426 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f38100cb130 con 0x55ee69303390 2024-07-27T03:16:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.899+0000 7f3833907740 10 client.15546 fetch_fsmap learned FSMap version 30 2024-07-27T03:16:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.899+0000 7f3833907740 1 -- 172.21.15.67:0/2402319426 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ee6974dd70 con 0x55ee69303390 2024-07-27T03:16:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.899+0000 7f380e7fc640 1 -- 172.21.15.67:0/2402319426 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38100cb830 con 0x55ee69303390 2024-07-27T03:16:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.900+0000 7f380e7fc640 1 -- 172.21.15.67:0/2402319426 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f38100cb390 con 0x55ee69303390 2024-07-27T03:16:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.900+0000 7f3833907740 10 client.15546 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:16:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.900+0000 7f3833907740 10 client.15546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:16:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.900+0000 7f3833907740 20 client.15546 populate_metadata read hostname 'smithi067' 2024-07-27T03:16:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.900+0000 7f3833907740 1 --2- 172.21.15.67:0/2402319426 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55ee692b7da0 0x55ee69751400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:16:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.900+0000 7f3833907740 4 client.15546 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-27T03:16:56.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.900+0000 7f3833907740 1 -- 172.21.15.67:0/2402319426 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ee6963d4c0 con 0x55ee692b7da0 2024-07-27T03:16:56.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.900+0000 7f3826ffd640 1 --2- 172.21.15.67:0/2402319426 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55ee692b7da0 0x55ee69751400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:16:56.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.900+0000 7f3826ffd640 1 --2- 172.21.15.67:0/2402319426 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55ee692b7da0 0x55ee69751400 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c82d800625ad718e server_cookie=29cff95282b3f789 in_seq=0 out_seq=0 2024-07-27T03:16:56.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.900+0000 7f380e7fc640 10 client.15546 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:16:56.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.901+0000 7f380e7fc640 1 -- 172.21.15.67:0/2402319426 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ee6963d4c0 con 0x55ee692b7da0 2024-07-27T03:16:56.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.901+0000 7f380e7fc640 10 client.15546 handle_command_reply: tid=0 2024-07-27T03:16:56.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.901+0000 7f3833907740 1 client.15546 shutdown 2024-07-27T03:16:56.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.901+0000 7f3833907740 1 -- 172.21.15.67:0/2402319426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee69303390 msgr2=0x55ee696d8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:56.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.901+0000 7f3833907740 1 --2- 172.21.15.67:0/2402319426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee69303390 0x55ee696d8aa0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f38100c83a0 tx=0x7f38100cf730 comp rx=0 tx=0).stop 2024-07-27T03:16:56.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.901+0000 7f3833907740 1 -- 172.21.15.67:0/2402319426 shutdown_connections 2024-07-27T03:16:56.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.901+0000 7f3833907740 1 --2- 172.21.15.67:0/2402319426 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55ee692b7da0 0x55ee69751400 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.901+0000 7f3833907740 1 --2- 172.21.15.67:0/2402319426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ee69324070 0x55ee696d8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.901+0000 7f3833907740 1 --2- 172.21.15.67:0/2402319426 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ee69303390 0x55ee696d8aa0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.908 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.901+0000 7f3833907740 1 --2- 172.21.15.67:0/2402319426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ee692fb1b0 0x55ee696d8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.901+0000 7f3833907740 1 -- 172.21.15.67:0/2402319426 >> 172.21.15.67:0/2402319426 conn(0x55ee69318460 msgr2=0x55ee696c9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:56.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.901+0000 7f380e7fc640 0 client.15546 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:16:56.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.901+0000 7f3833907740 1 -- 172.21.15.67:0/2402319426 shutdown_connections 2024-07-27T03:16:56.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.901+0000 7f3833907740 1 -- 172.21.15.67:0/2402319426 wait complete. 2024-07-27T03:16:56.909 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.901+0000 7f3833907740 20 client.15546 trim_cache size 0 max 16384 2024-07-27T03:16:56.909 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:16:56.909 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "IDLE (waiting for more scrubs)", 2024-07-27T03:16:56.909 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": { 2024-07-27T03:16:56.909 INFO:teuthology.orchestra.run.smithi067.stdout: "0514564b-71b1-453f-86f8-49f921df7979": { 2024-07-27T03:16:56.909 INFO:teuthology.orchestra.run.smithi067.stdout: "path": "/", 2024-07-27T03:16:56.909 INFO:teuthology.orchestra.run.smithi067.stdout: "tag": "0514564b-71b1-453f-86f8-49f921df7979", 2024-07-27T03:16:56.910 INFO:teuthology.orchestra.run.smithi067.stdout: "options": "recursive,force" 2024-07-27T03:16:56.910 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:16:56.910 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:16:56.910 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:16:56.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.902+0000 7f382d382640 1 -- 172.21.15.67:0/267763227 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f38140b3bd0 msgr2=0x7f38140b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:56.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.902+0000 7f382d382640 1 --2- 172.21.15.67:0/267763227 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f38140b3bd0 0x7f38140b60e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3818005fd0 tx=0x7f3818009740 comp rx=0 tx=0).stop 2024-07-27T03:16:56.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.902+0000 7f382d382640 1 -- 172.21.15.67:0/267763227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3828107e20 msgr2=0x7f382819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:16:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.902+0000 7f382d382640 1 --2- 172.21.15.67:0/267763227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3828107e20 0x7f382819a4c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3810032fd0 tx=0x7f3810004270 comp rx=0 tx=0).stop 2024-07-27T03:16:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.902+0000 7f382d382640 1 -- 172.21.15.67:0/267763227 shutdown_connections 2024-07-27T03:16:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.902+0000 7f382d382640 1 --2- 172.21.15.67:0/267763227 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f38140b3bd0 0x7f38140b60e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.902+0000 7f382d382640 1 --2- 172.21.15.67:0/267763227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3828108770 0x7f382819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.902+0000 7f382d382640 1 --2- 172.21.15.67:0/267763227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3828107e20 0x7f382819a4c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.902+0000 7f382d382640 1 --2- 172.21.15.67:0/267763227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f38281024e0 0x7f3828199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:16:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.902+0000 7f382d382640 1 -- 172.21.15.67:0/267763227 >> 172.21.15.67:0/267763227 conn(0x7f38280791a0 msgr2=0x7f382806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:16:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.902+0000 7f382d382640 1 -- 172.21.15.67:0/267763227 shutdown_connections 2024-07-27T03:16:56.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:16:56.902+0000 7f382d382640 1 -- 172.21.15.67:0/267763227 wait complete. 2024-07-27T03:16:57.569 INFO:tasks.cephfs.filesystem:scrub status for tag:0514564b-71b1-453f-86f8-49f921df7979 - {'path': '/', 'tag': '0514564b-71b1-453f-86f8-49f921df7979', 'options': 'recursive,force'} 2024-07-27T03:16:58.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:16:58 smithi067 ceph-mon[32814]: pgmap v442: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-27T03:16:58.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:16:58 smithi149 ceph-mon[37173]: pgmap v442: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-27T03:16:58.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:16:58 smithi175 ceph-mon[37125]: pgmap v442: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-07-27T03:17:00.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:00 smithi067 ceph-mon[32814]: pgmap v443: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 9 op/s 2024-07-27T03:17:00.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:00 smithi149 ceph-mon[37173]: pgmap v443: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 9 op/s 2024-07-27T03:17:00.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:00 smithi175 ceph-mon[37125]: pgmap v443: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 9 op/s 2024-07-27T03:17:02.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:02 smithi067 ceph-mon[32814]: pgmap v444: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1023 B/s wr, 12 op/s 2024-07-27T03:17:02.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:02 smithi149 ceph-mon[37173]: pgmap v444: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1023 B/s wr, 12 op/s 2024-07-27T03:17:02.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:02 smithi175 ceph-mon[37125]: pgmap v444: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1023 B/s wr, 12 op/s 2024-07-27T03:17:04.677 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:04 smithi175 ceph-mon[37125]: pgmap v445: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 9 op/s 2024-07-27T03:17:04.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:04 smithi067 ceph-mon[32814]: pgmap v445: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 9 op/s 2024-07-27T03:17:04.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:04 smithi149 ceph-mon[37173]: pgmap v445: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1023 B/s wr, 9 op/s 2024-07-27T03:17:06.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:06 smithi067 ceph-mon[32814]: pgmap v446: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 12 op/s 2024-07-27T03:17:06.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:06 smithi067 ceph-mon[32814]: scrub summary: idle 2024-07-27T03:17:06.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:06 smithi149 ceph-mon[37173]: pgmap v446: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 12 op/s 2024-07-27T03:17:06.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:06 smithi149 ceph-mon[37173]: scrub summary: idle 2024-07-27T03:17:06.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:06 smithi175 ceph-mon[37125]: pgmap v446: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 12 op/s 2024-07-27T03:17:06.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:06 smithi175 ceph-mon[37125]: scrub summary: idle 2024-07-27T03:17:08.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:08 smithi067 ceph-mon[32814]: pgmap v447: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-27T03:17:08.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:08 smithi149 ceph-mon[37173]: pgmap v447: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-27T03:17:08.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:08 smithi175 ceph-mon[37125]: pgmap v447: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-07-27T03:17:10.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:10 smithi067 ceph-mon[32814]: pgmap v448: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:10.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:10 smithi149 ceph-mon[37173]: pgmap v448: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:10.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:10 smithi175 ceph-mon[37125]: pgmap v448: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:12.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:12 smithi067 ceph-mon[32814]: pgmap v449: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-07-27T03:17:12.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:12 smithi149 ceph-mon[37173]: pgmap v449: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-07-27T03:17:12.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:12 smithi175 ceph-mon[37125]: pgmap v449: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-07-27T03:17:14.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:14 smithi067 ceph-mon[32814]: pgmap v450: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:14.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:17:14.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:17:14.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:17:14.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:17:14.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:14 smithi149 ceph-mon[37173]: pgmap v450: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:14.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:17:14.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:17:14.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:17:14.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:17:14.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:14 smithi175 ceph-mon[37125]: pgmap v450: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:17:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:17:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:17:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:17:16.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:16 smithi067 ceph-mon[32814]: pgmap v451: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:17:16.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:16 smithi149 ceph-mon[37173]: pgmap v451: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:17:16.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:16 smithi175 ceph-mon[37125]: pgmap v451: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:17:18.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:18 smithi067 ceph-mon[32814]: pgmap v452: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:17:18.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:18 smithi149 ceph-mon[37173]: pgmap v452: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:17:18.876 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:18 smithi175 ceph-mon[37125]: pgmap v452: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:17:20.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:20 smithi067 ceph-mon[32814]: pgmap v453: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:20.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:20 smithi149 ceph-mon[37173]: pgmap v453: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:20.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:20 smithi175 ceph-mon[37125]: pgmap v453: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:22.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:22 smithi067 ceph-mon[32814]: pgmap v454: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-07-27T03:17:22.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:22 smithi149 ceph-mon[37173]: pgmap v454: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-07-27T03:17:22.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:22 smithi175 ceph-mon[37125]: pgmap v454: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-07-27T03:17:24.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:24 smithi067 ceph-mon[32814]: pgmap v455: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:24.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:24 smithi149 ceph-mon[37173]: pgmap v455: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:24.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:24 smithi175 ceph-mon[37125]: pgmap v455: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:26.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:26 smithi067 ceph-mon[32814]: pgmap v456: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:17:26.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:26 smithi149 ceph-mon[37173]: pgmap v456: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:17:26.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:26 smithi175 ceph-mon[37125]: pgmap v456: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:17:27.571 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-27T03:17:27.861 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:17:28.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:28 smithi067 ceph-mon[32814]: pgmap v457: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:17:28.749 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:28 smithi149 ceph-mon[37173]: pgmap v457: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:17:28.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:28 smithi175 ceph-mon[37125]: pgmap v457: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:17:29.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.706+0000 7f29e6232640 1 -- 172.21.15.67:0/2021171520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29e0108f50 msgr2=0x7f29e0109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:29.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.706+0000 7f29e6232640 1 --2- 172.21.15.67:0/2021171520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29e0108f50 0x7f29e0109360 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009900 tx=0x7f29d402a410 comp rx=0 tx=0).stop 2024-07-27T03:17:29.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.706+0000 7f29e6232640 1 -- 172.21.15.67:0/2021171520 shutdown_connections 2024-07-27T03:17:29.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.706+0000 7f29e6232640 1 --2- 172.21.15.67:0/2021171520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29e01098a0 0x7f29e01100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.706+0000 7f29e6232640 1 --2- 172.21.15.67:0/2021171520 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29e0108f50 0x7f29e0109360 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.706+0000 7f29e6232640 1 --2- 172.21.15.67:0/2021171520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29e0103610 0x7f29e0103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.706+0000 7f29e6232640 1 -- 172.21.15.67:0/2021171520 >> 172.21.15.67:0/2021171520 conn(0x7f29e00ff0a0 msgr2=0x7f29e01014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:29.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.706+0000 7f29e6232640 1 -- 172.21.15.67:0/2021171520 shutdown_connections 2024-07-27T03:17:29.710 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.706+0000 7f29e6232640 1 -- 172.21.15.67:0/2021171520 wait complete. 2024-07-27T03:17:29.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.707+0000 7f29e6232640 1 Processor -- start 2024-07-27T03:17:29.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.707+0000 7f29e6232640 1 -- start start 2024-07-27T03:17:29.711 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.708+0000 7f29e6232640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29e0103610 0x7f29e019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:29.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.708+0000 7f29df7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29e0103610 0x7f29e019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:29.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.708+0000 7f29df7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29e0103610 0x7f29e019a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33998/0 (socket says 172.21.15.67:33998) 2024-07-27T03:17:29.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.708+0000 7f29df7fe640 1 -- 172.21.15.67:0/530310809 learned_addr learned my addr 172.21.15.67:0/530310809 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:29.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.708+0000 7f29e6232640 1 --2- 172.21.15.67:0/530310809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29e0108f50 0x7f29e019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:29.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.708+0000 7f29e6232640 1 --2- 172.21.15.67:0/530310809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29e01098a0 0x7f29e019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:29.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.708+0000 7f29e6232640 1 -- 172.21.15.67:0/530310809 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f29e0113510 con 0x7f29e0103610 2024-07-27T03:17:29.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.708+0000 7f29e6232640 1 -- 172.21.15.67:0/530310809 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f29e0113390 con 0x7f29e01098a0 2024-07-27T03:17:29.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.708+0000 7f29e6232640 1 -- 172.21.15.67:0/530310809 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29e0113690 con 0x7f29e0108f50 2024-07-27T03:17:29.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.708+0000 7f29deffd640 1 --2- 172.21.15.67:0/530310809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29e0108f50 0x7f29e019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:29.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.709+0000 7f29dffff640 1 --2- 172.21.15.67:0/530310809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29e01098a0 0x7f29e019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:29.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.709+0000 7f29df7fe640 1 -- 172.21.15.67:0/530310809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29e0108f50 msgr2=0x7f29e019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:29.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.709+0000 7f29df7fe640 1 --2- 172.21.15.67:0/530310809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29e0108f50 0x7f29e019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.709+0000 7f29df7fe640 1 -- 172.21.15.67:0/530310809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29e01098a0 msgr2=0x7f29e019f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:29.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.709+0000 7f29df7fe640 1 --2- 172.21.15.67:0/530310809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29e01098a0 0x7f29e019f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.709+0000 7f29df7fe640 1 -- 172.21.15.67:0/530310809 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e019fed0 con 0x7f29e0103610 2024-07-27T03:17:29.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.709+0000 7f29df7fe640 1 --2- 172.21.15.67:0/530310809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29e0103610 0x7f29e019a020 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f29d000eab0 tx=0x7f29d000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:29.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.709+0000 7f29dcff9640 1 -- 172.21.15.67:0/530310809 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d000ce40 con 0x7f29e0103610 2024-07-27T03:17:29.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.710+0000 7f29dcff9640 1 -- 172.21.15.67:0/530310809 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d0011920 con 0x7f29e0103610 2024-07-27T03:17:29.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.710+0000 7f29e6232640 1 -- 172.21.15.67:0/530310809 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e01a01c0 con 0x7f29e0103610 2024-07-27T03:17:29.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.710+0000 7f29e6232640 1 -- 172.21.15.67:0/530310809 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e0075060 con 0x7f29e0103610 2024-07-27T03:17:29.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.711+0000 7f29dcff9640 1 -- 172.21.15.67:0/530310809 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d0010a30 con 0x7f29e0103610 2024-07-27T03:17:29.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.713+0000 7f29ec7b7740 1 Processor -- start 2024-07-27T03:17:29.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.713+0000 7f29ec7b7740 1 -- start start 2024-07-27T03:17:29.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.713+0000 7f29ec7b7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e9d070 0x557147241d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:29.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.713+0000 7f29ec7b7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557146e741b0 0x557147242260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:29.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.714+0000 7f29ec7b7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e7c390 0x5571472427a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:29.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.714+0000 7f29ec7b7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55714712a3e0 con 0x557146e9d070 2024-07-27T03:17:29.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.714+0000 7f29ec7b7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5571471b8b00 con 0x557146e741b0 2024-07-27T03:17:29.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.714+0000 7f29ec7b7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5571471a8000 con 0x557146e7c390 2024-07-27T03:17:29.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.714+0000 7f29dffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e9d070 0x557147241d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:29.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.714+0000 7f29dffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e9d070 0x557147241d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34000/0 (socket says 172.21.15.67:34000) 2024-07-27T03:17:29.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.714+0000 7f29dffff640 1 -- 172.21.15.67:0/2553082295 learned_addr learned my addr 172.21.15.67:0/2553082295 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:29.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.714+0000 7f29deffd640 1 --2- 172.21.15.67:0/2553082295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e7c390 0x5571472427a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:29.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.714+0000 7f29dffff640 1 -- 172.21.15.67:0/2553082295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e7c390 msgr2=0x5571472427a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:29.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.714+0000 7f29dffff640 1 --2- 172.21.15.67:0/2553082295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e7c390 0x5571472427a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.714+0000 7f29dffff640 1 -- 172.21.15.67:0/2553082295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557146e741b0 msgr2=0x557147242260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-27T03:17:29.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.714+0000 7f29dffff640 1 --2- 172.21.15.67:0/2553082295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557146e741b0 0x557147242260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.714+0000 7f29dffff640 1 -- 172.21.15.67:0/2553082295 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55714713d750 con 0x557146e9d070 2024-07-27T03:17:29.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.715+0000 7f29dcff9640 1 -- 172.21.15.67:0/530310809 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f29d0010bd0 con 0x7f29e0103610 2024-07-27T03:17:29.720 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.715+0000 7f29dffff640 1 --2- 172.21.15.67:0/2553082295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e9d070 0x557147241d20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f29d800cb40 tx=0x7f29d800e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a052ed2ac75de56 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:29.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.715+0000 7f29c5ffb640 1 -- 172.21.15.67:0/2553082295 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8005020 con 0x557146e9d070 2024-07-27T03:17:29.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.715+0000 7f29c5ffb640 1 -- 172.21.15.67:0/2553082295 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d80047a0 con 0x557146e9d070 2024-07-27T03:17:29.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.715+0000 7f29c5ffb640 1 -- 172.21.15.67:0/2553082295 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8010690 con 0x557146e9d070 2024-07-27T03:17:29.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.716+0000 7f29dcff9640 1 --2- 172.21.15.67:0/530310809 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f29c80b3d00 0x7f29c80b6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:29.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.716+0000 7f29ec7b7740 1 -- 172.21.15.67:0/2553082295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e9d070 msgr2=0x557147241d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:29.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.716+0000 7f29ec7b7740 1 --2- 172.21.15.67:0/2553082295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e9d070 0x557147241d20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f29d800cb40 tx=0x7f29d800e350 comp rx=0 tx=0).stop 2024-07-27T03:17:29.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.716+0000 7f29dcff9640 1 -- 172.21.15.67:0/530310809 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f29d00e45d0 con 0x7f29e0103610 2024-07-27T03:17:29.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.716+0000 7f29df7fe640 1 --2- 172.21.15.67:0/530310809 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f29c80b3d00 0x7f29c80b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:29.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.716+0000 7f29ec7b7740 1 -- 172.21.15.67:0/2553082295 shutdown_connections 2024-07-27T03:17:29.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.716+0000 7f29ec7b7740 1 --2- 172.21.15.67:0/2553082295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e7c390 0x5571472427a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.716+0000 7f29ec7b7740 1 --2- 172.21.15.67:0/2553082295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557146e741b0 0x557147242260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.717+0000 7f29ec7b7740 1 --2- 172.21.15.67:0/2553082295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e9d070 0x557147241d20 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.721 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.717+0000 7f29ec7b7740 1 -- 172.21.15.67:0/2553082295 >> 172.21.15.67:0/2553082295 conn(0x557146e91460 msgr2=0x55714723fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:29.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.717+0000 7f29df7fe640 1 --2- 172.21.15.67:0/530310809 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f29c80b3d00 0x7f29c80b6210 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f29d00aae10 tx=0x7f29d00104d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:29.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.717+0000 7f29ec7b7740 1 -- 172.21.15.67:0/2553082295 shutdown_connections 2024-07-27T03:17:29.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.717+0000 7f29ec7b7740 1 -- 172.21.15.67:0/2553082295 wait complete. 2024-07-27T03:17:29.722 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.718+0000 7f29ec7b7740 1 -- start start 2024-07-27T03:17:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.718+0000 7f29ec7b7740 1 Processor -- start 2024-07-27T03:17:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.719+0000 7f29ec7b7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557146e741b0 0x557147256110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.719+0000 7f29ec7b7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e7c390 0x557147256670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.719+0000 7f29ec7b7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e9d070 0x557147261e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:29.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.719+0000 7f29ec7b7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55714714a8e0 con 0x557146e9d070 2024-07-27T03:17:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.719+0000 7f29ec7b7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5571471ac040 con 0x557146e741b0 2024-07-27T03:17:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.719+0000 7f29ec7b7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55714700b700 con 0x557146e7c390 2024-07-27T03:17:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.719+0000 7f29ec7b7740 10 client.0 _ll_register_callbacks cb 0x557147152d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:17:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.719+0000 7f29dffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e9d070 0x557147261e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.719+0000 7f29dffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e9d070 0x557147261e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34006/0 (socket says 172.21.15.67:34006) 2024-07-27T03:17:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.719+0000 7f29dffff640 1 -- 172.21.15.67:0/3372146197 learned_addr learned my addr 172.21.15.67:0/3372146197 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.719+0000 7f29deffd640 1 --2- 172.21.15.67:0/3372146197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e7c390 0x557147256670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.719+0000 7f29dffff640 1 --2- 172.21.15.67:0/3372146197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557146e741b0 0x557147256110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.719+0000 7f29dffff640 1 -- 172.21.15.67:0/3372146197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e7c390 msgr2=0x557147256670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.719+0000 7f29dffff640 1 --2- 172.21.15.67:0/3372146197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e7c390 0x557147256670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.719+0000 7f29dffff640 1 -- 172.21.15.67:0/3372146197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e9d070 msgr2=0x557147261e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.719+0000 7f29dffff640 1 --2- 172.21.15.67:0/3372146197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e9d070 0x557147261e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.719+0000 7f29dffff640 1 -- 172.21.15.67:0/3372146197 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55714714ebd0 con 0x557146e741b0 2024-07-27T03:17:29.725 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.720+0000 7f29dffff640 1 --2- 172.21.15.67:0/3372146197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557146e741b0 0x557147256110 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f29d8005940 tx=0x7f29d800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=146380c24fa7164a server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:29.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.720+0000 7f29beffd640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-07-27T03:17:29.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.720+0000 7f29beffd640 1 -- 172.21.15.67:0/3372146197 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8023040 con 0x557146e741b0 2024-07-27T03:17:29.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.720+0000 7f29ec7b7740 1 -- 172.21.15.67:0/3372146197 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29d80047a0 con 0x557146e741b0 2024-07-27T03:17:29.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.720+0000 7f29beffd640 1 -- 172.21.15.67:0/3372146197 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f29d8011d30 con 0x557146e741b0 2024-07-27T03:17:29.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.720+0000 7f29beffd640 1 -- 172.21.15.67:0/3372146197 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f29d80106d0 con 0x557146e741b0 2024-07-27T03:17:29.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.721+0000 7f29ec7b7740 10 client.25202 fetch_fsmap learned FSMap version 30 2024-07-27T03:17:29.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.721+0000 7f29ec7b7740 1 -- 172.21.15.67:0/3372146197 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557146f08650 con 0x557146e741b0 2024-07-27T03:17:29.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.721+0000 7f29beffd640 1 -- 172.21.15.67:0/3372146197 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d80047a0 con 0x557146e741b0 2024-07-27T03:17:29.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.721+0000 7f29beffd640 1 -- 172.21.15.67:0/3372146197 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f29d802a6a0 con 0x557146e741b0 2024-07-27T03:17:29.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.721+0000 7f29ec7b7740 10 client.25202 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:17:29.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.721+0000 7f29ec7b7740 10 client.25202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:17:29.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.721+0000 7f29ec7b7740 20 client.25202 populate_metadata read hostname 'smithi067' 2024-07-27T03:17:29.726 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.722+0000 7f29ec7b7740 1 --2- 172.21.15.67:0/3372146197 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x557146e5b9f0 0x5571472682d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:29.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.722+0000 7f29ec7b7740 4 client.25202 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:17:29.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.722+0000 7f29ec7b7740 1 -- 172.21.15.67:0/3372146197 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571471baa50 con 0x557146e5b9f0 2024-07-27T03:17:29.727 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.722+0000 7f29deffd640 1 --2- 172.21.15.67:0/3372146197 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x557146e5b9f0 0x5571472682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:29.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.723+0000 7f29deffd640 1 --2- 172.21.15.67:0/3372146197 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x557146e5b9f0 0x5571472682d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad4d9976cdf9005b server_cookie=9716ec8e67afb337 in_seq=0 out_seq=0 2024-07-27T03:17:29.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.723+0000 7f29beffd640 10 client.25202 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:17:29.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.723+0000 7f29beffd640 1 -- 172.21.15.67:0/3372146197 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571471baa50 con 0x557146e5b9f0 2024-07-27T03:17:29.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.723+0000 7f29beffd640 10 client.25202 handle_command_reply: tid=0 2024-07-27T03:17:29.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.724+0000 7f29ec7b7740 1 client.25202 shutdown 2024-07-27T03:17:29.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.724+0000 7f29ec7b7740 1 -- 172.21.15.67:0/3372146197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557146e741b0 msgr2=0x557147256110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:29.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.724+0000 7f29ec7b7740 1 --2- 172.21.15.67:0/3372146197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557146e741b0 0x557147256110 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f29d8005940 tx=0x7f29d800c6a0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.724+0000 7f29ec7b7740 1 -- 172.21.15.67:0/3372146197 shutdown_connections 2024-07-27T03:17:29.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.724+0000 7f29ec7b7740 1 --2- 172.21.15.67:0/3372146197 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x557146e5b9f0 0x5571472682d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.724+0000 7f29ec7b7740 1 --2- 172.21.15.67:0/3372146197 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e9d070 0x557147261e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.724+0000 7f29ec7b7740 1 --2- 172.21.15.67:0/3372146197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e7c390 0x557147256670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.724+0000 7f29beffd640 0 client.25202 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:17:29.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.724+0000 7f29ec7b7740 1 --2- 172.21.15.67:0/3372146197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557146e741b0 0x557147256110 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.724+0000 7f29ec7b7740 1 -- 172.21.15.67:0/3372146197 >> 172.21.15.67:0/3372146197 conn(0x557146e91460 msgr2=0x55714723e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:29.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.725+0000 7f29ec7b7740 1 -- 172.21.15.67:0/3372146197 shutdown_connections 2024-07-27T03:17:29.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.725+0000 7f29ec7b7740 1 -- 172.21.15.67:0/3372146197 wait complete. 2024-07-27T03:17:29.729 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.725+0000 7f29ec7b7740 20 client.25202 trim_cache size 0 max 16384 2024-07-27T03:17:29.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.758+0000 7f29ec7b7740 1 Processor -- start 2024-07-27T03:17:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.758+0000 7f29ec7b7740 1 -- start start 2024-07-27T03:17:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.759+0000 7f29ec7b7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e5b9f0 0x557147257df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.759+0000 7f29ec7b7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557146e741b0 0x557147243460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.759+0000 7f29ec7b7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e7c390 0x557147261c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.759+0000 7f29ec7b7740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01ae3e0 con 0x557146e7c390 2024-07-27T03:17:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.759+0000 7f29ec7b7740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01aff20 con 0x557146e741b0 2024-07-27T03:17:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.759+0000 7f29ec7b7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01aeab0 con 0x557146e5b9f0 2024-07-27T03:17:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.759+0000 7f29dffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e5b9f0 0x557147257df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.759+0000 7f29dffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e7c390 0x557147261c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.759+0000 7f29dffff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e5b9f0 0x557147257df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.67:42088/0 (socket says 172.21.15.67:42088) 2024-07-27T03:17:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.759+0000 7f29dffff640 1 -- 172.21.15.67:0/602684848 learned_addr learned my addr 172.21.15.67:0/602684848 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:29.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.759+0000 7f29deffd640 1 --2- 172.21.15.67:0/602684848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557146e741b0 0x557147243460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.759+0000 7f29dffff640 1 -- 172.21.15.67:0/602684848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557146e741b0 msgr2=0x557147243460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.759+0000 7f29dffff640 1 --2- 172.21.15.67:0/602684848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557146e741b0 0x557147243460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.759+0000 7f29dffff640 1 -- 172.21.15.67:0/602684848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e7c390 msgr2=0x557147261c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.759+0000 7f29dffff640 1 --2- 172.21.15.67:0/602684848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e7c390 0x557147261c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.759+0000 7f29dffff640 1 -- 172.21.15.67:0/602684848 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557147239da0 con 0x557146e5b9f0 2024-07-27T03:17:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.760+0000 7f29dffff640 1 --2- 172.21.15.67:0/602684848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e5b9f0 0x557147257df0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f29d802aaf0 tx=0x7f29d800e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b405a840eec763b server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.760+0000 7f29c57fa640 1 -- 172.21.15.67:0/602684848 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8040070 con 0x557146e5b9f0 2024-07-27T03:17:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.760+0000 7f29c57fa640 1 -- 172.21.15.67:0/602684848 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d801e070 con 0x557146e5b9f0 2024-07-27T03:17:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.760+0000 7f29c57fa640 1 -- 172.21.15.67:0/602684848 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d802a690 con 0x557146e5b9f0 2024-07-27T03:17:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.761+0000 7f29ec7b7740 1 -- 172.21.15.67:0/602684848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e5b9f0 msgr2=0x557147257df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:29.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.761+0000 7f29ec7b7740 1 --2- 172.21.15.67:0/602684848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e5b9f0 0x557147257df0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f29d802aaf0 tx=0x7f29d800e350 comp rx=0 tx=0).stop 2024-07-27T03:17:29.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.761+0000 7f29ec7b7740 1 -- 172.21.15.67:0/602684848 shutdown_connections 2024-07-27T03:17:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.762+0000 7f29ec7b7740 1 --2- 172.21.15.67:0/602684848 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e7c390 0x557147261c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.762+0000 7f29ec7b7740 1 --2- 172.21.15.67:0/602684848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557146e741b0 0x557147243460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.762+0000 7f29ec7b7740 1 --2- 172.21.15.67:0/602684848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e5b9f0 0x557147257df0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.762+0000 7f29ec7b7740 1 -- 172.21.15.67:0/602684848 >> 172.21.15.67:0/602684848 conn(0x557146e91460 msgr2=0x557147242d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.762+0000 7f29ec7b7740 1 -- 172.21.15.67:0/602684848 shutdown_connections 2024-07-27T03:17:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.762+0000 7f29ec7b7740 1 -- 172.21.15.67:0/602684848 wait complete. 2024-07-27T03:17:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.762+0000 7f29ec7b7740 1 -- start start 2024-07-27T03:17:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.763+0000 7f29ec7b7740 1 Processor -- start 2024-07-27T03:17:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.764+0000 7f29ec7b7740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557146e5b9f0 0x557147258e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.764+0000 7f29ec7b7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e741b0 0x557147259380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.764+0000 7f29ec7b7740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e7c390 0x5571472598e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:29.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.764+0000 7f29dffff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557146e5b9f0 0x557147258e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.764+0000 7f29dffff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557146e5b9f0 0x557147258e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.67:50114/0 (socket says 172.21.15.67:50114) 2024-07-27T03:17:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.764+0000 7f29deffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e741b0 0x557147259380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.764+0000 7f29deffd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e741b0 0x557147259380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.67:42092/0 (socket says 172.21.15.67:42092) 2024-07-27T03:17:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.764+0000 7f29dffff640 1 -- 172.21.15.67:0/2001960373 learned_addr learned my addr 172.21.15.67:0/2001960373 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.764+0000 7f29ec7b7740 1 -- 172.21.15.67:0/2001960373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01af850 con 0x557146e7c390 2024-07-27T03:17:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.764+0000 7f29ec7b7740 1 -- 172.21.15.67:0/2001960373 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01af180 con 0x557146e5b9f0 2024-07-27T03:17:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.764+0000 7f29ec7b7740 1 -- 172.21.15.67:0/2001960373 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01ac3c0 con 0x557146e741b0 2024-07-27T03:17:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.764+0000 7f29dffff640 1 --2- 172.21.15.67:0/2001960373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e7c390 0x5571472598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.764+0000 7f29dffff640 1 -- 172.21.15.67:0/2001960373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e741b0 msgr2=0x557147259380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.764+0000 7f29dffff640 1 --2- 172.21.15.67:0/2001960373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e741b0 0x557147259380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.764+0000 7f29dffff640 1 -- 172.21.15.67:0/2001960373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e7c390 msgr2=0x5571472598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.764+0000 7f29dffff640 1 --2- 172.21.15.67:0/2001960373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e7c390 0x5571472598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.764+0000 7f29dffff640 1 -- 172.21.15.67:0/2001960373 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571472407d0 con 0x557146e5b9f0 2024-07-27T03:17:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.765+0000 7f29ec7b7740 10 client.0 _ll_register_callbacks cb 0x7f29d8014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:17:29.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.765+0000 7f29dffff640 1 --2- 172.21.15.67:0/2001960373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557146e5b9f0 0x557147258e40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f29d800ea30 tx=0x7f29d800c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94f4eaa862e2728b server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:29.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.765+0000 7f29beffd640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-07-27T03:17:29.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.765+0000 7f29ec7b7740 1 -- 172.21.15.67:0/2001960373 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29d801e070 con 0x557146e5b9f0 2024-07-27T03:17:29.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.765+0000 7f29beffd640 1 -- 172.21.15.67:0/2001960373 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8040040 con 0x557146e5b9f0 2024-07-27T03:17:29.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.765+0000 7f29beffd640 1 -- 172.21.15.67:0/2001960373 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f29d8011b10 con 0x557146e5b9f0 2024-07-27T03:17:29.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.765+0000 7f29beffd640 1 -- 172.21.15.67:0/2001960373 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f29d8011d10 con 0x557146e5b9f0 2024-07-27T03:17:29.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.765+0000 7f29ec7b7740 10 client.25208 fetch_fsmap learned FSMap version 30 2024-07-27T03:17:29.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.765+0000 7f29ec7b7740 1 -- 172.21.15.67:0/2001960373 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5571472cfa70 con 0x557146e5b9f0 2024-07-27T03:17:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.766+0000 7f29beffd640 1 -- 172.21.15.67:0/2001960373 <== mon.2 v2:172.21.15.149:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f29d8046070 con 0x557146e5b9f0 2024-07-27T03:17:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.766+0000 7f29ec7b7740 10 client.25208 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:17:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.766+0000 7f29ec7b7740 10 client.25208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:17:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.766+0000 7f29ec7b7740 20 client.25208 populate_metadata read hostname 'smithi067' 2024-07-27T03:17:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.766+0000 7f29ec7b7740 1 --2- 172.21.15.67:0/2001960373 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x557146e9d070 0x5571472d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.766+0000 7f29ec7b7740 4 client.25208 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-27T03:17:29.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.766+0000 7f29ec7b7740 1 -- 172.21.15.67:0/2001960373 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f29d801e070 con 0x557146e9d070 2024-07-27T03:17:29.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.766+0000 7f29deffd640 1 --2- 172.21.15.67:0/2001960373 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x557146e9d070 0x5571472d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:29.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.767+0000 7f29beffd640 10 client.25208 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:17:29.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.767+0000 7f29deffd640 1 --2- 172.21.15.67:0/2001960373 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x557146e9d070 0x5571472d2120 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac26e7802bb828c6 server_cookie=87e89ff9531cffed in_seq=0 out_seq=0 2024-07-27T03:17:29.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.767+0000 7f29beffd640 1 -- 172.21.15.67:0/2001960373 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f29d801e070 con 0x557146e9d070 2024-07-27T03:17:29.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.767+0000 7f29beffd640 10 client.25208 handle_command_reply: tid=0 2024-07-27T03:17:29.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.767+0000 7f29ec7b7740 1 client.25208 shutdown 2024-07-27T03:17:29.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.767+0000 7f29ec7b7740 1 -- 172.21.15.67:0/2001960373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557146e5b9f0 msgr2=0x557147258e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:29.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.767+0000 7f29ec7b7740 1 --2- 172.21.15.67:0/2001960373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557146e5b9f0 0x557147258e40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f29d800ea30 tx=0x7f29d800c440 comp rx=0 tx=0).stop 2024-07-27T03:17:29.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.768+0000 7f29ec7b7740 1 -- 172.21.15.67:0/2001960373 shutdown_connections 2024-07-27T03:17:29.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.768+0000 7f29ec7b7740 1 --2- 172.21.15.67:0/2001960373 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x557146e9d070 0x5571472d2120 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.768+0000 7f29ec7b7740 1 --2- 172.21.15.67:0/2001960373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557146e7c390 0x5571472598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.768+0000 7f29ec7b7740 1 --2- 172.21.15.67:0/2001960373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557146e741b0 0x557147259380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.768+0000 7f29ec7b7740 1 --2- 172.21.15.67:0/2001960373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557146e5b9f0 0x557147258e40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.768+0000 7f29ec7b7740 1 -- 172.21.15.67:0/2001960373 >> 172.21.15.67:0/2001960373 conn(0x557146e91460 msgr2=0x557147257bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:29.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.768+0000 7f29beffd640 0 client.25208 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:17:29.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.768+0000 7f29ec7b7740 1 -- 172.21.15.67:0/2001960373 shutdown_connections 2024-07-27T03:17:29.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.768+0000 7f29ec7b7740 1 -- 172.21.15.67:0/2001960373 wait complete. 2024-07-27T03:17:29.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.768+0000 7f29ec7b7740 20 client.25208 trim_cache size 0 max 16384 2024-07-27T03:17:29.774 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:17:29.774 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-27T03:17:29.774 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-27T03:17:29.774 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:17:29.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.769+0000 7f29e6232640 1 -- 172.21.15.67:0/530310809 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f29c80b3d00 msgr2=0x7f29c80b6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:29.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.769+0000 7f29e6232640 1 --2- 172.21.15.67:0/530310809 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f29c80b3d00 0x7f29c80b6210 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f29d00aae10 tx=0x7f29d00104d0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.769+0000 7f29e6232640 1 -- 172.21.15.67:0/530310809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29e0103610 msgr2=0x7f29e019a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:29.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.769+0000 7f29e6232640 1 --2- 172.21.15.67:0/530310809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29e0103610 0x7f29e019a020 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f29d000eab0 tx=0x7f29d000ef70 comp rx=0 tx=0).stop 2024-07-27T03:17:29.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.769+0000 7f29e6232640 1 -- 172.21.15.67:0/530310809 shutdown_connections 2024-07-27T03:17:29.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.769+0000 7f29e6232640 1 --2- 172.21.15.67:0/530310809 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f29c80b3d00 0x7f29c80b6210 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.769+0000 7f29e6232640 1 --2- 172.21.15.67:0/530310809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29e01098a0 0x7f29e019f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.769+0000 7f29e6232640 1 --2- 172.21.15.67:0/530310809 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29e0108f50 0x7f29e019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:29.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.769+0000 7f29e6232640 1 --2- 172.21.15.67:0/530310809 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f29e0103610 0x7f29e019a020 secure :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f29d000eab0 tx=0x7f29d000ef70 comp rx=0 tx=0).stop 2024-07-27T03:17:29.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.769+0000 7f29e6232640 1 -- 172.21.15.67:0/530310809 >> 172.21.15.67:0/530310809 conn(0x7f29e00ff0a0 msgr2=0x7f29e01014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:29.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.769+0000 7f29e6232640 1 -- 172.21.15.67:0/530310809 shutdown_connections 2024-07-27T03:17:29.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:29.769+0000 7f29e6232640 1 -- 172.21.15.67:0/530310809 wait complete. 2024-07-27T03:17:30.553 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-27T03:17:30.554 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:17:30.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:30 smithi067 ceph-mon[32814]: pgmap v458: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:30.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:30 smithi149 ceph-mon[37173]: pgmap v458: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:30.847 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:17:30.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:30 smithi175 ceph-mon[37125]: pgmap v458: 129 pgs: 129 active+clean; 657 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:32.603 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:32 smithi067 ceph-mon[32814]: pgmap v459: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-07-27T03:17:32.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.745+0000 7fa51fff5640 1 -- 172.21.15.67:0/2133900643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa518108f20 msgr2=0x7fa518109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:32.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.745+0000 7fa51fff5640 1 --2- 172.21.15.67:0/2133900643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa518108f20 0x7fa518109330 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa508009900 tx=0x7fa50802a3f0 comp rx=0 tx=0).stop 2024-07-27T03:17:32.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.746+0000 7fa51fff5640 1 -- 172.21.15.67:0/2133900643 shutdown_connections 2024-07-27T03:17:32.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.746+0000 7fa51fff5640 1 --2- 172.21.15.67:0/2133900643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa518109870 0x7fa5181100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:32.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.746+0000 7fa51fff5640 1 --2- 172.21.15.67:0/2133900643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa518108f20 0x7fa518109330 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:32.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.746+0000 7fa51fff5640 1 --2- 172.21.15.67:0/2133900643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5181019e0 0x7fa518101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:32.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.746+0000 7fa51fff5640 1 -- 172.21.15.67:0/2133900643 >> 172.21.15.67:0/2133900643 conn(0x7fa5180776d0 msgr2=0x7fa518077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:32.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.746+0000 7fa51fff5640 1 -- 172.21.15.67:0/2133900643 shutdown_connections 2024-07-27T03:17:32.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.746+0000 7fa51fff5640 1 -- 172.21.15.67:0/2133900643 wait complete. 2024-07-27T03:17:32.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:32 smithi149 ceph-mon[37173]: pgmap v459: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-07-27T03:17:32.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.747+0000 7fa51fff5640 1 Processor -- start 2024-07-27T03:17:32.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.747+0000 7fa51fff5640 1 -- start start 2024-07-27T03:17:32.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.747+0000 7fa51fff5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5181019e0 0x7fa518102980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:32.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.747+0000 7fa51fff5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa518108f20 0x7fa518102ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:32.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.748+0000 7fa51fff5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa518109870 0x7fa518103400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:32.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.748+0000 7fa51fff5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa518113530 con 0x7fa5181019e0 2024-07-27T03:17:32.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.748+0000 7fa51d590640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa518108f20 0x7fa518102ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:32.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.748+0000 7fa51d590640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa518108f20 0x7fa518102ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.67:42112/0 (socket says 172.21.15.67:42112) 2024-07-27T03:17:32.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.748+0000 7fa51dd91640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5181019e0 0x7fa518102980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:32.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.748+0000 7fa51d590640 1 -- 172.21.15.67:0/2856476139 learned_addr learned my addr 172.21.15.67:0/2856476139 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:32.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.748+0000 7fa51e592640 1 --2- 172.21.15.67:0/2856476139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa518109870 0x7fa518103400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:32.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.748+0000 7fa51fff5640 1 -- 172.21.15.67:0/2856476139 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa5181133b0 con 0x7fa518109870 2024-07-27T03:17:32.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.748+0000 7fa51fff5640 1 -- 172.21.15.67:0/2856476139 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa5181136b0 con 0x7fa518108f20 2024-07-27T03:17:32.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.748+0000 7fa51d590640 1 -- 172.21.15.67:0/2856476139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa518109870 msgr2=0x7fa518103400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:32.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.748+0000 7fa51d590640 1 --2- 172.21.15.67:0/2856476139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa518109870 0x7fa518103400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:32.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.748+0000 7fa51d590640 1 -- 172.21.15.67:0/2856476139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5181019e0 msgr2=0x7fa518102980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:32.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.748+0000 7fa51d590640 1 --2- 172.21.15.67:0/2856476139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5181019e0 0x7fa518102980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:32.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.748+0000 7fa51d590640 1 -- 172.21.15.67:0/2856476139 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5181a7fc0 con 0x7fa518108f20 2024-07-27T03:17:32.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.749+0000 7fa51d590640 1 --2- 172.21.15.67:0/2856476139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa518108f20 0x7fa518102ec0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa50802a9e0 tx=0x7fa5080305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:32.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.749+0000 7fa51dd91640 1 --2- 172.21.15.67:0/2856476139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5181019e0 0x7fa518102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:17:32.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.749+0000 7fa50effd640 1 -- 172.21.15.67:0/2856476139 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50803f030 con 0x7fa518108f20 2024-07-27T03:17:32.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.749+0000 7fa50effd640 1 -- 172.21.15.67:0/2856476139 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa508030e00 con 0x7fa518108f20 2024-07-27T03:17:32.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.749+0000 7fa50effd640 1 -- 172.21.15.67:0/2856476139 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508035070 con 0x7fa518108f20 2024-07-27T03:17:32.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.749+0000 7fa51fff5640 1 -- 172.21.15.67:0/2856476139 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5181a8250 con 0x7fa518108f20 2024-07-27T03:17:32.754 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.750+0000 7fa51fff5640 1 -- 172.21.15.67:0/2856476139 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa518199be0 con 0x7fa518108f20 2024-07-27T03:17:32.755 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.751+0000 7fa51fff5640 1 -- 172.21.15.67:0/2856476139 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa518103c80 con 0x7fa518108f20 2024-07-27T03:17:32.756 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.753+0000 7fa50effd640 1 -- 172.21.15.67:0/2856476139 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7fa508042030 con 0x7fa518108f20 2024-07-27T03:17:32.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.755+0000 7fa50effd640 1 --2- 172.21.15.67:0/2856476139 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa5040b3c10 0x7fa5040b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:32.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.755+0000 7fa51dd91640 1 --2- 172.21.15.67:0/2856476139 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa5040b3c10 0x7fa5040b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:32.757 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.755+0000 7fa50effd640 1 -- 172.21.15.67:0/2856476139 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fa508104aa0 con 0x7fa518108f20 2024-07-27T03:17:32.758 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.756+0000 7fa51dd91640 1 --2- 172.21.15.67:0/2856476139 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa5040b3c10 0x7fa5040b6120 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa5181043d0 tx=0x7fa514005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:32.759 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:32.757+0000 7fa50effd640 1 -- 172.21.15.67:0/2856476139 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fa50803b460 con 0x7fa518108f20 2024-07-27T03:17:32.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:32 smithi175 ceph-mon[37125]: pgmap v459: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-07-27T03:17:33.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:33.209+0000 7fa51fff5640 1 -- 172.21.15.67:0/2856476139 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa518199ea0 con 0x7fa518108f20 2024-07-27T03:17:33.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:33.210+0000 7fa50effd640 1 -- 172.21.15.67:0/2856476139 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11983 (secure 0 0 0) 0x7fa5080cb690 con 0x7fa518108f20 2024-07-27T03:17:33.213 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:17:33.213 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"btime":"2024-07-27T03:16:40:350577+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14679,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/252686369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":252686369},{"type":"v1","addr":"172.21.15.67:6835","nonce":252686369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/3358832762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3358832762},{"type":"v1","addr":"172.21.15.67:6837","nonce":3358832762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-27T03:12:34.320208+0000","modified":"2024-07-27T03:16:39.741149+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24620,"mds_1":24587,"mds_2":24583,"mds_3":24581,"mds_4":24563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14736":{"gid":14736,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/2162708317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2162708317},{"type":"v1","addr":"172.21.15.67:6839","nonce":2162708317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14748":{"gid":14748,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/795704244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":795704244},{"type":"v1","addr":"172.21.15.67:6841","nonce":795704244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/3868325215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3868325215},{"type":"v1","addr":"172.21.15.149:6833","nonce":3868325215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2462749027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2462749027},{"type":"v1","addr":"172.21.15.175:6835","nonce":2462749027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.149:6835/3730964806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3730964806},{"type":"v1","addr":"172.21.15.149:6835","nonce":3730964806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.175:6837/1304803502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1304803502},{"type":"v1","addr":"172.21.15.175:6837","nonce":1304803502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.175:6839/1670568853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1670568853},{"type":"v1","addr":"172.21.15.175:6839","nonce":1670568853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24587":{"gid":24587,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.149:6837/2932951845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2932951845},{"type":"v1","addr":"172.21.15.149:6837","nonce":2932951845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.149:6839/281746498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":281746498},{"type":"v1","addr":"172.21.15.149:6839","nonce":281746498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24620,"qdb_cluster":[24587,24583,24620,24581,24563]},"id":1}]} 2024-07-27T03:17:33.213 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 30 2024-07-27T03:17:33.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:33.213+0000 7fa51fff5640 1 -- 172.21.15.67:0/2856476139 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa5040b3c10 msgr2=0x7fa5040b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:33.213+0000 7fa51fff5640 1 --2- 172.21.15.67:0/2856476139 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa5040b3c10 0x7fa5040b6120 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa5181043d0 tx=0x7fa514005f90 comp rx=0 tx=0).stop 2024-07-27T03:17:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:33.213+0000 7fa51fff5640 1 -- 172.21.15.67:0/2856476139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa518108f20 msgr2=0x7fa518102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:33.213+0000 7fa51fff5640 1 --2- 172.21.15.67:0/2856476139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa518108f20 0x7fa518102ec0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa50802a9e0 tx=0x7fa5080305d0 comp rx=0 tx=0).stop 2024-07-27T03:17:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:33.213+0000 7fa51fff5640 1 -- 172.21.15.67:0/2856476139 shutdown_connections 2024-07-27T03:17:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:33.213+0000 7fa51fff5640 1 --2- 172.21.15.67:0/2856476139 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa5040b3c10 0x7fa5040b6120 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:33.213+0000 7fa51fff5640 1 --2- 172.21.15.67:0/2856476139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa518109870 0x7fa518103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:33.213+0000 7fa51fff5640 1 --2- 172.21.15.67:0/2856476139 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa518108f20 0x7fa518102ec0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:33.213+0000 7fa51fff5640 1 --2- 172.21.15.67:0/2856476139 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5181019e0 0x7fa518102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:33.213+0000 7fa51fff5640 1 -- 172.21.15.67:0/2856476139 >> 172.21.15.67:0/2856476139 conn(0x7fa5180776d0 msgr2=0x7fa518107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:33.214+0000 7fa51fff5640 1 -- 172.21.15.67:0/2856476139 shutdown_connections 2024-07-27T03:17:33.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:33.214+0000 7fa51fff5640 1 -- 172.21.15.67:0/2856476139 wait complete. 2024-07-27T03:17:33.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:33 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2856476139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:17:33.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:33 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2856476139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:17:33.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:33 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2856476139' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:17:33.994 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-27T03:17:34.411 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:17:34.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:34 smithi067 ceph-mon[32814]: pgmap v460: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-07-27T03:17:34.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:34 smithi149 ceph-mon[37173]: pgmap v460: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-07-27T03:17:34.776 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:34 smithi175 ceph-mon[37125]: pgmap v460: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-07-27T03:17:36.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.225+0000 7f25e2e27640 1 -- 172.21.15.67:0/555609598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25dc108760 msgr2=0x7f25dc10ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:36.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.225+0000 7f25e2e27640 1 --2- 172.21.15.67:0/555609598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25dc108760 0x7f25dc10ef90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f25d4009ca0 tx=0x7f25d402a8c0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.225+0000 7f25e2e27640 1 -- 172.21.15.67:0/555609598 shutdown_connections 2024-07-27T03:17:36.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.225+0000 7f25e2e27640 1 --2- 172.21.15.67:0/555609598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25dc108760 0x7f25dc10ef90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.225+0000 7f25e2e27640 1 --2- 172.21.15.67:0/555609598 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25dc107e10 0x7f25dc108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.225+0000 7f25e2e27640 1 --2- 172.21.15.67:0/555609598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25dc1024d0 0x7f25dc1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.225+0000 7f25e2e27640 1 -- 172.21.15.67:0/555609598 >> 172.21.15.67:0/555609598 conn(0x7f25dc0791a0 msgr2=0x7f25dc0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:36.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.225+0000 7f25e2e27640 1 -- 172.21.15.67:0/555609598 shutdown_connections 2024-07-27T03:17:36.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.225+0000 7f25e2e27640 1 -- 172.21.15.67:0/555609598 wait complete. 2024-07-27T03:17:36.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.226+0000 7f25e2e27640 1 Processor -- start 2024-07-27T03:17:36.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.226+0000 7f25e2e27640 1 -- start start 2024-07-27T03:17:36.267 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.227+0000 7f25e2e27640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25dc1024d0 0x7f25dc197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:36.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.227+0000 7f25e2e27640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25dc107e10 0x7f25dc198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:36.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.227+0000 7f25e2e27640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25dc108760 0x7f25dc19d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:36.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.227+0000 7f25e2e27640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc1123c0 con 0x7f25dc108760 2024-07-27T03:17:36.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.227+0000 7f25e2e27640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc112240 con 0x7f25dc1024d0 2024-07-27T03:17:36.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.227+0000 7f25e2e27640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc112540 con 0x7f25dc107e10 2024-07-27T03:17:36.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.227+0000 7f25e13c4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25dc108760 0x7f25dc19d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:36.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.227+0000 7f25e13c4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25dc108760 0x7f25dc19d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42730/0 (socket says 172.21.15.67:42730) 2024-07-27T03:17:36.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.227+0000 7f25e13c4640 1 -- 172.21.15.67:0/3026969422 learned_addr learned my addr 172.21.15.67:0/3026969422 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:36.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.227+0000 7f25dbfff640 1 --2- 172.21.15.67:0/3026969422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25dc107e10 0x7f25dc198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:36.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.227+0000 7f25e0bc3640 1 --2- 172.21.15.67:0/3026969422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25dc1024d0 0x7f25dc197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:36.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.227+0000 7f25e13c4640 1 -- 172.21.15.67:0/3026969422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25dc107e10 msgr2=0x7f25dc198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:36.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.227+0000 7f25e13c4640 1 --2- 172.21.15.67:0/3026969422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25dc107e10 0x7f25dc198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.227+0000 7f25e13c4640 1 -- 172.21.15.67:0/3026969422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25dc1024d0 msgr2=0x7f25dc197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:36.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.227+0000 7f25e13c4640 1 --2- 172.21.15.67:0/3026969422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25dc1024d0 0x7f25dc197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.227+0000 7f25e13c4640 1 -- 172.21.15.67:0/3026969422 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25dc19dc10 con 0x7f25dc108760 2024-07-27T03:17:36.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.228+0000 7f25dbfff640 1 --2- 172.21.15.67:0/3026969422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25dc107e10 0x7f25dc198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:17:36.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.228+0000 7f25e13c4640 1 --2- 172.21.15.67:0/3026969422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25dc108760 0x7f25dc19d4e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f25d40098d0 tx=0x7f25d40317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:36.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.228+0000 7f25d9ffb640 1 -- 172.21.15.67:0/3026969422 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d4003b70 con 0x7f25dc108760 2024-07-27T03:17:36.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.228+0000 7f25d9ffb640 1 -- 172.21.15.67:0/3026969422 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d4003d10 con 0x7f25dc108760 2024-07-27T03:17:36.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.228+0000 7f25e2e27640 1 -- 172.21.15.67:0/3026969422 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25dc19de40 con 0x7f25dc108760 2024-07-27T03:17:36.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.228+0000 7f25d9ffb640 1 -- 172.21.15.67:0/3026969422 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d402fa20 con 0x7f25dc108760 2024-07-27T03:17:36.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.229+0000 7f25e2e27640 1 -- 172.21.15.67:0/3026969422 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25dc19e650 con 0x7f25dc108760 2024-07-27T03:17:36.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.232+0000 7f25e93ac740 1 Processor -- start 2024-07-27T03:17:36.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.233+0000 7f25d9ffb640 1 -- 172.21.15.67:0/3026969422 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f25d402fc40 con 0x7f25dc108760 2024-07-27T03:17:36.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.233+0000 7f25e93ac740 1 -- start start 2024-07-27T03:17:36.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.233+0000 7f25e93ac740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e633070 0x55c56e9d7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:36.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.233+0000 7f25e93ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e60a1b0 0x55c56e9d8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:36.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.233+0000 7f25e93ac740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c56e612390 0x55c56e9d88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:36.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.233+0000 7f25e93ac740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c56e8c0400 con 0x55c56e633070 2024-07-27T03:17:36.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.233+0000 7f25e93ac740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c56e94eb20 con 0x55c56e612390 2024-07-27T03:17:36.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.233+0000 7f25e93ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c56e93e020 con 0x55c56e60a1b0 2024-07-27T03:17:36.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.234+0000 7f25dbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e633070 0x55c56e9d7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:36.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.234+0000 7f25dbfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e633070 0x55c56e9d7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42742/0 (socket says 172.21.15.67:42742) 2024-07-27T03:17:36.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.234+0000 7f25dbfff640 1 -- 172.21.15.67:0/3501313046 learned_addr learned my addr 172.21.15.67:0/3501313046 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:36.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.234+0000 7f25d9ffb640 1 --2- 172.21.15.67:0/3026969422 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f25cc0b3c50 0x7f25cc0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:36.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.234+0000 7f25e0bc3640 1 --2- 172.21.15.67:0/3501313046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e60a1b0 0x55c56e9d8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:36.270 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.234+0000 7f25d9ffb640 1 -- 172.21.15.67:0/3026969422 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f25d4105420 con 0x7f25dc108760 2024-07-27T03:17:36.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.234+0000 7f25e0bc3640 1 -- 172.21.15.67:0/3501313046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c56e612390 msgr2=0x55c56e9d88b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-27T03:17:36.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.234+0000 7f25e0bc3640 1 --2- 172.21.15.67:0/3501313046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c56e612390 0x55c56e9d88b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.234+0000 7f25e0bc3640 1 -- 172.21.15.67:0/3501313046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e633070 msgr2=0x55c56e9d7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:36.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.234+0000 7f25e0bc3640 1 --2- 172.21.15.67:0/3501313046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e633070 0x55c56e9d7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.234+0000 7f25e0bc3640 1 -- 172.21.15.67:0/3501313046 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c56e8d3770 con 0x55c56e60a1b0 2024-07-27T03:17:36.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.234+0000 7f25e13c4640 1 --2- 172.21.15.67:0/3026969422 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f25cc0b3c50 0x7f25cc0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:36.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.235+0000 7f25dbfff640 1 --2- 172.21.15.67:0/3501313046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e633070 0x55c56e9d7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:17:36.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.235+0000 7f25e0bc3640 1 --2- 172.21.15.67:0/3501313046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e60a1b0 0x55c56e9d8370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009bf0 tx=0x7f25d0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a92e0849758ebc79 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:36.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.235+0000 7f25e13c4640 1 --2- 172.21.15.67:0/3026969422 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f25cc0b3c50 0x7f25cc0b6160 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f25d402f550 tx=0x7f25d40095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:36.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.235+0000 7f25c6ffd640 1 -- 172.21.15.67:0/3501313046 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d001e070 con 0x55c56e60a1b0 2024-07-27T03:17:36.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.235+0000 7f25c6ffd640 1 -- 172.21.15.67:0/3501313046 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d0009da0 con 0x55c56e60a1b0 2024-07-27T03:17:36.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.235+0000 7f25c6ffd640 1 -- 172.21.15.67:0/3501313046 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0019410 con 0x55c56e60a1b0 2024-07-27T03:17:36.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.236+0000 7f25e93ac740 1 -- 172.21.15.67:0/3501313046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e60a1b0 msgr2=0x55c56e9d8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:36.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.236+0000 7f25e93ac740 1 --2- 172.21.15.67:0/3501313046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e60a1b0 0x55c56e9d8370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009bf0 tx=0x7f25d0011810 comp rx=0 tx=0).stop 2024-07-27T03:17:36.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.236+0000 7f25e93ac740 1 -- 172.21.15.67:0/3501313046 shutdown_connections 2024-07-27T03:17:36.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.236+0000 7f25e93ac740 1 --2- 172.21.15.67:0/3501313046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c56e612390 0x55c56e9d88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.236+0000 7f25e93ac740 1 --2- 172.21.15.67:0/3501313046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e60a1b0 0x55c56e9d8370 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.236+0000 7f25e93ac740 1 --2- 172.21.15.67:0/3501313046 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e633070 0x55c56e9d7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.236+0000 7f25e93ac740 1 -- 172.21.15.67:0/3501313046 >> 172.21.15.67:0/3501313046 conn(0x55c56e627460 msgr2=0x55c56e9d5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:36.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.237+0000 7f25e93ac740 1 -- 172.21.15.67:0/3501313046 shutdown_connections 2024-07-27T03:17:36.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.237+0000 7f25e93ac740 1 -- 172.21.15.67:0/3501313046 wait complete. 2024-07-27T03:17:36.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.237+0000 7f25e93ac740 1 -- start start 2024-07-27T03:17:36.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.237+0000 7f25e93ac740 1 Processor -- start 2024-07-27T03:17:36.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.238+0000 7f25e93ac740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c56e60a1b0 0x55c56e9e3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:36.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.238+0000 7f25e93ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e612390 0x55c56e9e4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:36.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.238+0000 7f25e93ac740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e633070 0x55c56e9e9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:36.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.238+0000 7f25e93ac740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c56e8e0900 con 0x55c56e633070 2024-07-27T03:17:36.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.238+0000 7f25e93ac740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c56e942060 con 0x55c56e60a1b0 2024-07-27T03:17:36.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.238+0000 7f25e93ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c56e7a1700 con 0x55c56e612390 2024-07-27T03:17:36.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.239+0000 7f25e0bc3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e612390 0x55c56e9e4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:36.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.239+0000 7f25e0bc3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e612390 0x55c56e9e4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.67:32842/0 (socket says 172.21.15.67:32842) 2024-07-27T03:17:36.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.239+0000 7f25e0bc3640 1 -- 172.21.15.67:0/1968908908 learned_addr learned my addr 172.21.15.67:0/1968908908 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:36.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.239+0000 7f25e93ac740 10 client.0 _ll_register_callbacks cb 0x55c56e8e8d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:17:36.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.239+0000 7f25dbfff640 1 --2- 172.21.15.67:0/1968908908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c56e60a1b0 0x55c56e9e3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:36.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.239+0000 7f25dbfff640 1 --2- 172.21.15.67:0/1968908908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e633070 0x55c56e9e9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:36.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.239+0000 7f25e0bc3640 1 -- 172.21.15.67:0/1968908908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c56e60a1b0 msgr2=0x55c56e9e3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:36.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.239+0000 7f25e0bc3640 1 --2- 172.21.15.67:0/1968908908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c56e60a1b0 0x55c56e9e3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.239+0000 7f25e0bc3640 1 -- 172.21.15.67:0/1968908908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e633070 msgr2=0x55c56e9e9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:36.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.239+0000 7f25e0bc3640 1 --2- 172.21.15.67:0/1968908908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e633070 0x55c56e9e9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.239+0000 7f25e0bc3640 1 -- 172.21.15.67:0/1968908908 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c56e8e4bf0 con 0x55c56e612390 2024-07-27T03:17:36.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.239+0000 7f25e93ac740 1 -- 172.21.15.67:0/1968908908 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25d0009da0 con 0x55c56e612390 2024-07-27T03:17:36.273 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.239+0000 7f25dbfff640 1 --2- 172.21.15.67:0/1968908908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e633070 0x55c56e9e9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:17:36.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.239+0000 7f25e0bc3640 1 --2- 172.21.15.67:0/1968908908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e612390 0x55c56e9e4110 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f25d0003b00 tx=0x7f25d0003a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fe49cacd4438c07 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:36.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.239+0000 7f25bbfff640 10 client.25195 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-27T03:17:36.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.240+0000 7f25bbfff640 1 -- 172.21.15.67:0/1968908908 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0026020 con 0x55c56e612390 2024-07-27T03:17:36.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.240+0000 7f25bbfff640 1 -- 172.21.15.67:0/1968908908 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f25d00054a0 con 0x55c56e612390 2024-07-27T03:17:36.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.240+0000 7f25bbfff640 1 -- 172.21.15.67:0/1968908908 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f25d001e040 con 0x55c56e612390 2024-07-27T03:17:36.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.240+0000 7f25e93ac740 10 client.25195 fetch_fsmap learned FSMap version 30 2024-07-27T03:17:36.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.240+0000 7f25e93ac740 1 -- 172.21.15.67:0/1968908908 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55c56e69e650 con 0x55c56e612390 2024-07-27T03:17:36.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.240+0000 7f25bbfff640 1 -- 172.21.15.67:0/1968908908 <== mon.1 v2:172.21.15.175:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f25d0021030 con 0x55c56e612390 2024-07-27T03:17:36.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.241+0000 7f25e93ac740 10 client.25195 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:17:36.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.241+0000 7f25e93ac740 10 client.25195 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-07-27T03:17:36.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.241+0000 7f25e93ac740 20 client.25195 populate_metadata read hostname 'smithi067' 2024-07-27T03:17:36.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.241+0000 7f25e93ac740 1 --2- 172.21.15.67:0/1968908908 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55c56e5f19f0 0x55c56e9f5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:36.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.241+0000 7f25e93ac740 4 client.25195 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:17:36.274 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.241+0000 7f25e93ac740 1 -- 172.21.15.67:0/1968908908 --> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c56e950a70 con 0x55c56e5f19f0 2024-07-27T03:17:36.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.241+0000 7f25dbfff640 1 --2- 172.21.15.67:0/1968908908 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55c56e5f19f0 0x55c56e9f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:36.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.241+0000 7f25dbfff640 1 --2- 172.21.15.67:0/1968908908 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55c56e5f19f0 0x55c56e9f5730 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=94713520d71a91 server_cookie=535926d7b6a71e88 in_seq=0 out_seq=0 2024-07-27T03:17:36.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.241+0000 7f25bbfff640 10 client.25195 ms_handle_connect on v2:172.21.15.149:6834/3730964806 2024-07-27T03:17:36.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.242+0000 7f25bbfff640 1 -- 172.21.15.67:0/1968908908 <== mds.4 v2:172.21.15.149:6834/3730964806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c56e950a70 con 0x55c56e5f19f0 2024-07-27T03:17:36.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.242+0000 7f25bbfff640 10 client.25195 handle_command_reply: tid=0 2024-07-27T03:17:36.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.242+0000 7f25e93ac740 1 client.25195 shutdown 2024-07-27T03:17:36.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.242+0000 7f25e93ac740 1 -- 172.21.15.67:0/1968908908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e612390 msgr2=0x55c56e9e4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:36.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.242+0000 7f25e93ac740 1 --2- 172.21.15.67:0/1968908908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e612390 0x55c56e9e4110 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f25d0003b00 tx=0x7f25d0003a20 comp rx=0 tx=0).stop 2024-07-27T03:17:36.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.242+0000 7f25e93ac740 1 -- 172.21.15.67:0/1968908908 shutdown_connections 2024-07-27T03:17:36.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.242+0000 7f25e93ac740 1 --2- 172.21.15.67:0/1968908908 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55c56e5f19f0 0x55c56e9f5730 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.243+0000 7f25e93ac740 1 --2- 172.21.15.67:0/1968908908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e633070 0x55c56e9e9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.243+0000 7f25bbfff640 0 client.25195 ms_handle_reset on v2:172.21.15.149:6834/3730964806 2024-07-27T03:17:36.275 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.243+0000 7f25e93ac740 1 --2- 172.21.15.67:0/1968908908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e612390 0x55c56e9e4110 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.243+0000 7f25e93ac740 1 --2- 172.21.15.67:0/1968908908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c56e60a1b0 0x55c56e9e3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.243+0000 7f25e93ac740 1 -- 172.21.15.67:0/1968908908 >> 172.21.15.67:0/1968908908 conn(0x55c56e627460 msgr2=0x55c56e9db010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:36.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.243+0000 7f25e93ac740 1 -- 172.21.15.67:0/1968908908 shutdown_connections 2024-07-27T03:17:36.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.243+0000 7f25e93ac740 1 -- 172.21.15.67:0/1968908908 wait complete. 2024-07-27T03:17:36.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.243+0000 7f25e93ac740 20 client.25195 trim_cache size 0 max 16384 2024-07-27T03:17:36.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.256+0000 7f25e93ac740 1 Processor -- start 2024-07-27T03:17:36.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.256+0000 7f25e93ac740 1 -- start start 2024-07-27T03:17:36.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.256+0000 7f25e93ac740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c56e5f19f0 0x55c56e9e96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:36.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.256+0000 7f25e93ac740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e60a1b0 0x55c56e9d5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:36.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.256+0000 7f25e93ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e612390 0x55c56e9d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:36.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.256+0000 7f25e93ac740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc1bf320 con 0x55c56e60a1b0 2024-07-27T03:17:36.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.256+0000 7f25e93ac740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc1c0e60 con 0x55c56e5f19f0 2024-07-27T03:17:36.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.256+0000 7f25e93ac740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc1bf9f0 con 0x55c56e612390 2024-07-27T03:17:36.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.257+0000 7f25e0bc3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e60a1b0 0x55c56e9d5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:36.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.257+0000 7f25e0bc3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e60a1b0 0x55c56e9d5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42754/0 (socket says 172.21.15.67:42754) 2024-07-27T03:17:36.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.257+0000 7f25e0bc3640 1 -- 172.21.15.67:0/480041030 learned_addr learned my addr 172.21.15.67:0/480041030 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:36.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.257+0000 7f25dbfff640 1 --2- 172.21.15.67:0/480041030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c56e5f19f0 0x55c56e9e96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:36.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.257+0000 7f25dbfff640 1 --2- 172.21.15.67:0/480041030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e612390 0x55c56e9d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:36.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.257+0000 7f25e0bc3640 1 -- 172.21.15.67:0/480041030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e612390 msgr2=0x55c56e9d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:36.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.257+0000 7f25e0bc3640 1 --2- 172.21.15.67:0/480041030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e612390 0x55c56e9d5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.257+0000 7f25e0bc3640 1 -- 172.21.15.67:0/480041030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c56e5f19f0 msgr2=0x55c56e9e96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:36.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.257+0000 7f25e0bc3640 1 --2- 172.21.15.67:0/480041030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c56e5f19f0 0x55c56e9e96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.257+0000 7f25e0bc3640 1 -- 172.21.15.67:0/480041030 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c56e9da120 con 0x55c56e60a1b0 2024-07-27T03:17:36.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.257+0000 7f25e0bc3640 1 --2- 172.21.15.67:0/480041030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e60a1b0 0x55c56e9d5310 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009b10 tx=0x7f25d0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea4dc579869cf0f server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:36.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.257+0000 7f25c67fc640 1 -- 172.21.15.67:0/480041030 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0026070 con 0x55c56e60a1b0 2024-07-27T03:17:36.277 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.258+0000 7f25c67fc640 1 -- 172.21.15.67:0/480041030 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d0009da0 con 0x55c56e60a1b0 2024-07-27T03:17:36.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.258+0000 7f25c67fc640 1 -- 172.21.15.67:0/480041030 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d00041d0 con 0x55c56e60a1b0 2024-07-27T03:17:36.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.258+0000 7f25e93ac740 1 -- 172.21.15.67:0/480041030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e60a1b0 msgr2=0x55c56e9d5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:36.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.258+0000 7f25e93ac740 1 --2- 172.21.15.67:0/480041030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e60a1b0 0x55c56e9d5310 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009b10 tx=0x7f25d0016000 comp rx=0 tx=0).stop 2024-07-27T03:17:36.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.259+0000 7f25e93ac740 1 -- 172.21.15.67:0/480041030 shutdown_connections 2024-07-27T03:17:36.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.259+0000 7f25e93ac740 1 --2- 172.21.15.67:0/480041030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e612390 0x55c56e9d5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.259+0000 7f25e93ac740 1 --2- 172.21.15.67:0/480041030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e60a1b0 0x55c56e9d5310 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.259+0000 7f25e93ac740 1 --2- 172.21.15.67:0/480041030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c56e5f19f0 0x55c56e9e96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.259+0000 7f25e93ac740 1 -- 172.21.15.67:0/480041030 >> 172.21.15.67:0/480041030 conn(0x55c56e627460 msgr2=0x55c56e9d9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:36.278 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.259+0000 7f25e93ac740 1 -- 172.21.15.67:0/480041030 shutdown_connections 2024-07-27T03:17:36.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.259+0000 7f25e93ac740 1 -- 172.21.15.67:0/480041030 wait complete. 2024-07-27T03:17:36.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.259+0000 7f25e93ac740 1 -- start start 2024-07-27T03:17:36.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.259+0000 7f25e93ac740 1 Processor -- start 2024-07-27T03:17:36.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.260+0000 7f25e93ac740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c56e5f19f0 0x55c56e9ed610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:36.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.260+0000 7f25e93ac740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e60a1b0 0x55c56e9edbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:36.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.260+0000 7f25e93ac740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e612390 0x55c56ea54c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:36.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.260+0000 7f25e93ac740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc1c0790 con 0x55c56e60a1b0 2024-07-27T03:17:36.279 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.260+0000 7f25e0bc3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e60a1b0 0x55c56e9edbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:36.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.260+0000 7f25e0bc3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e60a1b0 0x55c56e9edbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42768/0 (socket says 172.21.15.67:42768) 2024-07-27T03:17:36.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.260+0000 7f25e0bc3640 1 -- 172.21.15.67:0/825450391 learned_addr learned my addr 172.21.15.67:0/825450391 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:36.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.260+0000 7f25e93ac740 1 -- 172.21.15.67:0/825450391 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc1c00c0 con 0x55c56e5f19f0 2024-07-27T03:17:36.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.260+0000 7f25e93ac740 1 -- 172.21.15.67:0/825450391 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc1bd300 con 0x55c56e612390 2024-07-27T03:17:36.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.260+0000 7f25dbfff640 1 --2- 172.21.15.67:0/825450391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c56e5f19f0 0x55c56e9ed610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:36.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.261+0000 7f25dbfff640 1 --2- 172.21.15.67:0/825450391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e612390 0x55c56ea54c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:36.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.261+0000 7f25e93ac740 10 client.0 _ll_register_callbacks cb 0x7f25d001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:17:36.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.261+0000 7f25e0bc3640 1 -- 172.21.15.67:0/825450391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e612390 msgr2=0x55c56ea54c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:36.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.261+0000 7f25e0bc3640 1 --2- 172.21.15.67:0/825450391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e612390 0x55c56ea54c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.261+0000 7f25e0bc3640 1 -- 172.21.15.67:0/825450391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c56e5f19f0 msgr2=0x55c56e9ed610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:36.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.261+0000 7f25e0bc3640 1 --2- 172.21.15.67:0/825450391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c56e5f19f0 0x55c56e9ed610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.261+0000 7f25e0bc3640 1 -- 172.21.15.67:0/825450391 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c56ea552a0 con 0x55c56e60a1b0 2024-07-27T03:17:36.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.261+0000 7f25e93ac740 1 -- 172.21.15.67:0/825450391 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25d0009da0 con 0x55c56e60a1b0 2024-07-27T03:17:36.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.261+0000 7f25e0bc3640 1 --2- 172.21.15.67:0/825450391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e60a1b0 0x55c56e9edbb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f25d0005fd0 tx=0x7f25d0005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85fb7a566a51095 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:36.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.261+0000 7f25bbfff640 10 client.15585 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:17:36.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.261+0000 7f25bbfff640 1 -- 172.21.15.67:0/825450391 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0026070 con 0x55c56e60a1b0 2024-07-27T03:17:36.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.262+0000 7f25bbfff640 1 -- 172.21.15.67:0/825450391 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f25d0004a90 con 0x55c56e60a1b0 2024-07-27T03:17:36.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.262+0000 7f25bbfff640 1 -- 172.21.15.67:0/825450391 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f25d0004d10 con 0x55c56e60a1b0 2024-07-27T03:17:36.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.262+0000 7f25e93ac740 10 client.15585 fetch_fsmap learned FSMap version 30 2024-07-27T03:17:36.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.262+0000 7f25e93ac740 1 -- 172.21.15.67:0/825450391 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55c56ea5d6c0 con 0x55c56e60a1b0 2024-07-27T03:17:36.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.262+0000 7f25bbfff640 1 -- 172.21.15.67:0/825450391 <== mon.0 v2:172.21.15.67:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f25d0021070 con 0x55c56e60a1b0 2024-07-27T03:17:36.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.262+0000 7f25e93ac740 10 client.15585 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:17:36.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.262+0000 7f25e93ac740 10 client.15585 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-07-27T03:17:36.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.262+0000 7f25e93ac740 20 client.15585 populate_metadata read hostname 'smithi067' 2024-07-27T03:17:36.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.262+0000 7f25e93ac740 1 --2- 172.21.15.67:0/825450391 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55c56e633070 0x55c56ea5fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:36.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.262+0000 7f25e93ac740 4 client.15585 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-27T03:17:36.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.262+0000 7f25e93ac740 1 -- 172.21.15.67:0/825450391 --> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c56e94c510 con 0x55c56e633070 2024-07-27T03:17:36.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.263+0000 7f25dbfff640 1 --2- 172.21.15.67:0/825450391 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55c56e633070 0x55c56ea5fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:36.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.264+0000 7f25dbfff640 1 --2- 172.21.15.67:0/825450391 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55c56e633070 0x55c56ea5fa70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=187bd78273146dfd server_cookie=517cddedf3aa4afb in_seq=0 out_seq=0 2024-07-27T03:17:36.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.264+0000 7f25bbfff640 10 client.15585 ms_handle_connect on v2:172.21.15.149:6834/3730964806 2024-07-27T03:17:36.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.264+0000 7f25bbfff640 1 -- 172.21.15.67:0/825450391 <== mds.4 v2:172.21.15.149:6834/3730964806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c56e94c510 con 0x55c56e633070 2024-07-27T03:17:36.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.264+0000 7f25bbfff640 10 client.15585 handle_command_reply: tid=0 2024-07-27T03:17:36.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.264+0000 7f25e93ac740 1 client.15585 shutdown 2024-07-27T03:17:36.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.265+0000 7f25e93ac740 1 -- 172.21.15.67:0/825450391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e60a1b0 msgr2=0x55c56e9edbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:36.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.265+0000 7f25e93ac740 1 --2- 172.21.15.67:0/825450391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e60a1b0 0x55c56e9edbb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f25d0005fd0 tx=0x7f25d0005c50 comp rx=0 tx=0).stop 2024-07-27T03:17:36.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.265+0000 7f25e93ac740 1 -- 172.21.15.67:0/825450391 shutdown_connections 2024-07-27T03:17:36.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.265+0000 7f25e93ac740 1 --2- 172.21.15.67:0/825450391 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55c56e633070 0x55c56ea5fa70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.265+0000 7f25bbfff640 0 client.15585 ms_handle_reset on v2:172.21.15.149:6834/3730964806 2024-07-27T03:17:36.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.265+0000 7f25e93ac740 1 --2- 172.21.15.67:0/825450391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c56e612390 0x55c56ea54c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.265+0000 7f25e93ac740 1 --2- 172.21.15.67:0/825450391 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c56e60a1b0 0x55c56e9edbb0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.265+0000 7f25e93ac740 1 --2- 172.21.15.67:0/825450391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c56e5f19f0 0x55c56e9ed610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.265+0000 7f25e93ac740 1 -- 172.21.15.67:0/825450391 >> 172.21.15.67:0/825450391 conn(0x55c56e627460 msgr2=0x55c56e9e2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:36.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.265+0000 7f25e93ac740 1 -- 172.21.15.67:0/825450391 shutdown_connections 2024-07-27T03:17:36.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.265+0000 7f25e93ac740 1 -- 172.21.15.67:0/825450391 wait complete. 2024-07-27T03:17:36.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.266+0000 7f25e93ac740 20 client.15585 trim_cache size 0 max 16384 2024-07-27T03:17:36.284 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-27T03:17:36.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.266+0000 7f25e2e27640 1 -- 172.21.15.67:0/3026969422 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f25cc0b3c50 msgr2=0x7f25cc0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:36.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.266+0000 7f25e2e27640 1 --2- 172.21.15.67:0/3026969422 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f25cc0b3c50 0x7f25cc0b6160 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f25d402f550 tx=0x7f25d40095c0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.267+0000 7f25e2e27640 1 -- 172.21.15.67:0/3026969422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25dc108760 msgr2=0x7f25dc19d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:36.285 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.267+0000 7f25e2e27640 1 --2- 172.21.15.67:0/3026969422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25dc108760 0x7f25dc19d4e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f25d40098d0 tx=0x7f25d40317b0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.267+0000 7f25e2e27640 1 -- 172.21.15.67:0/3026969422 shutdown_connections 2024-07-27T03:17:36.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.267+0000 7f25e2e27640 1 --2- 172.21.15.67:0/3026969422 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f25cc0b3c50 0x7f25cc0b6160 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.267+0000 7f25e2e27640 1 --2- 172.21.15.67:0/3026969422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f25dc108760 0x7f25dc19d4e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.267+0000 7f25e2e27640 1 --2- 172.21.15.67:0/3026969422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25dc107e10 0x7f25dc198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.267+0000 7f25e2e27640 1 --2- 172.21.15.67:0/3026969422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25dc1024d0 0x7f25dc197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:36.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.267+0000 7f25e2e27640 1 -- 172.21.15.67:0/3026969422 >> 172.21.15.67:0/3026969422 conn(0x7f25dc0791a0 msgr2=0x7f25dc103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:36.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.267+0000 7f25e2e27640 1 -- 172.21.15.67:0/3026969422 shutdown_connections 2024-07-27T03:17:36.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:36.267+0000 7f25e2e27640 1 -- 172.21.15.67:0/3026969422 wait complete. 2024-07-27T03:17:36.466 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:36 smithi067 ceph-mon[32814]: pgmap v461: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-07-27T03:17:36.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:36 smithi149 ceph-mon[37173]: pgmap v461: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-07-27T03:17:36.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:36 smithi175 ceph-mon[37125]: pgmap v461: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-07-27T03:17:37.067 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-27T03:17:37.476 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:17:38.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:38 smithi067 ceph-mon[32814]: pgmap v462: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-07-27T03:17:38.749 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:38 smithi149 ceph-mon[37173]: pgmap v462: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-07-27T03:17:38.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:38 smithi175 ceph-mon[37125]: pgmap v462: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-07-27T03:17:39.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.389+0000 7fed243d5640 1 -- 172.21.15.67:0/3257371388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed1c101dd0 msgr2=0x7fed1c1021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:39.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.389+0000 7fed243d5640 1 --2- 172.21.15.67:0/3257371388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed1c101dd0 0x7fed1c1021e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009900 tx=0x7fed0c02a410 comp rx=0 tx=0).stop 2024-07-27T03:17:39.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.390+0000 7fed243d5640 1 -- 172.21.15.67:0/3257371388 shutdown_connections 2024-07-27T03:17:39.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.390+0000 7fed243d5640 1 --2- 172.21.15.67:0/3257371388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed1c110010 0x7fed1c112430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.390+0000 7fed243d5640 1 --2- 172.21.15.67:0/3257371388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed1c101dd0 0x7fed1c1021e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.390+0000 7fed243d5640 1 --2- 172.21.15.67:0/3257371388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed1c104320 0x7fed1c101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.390+0000 7fed243d5640 1 -- 172.21.15.67:0/3257371388 >> 172.21.15.67:0/3257371388 conn(0x7fed1c077730 msgr2=0x7fed1c077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:39.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.390+0000 7fed243d5640 1 -- 172.21.15.67:0/3257371388 shutdown_connections 2024-07-27T03:17:39.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.390+0000 7fed243d5640 1 -- 172.21.15.67:0/3257371388 wait complete. 2024-07-27T03:17:39.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.391+0000 7fed243d5640 1 Processor -- start 2024-07-27T03:17:39.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.391+0000 7fed243d5640 1 -- start start 2024-07-27T03:17:39.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.391+0000 7fed243d5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed1c101dd0 0x7fed1c19c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:39.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.391+0000 7fed243d5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed1c104320 0x7fed1c19c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:39.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.392+0000 7fed243d5640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed1c110010 0x7fed1c1a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:39.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.392+0000 7fed243d5640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c115900 con 0x7fed1c104320 2024-07-27T03:17:39.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.392+0000 7fed243d5640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c115780 con 0x7fed1c101dd0 2024-07-27T03:17:39.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.392+0000 7fed243d5640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c115a80 con 0x7fed1c110010 2024-07-27T03:17:39.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.392+0000 7fed21970640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed1c104320 0x7fed1c19c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:39.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.392+0000 7fed21970640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed1c104320 0x7fed1c19c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42788/0 (socket says 172.21.15.67:42788) 2024-07-27T03:17:39.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.392+0000 7fed21970640 1 -- 172.21.15.67:0/1824041126 learned_addr learned my addr 172.21.15.67:0/1824041126 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:39.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.392+0000 7fed22171640 1 --2- 172.21.15.67:0/1824041126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed1c101dd0 0x7fed1c19c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:39.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.392+0000 7fed22972640 1 --2- 172.21.15.67:0/1824041126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed1c110010 0x7fed1c1a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:39.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.392+0000 7fed22972640 1 -- 172.21.15.67:0/1824041126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed1c101dd0 msgr2=0x7fed1c19c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:39.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.392+0000 7fed22972640 1 --2- 172.21.15.67:0/1824041126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed1c101dd0 0x7fed1c19c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.392+0000 7fed22972640 1 -- 172.21.15.67:0/1824041126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed1c104320 msgr2=0x7fed1c19c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:39.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.392+0000 7fed22972640 1 --2- 172.21.15.67:0/1824041126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed1c104320 0x7fed1c19c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.392+0000 7fed22972640 1 -- 172.21.15.67:0/1824041126 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1c1a2280 con 0x7fed1c110010 2024-07-27T03:17:39.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.392+0000 7fed22171640 1 --2- 172.21.15.67:0/1824041126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed1c101dd0 0x7fed1c19c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:17:39.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.393+0000 7fed21970640 1 --2- 172.21.15.67:0/1824041126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed1c104320 0x7fed1c19c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:17:39.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.393+0000 7fed22972640 1 --2- 172.21.15.67:0/1824041126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed1c110010 0x7fed1c1a1b50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fed1400e940 tx=0x7fed1400ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:39.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.393+0000 7fed137fe640 1 -- 172.21.15.67:0/1824041126 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed14004d20 con 0x7fed1c110010 2024-07-27T03:17:39.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.393+0000 7fed137fe640 1 -- 172.21.15.67:0/1824041126 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed14005020 con 0x7fed1c110010 2024-07-27T03:17:39.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.393+0000 7fed137fe640 1 -- 172.21.15.67:0/1824041126 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed140116d0 con 0x7fed1c110010 2024-07-27T03:17:39.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.393+0000 7fed243d5640 1 -- 172.21.15.67:0/1824041126 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed1c1a2570 con 0x7fed1c110010 2024-07-27T03:17:39.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.393+0000 7fed243d5640 1 -- 172.21.15.67:0/1824041126 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed1c1a8cd0 con 0x7fed1c110010 2024-07-27T03:17:39.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.396+0000 7fed2a95a740 1 Processor -- start 2024-07-27T03:17:39.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.397+0000 7fed137fe640 1 -- 172.21.15.67:0/1824041126 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7fed1400cb20 con 0x7fed1c110010 2024-07-27T03:17:39.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.397+0000 7fed2a95a740 1 -- start start 2024-07-27T03:17:39.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.397+0000 7fed2a95a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3f9070 0x55fb4f79dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:39.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.397+0000 7fed2a95a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3d01b0 0x55fb4f79e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:39.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.398+0000 7fed2a95a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb4f3d8390 0x55fb4f79e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:39.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.398+0000 7fed2a95a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fb4f686400 con 0x55fb4f3d8390 2024-07-27T03:17:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.398+0000 7fed2a95a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fb4f714b20 con 0x55fb4f3d01b0 2024-07-27T03:17:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.398+0000 7fed2a95a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fb4f704020 con 0x55fb4f3f9070 2024-07-27T03:17:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.398+0000 7fed22171640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3d01b0 0x55fb4f79e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.398+0000 7fed21970640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3f9070 0x55fb4f79dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.398+0000 7fed22171640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3d01b0 0x55fb4f79e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.67:58444/0 (socket says 172.21.15.67:58444) 2024-07-27T03:17:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.398+0000 7fed22171640 1 -- 172.21.15.67:0/2835983096 learned_addr learned my addr 172.21.15.67:0/2835983096 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.398+0000 7fed21970640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3f9070 0x55fb4f79dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.67:32882/0 (socket says 172.21.15.67:32882) 2024-07-27T03:17:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.398+0000 7fed22171640 1 -- 172.21.15.67:0/2835983096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3f9070 msgr2=0x55fb4f79dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:39.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.398+0000 7fed22171640 1 --2- 172.21.15.67:0/2835983096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3f9070 0x55fb4f79dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.398+0000 7fed22171640 1 -- 172.21.15.67:0/2835983096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb4f3d8390 msgr2=0x55fb4f79e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:39.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.398+0000 7fed22171640 1 --2- 172.21.15.67:0/2835983096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb4f3d8390 0x55fb4f79e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.398+0000 7fed22171640 1 -- 172.21.15.67:0/2835983096 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb4f699770 con 0x55fb4f3d01b0 2024-07-27T03:17:39.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.398+0000 7fed137fe640 1 --2- 172.21.15.67:0/1824041126 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fed080b3bd0 0x7fed080b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:39.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.399+0000 7fed22171640 1 --2- 172.21.15.67:0/2835983096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3d01b0 0x55fb4f79e280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fed180104d0 tx=0x7fed180108d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=602174f1e8850d28 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:39.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.399+0000 7fed137fe640 1 -- 172.21.15.67:0/1824041126 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fed140e4000 con 0x7fed1c110010 2024-07-27T03:17:39.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.399+0000 7fed21970640 1 --2- 172.21.15.67:0/1824041126 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fed080b3bd0 0x7fed080b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:39.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.399+0000 7fed10d9a640 1 -- 172.21.15.67:0/2835983096 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed18004220 con 0x55fb4f3d01b0 2024-07-27T03:17:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.399+0000 7fed10d9a640 1 -- 172.21.15.67:0/2835983096 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed180043c0 con 0x55fb4f3d01b0 2024-07-27T03:17:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.399+0000 7fed10d9a640 1 -- 172.21.15.67:0/2835983096 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed18011aa0 con 0x55fb4f3d01b0 2024-07-27T03:17:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.400+0000 7fed2a95a740 1 -- 172.21.15.67:0/2835983096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3d01b0 msgr2=0x55fb4f79e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.400+0000 7fed21970640 1 --2- 172.21.15.67:0/1824041126 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fed080b3bd0 0x7fed080b60e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fed1c110910 tx=0x7fed0c009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.400+0000 7fed2a95a740 1 --2- 172.21.15.67:0/2835983096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3d01b0 0x55fb4f79e280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fed180104d0 tx=0x7fed180108d0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.400+0000 7fed2a95a740 1 -- 172.21.15.67:0/2835983096 shutdown_connections 2024-07-27T03:17:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.400+0000 7fed2a95a740 1 --2- 172.21.15.67:0/2835983096 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb4f3d8390 0x55fb4f79e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.400+0000 7fed2a95a740 1 --2- 172.21.15.67:0/2835983096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3d01b0 0x55fb4f79e280 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.400+0000 7fed2a95a740 1 --2- 172.21.15.67:0/2835983096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3f9070 0x55fb4f79dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.400+0000 7fed2a95a740 1 -- 172.21.15.67:0/2835983096 >> 172.21.15.67:0/2835983096 conn(0x55fb4f3ed460 msgr2=0x55fb4f79bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.400+0000 7fed2a95a740 1 -- 172.21.15.67:0/2835983096 shutdown_connections 2024-07-27T03:17:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.400+0000 7fed2a95a740 1 -- 172.21.15.67:0/2835983096 wait complete. 2024-07-27T03:17:39.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.401+0000 7fed2a95a740 1 -- start start 2024-07-27T03:17:39.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.401+0000 7fed2a95a740 1 Processor -- start 2024-07-27T03:17:39.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.401+0000 7fed2a95a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3d01b0 0x55fb4f7a9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:39.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.401+0000 7fed2a95a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb4f3d8390 0x55fb4f7a9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:39.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.401+0000 7fed2a95a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3f9070 0x55fb4f7b5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:39.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.401+0000 7fed2a95a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fb4f6a6900 con 0x55fb4f3d8390 2024-07-27T03:17:39.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.401+0000 7fed2a95a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fb4f708060 con 0x55fb4f3d01b0 2024-07-27T03:17:39.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.401+0000 7fed2a95a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fb4f567700 con 0x55fb4f3f9070 2024-07-27T03:17:39.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.402+0000 7fed2a95a740 10 client.0 _ll_register_callbacks cb 0x55fb4f6aed90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:17:39.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.402+0000 7fed22171640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3d01b0 0x55fb4f7a9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:39.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.402+0000 7fed21970640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb4f3d8390 0x55fb4f7a9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:39.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.402+0000 7fed22171640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3d01b0 0x55fb4f7a9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.67:58448/0 (socket says 172.21.15.67:58448) 2024-07-27T03:17:39.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.402+0000 7fed22171640 1 -- 172.21.15.67:0/1414478804 learned_addr learned my addr 172.21.15.67:0/1414478804 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:39.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.402+0000 7fed22972640 1 --2- 172.21.15.67:0/1414478804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3f9070 0x55fb4f7b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:39.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.402+0000 7fed22171640 1 -- 172.21.15.67:0/1414478804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3f9070 msgr2=0x55fb4f7b5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:39.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.402+0000 7fed22171640 1 --2- 172.21.15.67:0/1414478804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3f9070 0x55fb4f7b5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.402+0000 7fed22171640 1 -- 172.21.15.67:0/1414478804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb4f3d8390 msgr2=0x55fb4f7a9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:39.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.402+0000 7fed22171640 1 --2- 172.21.15.67:0/1414478804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb4f3d8390 0x55fb4f7a9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.402+0000 7fed22171640 1 -- 172.21.15.67:0/1414478804 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb4f6aabf0 con 0x55fb4f3d01b0 2024-07-27T03:17:39.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.402+0000 7fed22972640 1 --2- 172.21.15.67:0/1414478804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3f9070 0x55fb4f7b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:17:39.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.403+0000 7fed22171640 1 --2- 172.21.15.67:0/1414478804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3d01b0 0x55fb4f7a9950 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fed18005d00 tx=0x7fed18004230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a7f9e1176c90f94 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:39.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.403+0000 7fed05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-07-27T03:17:39.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.403+0000 7fed05ffb640 1 -- 172.21.15.67:0/1414478804 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed18029060 con 0x55fb4f3d01b0 2024-07-27T03:17:39.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.403+0000 7fed05ffb640 1 -- 172.21.15.67:0/1414478804 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fed18005510 con 0x55fb4f3d01b0 2024-07-27T03:17:39.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.403+0000 7fed2a95a740 1 -- 172.21.15.67:0/1414478804 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed180043c0 con 0x55fb4f3d01b0 2024-07-27T03:17:39.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.403+0000 7fed05ffb640 1 -- 172.21.15.67:0/1414478804 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed18030050 con 0x55fb4f3d01b0 2024-07-27T03:17:39.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.403+0000 7fed2a95a740 10 client.25238 fetch_fsmap learned FSMap version 30 2024-07-27T03:17:39.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.404+0000 7fed2a95a740 1 -- 172.21.15.67:0/1414478804 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fb4f464650 con 0x55fb4f3d01b0 2024-07-27T03:17:39.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.404+0000 7fed05ffb640 1 -- 172.21.15.67:0/1414478804 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed180043c0 con 0x55fb4f3d01b0 2024-07-27T03:17:39.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.404+0000 7fed05ffb640 1 -- 172.21.15.67:0/1414478804 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fed18023030 con 0x55fb4f3d01b0 2024-07-27T03:17:39.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.404+0000 7fed2a95a740 10 client.25238 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:17:39.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.404+0000 7fed2a95a740 10 client.25238 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-27T03:17:39.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.404+0000 7fed2a95a740 20 client.25238 populate_metadata read hostname 'smithi067' 2024-07-27T03:17:39.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.404+0000 7fed2a95a740 1 --2- 172.21.15.67:0/1414478804 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55fb4f3b79f0 0x55fb4f7bba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:39.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.404+0000 7fed2a95a740 4 client.25238 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:17:39.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.404+0000 7fed2a95a740 1 -- 172.21.15.67:0/1414478804 --> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb4f716a70 con 0x55fb4f3b79f0 2024-07-27T03:17:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.405+0000 7fed21970640 1 --2- 172.21.15.67:0/1414478804 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55fb4f3b79f0 0x55fb4f7bba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.405+0000 7fed21970640 1 --2- 172.21.15.67:0/1414478804 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55fb4f3b79f0 0x55fb4f7bba90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9d52c3ead4304fd9 server_cookie=e9a2f5976ccb4615 in_seq=0 out_seq=0 2024-07-27T03:17:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.405+0000 7fed05ffb640 10 client.25238 ms_handle_connect on v2:172.21.15.175:6836/1304803502 2024-07-27T03:17:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.406+0000 7fed05ffb640 1 -- 172.21.15.67:0/1414478804 <== mds.3 v2:172.21.15.175:6836/1304803502 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb4f716a70 con 0x55fb4f3b79f0 2024-07-27T03:17:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.406+0000 7fed05ffb640 10 client.25238 handle_command_reply: tid=0 2024-07-27T03:17:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.406+0000 7fed2a95a740 1 client.25238 shutdown 2024-07-27T03:17:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.406+0000 7fed2a95a740 1 -- 172.21.15.67:0/1414478804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3d01b0 msgr2=0x55fb4f7a9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.406+0000 7fed2a95a740 1 --2- 172.21.15.67:0/1414478804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3d01b0 0x55fb4f7a9950 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fed18005d00 tx=0x7fed18004230 comp rx=0 tx=0).stop 2024-07-27T03:17:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.406+0000 7fed2a95a740 1 -- 172.21.15.67:0/1414478804 shutdown_connections 2024-07-27T03:17:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.406+0000 7fed2a95a740 1 --2- 172.21.15.67:0/1414478804 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55fb4f3b79f0 0x55fb4f7bba90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.406+0000 7fed2a95a740 1 --2- 172.21.15.67:0/1414478804 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3f9070 0x55fb4f7b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.406+0000 7fed2a95a740 1 --2- 172.21.15.67:0/1414478804 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb4f3d8390 0x55fb4f7a9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.406+0000 7fed2a95a740 1 --2- 172.21.15.67:0/1414478804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3d01b0 0x55fb4f7a9950 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.406+0000 7fed2a95a740 1 -- 172.21.15.67:0/1414478804 >> 172.21.15.67:0/1414478804 conn(0x55fb4f3ed460 msgr2=0x55fb4f7a0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.406+0000 7fed05ffb640 0 client.25238 ms_handle_reset on v2:172.21.15.175:6836/1304803502 2024-07-27T03:17:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.406+0000 7fed2a95a740 1 -- 172.21.15.67:0/1414478804 shutdown_connections 2024-07-27T03:17:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.407+0000 7fed2a95a740 1 -- 172.21.15.67:0/1414478804 wait complete. 2024-07-27T03:17:39.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.407+0000 7fed2a95a740 20 client.25238 trim_cache size 0 max 16384 2024-07-27T03:17:39.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.433+0000 7fed2a95a740 1 Processor -- start 2024-07-27T03:17:39.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.433+0000 7fed2a95a740 1 -- start start 2024-07-27T03:17:39.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.433+0000 7fed2a95a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3d01b0 0x55fb4f7b5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:39.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.433+0000 7fed2a95a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb4f3d8390 0x55fb4f7a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:39.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.433+0000 7fed2a95a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3f9070 0x55fb4f7a3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:39.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.433+0000 7fed2a95a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c1ae460 con 0x55fb4f3d8390 2024-07-27T03:17:39.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.433+0000 7fed2a95a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c1affa0 con 0x55fb4f3f9070 2024-07-27T03:17:39.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.433+0000 7fed2a95a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c1aeb30 con 0x55fb4f3d01b0 2024-07-27T03:17:39.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.433+0000 7fed21970640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb4f3d8390 0x55fb4f7a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:39.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.433+0000 7fed21970640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb4f3d8390 0x55fb4f7a2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42818/0 (socket says 172.21.15.67:42818) 2024-07-27T03:17:39.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.433+0000 7fed21970640 1 -- 172.21.15.67:0/3985709786 learned_addr learned my addr 172.21.15.67:0/3985709786 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:39.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.434+0000 7fed22972640 1 --2- 172.21.15.67:0/3985709786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3f9070 0x55fb4f7a3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:39.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.434+0000 7fed22171640 1 --2- 172.21.15.67:0/3985709786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3d01b0 0x55fb4f7b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:39.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.434+0000 7fed22972640 1 -- 172.21.15.67:0/3985709786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3d01b0 msgr2=0x55fb4f7b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:39.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.434+0000 7fed22972640 1 --2- 172.21.15.67:0/3985709786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3d01b0 0x55fb4f7b5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.434+0000 7fed22972640 1 -- 172.21.15.67:0/3985709786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb4f3d8390 msgr2=0x55fb4f7a2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:39.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.434+0000 7fed22972640 1 --2- 172.21.15.67:0/3985709786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb4f3d8390 0x55fb4f7a2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.434+0000 7fed22972640 1 -- 172.21.15.67:0/3985709786 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb4f7a0ba0 con 0x55fb4f3f9070 2024-07-27T03:17:39.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.434+0000 7fed21970640 1 --2- 172.21.15.67:0/3985709786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb4f3d8390 0x55fb4f7a2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:17:39.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.434+0000 7fed22972640 1 --2- 172.21.15.67:0/3985709786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3f9070 0x55fb4f7a3390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed140e1ac0 tx=0x7fed14005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a262d6068419e523 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:39.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.434+0000 7fed07fff640 1 -- 172.21.15.67:0/3985709786 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1401f040 con 0x55fb4f3f9070 2024-07-27T03:17:39.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.435+0000 7fed07fff640 1 -- 172.21.15.67:0/3985709786 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed140aa760 con 0x55fb4f3f9070 2024-07-27T03:17:39.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.435+0000 7fed07fff640 1 -- 172.21.15.67:0/3985709786 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed140920a0 con 0x55fb4f3f9070 2024-07-27T03:17:39.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.436+0000 7fed2a95a740 1 -- 172.21.15.67:0/3985709786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3f9070 msgr2=0x55fb4f7a3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:39.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.436+0000 7fed2a95a740 1 --2- 172.21.15.67:0/3985709786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3f9070 0x55fb4f7a3390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed140e1ac0 tx=0x7fed14005c00 comp rx=0 tx=0).stop 2024-07-27T03:17:39.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.436+0000 7fed2a95a740 1 -- 172.21.15.67:0/3985709786 shutdown_connections 2024-07-27T03:17:39.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.436+0000 7fed2a95a740 1 --2- 172.21.15.67:0/3985709786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3f9070 0x55fb4f7a3390 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.436+0000 7fed2a95a740 1 --2- 172.21.15.67:0/3985709786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb4f3d8390 0x55fb4f7a2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.436+0000 7fed2a95a740 1 --2- 172.21.15.67:0/3985709786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3d01b0 0x55fb4f7b5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.436+0000 7fed2a95a740 1 -- 172.21.15.67:0/3985709786 >> 172.21.15.67:0/3985709786 conn(0x55fb4f3ed460 msgr2=0x55fb4f7a7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:39.439 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.436+0000 7fed2a95a740 1 -- 172.21.15.67:0/3985709786 shutdown_connections 2024-07-27T03:17:39.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.436+0000 7fed2a95a740 1 -- 172.21.15.67:0/3985709786 wait complete. 2024-07-27T03:17:39.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.436+0000 7fed2a95a740 1 -- start start 2024-07-27T03:17:39.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.437+0000 7fed2a95a740 1 Processor -- start 2024-07-27T03:17:39.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.437+0000 7fed2a95a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3d01b0 0x55fb4f7ac5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:39.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.437+0000 7fed2a95a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3d8390 0x55fb4f7acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:39.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.437+0000 7fed2a95a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb4f3f9070 0x55fb4f7ad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:39.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.437+0000 7fed2a95a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c1af8d0 con 0x55fb4f3f9070 2024-07-27T03:17:39.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.437+0000 7fed2a95a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c1af200 con 0x55fb4f3d8390 2024-07-27T03:17:39.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.438+0000 7fed2a95a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fed1c1ac4e0 con 0x55fb4f3d01b0 2024-07-27T03:17:39.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.438+0000 7fed22972640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb4f3f9070 0x55fb4f7ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:39.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.438+0000 7fed2a95a740 10 client.0 _ll_register_callbacks cb 0x7fed1800bf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:17:39.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.438+0000 7fed22972640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb4f3f9070 0x55fb4f7ad030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42824/0 (socket says 172.21.15.67:42824) 2024-07-27T03:17:39.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.438+0000 7fed22171640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3d01b0 0x55fb4f7ac5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:39.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.438+0000 7fed22972640 1 -- 172.21.15.67:0/4159121302 learned_addr learned my addr 172.21.15.67:0/4159121302 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:39.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.438+0000 7fed21970640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3d8390 0x55fb4f7acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:39.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.438+0000 7fed22171640 1 -- 172.21.15.67:0/4159121302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3d8390 msgr2=0x55fb4f7acaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:39.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.438+0000 7fed22171640 1 --2- 172.21.15.67:0/4159121302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3d8390 0x55fb4f7acaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.438+0000 7fed22171640 1 -- 172.21.15.67:0/4159121302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb4f3f9070 msgr2=0x55fb4f7ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:39.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.438+0000 7fed22171640 1 --2- 172.21.15.67:0/4159121302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb4f3f9070 0x55fb4f7ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.438+0000 7fed22171640 1 -- 172.21.15.67:0/4159121302 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb4f822280 con 0x55fb4f3d01b0 2024-07-27T03:17:39.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.438+0000 7fed22171640 1 --2- 172.21.15.67:0/4159121302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3d01b0 0x55fb4f7ac5b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed180103f0 tx=0x7fed1800bac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42a39308778724e9 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:39.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.438+0000 7fed05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-27T03:17:39.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.439+0000 7fed05ffb640 1 -- 172.21.15.67:0/4159121302 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed18029040 con 0x55fb4f3d01b0 2024-07-27T03:17:39.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.439+0000 7fed2a95a740 1 -- 172.21.15.67:0/4159121302 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed140aa760 con 0x55fb4f3d01b0 2024-07-27T03:17:39.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.439+0000 7fed05ffb640 1 -- 172.21.15.67:0/4159121302 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fed18004660 con 0x55fb4f3d01b0 2024-07-27T03:17:39.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.439+0000 7fed05ffb640 1 -- 172.21.15.67:0/4159121302 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed180044a0 con 0x55fb4f3d01b0 2024-07-27T03:17:39.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.439+0000 7fed2a95a740 10 client.25219 fetch_fsmap learned FSMap version 30 2024-07-27T03:17:39.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.439+0000 7fed2a95a740 1 -- 172.21.15.67:0/4159121302 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fb4f823500 con 0x55fb4f3d01b0 2024-07-27T03:17:39.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.439+0000 7fed05ffb640 1 -- 172.21.15.67:0/4159121302 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed140aa760 con 0x55fb4f3d01b0 2024-07-27T03:17:39.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.440+0000 7fed05ffb640 1 -- 172.21.15.67:0/4159121302 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fed18023050 con 0x55fb4f3d01b0 2024-07-27T03:17:39.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.440+0000 7fed2a95a740 10 client.25219 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:17:39.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.440+0000 7fed2a95a740 10 client.25219 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-27T03:17:39.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.440+0000 7fed2a95a740 20 client.25219 populate_metadata read hostname 'smithi067' 2024-07-27T03:17:39.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.440+0000 7fed2a95a740 1 --2- 172.21.15.67:0/4159121302 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55fb4f38cda0 0x55fb4f825b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:39.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.440+0000 7fed2a95a740 4 client.25219 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-27T03:17:39.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.440+0000 7fed2a95a740 1 -- 172.21.15.67:0/4159121302 --> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fb4f712510 con 0x55fb4f38cda0 2024-07-27T03:17:39.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.440+0000 7fed21970640 1 --2- 172.21.15.67:0/4159121302 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55fb4f38cda0 0x55fb4f825b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:39.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.441+0000 7fed05ffb640 10 client.25219 ms_handle_connect on v2:172.21.15.175:6836/1304803502 2024-07-27T03:17:39.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.441+0000 7fed21970640 1 --2- 172.21.15.67:0/4159121302 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55fb4f38cda0 0x55fb4f825b90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=46ce6662160f0d8e server_cookie=abec28c10b214854 in_seq=0 out_seq=0 2024-07-27T03:17:39.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.441+0000 7fed05ffb640 1 -- 172.21.15.67:0/4159121302 <== mds.3 v2:172.21.15.175:6836/1304803502 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fb4f712510 con 0x55fb4f38cda0 2024-07-27T03:17:39.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.441+0000 7fed05ffb640 10 client.25219 handle_command_reply: tid=0 2024-07-27T03:17:39.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.441+0000 7fed2a95a740 1 client.25219 shutdown 2024-07-27T03:17:39.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.441+0000 7fed2a95a740 1 -- 172.21.15.67:0/4159121302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3d01b0 msgr2=0x55fb4f7ac5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:39.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.441+0000 7fed2a95a740 1 --2- 172.21.15.67:0/4159121302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3d01b0 0x55fb4f7ac5b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fed180103f0 tx=0x7fed1800bac0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.441+0000 7fed2a95a740 1 -- 172.21.15.67:0/4159121302 shutdown_connections 2024-07-27T03:17:39.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.441+0000 7fed2a95a740 1 --2- 172.21.15.67:0/4159121302 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55fb4f38cda0 0x55fb4f825b90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.441+0000 7fed2a95a740 1 --2- 172.21.15.67:0/4159121302 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fb4f3f9070 0x55fb4f7ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.441+0000 7fed05ffb640 0 client.25219 ms_handle_reset on v2:172.21.15.175:6836/1304803502 2024-07-27T03:17:39.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.441+0000 7fed2a95a740 1 --2- 172.21.15.67:0/4159121302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb4f3d8390 0x55fb4f7acaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.441+0000 7fed2a95a740 1 --2- 172.21.15.67:0/4159121302 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fb4f3d01b0 0x55fb4f7ac5b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.441+0000 7fed2a95a740 1 -- 172.21.15.67:0/4159121302 >> 172.21.15.67:0/4159121302 conn(0x55fb4f3ed460 msgr2=0x55fb4f7a2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:39.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.442+0000 7fed2a95a740 1 -- 172.21.15.67:0/4159121302 shutdown_connections 2024-07-27T03:17:39.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.442+0000 7fed2a95a740 1 -- 172.21.15.67:0/4159121302 wait complete. 2024-07-27T03:17:39.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.442+0000 7fed2a95a740 20 client.25219 trim_cache size 0 max 16384 2024-07-27T03:17:39.447 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-27T03:17:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.442+0000 7fed243d5640 1 -- 172.21.15.67:0/1824041126 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fed080b3bd0 msgr2=0x7fed080b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.442+0000 7fed243d5640 1 --2- 172.21.15.67:0/1824041126 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fed080b3bd0 0x7fed080b60e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fed1c110910 tx=0x7fed0c009560 comp rx=0 tx=0).stop 2024-07-27T03:17:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.442+0000 7fed243d5640 1 -- 172.21.15.67:0/1824041126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed1c110010 msgr2=0x7fed1c1a1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.442+0000 7fed243d5640 1 --2- 172.21.15.67:0/1824041126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed1c110010 0x7fed1c1a1b50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fed1400e940 tx=0x7fed1400ee00 comp rx=0 tx=0).stop 2024-07-27T03:17:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.443+0000 7fed243d5640 1 -- 172.21.15.67:0/1824041126 shutdown_connections 2024-07-27T03:17:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.443+0000 7fed243d5640 1 --2- 172.21.15.67:0/1824041126 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fed080b3bd0 0x7fed080b60e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.443+0000 7fed243d5640 1 --2- 172.21.15.67:0/1824041126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed1c110010 0x7fed1c1a1b50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.443+0000 7fed243d5640 1 --2- 172.21.15.67:0/1824041126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fed1c104320 0x7fed1c19c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.443+0000 7fed243d5640 1 --2- 172.21.15.67:0/1824041126 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed1c101dd0 0x7fed1c19c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.443+0000 7fed243d5640 1 -- 172.21.15.67:0/1824041126 >> 172.21.15.67:0/1824041126 conn(0x7fed1c077730 msgr2=0x7fed1c100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:39.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.443+0000 7fed243d5640 1 -- 172.21.15.67:0/1824041126 shutdown_connections 2024-07-27T03:17:39.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:39.443+0000 7fed243d5640 1 -- 172.21.15.67:0/1824041126 wait complete. 2024-07-27T03:17:40.153 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-27T03:17:40.451 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:17:40.740 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:40 smithi067 ceph-mon[32814]: pgmap v463: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-07-27T03:17:40.749 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:40 smithi149 ceph-mon[37173]: pgmap v463: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-07-27T03:17:40.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:40 smithi175 ceph-mon[37125]: pgmap v463: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-07-27T03:17:42.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.387+0000 7fd2fe061640 1 -- 172.21.15.67:0/2301567556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2f8107e10 msgr2=0x7fd2f8108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:42.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.387+0000 7fd2fe061640 1 --2- 172.21.15.67:0/2301567556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2f8107e10 0x7fd2f8108220 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009900 tx=0x7fd2ec02a3f0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.387+0000 7fd2fe061640 1 -- 172.21.15.67:0/2301567556 shutdown_connections 2024-07-27T03:17:42.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.387+0000 7fd2fe061640 1 --2- 172.21.15.67:0/2301567556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2f8108760 0x7fd2f810ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.387+0000 7fd2fe061640 1 --2- 172.21.15.67:0/2301567556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2f8107e10 0x7fd2f8108220 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.387+0000 7fd2fe061640 1 --2- 172.21.15.67:0/2301567556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2f81024d0 0x7fd2f81028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.387+0000 7fd2fe061640 1 -- 172.21.15.67:0/2301567556 >> 172.21.15.67:0/2301567556 conn(0x7fd2f80791a0 msgr2=0x7fd2f80795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:42.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.387+0000 7fd2fe061640 1 -- 172.21.15.67:0/2301567556 shutdown_connections 2024-07-27T03:17:42.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.387+0000 7fd2fe061640 1 -- 172.21.15.67:0/2301567556 wait complete. 2024-07-27T03:17:42.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.388+0000 7fd2fe061640 1 Processor -- start 2024-07-27T03:17:42.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.389+0000 7fd2fe061640 1 -- start start 2024-07-27T03:17:42.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.389+0000 7fd2fe061640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2f81024d0 0x7fd2f8198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:42.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.389+0000 7fd2fe061640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2f8107e10 0x7fd2f8199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:42.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.389+0000 7fd2fe061640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2f8108760 0x7fd2f819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:42.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.389+0000 7fd2fe061640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f81123e0 con 0x7fd2f8107e10 2024-07-27T03:17:42.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.389+0000 7fd2fe061640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f8112260 con 0x7fd2f8108760 2024-07-27T03:17:42.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.389+0000 7fd2fe061640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f8112560 con 0x7fd2f81024d0 2024-07-27T03:17:42.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.390+0000 7fd2f77fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2f81024d0 0x7fd2f8198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:42.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.390+0000 7fd2f77fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2f81024d0 0x7fd2f8198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.67:32940/0 (socket says 172.21.15.67:32940) 2024-07-27T03:17:42.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.390+0000 7fd2f77fe640 1 -- 172.21.15.67:0/892876300 learned_addr learned my addr 172.21.15.67:0/892876300 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:42.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.390+0000 7fd2f7fff640 1 --2- 172.21.15.67:0/892876300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2f8108760 0x7fd2f819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:42.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.390+0000 7fd2f6ffd640 1 --2- 172.21.15.67:0/892876300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2f8107e10 0x7fd2f8199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:42.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.390+0000 7fd2f77fe640 1 -- 172.21.15.67:0/892876300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2f8108760 msgr2=0x7fd2f819e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:42.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.390+0000 7fd2f77fe640 1 --2- 172.21.15.67:0/892876300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2f8108760 0x7fd2f819e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.390+0000 7fd2f77fe640 1 -- 172.21.15.67:0/892876300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2f8107e10 msgr2=0x7fd2f8199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:42.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.390+0000 7fd2f77fe640 1 --2- 172.21.15.67:0/892876300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2f8107e10 0x7fd2f8199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.390+0000 7fd2f77fe640 1 -- 172.21.15.67:0/892876300 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f819ed20 con 0x7fd2f81024d0 2024-07-27T03:17:42.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.390+0000 7fd2f77fe640 1 --2- 172.21.15.67:0/892876300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2f81024d0 0x7fd2f8198db0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800bea0 tx=0x7fd2e800bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:42.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.390+0000 7fd2f6ffd640 1 --2- 172.21.15.67:0/892876300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2f8107e10 0x7fd2f8199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:17:42.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.391+0000 7fd2f4ff9640 1 -- 172.21.15.67:0/892876300 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8016070 con 0x7fd2f81024d0 2024-07-27T03:17:42.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.391+0000 7fd2f4ff9640 1 -- 172.21.15.67:0/892876300 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2e8009c30 con 0x7fd2f81024d0 2024-07-27T03:17:42.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.391+0000 7fd2fe061640 1 -- 172.21.15.67:0/892876300 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2f819f010 con 0x7fd2f81024d0 2024-07-27T03:17:42.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.392+0000 7fd2fe061640 1 -- 172.21.15.67:0/892876300 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2f806a8d0 con 0x7fd2f81024d0 2024-07-27T03:17:42.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.392+0000 7fd2f4ff9640 1 -- 172.21.15.67:0/892876300 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80196a0 con 0x7fd2f81024d0 2024-07-27T03:17:42.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.395+0000 7fd2f4ff9640 1 -- 172.21.15.67:0/892876300 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7fd2e8019840 con 0x7fd2f81024d0 2024-07-27T03:17:42.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.395+0000 7fd3045e6740 1 Processor -- start 2024-07-27T03:17:42.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.395+0000 7fd3045e6740 1 -- start start 2024-07-27T03:17:42.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.396+0000 7fd3045e6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c911070 0x55924ccb5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:42.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.396+0000 7fd3045e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8e81b0 0x55924ccb6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:42.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.396+0000 7fd3045e6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55924c8f0390 0x55924ccb68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:42.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.396+0000 7fd3045e6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55924cb9e400 con 0x55924c911070 2024-07-27T03:17:42.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.396+0000 7fd3045e6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55924cc2cb20 con 0x55924c8f0390 2024-07-27T03:17:42.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.396+0000 7fd3045e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55924cc1c020 con 0x55924c8e81b0 2024-07-27T03:17:42.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.396+0000 7fd2f77fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8e81b0 0x55924ccb6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:42.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.396+0000 7fd2f77fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8e81b0 0x55924ccb6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.67:32942/0 (socket says 172.21.15.67:32942) 2024-07-27T03:17:42.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.396+0000 7fd2f77fe640 1 -- 172.21.15.67:0/3934835288 learned_addr learned my addr 172.21.15.67:0/3934835288 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:42.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.396+0000 7fd2f6ffd640 1 --2- 172.21.15.67:0/3934835288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55924c8f0390 0x55924ccb68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:42.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.396+0000 7fd2f7fff640 1 --2- 172.21.15.67:0/3934835288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c911070 0x55924ccb5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:42.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.396+0000 7fd2f6ffd640 1 -- 172.21.15.67:0/3934835288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8e81b0 msgr2=0x55924ccb6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:42.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.396+0000 7fd2f6ffd640 1 --2- 172.21.15.67:0/3934835288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8e81b0 0x55924ccb6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.396+0000 7fd2f6ffd640 1 -- 172.21.15.67:0/3934835288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c911070 msgr2=0x55924ccb5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:42.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.397+0000 7fd2f6ffd640 1 --2- 172.21.15.67:0/3934835288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c911070 0x55924ccb5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.397+0000 7fd2f6ffd640 1 -- 172.21.15.67:0/3934835288 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55924cbb1770 con 0x55924c8f0390 2024-07-27T03:17:42.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.397+0000 7fd2f6ffd640 1 --2- 172.21.15.67:0/3934835288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55924c8f0390 0x55924ccb68b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec02a910 tx=0x7fd2ec004170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af0a1929c3ce433b server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:42.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.397+0000 7fd2f7fff640 1 --2- 172.21.15.67:0/3934835288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c911070 0x55924ccb5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:17:42.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.397+0000 7fd2ddffb640 1 -- 172.21.15.67:0/3934835288 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec03f070 con 0x55924c8f0390 2024-07-27T03:17:42.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.397+0000 7fd2ddffb640 1 -- 172.21.15.67:0/3934835288 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2ec0054f0 con 0x55924c8f0390 2024-07-27T03:17:42.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.397+0000 7fd2ddffb640 1 -- 172.21.15.67:0/3934835288 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec035030 con 0x55924c8f0390 2024-07-27T03:17:42.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.398+0000 7fd2f4ff9640 1 --2- 172.21.15.67:0/892876300 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd2e00b3c50 0x7fd2e00b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:42.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.398+0000 7fd2f7fff640 1 --2- 172.21.15.67:0/892876300 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd2e00b3c50 0x7fd2e00b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:42.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.398+0000 7fd3045e6740 1 -- 172.21.15.67:0/3934835288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55924c8f0390 msgr2=0x55924ccb68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:42.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.398+0000 7fd3045e6740 1 --2- 172.21.15.67:0/3934835288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55924c8f0390 0x55924ccb68b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec02a910 tx=0x7fd2ec004170 comp rx=0 tx=0).stop 2024-07-27T03:17:42.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.399+0000 7fd2f4ff9640 1 -- 172.21.15.67:0/892876300 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fd2e80e5cc0 con 0x7fd2f81024d0 2024-07-27T03:17:42.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.399+0000 7fd3045e6740 1 -- 172.21.15.67:0/3934835288 shutdown_connections 2024-07-27T03:17:42.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.399+0000 7fd3045e6740 1 --2- 172.21.15.67:0/3934835288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55924c8f0390 0x55924ccb68b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.399+0000 7fd3045e6740 1 --2- 172.21.15.67:0/3934835288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8e81b0 0x55924ccb6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.399+0000 7fd3045e6740 1 --2- 172.21.15.67:0/3934835288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c911070 0x55924ccb5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.399+0000 7fd3045e6740 1 -- 172.21.15.67:0/3934835288 >> 172.21.15.67:0/3934835288 conn(0x55924c905460 msgr2=0x55924ccb3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.399+0000 7fd2f7fff640 1 --2- 172.21.15.67:0/892876300 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd2e00b3c50 0x7fd2e00b6160 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0005fd0 tx=0x7fd2f0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.399+0000 7fd3045e6740 1 -- 172.21.15.67:0/3934835288 shutdown_connections 2024-07-27T03:17:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.399+0000 7fd3045e6740 1 -- 172.21.15.67:0/3934835288 wait complete. 2024-07-27T03:17:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.400+0000 7fd3045e6740 1 -- start start 2024-07-27T03:17:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.400+0000 7fd3045e6740 1 Processor -- start 2024-07-27T03:17:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.401+0000 7fd3045e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8e81b0 0x55924ccc1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.401+0000 7fd3045e6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55924c8f0390 0x55924ccc1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.401+0000 7fd3045e6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c911070 0x55924cccd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.401+0000 7fd3045e6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55924cbbe900 con 0x55924c911070 2024-07-27T03:17:42.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.401+0000 7fd3045e6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55924cc20060 con 0x55924c8f0390 2024-07-27T03:17:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.401+0000 7fd3045e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55924ca7f700 con 0x55924c8e81b0 2024-07-27T03:17:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.401+0000 7fd3045e6740 10 client.0 _ll_register_callbacks cb 0x55924cbc6d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:17:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.401+0000 7fd2f77fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8e81b0 0x55924ccc1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.401+0000 7fd2f77fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8e81b0 0x55924ccc1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.67:32956/0 (socket says 172.21.15.67:32956) 2024-07-27T03:17:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.401+0000 7fd2f77fe640 1 -- 172.21.15.67:0/2661905914 learned_addr learned my addr 172.21.15.67:0/2661905914 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.401+0000 7fd2f7fff640 1 --2- 172.21.15.67:0/2661905914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c911070 0x55924cccd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.401+0000 7fd2f6ffd640 1 --2- 172.21.15.67:0/2661905914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55924c8f0390 0x55924ccc1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.402+0000 7fd2f77fe640 1 -- 172.21.15.67:0/2661905914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55924c8f0390 msgr2=0x55924ccc1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.402+0000 7fd2f77fe640 1 --2- 172.21.15.67:0/2661905914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55924c8f0390 0x55924ccc1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.402+0000 7fd2f77fe640 1 -- 172.21.15.67:0/2661905914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c911070 msgr2=0x55924cccd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.402+0000 7fd2f77fe640 1 --2- 172.21.15.67:0/2661905914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c911070 0x55924cccd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.402+0000 7fd2f77fe640 1 -- 172.21.15.67:0/2661905914 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55924cbc2bf0 con 0x55924c8e81b0 2024-07-27T03:17:42.407 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.402+0000 7fd2f7fff640 1 --2- 172.21.15.67:0/2661905914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c911070 0x55924cccd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:17:42.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.402+0000 7fd2f77fe640 1 --2- 172.21.15.67:0/2661905914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8e81b0 0x55924ccc1a40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80a88e0 tx=0x7fd2e800b660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4526fd95f10a5ce4 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:42.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.402+0000 7fd2d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-27T03:17:42.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.402+0000 7fd2d6ffd640 1 -- 172.21.15.67:0/2661905914 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80e7030 con 0x55924c8e81b0 2024-07-27T03:17:42.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.402+0000 7fd2d6ffd640 1 -- 172.21.15.67:0/2661905914 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fd2e80a8af0 con 0x55924c8e81b0 2024-07-27T03:17:42.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.402+0000 7fd3045e6740 1 -- 172.21.15.67:0/2661905914 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2ec0054f0 con 0x55924c8e81b0 2024-07-27T03:17:42.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.403+0000 7fd2d6ffd640 1 -- 172.21.15.67:0/2661905914 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd2e80ac1b0 con 0x55924c8e81b0 2024-07-27T03:17:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.403+0000 7fd3045e6740 10 client.25231 fetch_fsmap learned FSMap version 30 2024-07-27T03:17:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.403+0000 7fd3045e6740 1 -- 172.21.15.67:0/2661905914 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55924c97c650 con 0x55924c8e81b0 2024-07-27T03:17:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.403+0000 7fd2d6ffd640 1 -- 172.21.15.67:0/2661905914 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2ec0054f0 con 0x55924c8e81b0 2024-07-27T03:17:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.403+0000 7fd2d6ffd640 1 -- 172.21.15.67:0/2661905914 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd2e80abac0 con 0x55924c8e81b0 2024-07-27T03:17:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.403+0000 7fd3045e6740 10 client.25231 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:17:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.403+0000 7fd3045e6740 10 client.25231 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-27T03:17:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.403+0000 7fd3045e6740 20 client.25231 populate_metadata read hostname 'smithi067' 2024-07-27T03:17:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.403+0000 7fd3045e6740 1 --2- 172.21.15.67:0/2661905914 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55924c8cf9f0 0x55924ccd11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.403+0000 7fd3045e6740 4 client.25231 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:17:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.403+0000 7fd3045e6740 1 -- 172.21.15.67:0/2661905914 --> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55924cc2ea70 con 0x55924c8cf9f0 2024-07-27T03:17:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.403+0000 7fd2f6ffd640 1 --2- 172.21.15.67:0/2661905914 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55924c8cf9f0 0x55924ccd11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.404+0000 7fd2d6ffd640 10 client.25231 ms_handle_connect on v2:172.21.15.175:6838/1670568853 2024-07-27T03:17:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.405+0000 7fd2f6ffd640 1 --2- 172.21.15.67:0/2661905914 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55924c8cf9f0 0x55924ccd11b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7c43293f49873290 server_cookie=3b3899f89527ad06 in_seq=0 out_seq=0 2024-07-27T03:17:42.411 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.406+0000 7fd2d6ffd640 1 -- 172.21.15.67:0/2661905914 <== mds.2 v2:172.21.15.175:6838/1670568853 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55924cc2ea70 con 0x55924c8cf9f0 2024-07-27T03:17:42.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.406+0000 7fd2d6ffd640 10 client.25231 handle_command_reply: tid=0 2024-07-27T03:17:42.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.406+0000 7fd3045e6740 1 client.25231 shutdown 2024-07-27T03:17:42.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.407+0000 7fd3045e6740 1 -- 172.21.15.67:0/2661905914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8e81b0 msgr2=0x55924ccc1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:42.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.407+0000 7fd3045e6740 1 --2- 172.21.15.67:0/2661905914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8e81b0 0x55924ccc1a40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80a88e0 tx=0x7fd2e800b660 comp rx=0 tx=0).stop 2024-07-27T03:17:42.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.407+0000 7fd3045e6740 1 -- 172.21.15.67:0/2661905914 shutdown_connections 2024-07-27T03:17:42.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.407+0000 7fd3045e6740 1 --2- 172.21.15.67:0/2661905914 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55924c8cf9f0 0x55924ccd11b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.407+0000 7fd3045e6740 1 --2- 172.21.15.67:0/2661905914 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c911070 0x55924cccd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.407+0000 7fd2d6ffd640 0 client.25231 ms_handle_reset on v2:172.21.15.175:6838/1670568853 2024-07-27T03:17:42.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.407+0000 7fd3045e6740 1 --2- 172.21.15.67:0/2661905914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55924c8f0390 0x55924ccc1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.407+0000 7fd3045e6740 1 --2- 172.21.15.67:0/2661905914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8e81b0 0x55924ccc1a40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.407+0000 7fd3045e6740 1 -- 172.21.15.67:0/2661905914 >> 172.21.15.67:0/2661905914 conn(0x55924c905460 msgr2=0x55924ccb9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:42.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.407+0000 7fd3045e6740 1 -- 172.21.15.67:0/2661905914 shutdown_connections 2024-07-27T03:17:42.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.407+0000 7fd3045e6740 1 -- 172.21.15.67:0/2661905914 wait complete. 2024-07-27T03:17:42.412 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.408+0000 7fd3045e6740 20 client.25231 trim_cache size 0 max 16384 2024-07-27T03:17:42.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.433+0000 7fd3045e6740 1 Processor -- start 2024-07-27T03:17:42.435 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.433+0000 7fd3045e6740 1 -- start start 2024-07-27T03:17:42.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.433+0000 7fd3045e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8cf9f0 0x55924cccd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:42.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.433+0000 7fd3045e6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55924c8e81b0 0x55924ccd2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:42.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.433+0000 7fd3045e6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c8f0390 0x55924ccb3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:42.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.433+0000 7fd3045e6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f81ae370 con 0x55924c8f0390 2024-07-27T03:17:42.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.433+0000 7fd3045e6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f81afeb0 con 0x55924c8e81b0 2024-07-27T03:17:42.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.433+0000 7fd3045e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f81aea40 con 0x55924c8cf9f0 2024-07-27T03:17:42.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.433+0000 7fd2f77fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8cf9f0 0x55924cccd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:42.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.433+0000 7fd2f77fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8cf9f0 0x55924cccd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.67:32966/0 (socket says 172.21.15.67:32966) 2024-07-27T03:17:42.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.434+0000 7fd2f77fe640 1 -- 172.21.15.67:0/1527181402 learned_addr learned my addr 172.21.15.67:0/1527181402 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:42.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.434+0000 7fd2f6ffd640 1 --2- 172.21.15.67:0/1527181402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55924c8e81b0 0x55924ccd2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:42.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.434+0000 7fd2f7fff640 1 --2- 172.21.15.67:0/1527181402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c8f0390 0x55924ccb3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:42.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.434+0000 7fd2f77fe640 1 -- 172.21.15.67:0/1527181402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55924c8e81b0 msgr2=0x55924ccd2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:42.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.434+0000 7fd2f77fe640 1 --2- 172.21.15.67:0/1527181402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55924c8e81b0 0x55924ccd2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.434+0000 7fd2f77fe640 1 -- 172.21.15.67:0/1527181402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c8f0390 msgr2=0x55924ccb3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:42.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.434+0000 7fd2f77fe640 1 --2- 172.21.15.67:0/1527181402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c8f0390 0x55924ccb3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.434+0000 7fd2f77fe640 1 -- 172.21.15.67:0/1527181402 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55924ccbd060 con 0x55924c8cf9f0 2024-07-27T03:17:42.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.434+0000 7fd2f7fff640 1 --2- 172.21.15.67:0/1527181402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c8f0390 0x55924ccb3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:17:42.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.434+0000 7fd2f77fe640 1 --2- 172.21.15.67:0/1527181402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8cf9f0 0x55924cccd510 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8005fd0 tx=0x7fd2e800df00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1383e459a1fa240a server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:42.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.434+0000 7fd2dd7fa640 1 -- 172.21.15.67:0/1527181402 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80e7030 con 0x55924c8cf9f0 2024-07-27T03:17:42.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.435+0000 7fd2dd7fa640 1 -- 172.21.15.67:0/1527181402 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2e80b5250 con 0x55924c8cf9f0 2024-07-27T03:17:42.438 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.435+0000 7fd2dd7fa640 1 -- 172.21.15.67:0/1527181402 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80abfd0 con 0x55924c8cf9f0 2024-07-27T03:17:42.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.436+0000 7fd3045e6740 1 -- 172.21.15.67:0/1527181402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8cf9f0 msgr2=0x55924cccd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:42.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.436+0000 7fd3045e6740 1 --2- 172.21.15.67:0/1527181402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8cf9f0 0x55924cccd510 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8005fd0 tx=0x7fd2e800df00 comp rx=0 tx=0).stop 2024-07-27T03:17:42.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.436+0000 7fd3045e6740 1 -- 172.21.15.67:0/1527181402 shutdown_connections 2024-07-27T03:17:42.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.436+0000 7fd3045e6740 1 --2- 172.21.15.67:0/1527181402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c8f0390 0x55924ccb3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.436+0000 7fd3045e6740 1 --2- 172.21.15.67:0/1527181402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55924c8e81b0 0x55924ccd2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.436+0000 7fd3045e6740 1 --2- 172.21.15.67:0/1527181402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8cf9f0 0x55924cccd510 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.436+0000 7fd3045e6740 1 -- 172.21.15.67:0/1527181402 >> 172.21.15.67:0/1527181402 conn(0x55924c905460 msgr2=0x55924ccd1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:42.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.436+0000 7fd3045e6740 1 -- 172.21.15.67:0/1527181402 shutdown_connections 2024-07-27T03:17:42.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.436+0000 7fd3045e6740 1 -- 172.21.15.67:0/1527181402 wait complete. 2024-07-27T03:17:42.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.437+0000 7fd3045e6740 1 -- start start 2024-07-27T03:17:42.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.437+0000 7fd3045e6740 1 Processor -- start 2024-07-27T03:17:42.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.437+0000 7fd3045e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8cf9f0 0x55924ccc4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:42.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.437+0000 7fd3045e6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c8e81b0 0x55924ccc4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:42.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.438+0000 7fd2f6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c8e81b0 0x55924ccc4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:42.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.438+0000 7fd2f6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c8e81b0 0x55924ccc4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42896/0 (socket says 172.21.15.67:42896) 2024-07-27T03:17:42.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.438+0000 7fd2f6ffd640 1 -- 172.21.15.67:0/1090974660 learned_addr learned my addr 172.21.15.67:0/1090974660 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:42.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.438+0000 7fd3045e6740 1 --2- 172.21.15.67:0/1090974660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55924c8f0390 0x55924ccc50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:42.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.438+0000 7fd2f77fe640 1 --2- 172.21.15.67:0/1090974660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8cf9f0 0x55924ccc4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:42.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.438+0000 7fd3045e6740 1 -- 172.21.15.67:0/1090974660 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f81af7e0 con 0x55924c8e81b0 2024-07-27T03:17:42.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.438+0000 7fd3045e6740 1 -- 172.21.15.67:0/1090974660 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f81af110 con 0x55924c8f0390 2024-07-27T03:17:42.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.438+0000 7fd3045e6740 1 -- 172.21.15.67:0/1090974660 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f81ac350 con 0x55924c8cf9f0 2024-07-27T03:17:42.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.438+0000 7fd3045e6740 10 client.0 _ll_register_callbacks cb 0x7fd2e80a8050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:17:42.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.438+0000 7fd2f7fff640 1 --2- 172.21.15.67:0/1090974660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55924c8f0390 0x55924ccc50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:42.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.438+0000 7fd2f6ffd640 1 -- 172.21.15.67:0/1090974660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8cf9f0 msgr2=0x55924ccc4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:42.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.438+0000 7fd2f6ffd640 1 --2- 172.21.15.67:0/1090974660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8cf9f0 0x55924ccc4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.438+0000 7fd2f6ffd640 1 -- 172.21.15.67:0/1090974660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55924c8f0390 msgr2=0x55924ccc50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:42.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.438+0000 7fd2f6ffd640 1 --2- 172.21.15.67:0/1090974660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55924c8f0390 0x55924ccc50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.438+0000 7fd2f6ffd640 1 -- 172.21.15.67:0/1090974660 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55924ccb86c0 con 0x55924c8e81b0 2024-07-27T03:17:42.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.438+0000 7fd2f77fe640 1 --2- 172.21.15.67:0/1090974660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8cf9f0 0x55924ccc4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:17:42.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.438+0000 7fd2f6ffd640 1 --2- 172.21.15.67:0/1090974660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c8e81b0 0x55924ccc4b80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009900 tx=0x7fd2ec009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105bc820b1f0cd53 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:42.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.438+0000 7fd2d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:17:42.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.439+0000 7fd2d6ffd640 1 -- 172.21.15.67:0/1090974660 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec03f040 con 0x55924c8e81b0 2024-07-27T03:17:42.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.439+0000 7fd2d6ffd640 1 -- 172.21.15.67:0/1090974660 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fd2ec050430 con 0x55924c8e81b0 2024-07-27T03:17:42.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.439+0000 7fd3045e6740 1 -- 172.21.15.67:0/1090974660 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55924cc2a510 con 0x55924c8e81b0 2024-07-27T03:17:42.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.439+0000 7fd2d6ffd640 1 -- 172.21.15.67:0/1090974660 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd2ec035040 con 0x55924c8e81b0 2024-07-27T03:17:42.450 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.439+0000 7fd3045e6740 10 client.15612 fetch_fsmap learned FSMap version 30 2024-07-27T03:17:42.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.439+0000 7fd3045e6740 1 -- 172.21.15.67:0/1090974660 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55924cd3b8c0 con 0x55924c8e81b0 2024-07-27T03:17:42.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.440+0000 7fd2d6ffd640 1 -- 172.21.15.67:0/1090974660 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55924cc2a510 con 0x55924c8e81b0 2024-07-27T03:17:42.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.440+0000 7fd2d6ffd640 1 -- 172.21.15.67:0/1090974660 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd2ec0038f0 con 0x55924c8e81b0 2024-07-27T03:17:42.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.440+0000 7fd3045e6740 10 client.15612 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:17:42.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.440+0000 7fd3045e6740 10 client.15612 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-27T03:17:42.451 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.440+0000 7fd3045e6740 20 client.15612 populate_metadata read hostname 'smithi067' 2024-07-27T03:17:42.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.440+0000 7fd3045e6740 1 --2- 172.21.15.67:0/1090974660 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55924c911070 0x55924cd3dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:42.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.440+0000 7fd3045e6740 4 client.15612 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-27T03:17:42.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.440+0000 7fd3045e6740 1 -- 172.21.15.67:0/1090974660 --> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55924cc27c50 con 0x55924c911070 2024-07-27T03:17:42.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.440+0000 7fd2f77fe640 1 --2- 172.21.15.67:0/1090974660 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55924c911070 0x55924cd3dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:42.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.440+0000 7fd2d6ffd640 10 client.15612 ms_handle_connect on v2:172.21.15.175:6838/1670568853 2024-07-27T03:17:42.452 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.441+0000 7fd2f77fe640 1 --2- 172.21.15.67:0/1090974660 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55924c911070 0x55924cd3dc10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=334df476b8a24753 server_cookie=f8e3eb4313491bcd in_seq=0 out_seq=0 2024-07-27T03:17:42.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.442+0000 7fd2d6ffd640 1 -- 172.21.15.67:0/1090974660 <== mds.2 v2:172.21.15.175:6838/1670568853 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55924cc27c50 con 0x55924c911070 2024-07-27T03:17:42.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.442+0000 7fd2d6ffd640 10 client.15612 handle_command_reply: tid=0 2024-07-27T03:17:42.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.442+0000 7fd3045e6740 1 client.15612 shutdown 2024-07-27T03:17:42.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.442+0000 7fd3045e6740 1 -- 172.21.15.67:0/1090974660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c8e81b0 msgr2=0x55924ccc4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:42.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.442+0000 7fd3045e6740 1 --2- 172.21.15.67:0/1090974660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c8e81b0 0x55924ccc4b80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009900 tx=0x7fd2ec009560 comp rx=0 tx=0).stop 2024-07-27T03:17:42.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.442+0000 7fd3045e6740 1 -- 172.21.15.67:0/1090974660 shutdown_connections 2024-07-27T03:17:42.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.442+0000 7fd3045e6740 1 --2- 172.21.15.67:0/1090974660 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55924c911070 0x55924cd3dc10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.442+0000 7fd2d6ffd640 0 client.15612 ms_handle_reset on v2:172.21.15.175:6838/1670568853 2024-07-27T03:17:42.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.442+0000 7fd3045e6740 1 --2- 172.21.15.67:0/1090974660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55924c8f0390 0x55924ccc50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.442+0000 7fd3045e6740 1 --2- 172.21.15.67:0/1090974660 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55924c8e81b0 0x55924ccc4b80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.442+0000 7fd3045e6740 1 --2- 172.21.15.67:0/1090974660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55924c8cf9f0 0x55924ccc4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.442+0000 7fd3045e6740 1 -- 172.21.15.67:0/1090974660 >> 172.21.15.67:0/1090974660 conn(0x55924c905460 msgr2=0x55924ccb8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:42.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.442+0000 7fd3045e6740 1 -- 172.21.15.67:0/1090974660 shutdown_connections 2024-07-27T03:17:42.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.442+0000 7fd3045e6740 1 -- 172.21.15.67:0/1090974660 wait complete. 2024-07-27T03:17:42.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.443+0000 7fd3045e6740 20 client.15612 trim_cache size 0 max 16384 2024-07-27T03:17:42.455 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-27T03:17:42.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.443+0000 7fd2fe061640 1 -- 172.21.15.67:0/892876300 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd2e00b3c50 msgr2=0x7fd2e00b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:42.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.443+0000 7fd2fe061640 1 --2- 172.21.15.67:0/892876300 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd2e00b3c50 0x7fd2e00b6160 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0005fd0 tx=0x7fd2f0009500 comp rx=0 tx=0).stop 2024-07-27T03:17:42.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.443+0000 7fd2fe061640 1 -- 172.21.15.67:0/892876300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2f81024d0 msgr2=0x7fd2f8198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:42.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.443+0000 7fd2fe061640 1 --2- 172.21.15.67:0/892876300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2f81024d0 0x7fd2f8198db0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800bea0 tx=0x7fd2e800bf90 comp rx=0 tx=0).stop 2024-07-27T03:17:42.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.444+0000 7fd2fe061640 1 -- 172.21.15.67:0/892876300 shutdown_connections 2024-07-27T03:17:42.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.444+0000 7fd2fe061640 1 --2- 172.21.15.67:0/892876300 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fd2e00b3c50 0x7fd2e00b6160 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.444+0000 7fd2fe061640 1 --2- 172.21.15.67:0/892876300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2f8108760 0x7fd2f819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.444+0000 7fd2fe061640 1 --2- 172.21.15.67:0/892876300 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2f8107e10 0x7fd2f8199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.444+0000 7fd2fe061640 1 --2- 172.21.15.67:0/892876300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2f81024d0 0x7fd2f8198db0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:42.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.444+0000 7fd2fe061640 1 -- 172.21.15.67:0/892876300 >> 172.21.15.67:0/892876300 conn(0x7fd2f80791a0 msgr2=0x7fd2f8104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:42.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.444+0000 7fd2fe061640 1 -- 172.21.15.67:0/892876300 shutdown_connections 2024-07-27T03:17:42.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:42.444+0000 7fd2fe061640 1 -- 172.21.15.67:0/892876300 wait complete. 2024-07-27T03:17:42.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:42 smithi067 ceph-mon[32814]: pgmap v464: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-07-27T03:17:42.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:42 smithi149 ceph-mon[37173]: pgmap v464: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-07-27T03:17:42.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:42 smithi175 ceph-mon[37125]: pgmap v464: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-07-27T03:17:43.189 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-27T03:17:43.494 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:17:44.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:44 smithi149 ceph-mon[37173]: pgmap v465: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:44.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:44 smithi067 ceph-mon[32814]: pgmap v465: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:44.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:44 smithi175 ceph-mon[37125]: pgmap v465: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:45.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.502+0000 7fbbbc376640 1 -- 172.21.15.67:0/128437926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbb4108770 msgr2=0x7fbbb410efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:45.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.502+0000 7fbbbc376640 1 --2- 172.21.15.67:0/128437926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbb4108770 0x7fbbb410efa0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009ca0 tx=0x7fbbac02a8c0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.502+0000 7fbbbc376640 1 -- 172.21.15.67:0/128437926 shutdown_connections 2024-07-27T03:17:45.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.502+0000 7fbbbc376640 1 --2- 172.21.15.67:0/128437926 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbb4108770 0x7fbbb410efa0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.502+0000 7fbbbc376640 1 --2- 172.21.15.67:0/128437926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb4107e20 0x7fbbb4108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.502+0000 7fbbbc376640 1 --2- 172.21.15.67:0/128437926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbb4102bb0 0x7fbbb4102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.502+0000 7fbbbc376640 1 -- 172.21.15.67:0/128437926 >> 172.21.15.67:0/128437926 conn(0x7fbbb40776d0 msgr2=0x7fbbb4077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:45.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.503+0000 7fbbbc376640 1 -- 172.21.15.67:0/128437926 shutdown_connections 2024-07-27T03:17:45.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.503+0000 7fbbbc376640 1 -- 172.21.15.67:0/128437926 wait complete. 2024-07-27T03:17:45.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.504+0000 7fbbbc376640 1 Processor -- start 2024-07-27T03:17:45.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.504+0000 7fbbbc376640 1 -- start start 2024-07-27T03:17:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.504+0000 7fbbbc376640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb4102bb0 0x7fbbb419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.504+0000 7fbbbc376640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbb4107e20 0x7fbbb419afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.504+0000 7fbbbc376640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbb4108770 0x7fbbb419b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.505+0000 7fbbba913640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbb4108770 0x7fbbb419b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.505+0000 7fbbba913640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbb4108770 0x7fbbb419b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41528/0 (socket says 172.21.15.67:41528) 2024-07-27T03:17:45.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.505+0000 7fbbb9911640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbb4107e20 0x7fbbb419afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.505+0000 7fbbba112640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb4102bb0 0x7fbbb419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.505+0000 7fbbb9911640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbb4107e20 0x7fbbb419afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.67:40878/0 (socket says 172.21.15.67:40878) 2024-07-27T03:17:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.505+0000 7fbbba913640 1 -- 172.21.15.67:0/1935395187 learned_addr learned my addr 172.21.15.67:0/1935395187 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.505+0000 7fbbbc376640 1 -- 172.21.15.67:0/1935395187 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb4112440 con 0x7fbbb4108770 2024-07-27T03:17:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.505+0000 7fbbbc376640 1 -- 172.21.15.67:0/1935395187 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41122c0 con 0x7fbbb4102bb0 2024-07-27T03:17:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.505+0000 7fbbbc376640 1 -- 172.21.15.67:0/1935395187 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41125c0 con 0x7fbbb4107e20 2024-07-27T03:17:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.506+0000 7fbbba913640 1 -- 172.21.15.67:0/1935395187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbb4107e20 msgr2=0x7fbbb419afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.506+0000 7fbbba913640 1 --2- 172.21.15.67:0/1935395187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbb4107e20 0x7fbbb419afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.506+0000 7fbbba913640 1 -- 172.21.15.67:0/1935395187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb4102bb0 msgr2=0x7fbbb419aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.506+0000 7fbbba913640 1 --2- 172.21.15.67:0/1935395187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb4102bb0 0x7fbbb419aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.506+0000 7fbbba913640 1 -- 172.21.15.67:0/1935395187 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb4198990 con 0x7fbbb4108770 2024-07-27T03:17:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.506+0000 7fbbba913640 1 --2- 172.21.15.67:0/1935395187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbb4108770 0x7fbbb419b520 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0316b0 tx=0x7fbbac0316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.506+0000 7fbbba112640 1 --2- 172.21.15.67:0/1935395187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb4102bb0 0x7fbbb419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:17:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.506+0000 7fbbb9911640 1 --2- 172.21.15.67:0/1935395187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbb4107e20 0x7fbbb419afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:17:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.507+0000 7fbbab7fe640 1 -- 172.21.15.67:0/1935395187 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac051070 con 0x7fbbb4108770 2024-07-27T03:17:45.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.507+0000 7fbbab7fe640 1 -- 172.21.15.67:0/1935395187 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbac003e20 con 0x7fbbb4108770 2024-07-27T03:17:45.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.507+0000 7fbbbc376640 1 -- 172.21.15.67:0/1935395187 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb4198c20 con 0x7fbbb4108770 2024-07-27T03:17:45.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.507+0000 7fbbab7fe640 1 -- 172.21.15.67:0/1935395187 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac02f660 con 0x7fbbb4108770 2024-07-27T03:17:45.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.507+0000 7fbbbc376640 1 -- 172.21.15.67:0/1935395187 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb41991b0 con 0x7fbbb4108770 2024-07-27T03:17:45.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.510+0000 7fbbc28fb740 1 Processor -- start 2024-07-27T03:17:45.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.513+0000 7fbbab7fe640 1 -- 172.21.15.67:0/1935395187 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7fbbac054070 con 0x7fbbb4108770 2024-07-27T03:17:45.516 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.513+0000 7fbbc28fb740 1 -- start start 2024-07-27T03:17:45.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.513+0000 7fbbc28fb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b4684070 0x55d0b4a28e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:45.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.514+0000 7fbbc28fb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b465b1b0 0x55d0b4a29370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:45.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.514+0000 7fbbc28fb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b4663390 0x55d0b4a298b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:45.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.514+0000 7fbbc28fb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d0b4911400 con 0x55d0b465b1b0 2024-07-27T03:17:45.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.514+0000 7fbbc28fb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d0b499fb20 con 0x55d0b4684070 2024-07-27T03:17:45.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.514+0000 7fbbc28fb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d0b498f020 con 0x55d0b4663390 2024-07-27T03:17:45.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.514+0000 7fbbb9911640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b4684070 0x55d0b4a28e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:45.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.514+0000 7fbbb9911640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b4684070 0x55d0b4a28e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.67:48892/0 (socket says 172.21.15.67:48892) 2024-07-27T03:17:45.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.514+0000 7fbbb9911640 1 -- 172.21.15.67:0/3263699894 learned_addr learned my addr 172.21.15.67:0/3263699894 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:45.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.514+0000 7fbbb9911640 1 --2- 172.21.15.67:0/3263699894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b4663390 0x55d0b4a298b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:45.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.514+0000 7fbbba112640 1 --2- 172.21.15.67:0/3263699894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b465b1b0 0x55d0b4a29370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:45.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.515+0000 7fbbb9911640 1 -- 172.21.15.67:0/3263699894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b4663390 msgr2=0x55d0b4a298b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:45.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.515+0000 7fbbb9911640 1 --2- 172.21.15.67:0/3263699894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b4663390 0x55d0b4a298b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.515+0000 7fbbb9911640 1 -- 172.21.15.67:0/3263699894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b465b1b0 msgr2=0x55d0b4a29370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:45.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.515+0000 7fbbb9911640 1 --2- 172.21.15.67:0/3263699894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b465b1b0 0x55d0b4a29370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.515+0000 7fbbb9911640 1 -- 172.21.15.67:0/3263699894 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0b4924770 con 0x55d0b4684070 2024-07-27T03:17:45.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.515+0000 7fbbab7fe640 1 --2- 172.21.15.67:0/1935395187 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbba00b3cd0 0x7fbba00b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:45.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.515+0000 7fbbba112640 1 --2- 172.21.15.67:0/1935395187 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbba00b3cd0 0x7fbba00b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:45.519 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.515+0000 7fbbab7fe640 1 -- 172.21.15.67:0/1935395187 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fbbac10cda0 con 0x7fbbb4108770 2024-07-27T03:17:45.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.516+0000 7fbbb9911640 1 --2- 172.21.15.67:0/3263699894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b4684070 0x55d0b4a28e30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbba4014460 tx=0x7fbba40095d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b7189c30850f92b server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:45.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.516+0000 7fbba8d9a640 1 -- 172.21.15.67:0/3263699894 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba401a020 con 0x55d0b4684070 2024-07-27T03:17:45.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.516+0000 7fbba8d9a640 1 -- 172.21.15.67:0/3263699894 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbba4004230 con 0x55d0b4684070 2024-07-27T03:17:45.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.516+0000 7fbba8d9a640 1 -- 172.21.15.67:0/3263699894 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba4010660 con 0x55d0b4684070 2024-07-27T03:17:45.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.516+0000 7fbbba112640 1 --2- 172.21.15.67:0/1935395187 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbba00b3cd0 0x7fbba00b61e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55d0b4a27ab0 tx=0x7fbbb0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:45.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.517+0000 7fbbc28fb740 1 -- 172.21.15.67:0/3263699894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b4684070 msgr2=0x55d0b4a28e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:45.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.517+0000 7fbbc28fb740 1 --2- 172.21.15.67:0/3263699894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b4684070 0x55d0b4a28e30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbba4014460 tx=0x7fbba40095d0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.517+0000 7fbbc28fb740 1 -- 172.21.15.67:0/3263699894 shutdown_connections 2024-07-27T03:17:45.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.517+0000 7fbbc28fb740 1 --2- 172.21.15.67:0/3263699894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b4663390 0x55d0b4a298b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.517+0000 7fbbc28fb740 1 --2- 172.21.15.67:0/3263699894 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b465b1b0 0x55d0b4a29370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.517+0000 7fbbc28fb740 1 --2- 172.21.15.67:0/3263699894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b4684070 0x55d0b4a28e30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.517+0000 7fbbc28fb740 1 -- 172.21.15.67:0/3263699894 >> 172.21.15.67:0/3263699894 conn(0x55d0b4678460 msgr2=0x55d0b4a26d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:45.521 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.517+0000 7fbbc28fb740 1 -- 172.21.15.67:0/3263699894 shutdown_connections 2024-07-27T03:17:45.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.517+0000 7fbbc28fb740 1 -- 172.21.15.67:0/3263699894 wait complete. 2024-07-27T03:17:45.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.518+0000 7fbbc28fb740 1 -- start start 2024-07-27T03:17:45.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.518+0000 7fbbc28fb740 1 Processor -- start 2024-07-27T03:17:45.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.518+0000 7fbbc28fb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b465b1b0 0x55d0b4a34a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:45.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.518+0000 7fbbc28fb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b4663390 0x55d0b4a34f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:45.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.518+0000 7fbbc28fb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b4684070 0x55d0b4a40780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:45.522 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.518+0000 7fbbc28fb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d0b4931900 con 0x55d0b465b1b0 2024-07-27T03:17:45.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.518+0000 7fbbc28fb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d0b4993060 con 0x55d0b4663390 2024-07-27T03:17:45.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.518+0000 7fbbc28fb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d0b47f2700 con 0x55d0b4684070 2024-07-27T03:17:45.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.519+0000 7fbbc28fb740 10 client.0 _ll_register_callbacks cb 0x55d0b4939d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:17:45.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.519+0000 7fbbba112640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b465b1b0 0x55d0b4a34a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:45.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.519+0000 7fbbba112640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b465b1b0 0x55d0b4a34a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41546/0 (socket says 172.21.15.67:41546) 2024-07-27T03:17:45.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.519+0000 7fbbba112640 1 -- 172.21.15.67:0/2161724953 learned_addr learned my addr 172.21.15.67:0/2161724953 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:45.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.519+0000 7fbbba913640 1 --2- 172.21.15.67:0/2161724953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b4684070 0x55d0b4a40780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:45.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.519+0000 7fbbb9911640 1 --2- 172.21.15.67:0/2161724953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b4663390 0x55d0b4a34f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:45.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.519+0000 7fbbba112640 1 -- 172.21.15.67:0/2161724953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b4684070 msgr2=0x55d0b4a40780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:45.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.519+0000 7fbbba112640 1 --2- 172.21.15.67:0/2161724953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b4684070 0x55d0b4a40780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.519+0000 7fbbba112640 1 -- 172.21.15.67:0/2161724953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b4663390 msgr2=0x55d0b4a34f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:45.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.519+0000 7fbbba112640 1 --2- 172.21.15.67:0/2161724953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b4663390 0x55d0b4a34f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.519+0000 7fbbba112640 1 -- 172.21.15.67:0/2161724953 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0b4935bf0 con 0x55d0b465b1b0 2024-07-27T03:17:45.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.520+0000 7fbbba112640 1 --2- 172.21.15.67:0/2161724953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b465b1b0 0x55d0b4a34a50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fbbb001dc60 tx=0x7fbbb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18e2fad15821253 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:45.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.520+0000 7fbb9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:17:45.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.520+0000 7fbb9dffb640 1 -- 172.21.15.67:0/2161724953 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb001d250 con 0x55d0b465b1b0 2024-07-27T03:17:45.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.520+0000 7fbb9dffb640 1 -- 172.21.15.67:0/2161724953 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fbbb001b460 con 0x55d0b465b1b0 2024-07-27T03:17:45.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.520+0000 7fbbc28fb740 1 -- 172.21.15.67:0/2161724953 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbba4004230 con 0x55d0b465b1b0 2024-07-27T03:17:45.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.522+0000 7fbb9dffb640 1 -- 172.21.15.67:0/2161724953 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbbb001b660 con 0x55d0b465b1b0 2024-07-27T03:17:45.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.522+0000 7fbbc28fb740 10 client.15624 fetch_fsmap learned FSMap version 30 2024-07-27T03:17:45.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.522+0000 7fbbc28fb740 1 -- 172.21.15.67:0/2161724953 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d0b46ef650 con 0x55d0b465b1b0 2024-07-27T03:17:45.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.522+0000 7fbb9dffb640 1 -- 172.21.15.67:0/2161724953 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbba4004230 con 0x55d0b465b1b0 2024-07-27T03:17:45.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.522+0000 7fbb9dffb640 1 -- 172.21.15.67:0/2161724953 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fbbb0024070 con 0x55d0b465b1b0 2024-07-27T03:17:45.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.522+0000 7fbbc28fb740 10 client.15624 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:17:45.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.522+0000 7fbbc28fb740 10 client.15624 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-07-27T03:17:45.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.522+0000 7fbbc28fb740 20 client.15624 populate_metadata read hostname 'smithi067' 2024-07-27T03:17:45.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.522+0000 7fbbc28fb740 1 --2- 172.21.15.67:0/2161724953 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55d0b46429f0 0x55d0b4a46890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:45.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.522+0000 7fbbc28fb740 4 client.15624 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:17:45.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.522+0000 7fbbc28fb740 1 -- 172.21.15.67:0/2161724953 --> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0b49a1a70 con 0x55d0b46429f0 2024-07-27T03:17:45.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.523+0000 7fbbb9911640 1 --2- 172.21.15.67:0/2161724953 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55d0b46429f0 0x55d0b4a46890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:45.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.524+0000 7fbb9dffb640 10 client.15624 ms_handle_connect on v2:172.21.15.149:6836/2932951845 2024-07-27T03:17:45.527 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.524+0000 7fbbb9911640 1 --2- 172.21.15.67:0/2161724953 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55d0b46429f0 0x55d0b4a46890 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=742d2fc234821523 server_cookie=8b8121906782ffe9 in_seq=0 out_seq=0 2024-07-27T03:17:45.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.524+0000 7fbb9dffb640 1 -- 172.21.15.67:0/2161724953 <== mds.1 v2:172.21.15.149:6836/2932951845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0b49a1a70 con 0x55d0b46429f0 2024-07-27T03:17:45.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.524+0000 7fbb9dffb640 10 client.15624 handle_command_reply: tid=0 2024-07-27T03:17:45.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.525+0000 7fbbc28fb740 1 client.15624 shutdown 2024-07-27T03:17:45.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.525+0000 7fbbc28fb740 1 -- 172.21.15.67:0/2161724953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b465b1b0 msgr2=0x55d0b4a34a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:45.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.525+0000 7fbbc28fb740 1 --2- 172.21.15.67:0/2161724953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b465b1b0 0x55d0b4a34a50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fbbb001dc60 tx=0x7fbbb00046c0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.525+0000 7fbbc28fb740 1 -- 172.21.15.67:0/2161724953 shutdown_connections 2024-07-27T03:17:45.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.525+0000 7fbbc28fb740 1 --2- 172.21.15.67:0/2161724953 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55d0b46429f0 0x55d0b4a46890 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.525+0000 7fbb9dffb640 0 client.15624 ms_handle_reset on v2:172.21.15.149:6836/2932951845 2024-07-27T03:17:45.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.525+0000 7fbbc28fb740 1 --2- 172.21.15.67:0/2161724953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b4684070 0x55d0b4a40780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.525+0000 7fbbc28fb740 1 --2- 172.21.15.67:0/2161724953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b4663390 0x55d0b4a34f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.525+0000 7fbbc28fb740 1 --2- 172.21.15.67:0/2161724953 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b465b1b0 0x55d0b4a34a50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.525+0000 7fbbc28fb740 1 -- 172.21.15.67:0/2161724953 >> 172.21.15.67:0/2161724953 conn(0x55d0b4678460 msgr2=0x55d0b4a2c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:45.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.525+0000 7fbbc28fb740 1 -- 172.21.15.67:0/2161724953 shutdown_connections 2024-07-27T03:17:45.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.525+0000 7fbbc28fb740 1 -- 172.21.15.67:0/2161724953 wait complete. 2024-07-27T03:17:45.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.525+0000 7fbbc28fb740 20 client.15624 trim_cache size 0 max 16384 2024-07-27T03:17:45.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.539+0000 7fbbc28fb740 1 Processor -- start 2024-07-27T03:17:45.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.539+0000 7fbbc28fb740 1 -- start start 2024-07-27T03:17:45.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.539+0000 7fbbc28fb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b46429f0 0x55d0b4a40570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:45.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.539+0000 7fbbc28fb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b465b1b0 0x55d0b4a40ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:45.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.539+0000 7fbbc28fb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b4663390 0x55d0b4a4c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:45.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.539+0000 7fbbb9911640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b465b1b0 0x55d0b4a40ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:45.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.539+0000 7fbbb9911640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b465b1b0 0x55d0b4a40ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41560/0 (socket says 172.21.15.67:41560) 2024-07-27T03:17:45.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.539+0000 7fbbb9911640 1 -- 172.21.15.67:0/1990908430 learned_addr learned my addr 172.21.15.67:0/1990908430 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:45.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.539+0000 7fbbc28fb740 1 -- 172.21.15.67:0/1990908430 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41ae430 con 0x55d0b465b1b0 2024-07-27T03:17:45.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.539+0000 7fbbc28fb740 1 -- 172.21.15.67:0/1990908430 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41aff70 con 0x55d0b46429f0 2024-07-27T03:17:45.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.539+0000 7fbbc28fb740 1 -- 172.21.15.67:0/1990908430 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41aeb00 con 0x55d0b4663390 2024-07-27T03:17:45.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.539+0000 7fbbba112640 1 --2- 172.21.15.67:0/1990908430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b46429f0 0x55d0b4a40570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:45.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.540+0000 7fbbba913640 1 --2- 172.21.15.67:0/1990908430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b4663390 0x55d0b4a4c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:45.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.540+0000 7fbbb9911640 1 -- 172.21.15.67:0/1990908430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b4663390 msgr2=0x55d0b4a4c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:45.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.540+0000 7fbbb9911640 1 --2- 172.21.15.67:0/1990908430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b4663390 0x55d0b4a4c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.540+0000 7fbbb9911640 1 -- 172.21.15.67:0/1990908430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b46429f0 msgr2=0x55d0b4a40570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:45.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.540+0000 7fbbb9911640 1 --2- 172.21.15.67:0/1990908430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b46429f0 0x55d0b4a40570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.545 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.540+0000 7fbbb9911640 1 -- 172.21.15.67:0/1990908430 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0b4a2b120 con 0x55d0b465b1b0 2024-07-27T03:17:45.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.540+0000 7fbbba112640 1 --2- 172.21.15.67:0/1990908430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b46429f0 0x55d0b4a40570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:17:45.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.540+0000 7fbbb9911640 1 --2- 172.21.15.67:0/1990908430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b465b1b0 0x55d0b4a40ab0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbba4014430 tx=0x7fbba40097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2092f0bc9babf90 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:45.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.540+0000 7fbb9ffff640 1 -- 172.21.15.67:0/1990908430 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba403e020 con 0x55d0b465b1b0 2024-07-27T03:17:45.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.540+0000 7fbb9ffff640 1 -- 172.21.15.67:0/1990908430 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbba40054f0 con 0x55d0b465b1b0 2024-07-27T03:17:45.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.540+0000 7fbb9ffff640 1 -- 172.21.15.67:0/1990908430 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba401a040 con 0x55d0b465b1b0 2024-07-27T03:17:45.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.542+0000 7fbbc28fb740 1 -- 172.21.15.67:0/1990908430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b465b1b0 msgr2=0x55d0b4a40ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:45.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.542+0000 7fbbc28fb740 1 --2- 172.21.15.67:0/1990908430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b465b1b0 0x55d0b4a40ab0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbba4014430 tx=0x7fbba40097b0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.549 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.542+0000 7fbbc28fb740 1 -- 172.21.15.67:0/1990908430 shutdown_connections 2024-07-27T03:17:45.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.542+0000 7fbbc28fb740 1 --2- 172.21.15.67:0/1990908430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b4663390 0x55d0b4a4c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.542+0000 7fbbc28fb740 1 --2- 172.21.15.67:0/1990908430 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b465b1b0 0x55d0b4a40ab0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.542+0000 7fbbc28fb740 1 --2- 172.21.15.67:0/1990908430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b46429f0 0x55d0b4a40570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.542+0000 7fbbc28fb740 1 -- 172.21.15.67:0/1990908430 >> 172.21.15.67:0/1990908430 conn(0x55d0b4678460 msgr2=0x55d0b4a330d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:45.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.542+0000 7fbbc28fb740 1 -- 172.21.15.67:0/1990908430 shutdown_connections 2024-07-27T03:17:45.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.542+0000 7fbbc28fb740 1 -- 172.21.15.67:0/1990908430 wait complete. 2024-07-27T03:17:45.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.542+0000 7fbbc28fb740 1 -- start start 2024-07-27T03:17:45.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.542+0000 7fbbc28fb740 1 Processor -- start 2024-07-27T03:17:45.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.543+0000 7fbbc28fb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b46429f0 0x55d0b4a38e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:45.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.543+0000 7fbbc28fb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b465b1b0 0x55d0b4a39430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:45.550 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.543+0000 7fbbc28fb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b4663390 0x55d0b4aa7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.543+0000 7fbbc28fb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41af8a0 con 0x55d0b4663390 2024-07-27T03:17:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.543+0000 7fbbc28fb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41af1d0 con 0x55d0b46429f0 2024-07-27T03:17:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.543+0000 7fbbc28fb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41ac410 con 0x55d0b465b1b0 2024-07-27T03:17:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.543+0000 7fbbba913640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b4663390 0x55d0b4aa7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.543+0000 7fbbc28fb740 10 client.0 _ll_register_callbacks cb 0x7fbbb00099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:17:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.543+0000 7fbbba913640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b4663390 0x55d0b4aa7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41564/0 (socket says 172.21.15.67:41564) 2024-07-27T03:17:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.543+0000 7fbbba913640 1 -- 172.21.15.67:0/809339186 learned_addr learned my addr 172.21.15.67:0/809339186 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.543+0000 7fbbba112640 1 --2- 172.21.15.67:0/809339186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b46429f0 0x55d0b4a38e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.543+0000 7fbbb9911640 1 --2- 172.21.15.67:0/809339186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b465b1b0 0x55d0b4a39430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.543+0000 7fbbb9911640 1 -- 172.21.15.67:0/809339186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b46429f0 msgr2=0x55d0b4a38e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.544+0000 7fbbb9911640 1 --2- 172.21.15.67:0/809339186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b46429f0 0x55d0b4a38e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.544+0000 7fbbb9911640 1 -- 172.21.15.67:0/809339186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b4663390 msgr2=0x55d0b4aa7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:45.551 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.544+0000 7fbbb9911640 1 --2- 172.21.15.67:0/809339186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b4663390 0x55d0b4aa7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.544+0000 7fbbb9911640 1 -- 172.21.15.67:0/809339186 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0b4aa8660 con 0x55d0b465b1b0 2024-07-27T03:17:45.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.544+0000 7fbbba112640 1 --2- 172.21.15.67:0/809339186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b46429f0 0x55d0b4a38e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:17:45.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.544+0000 7fbbb9911640 1 --2- 172.21.15.67:0/809339186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b465b1b0 0x55d0b4a39430 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009ee0 tx=0x7fbba4009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13f71792b67d5fa0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:45.552 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.544+0000 7fbb9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-27T03:17:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.544+0000 7fbb9dffb640 1 -- 172.21.15.67:0/809339186 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba403e070 con 0x55d0b465b1b0 2024-07-27T03:17:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.544+0000 7fbb9dffb640 1 -- 172.21.15.67:0/809339186 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fbba40297c0 con 0x55d0b465b1b0 2024-07-27T03:17:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.544+0000 7fbbba913640 1 --2- 172.21.15.67:0/809339186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b4663390 0x55d0b4aa7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:17:45.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.544+0000 7fbbc28fb740 1 -- 172.21.15.67:0/809339186 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbba40054f0 con 0x55d0b465b1b0 2024-07-27T03:17:45.553 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-27T03:17:45.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.544+0000 7fbb9dffb640 1 -- 172.21.15.67:0/809339186 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbba401a040 con 0x55d0b465b1b0 2024-07-27T03:17:45.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.544+0000 7fbbc28fb740 10 client.25252 fetch_fsmap learned FSMap version 30 2024-07-27T03:17:45.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.544+0000 7fbbc28fb740 1 -- 172.21.15.67:0/809339186 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d0b4ad1970 con 0x55d0b465b1b0 2024-07-27T03:17:45.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.545+0000 7fbb9dffb640 1 -- 172.21.15.67:0/809339186 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbba4043400 con 0x55d0b465b1b0 2024-07-27T03:17:45.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.545+0000 7fbb9dffb640 1 -- 172.21.15.67:0/809339186 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fbba4032030 con 0x55d0b465b1b0 2024-07-27T03:17:45.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.545+0000 7fbbc28fb740 10 client.25252 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:17:45.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.545+0000 7fbbc28fb740 10 client.25252 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-07-27T03:17:45.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.545+0000 7fbbc28fb740 20 client.25252 populate_metadata read hostname 'smithi067' 2024-07-27T03:17:45.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.545+0000 7fbbc28fb740 1 --2- 172.21.15.67:0/809339186 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55d0b4684070 0x55d0b4ad3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:45.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.545+0000 7fbbc28fb740 4 client.25252 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-27T03:17:45.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.545+0000 7fbbc28fb740 1 -- 172.21.15.67:0/809339186 --> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fbba40054f0 con 0x55d0b4684070 2024-07-27T03:17:45.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.545+0000 7fbbba112640 1 --2- 172.21.15.67:0/809339186 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55d0b4684070 0x55d0b4ad3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:45.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.546+0000 7fbb9dffb640 10 client.25252 ms_handle_connect on v2:172.21.15.149:6836/2932951845 2024-07-27T03:17:45.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.546+0000 7fbbba112640 1 --2- 172.21.15.67:0/809339186 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55d0b4684070 0x55d0b4ad3e50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6cb898d963953323 server_cookie=3c08892b38a9a9f4 in_seq=0 out_seq=0 2024-07-27T03:17:45.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.546+0000 7fbb9dffb640 1 -- 172.21.15.67:0/809339186 <== mds.1 v2:172.21.15.149:6836/2932951845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fbba40054f0 con 0x55d0b4684070 2024-07-27T03:17:45.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.546+0000 7fbb9dffb640 10 client.25252 handle_command_reply: tid=0 2024-07-27T03:17:45.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.546+0000 7fbbc28fb740 1 client.25252 shutdown 2024-07-27T03:17:45.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.546+0000 7fbbc28fb740 1 -- 172.21.15.67:0/809339186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b465b1b0 msgr2=0x55d0b4a39430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:45.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.546+0000 7fbbc28fb740 1 --2- 172.21.15.67:0/809339186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b465b1b0 0x55d0b4a39430 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009ee0 tx=0x7fbba4009610 comp rx=0 tx=0).stop 2024-07-27T03:17:45.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.546+0000 7fbbc28fb740 1 -- 172.21.15.67:0/809339186 shutdown_connections 2024-07-27T03:17:45.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.546+0000 7fbbc28fb740 1 --2- 172.21.15.67:0/809339186 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55d0b4684070 0x55d0b4ad3e50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.546+0000 7fbbc28fb740 1 --2- 172.21.15.67:0/809339186 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0b4663390 0x55d0b4aa7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.546+0000 7fbbc28fb740 1 --2- 172.21.15.67:0/809339186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0b465b1b0 0x55d0b4a39430 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.546+0000 7fbbc28fb740 1 --2- 172.21.15.67:0/809339186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0b46429f0 0x55d0b4a38e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.546+0000 7fbbc28fb740 1 -- 172.21.15.67:0/809339186 >> 172.21.15.67:0/809339186 conn(0x55d0b4678460 msgr2=0x55d0b4a29e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:45.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.546+0000 7fbb9dffb640 0 client.25252 ms_handle_reset on v2:172.21.15.149:6836/2932951845 2024-07-27T03:17:45.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.546+0000 7fbbc28fb740 1 -- 172.21.15.67:0/809339186 shutdown_connections 2024-07-27T03:17:45.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.546+0000 7fbbc28fb740 1 -- 172.21.15.67:0/809339186 wait complete. 2024-07-27T03:17:45.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.547+0000 7fbbc28fb740 20 client.25252 trim_cache size 0 max 16384 2024-07-27T03:17:45.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.547+0000 7fbbbc376640 1 -- 172.21.15.67:0/1935395187 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbba00b3cd0 msgr2=0x7fbba00b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:45.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.547+0000 7fbbbc376640 1 --2- 172.21.15.67:0/1935395187 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbba00b3cd0 0x7fbba00b61e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55d0b4a27ab0 tx=0x7fbbb0005f90 comp rx=0 tx=0).stop 2024-07-27T03:17:45.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.547+0000 7fbbbc376640 1 -- 172.21.15.67:0/1935395187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbb4108770 msgr2=0x7fbbb419b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:45.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.547+0000 7fbbbc376640 1 --2- 172.21.15.67:0/1935395187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbb4108770 0x7fbbb419b520 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0316b0 tx=0x7fbbac0316e0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.547+0000 7fbbbc376640 1 -- 172.21.15.67:0/1935395187 shutdown_connections 2024-07-27T03:17:45.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.547+0000 7fbbbc376640 1 --2- 172.21.15.67:0/1935395187 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fbba00b3cd0 0x7fbba00b61e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.547+0000 7fbbbc376640 1 --2- 172.21.15.67:0/1935395187 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbbb4108770 0x7fbbb419b520 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.547+0000 7fbbbc376640 1 --2- 172.21.15.67:0/1935395187 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbb4107e20 0x7fbbb419afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.547+0000 7fbbbc376640 1 --2- 172.21.15.67:0/1935395187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb4102bb0 0x7fbbb419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:45.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.547+0000 7fbbbc376640 1 -- 172.21.15.67:0/1935395187 >> 172.21.15.67:0/1935395187 conn(0x7fbbb40776d0 msgr2=0x7fbbb40fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:45.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.548+0000 7fbbbc376640 1 -- 172.21.15.67:0/1935395187 shutdown_connections 2024-07-27T03:17:45.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:45.548+0000 7fbbbc376640 1 -- 172.21.15.67:0/1935395187 wait complete. 2024-07-27T03:17:46.389 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-27T03:17:46.630 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:46 smithi067 ceph-mon[32814]: pgmap v466: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:17:46.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:46 smithi149 ceph-mon[37173]: pgmap v466: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:17:46.759 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:17:46.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:46 smithi175 ceph-mon[37125]: pgmap v466: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:17:48.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:48 smithi067 ceph-mon[32814]: pgmap v467: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:17:48.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:48 smithi149 ceph-mon[37173]: pgmap v467: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:17:48.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:48 smithi175 ceph-mon[37125]: pgmap v467: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:17:49.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.350+0000 7f7b8e82c640 1 -- 172.21.15.67:0/1514246974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b881042f0 msgr2=0x7f7b88104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:49.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.350+0000 7f7b8e82c640 1 --2- 172.21.15.67:0/1514246974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b881042f0 0x7f7b88104700 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009900 tx=0x7f7b7802a410 comp rx=0 tx=0).stop 2024-07-27T03:17:49.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.350+0000 7f7b8e82c640 1 -- 172.21.15.67:0/1514246974 shutdown_connections 2024-07-27T03:17:49.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.350+0000 7f7b8e82c640 1 --2- 172.21.15.67:0/1514246974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b8810d8a0 0x7f7b8810fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.350+0000 7f7b8e82c640 1 --2- 172.21.15.67:0/1514246974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b88104d20 0x7f7b8810d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.350+0000 7f7b8e82c640 1 --2- 172.21.15.67:0/1514246974 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b881042f0 0x7f7b88104700 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.350+0000 7f7b8e82c640 1 -- 172.21.15.67:0/1514246974 >> 172.21.15.67:0/1514246974 conn(0x7f7b880791a0 msgr2=0x7f7b880795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:49.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.350+0000 7f7b8e82c640 1 -- 172.21.15.67:0/1514246974 shutdown_connections 2024-07-27T03:17:49.354 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.351+0000 7f7b8e82c640 1 -- 172.21.15.67:0/1514246974 wait complete. 2024-07-27T03:17:49.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.351+0000 7f7b8e82c640 1 Processor -- start 2024-07-27T03:17:49.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.352+0000 7f7b8e82c640 1 -- start start 2024-07-27T03:17:49.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.352+0000 7f7b8e82c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b881042f0 0x7f7b88199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:49.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.352+0000 7f7b8e82c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b88104d20 0x7f7b8819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:49.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.352+0000 7f7b8e82c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b8810d8a0 0x7f7b8819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:49.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.352+0000 7f7b8e82c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7b88113510 con 0x7f7b8810d8a0 2024-07-27T03:17:49.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.352+0000 7f7b8e82c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7b88113390 con 0x7f7b881042f0 2024-07-27T03:17:49.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.352+0000 7f7b8e82c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b88113690 con 0x7f7b88104d20 2024-07-27T03:17:49.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.352+0000 7f7b8cdc9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b8810d8a0 0x7f7b8819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:49.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.352+0000 7f7b8cdc9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b8810d8a0 0x7f7b8819f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41580/0 (socket says 172.21.15.67:41580) 2024-07-27T03:17:49.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.352+0000 7f7b8cdc9640 1 -- 172.21.15.67:0/2216594745 learned_addr learned my addr 172.21.15.67:0/2216594745 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:49.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.353+0000 7f7b87fff640 1 --2- 172.21.15.67:0/2216594745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b881042f0 0x7f7b88199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:49.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.353+0000 7f7b877fe640 1 --2- 172.21.15.67:0/2216594745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b88104d20 0x7f7b8819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:49.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.353+0000 7f7b8cdc9640 1 -- 172.21.15.67:0/2216594745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b88104d20 msgr2=0x7f7b8819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:49.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.353+0000 7f7b8cdc9640 1 --2- 172.21.15.67:0/2216594745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b88104d20 0x7f7b8819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.353+0000 7f7b8cdc9640 1 -- 172.21.15.67:0/2216594745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b881042f0 msgr2=0x7f7b88199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:49.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.353+0000 7f7b8cdc9640 1 --2- 172.21.15.67:0/2216594745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b881042f0 0x7f7b88199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.353+0000 7f7b8cdc9640 1 -- 172.21.15.67:0/2216594745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b8819fe90 con 0x7f7b8810d8a0 2024-07-27T03:17:49.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.353+0000 7f7b8cdc9640 1 --2- 172.21.15.67:0/2216594745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b8810d8a0 0x7f7b8819f760 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000eb10 tx=0x7f7b8000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:49.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.353+0000 7f7b857fa640 1 -- 172.21.15.67:0/2216594745 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80004b20 con 0x7f7b8810d8a0 2024-07-27T03:17:49.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.354+0000 7f7b857fa640 1 -- 172.21.15.67:0/2216594745 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b80004cc0 con 0x7f7b8810d8a0 2024-07-27T03:17:49.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.354+0000 7f7b857fa640 1 -- 172.21.15.67:0/2216594745 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80010670 con 0x7f7b8810d8a0 2024-07-27T03:17:49.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.354+0000 7f7b8e82c640 1 -- 172.21.15.67:0/2216594745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b881a0180 con 0x7f7b8810d8a0 2024-07-27T03:17:49.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.355+0000 7f7b8e82c640 1 -- 172.21.15.67:0/2216594745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b881a68e0 con 0x7f7b8810d8a0 2024-07-27T03:17:49.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.357+0000 7f7b857fa640 1 -- 172.21.15.67:0/2216594745 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f7b80005000 con 0x7f7b8810d8a0 2024-07-27T03:17:49.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.358+0000 7f7b857fa640 1 --2- 172.21.15.67:0/2216594745 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7b700b3c50 0x7f7b700b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:49.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.358+0000 7f7b857fa640 1 -- 172.21.15.67:0/2216594745 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f7b800e44b0 con 0x7f7b8810d8a0 2024-07-27T03:17:49.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.358+0000 7f7b87fff640 1 --2- 172.21.15.67:0/2216594745 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7b700b3c50 0x7f7b700b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:49.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.358+0000 7f7b94db1740 1 Processor -- start 2024-07-27T03:17:49.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.358+0000 7f7b94db1740 1 -- start start 2024-07-27T03:17:49.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.359+0000 7f7b94db1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd397c070 0x559bd3d20d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:49.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.359+0000 7f7b94db1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd39531b0 0x559bd3d21280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.359+0000 7f7b94db1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd395b390 0x559bd3d217c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.359+0000 7f7b94db1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559bd3c09400 con 0x559bd395b390 2024-07-27T03:17:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.359+0000 7f7b94db1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559bd3c97b20 con 0x559bd39531b0 2024-07-27T03:17:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.359+0000 7f7b94db1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559bd3c87020 con 0x559bd397c070 2024-07-27T03:17:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.359+0000 7f7b8cdc9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd395b390 0x559bd3d217c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.359+0000 7f7b8cdc9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd395b390 0x559bd3d217c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41586/0 (socket says 172.21.15.67:41586) 2024-07-27T03:17:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.359+0000 7f7b8cdc9640 1 -- 172.21.15.67:0/2157319551 learned_addr learned my addr 172.21.15.67:0/2157319551 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.359+0000 7f7b877fe640 1 --2- 172.21.15.67:0/2157319551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd39531b0 0x559bd3d21280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.360+0000 7f7b87fff640 1 --2- 172.21.15.67:0/2216594745 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7b700b3c50 0x7f7b700b6160 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7b78006010 tx=0x7f7b78005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.360+0000 7f7b87fff640 1 --2- 172.21.15.67:0/2157319551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd397c070 0x559bd3d20d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.360+0000 7f7b8cdc9640 1 -- 172.21.15.67:0/2157319551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd397c070 msgr2=0x559bd3d20d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.360+0000 7f7b8cdc9640 1 --2- 172.21.15.67:0/2157319551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd397c070 0x559bd3d20d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.360+0000 7f7b8cdc9640 1 -- 172.21.15.67:0/2157319551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd39531b0 msgr2=0x559bd3d21280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.360+0000 7f7b8cdc9640 1 --2- 172.21.15.67:0/2157319551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd39531b0 0x559bd3d21280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.360+0000 7f7b8cdc9640 1 -- 172.21.15.67:0/2157319551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bd3c1c770 con 0x559bd395b390 2024-07-27T03:17:49.363 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.360+0000 7f7b8cdc9640 1 --2- 172.21.15.67:0/2157319551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd395b390 0x559bd3d217c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7b800aafe0 tx=0x7f7b8000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f0c547f88e3b83 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:49.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.360+0000 7f7b6e7fc640 1 -- 172.21.15.67:0/2157319551 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b800af430 con 0x559bd395b390 2024-07-27T03:17:49.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.360+0000 7f7b6e7fc640 1 -- 172.21.15.67:0/2157319551 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b800ab750 con 0x559bd395b390 2024-07-27T03:17:49.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.361+0000 7f7b6e7fc640 1 -- 172.21.15.67:0/2157319551 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b800b3c70 con 0x559bd395b390 2024-07-27T03:17:49.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.362+0000 7f7b94db1740 1 -- 172.21.15.67:0/2157319551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd395b390 msgr2=0x559bd3d217c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:49.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.362+0000 7f7b94db1740 1 --2- 172.21.15.67:0/2157319551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd395b390 0x559bd3d217c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7b800aafe0 tx=0x7f7b8000cf70 comp rx=0 tx=0).stop 2024-07-27T03:17:49.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.362+0000 7f7b94db1740 1 -- 172.21.15.67:0/2157319551 shutdown_connections 2024-07-27T03:17:49.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.362+0000 7f7b94db1740 1 --2- 172.21.15.67:0/2157319551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd395b390 0x559bd3d217c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.362+0000 7f7b94db1740 1 --2- 172.21.15.67:0/2157319551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd39531b0 0x559bd3d21280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.362+0000 7f7b94db1740 1 --2- 172.21.15.67:0/2157319551 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd397c070 0x559bd3d20d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.362+0000 7f7b94db1740 1 -- 172.21.15.67:0/2157319551 >> 172.21.15.67:0/2157319551 conn(0x559bd3970460 msgr2=0x559bd3d1ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:49.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.362+0000 7f7b94db1740 1 -- 172.21.15.67:0/2157319551 shutdown_connections 2024-07-27T03:17:49.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.362+0000 7f7b94db1740 1 -- 172.21.15.67:0/2157319551 wait complete. 2024-07-27T03:17:49.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.363+0000 7f7b94db1740 1 -- start start 2024-07-27T03:17:49.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.363+0000 7f7b94db1740 1 Processor -- start 2024-07-27T03:17:49.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.363+0000 7f7b94db1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd39531b0 0x559bd3d2c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:49.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.363+0000 7f7b94db1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd395b390 0x559bd3d2cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:49.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.363+0000 7f7b94db1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd397c070 0x559bd3d3cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:49.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.363+0000 7f7b94db1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x559bd3c29900 con 0x559bd397c070 2024-07-27T03:17:49.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.363+0000 7f7b94db1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x559bd3c8b060 con 0x559bd39531b0 2024-07-27T03:17:49.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.363+0000 7f7b94db1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559bd3aea700 con 0x559bd395b390 2024-07-27T03:17:49.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.363+0000 7f7b94db1740 10 client.0 _ll_register_callbacks cb 0x559bd3c31d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:17:49.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.364+0000 7f7b87fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd39531b0 0x559bd3d2c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:49.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.364+0000 7f7b8cdc9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd397c070 0x559bd3d3cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:49.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.364+0000 7f7b87fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd39531b0 0x559bd3d2c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.67:48958/0 (socket says 172.21.15.67:48958) 2024-07-27T03:17:49.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.364+0000 7f7b87fff640 1 -- 172.21.15.67:0/3935317283 learned_addr learned my addr 172.21.15.67:0/3935317283 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:49.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.364+0000 7f7b877fe640 1 --2- 172.21.15.67:0/3935317283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd395b390 0x559bd3d2cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.364+0000 7f7b8cdc9640 1 -- 172.21.15.67:0/3935317283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd395b390 msgr2=0x559bd3d2cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.364+0000 7f7b8cdc9640 1 --2- 172.21.15.67:0/3935317283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd395b390 0x559bd3d2cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.364+0000 7f7b8cdc9640 1 -- 172.21.15.67:0/3935317283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd39531b0 msgr2=0x559bd3d2c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:49.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.364+0000 7f7b8cdc9640 1 --2- 172.21.15.67:0/3935317283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd39531b0 0x559bd3d2c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.364+0000 7f7b8cdc9640 1 -- 172.21.15.67:0/3935317283 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bd3c2dbf0 con 0x559bd397c070 2024-07-27T03:17:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.365+0000 7f7b8cdc9640 1 --2- 172.21.15.67:0/3935317283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd397c070 0x559bd3d3cb00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7b800e1c50 tx=0x7f7b800aa5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66384fb7dd13a2a3 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.365+0000 7f7b677fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:17:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.365+0000 7f7b677fe640 1 -- 172.21.15.67:0/3935317283 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b800c9020 con 0x559bd397c070 2024-07-27T03:17:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.365+0000 7f7b677fe640 1 -- 172.21.15.67:0/3935317283 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f7b800c5b90 con 0x559bd397c070 2024-07-27T03:17:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.366+0000 7f7b94db1740 1 -- 172.21.15.67:0/3935317283 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b800ab750 con 0x559bd397c070 2024-07-27T03:17:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.366+0000 7f7b677fe640 1 -- 172.21.15.67:0/3935317283 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7b800af430 con 0x559bd397c070 2024-07-27T03:17:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.366+0000 7f7b94db1740 10 client.15639 fetch_fsmap learned FSMap version 30 2024-07-27T03:17:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.366+0000 7f7b94db1740 1 -- 172.21.15.67:0/3935317283 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559bd39e7650 con 0x559bd397c070 2024-07-27T03:17:49.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.366+0000 7f7b677fe640 1 -- 172.21.15.67:0/3935317283 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b800ab750 con 0x559bd397c070 2024-07-27T03:17:49.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.366+0000 7f7b677fe640 1 -- 172.21.15.67:0/3935317283 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7b800a9d20 con 0x559bd397c070 2024-07-27T03:17:49.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.367+0000 7f7b94db1740 10 client.15639 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:17:49.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.367+0000 7f7b94db1740 10 client.15639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:17:49.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.367+0000 7f7b94db1740 20 client.15639 populate_metadata read hostname 'smithi067' 2024-07-27T03:17:49.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.367+0000 7f7b94db1740 1 --2- 172.21.15.67:0/3935317283 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x559bd393a9f0 0x559bd3d42c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:49.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.367+0000 7f7b94db1740 4 client.15639 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:17:49.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.367+0000 7f7b94db1740 1 -- 172.21.15.67:0/3935317283 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bd3c99a70 con 0x559bd393a9f0 2024-07-27T03:17:49.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.367+0000 7f7b87fff640 1 --2- 172.21.15.67:0/3935317283 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x559bd393a9f0 0x559bd3d42c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:49.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.368+0000 7f7b677fe640 10 client.15639 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:17:49.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.368+0000 7f7b87fff640 1 --2- 172.21.15.67:0/3935317283 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x559bd393a9f0 0x559bd3d42c70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10eaefdb1ca6b94f server_cookie=83b9c9ce25fe67b in_seq=0 out_seq=0 2024-07-27T03:17:49.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.368+0000 7f7b677fe640 1 -- 172.21.15.67:0/3935317283 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559bd3c99a70 con 0x559bd393a9f0 2024-07-27T03:17:49.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.368+0000 7f7b677fe640 10 client.15639 handle_command_reply: tid=0 2024-07-27T03:17:49.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.368+0000 7f7b94db1740 1 client.15639 shutdown 2024-07-27T03:17:49.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.369+0000 7f7b94db1740 1 -- 172.21.15.67:0/3935317283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd397c070 msgr2=0x559bd3d3cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:49.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.369+0000 7f7b94db1740 1 --2- 172.21.15.67:0/3935317283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd397c070 0x559bd3d3cb00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7b800e1c50 tx=0x7f7b800aa5b0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.369+0000 7f7b94db1740 1 -- 172.21.15.67:0/3935317283 shutdown_connections 2024-07-27T03:17:49.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.369+0000 7f7b94db1740 1 --2- 172.21.15.67:0/3935317283 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x559bd393a9f0 0x559bd3d42c70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.369+0000 7f7b94db1740 1 --2- 172.21.15.67:0/3935317283 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd397c070 0x559bd3d3cb00 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.369+0000 7f7b94db1740 1 --2- 172.21.15.67:0/3935317283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd395b390 0x559bd3d2cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.369+0000 7f7b94db1740 1 --2- 172.21.15.67:0/3935317283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd39531b0 0x559bd3d2c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.369+0000 7f7b94db1740 1 -- 172.21.15.67:0/3935317283 >> 172.21.15.67:0/3935317283 conn(0x559bd3970460 msgr2=0x559bd3d23ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:49.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.369+0000 7f7b677fe640 0 client.15639 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:17:49.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.369+0000 7f7b94db1740 1 -- 172.21.15.67:0/3935317283 shutdown_connections 2024-07-27T03:17:49.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.369+0000 7f7b94db1740 1 -- 172.21.15.67:0/3935317283 wait complete. 2024-07-27T03:17:49.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.369+0000 7f7b94db1740 20 client.15639 trim_cache size 0 max 16384 2024-07-27T03:17:49.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.382+0000 7f7b94db1740 1 Processor -- start 2024-07-27T03:17:49.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.382+0000 7f7b94db1740 1 -- start start 2024-07-27T03:17:49.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.383+0000 7f7b94db1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd39531b0 0x559bd3d27ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:49.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.383+0000 7f7b94db1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd395b390 0x559bd3d3c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:49.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.383+0000 7f7b94db1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd397c070 0x559bd3d22480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:49.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.383+0000 7f7b8cdc9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd397c070 0x559bd3d22480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:49.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.383+0000 7f7b8cdc9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd397c070 0x559bd3d22480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41604/0 (socket says 172.21.15.67:41604) 2024-07-27T03:17:49.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.383+0000 7f7b87fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd39531b0 0x559bd3d27ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:49.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.383+0000 7f7b8cdc9640 1 -- 172.21.15.67:0/1011921367 learned_addr learned my addr 172.21.15.67:0/1011921367 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:49.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.383+0000 7f7b877fe640 1 --2- 172.21.15.67:0/1011921367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd395b390 0x559bd3d3c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:49.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.383+0000 7f7b94db1740 1 -- 172.21.15.67:0/1011921367 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881ae510 con 0x559bd397c070 2024-07-27T03:17:49.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.383+0000 7f7b94db1740 1 -- 172.21.15.67:0/1011921367 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881b0050 con 0x559bd395b390 2024-07-27T03:17:49.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.383+0000 7f7b94db1740 1 -- 172.21.15.67:0/1011921367 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881aebe0 con 0x559bd39531b0 2024-07-27T03:17:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.383+0000 7f7b87fff640 1 -- 172.21.15.67:0/1011921367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd395b390 msgr2=0x559bd3d3c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.383+0000 7f7b87fff640 1 --2- 172.21.15.67:0/1011921367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd395b390 0x559bd3d3c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.383+0000 7f7b87fff640 1 -- 172.21.15.67:0/1011921367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd397c070 msgr2=0x559bd3d22480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.383+0000 7f7b87fff640 1 --2- 172.21.15.67:0/1011921367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd397c070 0x559bd3d22480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.383+0000 7f7b87fff640 1 -- 172.21.15.67:0/1011921367 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bd3d2d7e0 con 0x559bd39531b0 2024-07-27T03:17:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.384+0000 7f7b8cdc9640 1 --2- 172.21.15.67:0/1011921367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd397c070 0x559bd3d22480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:17:49.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.384+0000 7f7b87fff640 1 --2- 172.21.15.67:0/1011921367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd39531b0 0x559bd3d27ef0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x559bd3d43410 tx=0x7f7b7803f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ca59ae0a620577d server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:49.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.384+0000 7f7b6dffb640 1 -- 172.21.15.67:0/1011921367 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b78035020 con 0x559bd39531b0 2024-07-27T03:17:49.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.384+0000 7f7b6dffb640 1 -- 172.21.15.67:0/1011921367 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b78057070 con 0x559bd39531b0 2024-07-27T03:17:49.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.384+0000 7f7b6dffb640 1 -- 172.21.15.67:0/1011921367 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b780316d0 con 0x559bd39531b0 2024-07-27T03:17:49.388 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.385+0000 7f7b94db1740 1 -- 172.21.15.67:0/1011921367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd39531b0 msgr2=0x559bd3d27ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:49.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.385+0000 7f7b94db1740 1 --2- 172.21.15.67:0/1011921367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd39531b0 0x559bd3d27ef0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x559bd3d43410 tx=0x7f7b7803f000 comp rx=0 tx=0).stop 2024-07-27T03:17:49.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.385+0000 7f7b94db1740 1 -- 172.21.15.67:0/1011921367 shutdown_connections 2024-07-27T03:17:49.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.385+0000 7f7b94db1740 1 --2- 172.21.15.67:0/1011921367 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd397c070 0x559bd3d22480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.385+0000 7f7b94db1740 1 --2- 172.21.15.67:0/1011921367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd395b390 0x559bd3d3c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.385+0000 7f7b94db1740 1 --2- 172.21.15.67:0/1011921367 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd39531b0 0x559bd3d27ef0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.385+0000 7f7b94db1740 1 -- 172.21.15.67:0/1011921367 >> 172.21.15.67:0/1011921367 conn(0x559bd3970460 msgr2=0x559bd3d21d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:49.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.385+0000 7f7b94db1740 1 -- 172.21.15.67:0/1011921367 shutdown_connections 2024-07-27T03:17:49.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.385+0000 7f7b94db1740 1 -- 172.21.15.67:0/1011921367 wait complete. 2024-07-27T03:17:49.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.386+0000 7f7b94db1740 1 -- start start 2024-07-27T03:17:49.389 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.386+0000 7f7b94db1740 1 Processor -- start 2024-07-27T03:17:49.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.387+0000 7f7b94db1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd39531b0 0x559bd3d38ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:49.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.387+0000 7f7b94db1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd395b390 0x559bd3d33a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:49.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.387+0000 7f7b94db1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd397c070 0x559bd3d33f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:49.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.387+0000 7f7b877fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd395b390 0x559bd3d33a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:49.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.387+0000 7f7b877fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd395b390 0x559bd3d33a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41606/0 (socket says 172.21.15.67:41606) 2024-07-27T03:17:49.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.387+0000 7f7b87fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd39531b0 0x559bd3d38ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:49.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.387+0000 7f7b87fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd39531b0 0x559bd3d38ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.67:48984/0 (socket says 172.21.15.67:48984) 2024-07-27T03:17:49.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.388+0000 7f7b877fe640 1 -- 172.21.15.67:0/1557364333 learned_addr learned my addr 172.21.15.67:0/1557364333 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:49.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.388+0000 7f7b94db1740 1 -- 172.21.15.67:0/1557364333 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881af980 con 0x559bd395b390 2024-07-27T03:17:49.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.388+0000 7f7b94db1740 1 -- 172.21.15.67:0/1557364333 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881af2b0 con 0x559bd39531b0 2024-07-27T03:17:49.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.388+0000 7f7b94db1740 1 -- 172.21.15.67:0/1557364333 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881ac4f0 con 0x559bd397c070 2024-07-27T03:17:49.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.388+0000 7f7b94db1740 10 client.0 _ll_register_callbacks cb 0x7f7b7c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:17:49.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.388+0000 7f7b8cdc9640 1 --2- 172.21.15.67:0/1557364333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd397c070 0x559bd3d33f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:49.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.388+0000 7f7b877fe640 1 -- 172.21.15.67:0/1557364333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd397c070 msgr2=0x559bd3d33f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:49.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.388+0000 7f7b877fe640 1 --2- 172.21.15.67:0/1557364333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd397c070 0x559bd3d33f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.388+0000 7f7b877fe640 1 -- 172.21.15.67:0/1557364333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd39531b0 msgr2=0x559bd3d38ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:49.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.388+0000 7f7b877fe640 1 --2- 172.21.15.67:0/1557364333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd39531b0 0x559bd3d38ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.388+0000 7f7b877fe640 1 -- 172.21.15.67:0/1557364333 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bd3d344a0 con 0x559bd395b390 2024-07-27T03:17:49.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.388+0000 7f7b87fff640 1 --2- 172.21.15.67:0/1557364333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd39531b0 0x559bd3d38ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:17:49.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.388+0000 7f7b877fe640 1 --2- 172.21.15.67:0/1557364333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd395b390 0x559bd3d33a20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7b8806a7a0 tx=0x7f7b7c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b657e27faef9fb server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:49.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.388+0000 7f7b94db1740 1 -- 172.21.15.67:0/1557364333 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b78057070 con 0x559bd395b390 2024-07-27T03:17:49.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.388+0000 7f7b677fe640 10 client.15645 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:17:49.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.389+0000 7f7b677fe640 1 -- 172.21.15.67:0/1557364333 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c01d640 con 0x559bd395b390 2024-07-27T03:17:49.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.389+0000 7f7b677fe640 1 -- 172.21.15.67:0/1557364333 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f7b7c01d7e0 con 0x559bd395b390 2024-07-27T03:17:49.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.389+0000 7f7b677fe640 1 -- 172.21.15.67:0/1557364333 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7b7c01da20 con 0x559bd395b390 2024-07-27T03:17:49.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.389+0000 7f7b94db1740 10 client.15645 fetch_fsmap learned FSMap version 30 2024-07-27T03:17:49.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.389+0000 7f7b94db1740 1 -- 172.21.15.67:0/1557364333 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({fsmap=30}) -- 0x559bd3d2f4c0 con 0x559bd395b390 2024-07-27T03:17:49.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.389+0000 7f7b677fe640 1 -- 172.21.15.67:0/1557364333 <== mon.0 v2:172.21.15.67:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7b7c025030 con 0x559bd395b390 2024-07-27T03:17:49.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.389+0000 7f7b94db1740 10 client.15645 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:17:49.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.389+0000 7f7b94db1740 10 client.15645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:17:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.389+0000 7f7b94db1740 20 client.15645 populate_metadata read hostname 'smithi067' 2024-07-27T03:17:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.389+0000 7f7b94db1740 1 --2- 172.21.15.67:0/1557364333 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x559bd390fda0 0x559bd3d31b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.389+0000 7f7b94db1740 4 client.15645 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-27T03:17:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.389+0000 7f7b94db1740 1 -- 172.21.15.67:0/1557364333 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559bd3c95510 con 0x559bd390fda0 2024-07-27T03:17:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.389+0000 7f7b87fff640 1 --2- 172.21.15.67:0/1557364333 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x559bd390fda0 0x559bd3d31b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.390+0000 7f7b87fff640 1 --2- 172.21.15.67:0/1557364333 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x559bd390fda0 0x559bd3d31b70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=855e9057f13515c9 server_cookie=1290994edc401288 in_seq=0 out_seq=0 2024-07-27T03:17:49.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.390+0000 7f7b677fe640 10 client.15645 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:17:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.390+0000 7f7b677fe640 1 -- 172.21.15.67:0/1557364333 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559bd3c95510 con 0x559bd390fda0 2024-07-27T03:17:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.390+0000 7f7b677fe640 10 client.15645 handle_command_reply: tid=0 2024-07-27T03:17:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.390+0000 7f7b94db1740 1 client.15645 shutdown 2024-07-27T03:17:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.391+0000 7f7b94db1740 1 -- 172.21.15.67:0/1557364333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd395b390 msgr2=0x559bd3d33a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.391+0000 7f7b94db1740 1 --2- 172.21.15.67:0/1557364333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd395b390 0x559bd3d33a20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7b8806a7a0 tx=0x7f7b7c0039c0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.391+0000 7f7b94db1740 1 -- 172.21.15.67:0/1557364333 shutdown_connections 2024-07-27T03:17:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.391+0000 7f7b94db1740 1 --2- 172.21.15.67:0/1557364333 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x559bd390fda0 0x559bd3d31b70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.391+0000 7f7b677fe640 0 client.15645 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:17:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.391+0000 7f7b94db1740 1 --2- 172.21.15.67:0/1557364333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559bd397c070 0x559bd3d33f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.391+0000 7f7b94db1740 1 --2- 172.21.15.67:0/1557364333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x559bd395b390 0x559bd3d33a20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.391+0000 7f7b94db1740 1 --2- 172.21.15.67:0/1557364333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559bd39531b0 0x559bd3d38ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.391+0000 7f7b94db1740 1 -- 172.21.15.67:0/1557364333 >> 172.21.15.67:0/1557364333 conn(0x559bd3970460 msgr2=0x559bd3d21d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.391+0000 7f7b94db1740 1 -- 172.21.15.67:0/1557364333 shutdown_connections 2024-07-27T03:17:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.391+0000 7f7b94db1740 1 -- 172.21.15.67:0/1557364333 wait complete. 2024-07-27T03:17:49.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.391+0000 7f7b94db1740 20 client.15645 trim_cache size 0 max 16384 2024-07-27T03:17:49.397 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-27T03:17:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.392+0000 7f7b8e82c640 1 -- 172.21.15.67:0/2216594745 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7b700b3c50 msgr2=0x7f7b700b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.392+0000 7f7b8e82c640 1 --2- 172.21.15.67:0/2216594745 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7b700b3c50 0x7f7b700b6160 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7b78006010 tx=0x7f7b78005d50 comp rx=0 tx=0).stop 2024-07-27T03:17:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.392+0000 7f7b8e82c640 1 -- 172.21.15.67:0/2216594745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b8810d8a0 msgr2=0x7f7b8819f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.392+0000 7f7b8e82c640 1 --2- 172.21.15.67:0/2216594745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b8810d8a0 0x7f7b8819f760 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000eb10 tx=0x7f7b8000c2d0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.392+0000 7f7b8e82c640 1 -- 172.21.15.67:0/2216594745 shutdown_connections 2024-07-27T03:17:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.392+0000 7f7b8e82c640 1 --2- 172.21.15.67:0/2216594745 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7b700b3c50 0x7f7b700b6160 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.392+0000 7f7b8e82c640 1 --2- 172.21.15.67:0/2216594745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b8810d8a0 0x7f7b8819f760 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.392+0000 7f7b8e82c640 1 --2- 172.21.15.67:0/2216594745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b88104d20 0x7f7b8819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.392+0000 7f7b8e82c640 1 --2- 172.21.15.67:0/2216594745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b881042f0 0x7f7b88199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.392+0000 7f7b8e82c640 1 -- 172.21.15.67:0/2216594745 >> 172.21.15.67:0/2216594745 conn(0x7f7b880791a0 msgr2=0x7f7b881073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:49.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.392+0000 7f7b8e82c640 1 -- 172.21.15.67:0/2216594745 shutdown_connections 2024-07-27T03:17:49.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:49.392+0000 7f7b8e82c640 1 -- 172.21.15.67:0/2216594745 wait complete. 2024-07-27T03:17:50.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:50 smithi067 ceph-mon[32814]: pgmap v468: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:50.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:50 smithi149 ceph-mon[37173]: pgmap v468: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:50.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:50 smithi175 ceph-mon[37125]: pgmap v468: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:51.414 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-27T03:17:51.414 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-27T03:17:51.688 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:17:52.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:52 smithi149 ceph-mon[37173]: pgmap v469: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-07-27T03:17:52.846 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:52 smithi067 ceph-mon[32814]: pgmap v469: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-07-27T03:17:52.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:52 smithi175 ceph-mon[37125]: pgmap v469: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-07-27T03:17:53.500 INFO:tasks.workunit.client.0.smithi067.stderr:Updating files: 88% (11209/12697) Updating files: 89% (11301/12697) Updating files: 90% (11428/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-07-27T03:17:53.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.486+0000 7f6014609640 1 -- 172.21.15.67:0/1977585621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f600c1024d0 msgr2=0x7f600c1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.486+0000 7f6014609640 1 --2- 172.21.15.67:0/1977585621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f600c1024d0 0x7f600c1028e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6008009900 tx=0x7f600802a3f0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.486+0000 7f6014609640 1 -- 172.21.15.67:0/1977585621 shutdown_connections 2024-07-27T03:17:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.486+0000 7f6014609640 1 --2- 172.21.15.67:0/1977585621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f600c108760 0x7f600c10ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.486+0000 7f6014609640 1 --2- 172.21.15.67:0/1977585621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f600c107e10 0x7f600c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.486+0000 7f6014609640 1 --2- 172.21.15.67:0/1977585621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f600c1024d0 0x7f600c1028e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.486+0000 7f6014609640 1 -- 172.21.15.67:0/1977585621 >> 172.21.15.67:0/1977585621 conn(0x7f600c0791a0 msgr2=0x7f600c0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.486+0000 7f6014609640 1 -- 172.21.15.67:0/1977585621 shutdown_connections 2024-07-27T03:17:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.486+0000 7f6014609640 1 -- 172.21.15.67:0/1977585621 wait complete. 2024-07-27T03:17:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.487+0000 7f6014609640 1 Processor -- start 2024-07-27T03:17:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.487+0000 7f6014609640 1 -- start start 2024-07-27T03:17:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.487+0000 7f6014609640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f600c1024d0 0x7f600c19ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:53.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.487+0000 7f6014609640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f600c107e10 0x7f600c19b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:53.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.487+0000 7f6014609640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f600c108760 0x7f600c19b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:53.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.487+0000 7f6014609640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f600c1123c0 con 0x7f600c107e10 2024-07-27T03:17:53.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.487+0000 7f6014609640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f600c112240 con 0x7f600c1024d0 2024-07-27T03:17:53.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.487+0000 7f6014609640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f600c112540 con 0x7f600c108760 2024-07-27T03:17:53.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.487+0000 7f6011ba4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f600c107e10 0x7f600c19b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:53.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.488+0000 7f60123a5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f600c1024d0 0x7f600c19ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:53.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.488+0000 7f60123a5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f600c1024d0 0x7f600c19ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.67:49008/0 (socket says 172.21.15.67:49008) 2024-07-27T03:17:53.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.488+0000 7f60123a5640 1 -- 172.21.15.67:0/3748422747 learned_addr learned my addr 172.21.15.67:0/3748422747 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:53.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.488+0000 7f6012ba6640 1 --2- 172.21.15.67:0/3748422747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f600c108760 0x7f600c19b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:53.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.488+0000 7f6011ba4640 1 -- 172.21.15.67:0/3748422747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f600c108760 msgr2=0x7f600c19b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:53.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.488+0000 7f6011ba4640 1 --2- 172.21.15.67:0/3748422747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f600c108760 0x7f600c19b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.488+0000 7f6011ba4640 1 -- 172.21.15.67:0/3748422747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f600c1024d0 msgr2=0x7f600c19ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:53.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.488+0000 7f6011ba4640 1 --2- 172.21.15.67:0/3748422747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f600c1024d0 0x7f600c19ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.488+0000 7f6011ba4640 1 -- 172.21.15.67:0/3748422747 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f600c1a5f10 con 0x7f600c107e10 2024-07-27T03:17:53.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.488+0000 7f60123a5640 1 --2- 172.21.15.67:0/3748422747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f600c1024d0 0x7f600c19ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:17:53.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.488+0000 7f6011ba4640 1 --2- 172.21.15.67:0/3748422747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f600c107e10 0x7f600c19b250 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6000007b20 tx=0x7f6000010050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:53.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.491+0000 7f5fff7fe640 1 -- 172.21.15.67:0/3748422747 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600000e2e0 con 0x7f600c107e10 2024-07-27T03:17:53.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.491+0000 7f5fff7fe640 1 -- 172.21.15.67:0/3748422747 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f600000e480 con 0x7f600c107e10 2024-07-27T03:17:53.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.491+0000 7f5fff7fe640 1 -- 172.21.15.67:0/3748422747 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600000cdd0 con 0x7f600c107e10 2024-07-27T03:17:53.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.491+0000 7f6014609640 1 -- 172.21.15.67:0/3748422747 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f600c198b60 con 0x7f600c107e10 2024-07-27T03:17:53.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.491+0000 7f6014609640 1 -- 172.21.15.67:0/3748422747 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f600c1990f0 con 0x7f600c107e10 2024-07-27T03:17:53.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.493+0000 7f601ab8e740 1 Processor -- start 2024-07-27T03:17:53.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.494+0000 7f601ab8e740 1 -- start start 2024-07-27T03:17:53.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.494+0000 7f5fff7fe640 1 -- 172.21.15.67:0/3748422747 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f6000010820 con 0x7f600c107e10 2024-07-27T03:17:53.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.494+0000 7f601ab8e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613d9b0c350 0x5613d9eb0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:53.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.494+0000 7f601ab8e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613d9ae3490 0x5613d9eb1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:53.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.494+0000 7f601ab8e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9aeb670 0x5613d9eb1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:53.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.494+0000 7f601ab8e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5613d9e27a60 con 0x5613d9aeb670 2024-07-27T03:17:53.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.494+0000 7f601ab8e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5613d9e16f60 con 0x5613d9ae3490 2024-07-27T03:17:53.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.494+0000 7f601ab8e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5613d9e1afa0 con 0x5613d9b0c350 2024-07-27T03:17:53.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.494+0000 7f6011ba4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9aeb670 0x5613d9eb1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:53.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.494+0000 7f60123a5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613d9ae3490 0x5613d9eb1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:53.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.494+0000 7f60123a5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613d9ae3490 0x5613d9eb1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.67:49022/0 (socket says 172.21.15.67:49022) 2024-07-27T03:17:53.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.494+0000 7f6011ba4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9aeb670 0x5613d9eb1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41644/0 (socket says 172.21.15.67:41644) 2024-07-27T03:17:53.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.494+0000 7f60123a5640 1 -- 172.21.15.67:0/2578135768 learned_addr learned my addr 172.21.15.67:0/2578135768 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:53.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.495+0000 7f6012ba6640 1 --2- 172.21.15.67:0/2578135768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613d9b0c350 0x5613d9eb0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:53.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.495+0000 7f6011ba4640 1 -- 172.21.15.67:0/2578135768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613d9b0c350 msgr2=0x5613d9eb0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.495+0000 7f6011ba4640 1 --2- 172.21.15.67:0/2578135768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613d9b0c350 0x5613d9eb0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.495+0000 7f6011ba4640 1 -- 172.21.15.67:0/2578135768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613d9ae3490 msgr2=0x5613d9eb1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.495+0000 7f6011ba4640 1 --2- 172.21.15.67:0/2578135768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613d9ae3490 0x5613d9eb1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.495+0000 7f6011ba4640 1 -- 172.21.15.67:0/2578135768 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d9b77930 con 0x5613d9aeb670 2024-07-27T03:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.496+0000 7f5fff7fe640 1 --2- 172.21.15.67:0/3748422747 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5ff80b3c80 0x7f5ff80b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.496+0000 7f6012ba6640 1 --2- 172.21.15.67:0/3748422747 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5ff80b3c80 0x7f5ff80b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.496+0000 7f6011ba4640 1 --2- 172.21.15.67:0/2578135768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9aeb670 0x5613d9eb1890 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f60000e4f90 tx=0x7f60000edf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae33ec50ecf2824b server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.496+0000 7f5ffcff9640 1 -- 172.21.15.67:0/2578135768 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60000eae60 con 0x5613d9aeb670 2024-07-27T03:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.496+0000 7f5fff7fe640 1 -- 172.21.15.67:0/3748422747 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f60000e7870 con 0x7f600c107e10 2024-07-27T03:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.496+0000 7f6012ba6640 1 --2- 172.21.15.67:0/3748422747 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5ff80b3c80 0x7f5ff80b6190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6004007e50 tx=0x7f6004007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.496+0000 7f5ffcff9640 1 -- 172.21.15.67:0/2578135768 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60000adf20 con 0x5613d9aeb670 2024-07-27T03:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.496+0000 7f5ffcff9640 1 -- 172.21.15.67:0/2578135768 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60000ae1e0 con 0x5613d9aeb670 2024-07-27T03:17:53.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.497+0000 7f601ab8e740 1 -- 172.21.15.67:0/2578135768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9aeb670 msgr2=0x5613d9eb1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.497+0000 7f601ab8e740 1 --2- 172.21.15.67:0/2578135768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9aeb670 0x5613d9eb1890 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f60000e4f90 tx=0x7f60000edf10 comp rx=0 tx=0).stop 2024-07-27T03:17:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.497+0000 7f601ab8e740 1 -- 172.21.15.67:0/2578135768 shutdown_connections 2024-07-27T03:17:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.497+0000 7f601ab8e740 1 --2- 172.21.15.67:0/2578135768 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9aeb670 0x5613d9eb1890 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.497+0000 7f601ab8e740 1 --2- 172.21.15.67:0/2578135768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613d9ae3490 0x5613d9eb1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.497+0000 7f601ab8e740 1 --2- 172.21.15.67:0/2578135768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613d9b0c350 0x5613d9eb0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.497+0000 7f601ab8e740 1 -- 172.21.15.67:0/2578135768 >> 172.21.15.67:0/2578135768 conn(0x5613d9b00740 msgr2=0x5613d9eaed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.498+0000 7f601ab8e740 1 -- 172.21.15.67:0/2578135768 shutdown_connections 2024-07-27T03:17:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.498+0000 7f601ab8e740 1 -- 172.21.15.67:0/2578135768 wait complete. 2024-07-27T03:17:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.498+0000 7f601ab8e740 1 -- start start 2024-07-27T03:17:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.498+0000 7f601ab8e740 1 Processor -- start 2024-07-27T03:17:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.498+0000 7f601ab8e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9ae3490 0x5613d9ebcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.498+0000 7f601ab8e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613d9aeb670 0x5613d9ebd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:53.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.498+0000 7f601ab8e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613d9b0c350 0x5613d9ecb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:53.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.498+0000 7f601ab8e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5613d9c7a890 con 0x5613d9ae3490 2024-07-27T03:17:53.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.498+0000 7f601ab8e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5613d9db9580 con 0x5613d9aeb670 2024-07-27T03:17:53.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.498+0000 7f601ab8e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5613d9ec7d10 con 0x5613d9b0c350 2024-07-27T03:17:53.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.499+0000 7f601ab8e740 10 client.0 _ll_register_callbacks cb 0x5613d9e10f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:17:53.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.499+0000 7f60123a5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9ae3490 0x5613d9ebcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:53.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.499+0000 7f6012ba6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613d9b0c350 0x5613d9ecb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:53.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.499+0000 7f60123a5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9ae3490 0x5613d9ebcb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41652/0 (socket says 172.21.15.67:41652) 2024-07-27T03:17:53.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.499+0000 7f60123a5640 1 -- 172.21.15.67:0/3142174138 learned_addr learned my addr 172.21.15.67:0/3142174138 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:53.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.499+0000 7f6011ba4640 1 --2- 172.21.15.67:0/3142174138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613d9aeb670 0x5613d9ebd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:53.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.499+0000 7f60123a5640 1 -- 172.21.15.67:0/3142174138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613d9b0c350 msgr2=0x5613d9ecb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:53.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.499+0000 7f60123a5640 1 --2- 172.21.15.67:0/3142174138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613d9b0c350 0x5613d9ecb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.499+0000 7f60123a5640 1 -- 172.21.15.67:0/3142174138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613d9aeb670 msgr2=0x5613d9ebd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:53.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.499+0000 7f60123a5640 1 --2- 172.21.15.67:0/3142174138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613d9aeb670 0x5613d9ebd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.499+0000 7f60123a5640 1 -- 172.21.15.67:0/3142174138 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d9ecb750 con 0x5613d9ae3490 2024-07-27T03:17:53.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.499+0000 7f5ff5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:17:53.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.499+0000 7f60123a5640 1 --2- 172.21.15.67:0/3142174138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9ae3490 0x5613d9ebcb70 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6008005940 tx=0x7f6008031a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8f3ee78c9a6018 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:53.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.499+0000 7f5ff5ffb640 1 -- 172.21.15.67:0/3142174138 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008035040 con 0x5613d9ae3490 2024-07-27T03:17:53.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.500+0000 7f601ab8e740 1 -- 172.21.15.67:0/3142174138 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60000adf20 con 0x5613d9ae3490 2024-07-27T03:17:53.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.500+0000 7f5ff5ffb640 1 -- 172.21.15.67:0/3142174138 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f6008005020 con 0x5613d9ae3490 2024-07-27T03:17:53.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.500+0000 7f5ff5ffb640 1 -- 172.21.15.67:0/3142174138 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6008035040 con 0x5613d9ae3490 2024-07-27T03:17:53.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.500+0000 7f601ab8e740 10 client.15657 fetch_fsmap learned FSMap version 30 2024-07-27T03:17:53.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.500+0000 7f601ab8e740 1 -- 172.21.15.67:0/3142174138 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5613d9ed4170 con 0x5613d9ae3490 2024-07-27T03:17:53.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.500+0000 7f5ff5ffb640 1 -- 172.21.15.67:0/3142174138 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60000adf20 con 0x5613d9ae3490 2024-07-27T03:17:53.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.500+0000 7f5ff5ffb640 1 -- 172.21.15.67:0/3142174138 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f60080035f0 con 0x5613d9ae3490 2024-07-27T03:17:53.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.500+0000 7f601ab8e740 10 client.15657 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:17:53.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.500+0000 7f601ab8e740 10 client.15657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:17:53.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.500+0000 7f601ab8e740 20 client.15657 populate_metadata read hostname 'smithi067' 2024-07-27T03:17:53.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.500+0000 7f601ab8e740 1 --2- 172.21.15.67:0/3142174138 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5613d9acacd0 0x5613d9ed75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.500+0000 7f601ab8e740 4 client.15657 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:17:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.500+0000 7f601ab8e740 1 -- 172.21.15.67:0/3142174138 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613d9e299b0 con 0x5613d9acacd0 2024-07-27T03:17:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.501+0000 7f6011ba4640 1 --2- 172.21.15.67:0/3142174138 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5613d9acacd0 0x5613d9ed75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.501+0000 7f6011ba4640 1 --2- 172.21.15.67:0/3142174138 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5613d9acacd0 0x5613d9ed75e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d71f129795825371 server_cookie=996ef52721887cd9 in_seq=0 out_seq=0 2024-07-27T03:17:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.501+0000 7f5ff5ffb640 10 client.15657 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:17:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.502+0000 7f5ff5ffb640 1 -- 172.21.15.67:0/3142174138 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5613d9e299b0 con 0x5613d9acacd0 2024-07-27T03:17:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.502+0000 7f5ff5ffb640 10 client.15657 handle_command_reply: tid=0 2024-07-27T03:17:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.502+0000 7f601ab8e740 1 client.15657 shutdown 2024-07-27T03:17:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.502+0000 7f601ab8e740 1 -- 172.21.15.67:0/3142174138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9ae3490 msgr2=0x5613d9ebcb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.502+0000 7f601ab8e740 1 --2- 172.21.15.67:0/3142174138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9ae3490 0x5613d9ebcb70 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6008005940 tx=0x7f6008031a00 comp rx=0 tx=0).stop 2024-07-27T03:17:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.502+0000 7f601ab8e740 1 -- 172.21.15.67:0/3142174138 shutdown_connections 2024-07-27T03:17:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.502+0000 7f601ab8e740 1 --2- 172.21.15.67:0/3142174138 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5613d9acacd0 0x5613d9ed75e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.502+0000 7f5ff5ffb640 0 client.15657 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:17:53.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.502+0000 7f601ab8e740 1 --2- 172.21.15.67:0/3142174138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613d9b0c350 0x5613d9ecb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.502+0000 7f601ab8e740 1 --2- 172.21.15.67:0/3142174138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613d9aeb670 0x5613d9ebd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.502+0000 7f601ab8e740 1 --2- 172.21.15.67:0/3142174138 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9ae3490 0x5613d9ebcb70 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.502+0000 7f601ab8e740 1 -- 172.21.15.67:0/3142174138 >> 172.21.15.67:0/3142174138 conn(0x5613d9b00740 msgr2=0x5613d9eb3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:53.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.503+0000 7f601ab8e740 1 -- 172.21.15.67:0/3142174138 shutdown_connections 2024-07-27T03:17:53.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.503+0000 7f601ab8e740 1 -- 172.21.15.67:0/3142174138 wait complete. 2024-07-27T03:17:53.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.503+0000 7f601ab8e740 20 client.15657 trim_cache size 0 max 16384 2024-07-27T03:17:53.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.521+0000 7f601ab8e740 1 Processor -- start 2024-07-27T03:17:53.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.521+0000 7f601ab8e740 1 -- start start 2024-07-27T03:17:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.521+0000 7f601ab8e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9ae3490 0x5613d9ecb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.521+0000 7f601ab8e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613d9aeb670 0x5613d9ed2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.521+0000 7f601ab8e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613d9b0c350 0x5613d9edcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.521+0000 7f601ab8e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f600c1ae330 con 0x5613d9ae3490 2024-07-27T03:17:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.521+0000 7f601ab8e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f600c1afe70 con 0x5613d9aeb670 2024-07-27T03:17:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.521+0000 7f601ab8e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f600c1aea00 con 0x5613d9b0c350 2024-07-27T03:17:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.521+0000 7f60123a5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9ae3490 0x5613d9ecb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.521+0000 7f60123a5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9ae3490 0x5613d9ecb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41662/0 (socket says 172.21.15.67:41662) 2024-07-27T03:17:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.521+0000 7f60123a5640 1 -- 172.21.15.67:0/3808821565 learned_addr learned my addr 172.21.15.67:0/3808821565 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.522+0000 7f6011ba4640 1 --2- 172.21.15.67:0/3808821565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613d9aeb670 0x5613d9ed2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.522+0000 7f6012ba6640 1 --2- 172.21.15.67:0/3808821565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613d9b0c350 0x5613d9edcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.522+0000 7f60123a5640 1 -- 172.21.15.67:0/3808821565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613d9b0c350 msgr2=0x5613d9edcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:53.525 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.522+0000 7f60123a5640 1 --2- 172.21.15.67:0/3808821565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613d9b0c350 0x5613d9edcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.522+0000 7f60123a5640 1 -- 172.21.15.67:0/3808821565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613d9aeb670 msgr2=0x5613d9ed2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.522+0000 7f60123a5640 1 --2- 172.21.15.67:0/3808821565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613d9aeb670 0x5613d9ed2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.522+0000 7f60123a5640 1 -- 172.21.15.67:0/3808821565 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d9ecb560 con 0x5613d9ae3490 2024-07-27T03:17:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.522+0000 7f6011ba4640 1 --2- 172.21.15.67:0/3808821565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613d9aeb670 0x5613d9ed2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:17:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.522+0000 7f6012ba6640 1 --2- 172.21.15.67:0/3808821565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613d9b0c350 0x5613d9edcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:17:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.522+0000 7f60123a5640 1 --2- 172.21.15.67:0/3808821565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9ae3490 0x5613d9ecb020 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6008031440 tx=0x7f600802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6864836a9c4f2f9c server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.522+0000 7f5ff7fff640 1 -- 172.21.15.67:0/3808821565 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008003c30 con 0x5613d9ae3490 2024-07-27T03:17:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.522+0000 7f5ff7fff640 1 -- 172.21.15.67:0/3808821565 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f600805a070 con 0x5613d9ae3490 2024-07-27T03:17:53.526 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.522+0000 7f5ff7fff640 1 -- 172.21.15.67:0/3808821565 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008005080 con 0x5613d9ae3490 2024-07-27T03:17:53.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.523+0000 7f601ab8e740 1 -- 172.21.15.67:0/3808821565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9ae3490 msgr2=0x5613d9ecb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:53.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.523+0000 7f601ab8e740 1 --2- 172.21.15.67:0/3808821565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9ae3490 0x5613d9ecb020 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6008031440 tx=0x7f600802a3f0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.523+0000 7f601ab8e740 1 -- 172.21.15.67:0/3808821565 shutdown_connections 2024-07-27T03:17:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.523+0000 7f601ab8e740 1 --2- 172.21.15.67:0/3808821565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613d9b0c350 0x5613d9edcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.523+0000 7f601ab8e740 1 --2- 172.21.15.67:0/3808821565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613d9aeb670 0x5613d9ed2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.523+0000 7f601ab8e740 1 --2- 172.21.15.67:0/3808821565 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9ae3490 0x5613d9ecb020 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.523+0000 7f601ab8e740 1 -- 172.21.15.67:0/3808821565 >> 172.21.15.67:0/3808821565 conn(0x5613d9b00740 msgr2=0x5613d9eaecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.523+0000 7f601ab8e740 1 -- 172.21.15.67:0/3808821565 shutdown_connections 2024-07-27T03:17:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.523+0000 7f601ab8e740 1 -- 172.21.15.67:0/3808821565 wait complete. 2024-07-27T03:17:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.524+0000 7f601ab8e740 1 -- start start 2024-07-27T03:17:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.524+0000 7f601ab8e740 1 Processor -- start 2024-07-27T03:17:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.524+0000 7f601ab8e740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9ae3490 0x5613d9ecd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.524+0000 7f601ab8e740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613d9aeb670 0x5613d9ecdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.524+0000 7f601ab8e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613d9b0c350 0x5613d9ece000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.524+0000 7f601ab8e740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f600c1af7a0 con 0x5613d9ae3490 2024-07-27T03:17:53.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.524+0000 7f601ab8e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f600c1af0d0 con 0x5613d9b0c350 2024-07-27T03:17:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.524+0000 7f601ab8e740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f600c1ac310 con 0x5613d9aeb670 2024-07-27T03:17:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.524+0000 7f601ab8e740 10 client.0 _ll_register_callbacks cb 0x7f6008009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:17:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.524+0000 7f60123a5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9ae3490 0x5613d9ecd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.525+0000 7f60123a5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9ae3490 0x5613d9ecd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:41676/0 (socket says 172.21.15.67:41676) 2024-07-27T03:17:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.525+0000 7f60123a5640 1 -- 172.21.15.67:0/1011756561 learned_addr learned my addr 172.21.15.67:0/1011756561 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.525+0000 7f6011ba4640 1 --2- 172.21.15.67:0/1011756561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613d9aeb670 0x5613d9ecdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.525+0000 7f6012ba6640 1 --2- 172.21.15.67:0/1011756561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613d9b0c350 0x5613d9ece000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.525+0000 7f60123a5640 1 -- 172.21.15.67:0/1011756561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613d9aeb670 msgr2=0x5613d9ecdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.525+0000 7f60123a5640 1 --2- 172.21.15.67:0/1011756561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613d9aeb670 0x5613d9ecdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.525+0000 7f60123a5640 1 -- 172.21.15.67:0/1011756561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613d9b0c350 msgr2=0x5613d9ece000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.525+0000 7f60123a5640 1 --2- 172.21.15.67:0/1011756561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613d9b0c350 0x5613d9ece000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.525+0000 7f60123a5640 1 -- 172.21.15.67:0/1011756561 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d9f3ed70 con 0x5613d9ae3490 2024-07-27T03:17:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.525+0000 7f60123a5640 1 --2- 172.21.15.67:0/1011756561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9ae3490 0x5613d9ecd580 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6008003a40 tx=0x7f6008003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fddb1304fc12e4f9 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:53.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.525+0000 7f5ff5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:17:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.525+0000 7f5ff5ffb640 1 -- 172.21.15.67:0/1011756561 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008063070 con 0x5613d9ae3490 2024-07-27T03:17:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.525+0000 7f601ab8e740 1 -- 172.21.15.67:0/1011756561 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f600805a070 con 0x5613d9ae3490 2024-07-27T03:17:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.525+0000 7f5ff5ffb640 1 -- 172.21.15.67:0/1011756561 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f60080319e0 con 0x5613d9ae3490 2024-07-27T03:17:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.526+0000 7f5ff5ffb640 1 -- 172.21.15.67:0/1011756561 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6008004bb0 con 0x5613d9ae3490 2024-07-27T03:17:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.526+0000 7f601ab8e740 10 client.15663 fetch_fsmap learned FSMap version 30 2024-07-27T03:17:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.526+0000 7f601ab8e740 1 -- 172.21.15.67:0/1011756561 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5613d9f3ff00 con 0x5613d9ae3490 2024-07-27T03:17:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.526+0000 7f5ff5ffb640 1 -- 172.21.15.67:0/1011756561 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f600805a070 con 0x5613d9ae3490 2024-07-27T03:17:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.526+0000 7f5ff5ffb640 1 -- 172.21.15.67:0/1011756561 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6008066020 con 0x5613d9ae3490 2024-07-27T03:17:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.526+0000 7f601ab8e740 10 client.15663 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:17:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.526+0000 7f601ab8e740 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:17:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.526+0000 7f601ab8e740 20 client.15663 populate_metadata read hostname 'smithi067' 2024-07-27T03:17:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.526+0000 7f601ab8e740 1 --2- 172.21.15.67:0/1011756561 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5613d9aa0080 0x5613d9f42440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.526+0000 7f601ab8e740 4 client.15663 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-27T03:17:53.532 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.526+0000 7f601ab8e740 1 -- 172.21.15.67:0/1011756561 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5613d9eb8500 con 0x5613d9aa0080 2024-07-27T03:17:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.526+0000 7f6011ba4640 1 --2- 172.21.15.67:0/1011756561 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5613d9aa0080 0x5613d9f42440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.527+0000 7f5ff5ffb640 10 client.15663 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:17:53.533 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.527+0000 7f6011ba4640 1 --2- 172.21.15.67:0/1011756561 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5613d9aa0080 0x5613d9f42440 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e5207c15db1c77 server_cookie=31731926b124aea6 in_seq=0 out_seq=0 2024-07-27T03:17:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.528+0000 7f5ff5ffb640 1 -- 172.21.15.67:0/1011756561 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5613d9eb8500 con 0x5613d9aa0080 2024-07-27T03:17:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.528+0000 7f5ff5ffb640 10 client.15663 handle_command_reply: tid=0 2024-07-27T03:17:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.528+0000 7f601ab8e740 1 client.15663 shutdown 2024-07-27T03:17:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.528+0000 7f601ab8e740 1 -- 172.21.15.67:0/1011756561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9ae3490 msgr2=0x5613d9ecd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.528+0000 7f601ab8e740 1 --2- 172.21.15.67:0/1011756561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9ae3490 0x5613d9ecd580 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6008003a40 tx=0x7f6008003ee0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.528+0000 7f601ab8e740 1 -- 172.21.15.67:0/1011756561 shutdown_connections 2024-07-27T03:17:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.528+0000 7f601ab8e740 1 --2- 172.21.15.67:0/1011756561 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5613d9aa0080 0x5613d9f42440 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.528+0000 7f601ab8e740 1 --2- 172.21.15.67:0/1011756561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613d9b0c350 0x5613d9ece000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.528+0000 7f601ab8e740 1 --2- 172.21.15.67:0/1011756561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5613d9aeb670 0x5613d9ecdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.528+0000 7f601ab8e740 1 --2- 172.21.15.67:0/1011756561 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5613d9ae3490 0x5613d9ecd580 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.528+0000 7f601ab8e740 1 -- 172.21.15.67:0/1011756561 >> 172.21.15.67:0/1011756561 conn(0x5613d9b00740 msgr2=0x5613d9eb66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.528+0000 7f5ff5ffb640 0 client.15663 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:17:53.534 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.529+0000 7f601ab8e740 1 -- 172.21.15.67:0/1011756561 shutdown_connections 2024-07-27T03:17:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.529+0000 7f601ab8e740 1 -- 172.21.15.67:0/1011756561 wait complete. 2024-07-27T03:17:53.535 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.529+0000 7f601ab8e740 20 client.15663 trim_cache size 0 max 16384 2024-07-27T03:17:53.535 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:17:53.535 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-27T03:17:53.535 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "ff1963ed-1500-4b79-bb32-22249d314226", 2024-07-27T03:17:53.535 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-27T03:17:53.535 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:17:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.529+0000 7f6014609640 1 -- 172.21.15.67:0/3748422747 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5ff80b3c80 msgr2=0x7f5ff80b6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.529+0000 7f6014609640 1 --2- 172.21.15.67:0/3748422747 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5ff80b3c80 0x7f5ff80b6190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6004007e50 tx=0x7f6004007520 comp rx=0 tx=0).stop 2024-07-27T03:17:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.529+0000 7f6014609640 1 -- 172.21.15.67:0/3748422747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f600c107e10 msgr2=0x7f600c19b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.529+0000 7f6014609640 1 --2- 172.21.15.67:0/3748422747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f600c107e10 0x7f600c19b250 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6000007b20 tx=0x7f6000010050 comp rx=0 tx=0).stop 2024-07-27T03:17:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.529+0000 7f6014609640 1 -- 172.21.15.67:0/3748422747 shutdown_connections 2024-07-27T03:17:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.529+0000 7f6014609640 1 --2- 172.21.15.67:0/3748422747 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5ff80b3c80 0x7f5ff80b6190 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.529+0000 7f6014609640 1 --2- 172.21.15.67:0/3748422747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f600c108760 0x7f600c19b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.536 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.529+0000 7f6014609640 1 --2- 172.21.15.67:0/3748422747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f600c107e10 0x7f600c19b250 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.529+0000 7f6014609640 1 --2- 172.21.15.67:0/3748422747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f600c1024d0 0x7f600c19ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.529+0000 7f6014609640 1 -- 172.21.15.67:0/3748422747 >> 172.21.15.67:0/3748422747 conn(0x7f600c0791a0 msgr2=0x7f600c100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.530+0000 7f6014609640 1 -- 172.21.15.67:0/3748422747 shutdown_connections 2024-07-27T03:17:53.537 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:53.530+0000 7f6014609640 1 -- 172.21.15.67:0/3748422747 wait complete. 2024-07-27T03:17:54.107 INFO:tasks.workunit.client.0.smithi067.stderr:Note: switching to 'af8fdbf079811fe158ce62927bd145c35b74d4b8'. 2024-07-27T03:17:54.107 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-07-27T03:17:54.107 INFO:tasks.workunit.client.0.smithi067.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-07-27T03:17:54.108 INFO:tasks.workunit.client.0.smithi067.stderr:changes and commit them, and you can discard any commits you make in this 2024-07-27T03:17:54.108 INFO:tasks.workunit.client.0.smithi067.stderr:state without impacting any branches by switching back to a branch. 2024-07-27T03:17:54.108 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-07-27T03:17:54.108 INFO:tasks.workunit.client.0.smithi067.stderr:If you want to create a new branch to retain commits you create, you may 2024-07-27T03:17:54.108 INFO:tasks.workunit.client.0.smithi067.stderr:do so (now or later) by using -c with the switch command. Example: 2024-07-27T03:17:54.108 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-07-27T03:17:54.108 INFO:tasks.workunit.client.0.smithi067.stderr: git switch -c 2024-07-27T03:17:54.108 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-07-27T03:17:54.108 INFO:tasks.workunit.client.0.smithi067.stderr:Or undo this operation with: 2024-07-27T03:17:54.108 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-07-27T03:17:54.108 INFO:tasks.workunit.client.0.smithi067.stderr: git switch - 2024-07-27T03:17:54.108 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-07-27T03:17:54.109 INFO:tasks.workunit.client.0.smithi067.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-07-27T03:17:54.109 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-07-27T03:17:54.109 INFO:tasks.workunit.client.0.smithi067.stderr:HEAD is now at af8fdbf0798 Merge PR #52373 into wip-pdonnell-testing-20240726.202642-debug 2024-07-27T03:17:54.111 DEBUG:teuthology.orchestra.run.smithi067:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-07-27T03:17:54.188 INFO:tasks.workunit.client.0.smithi067.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-07-27T03:17:54.191 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-07-27T03:17:54.191 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-07-27T03:17:54.283 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-27T03:17:54.550 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:17:54.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:54 smithi067 ceph-mon[32814]: pgmap v470: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:54.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:54 smithi067 ceph-mon[32814]: scrub queued for path: / 2024-07-27T03:17:54.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:54 smithi067 ceph-mon[32814]: scrub summary: idle+waiting paths [/] 2024-07-27T03:17:54.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:54 smithi067 ceph-mon[32814]: scrub summary: active paths [/] 2024-07-27T03:17:54.673 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:54 smithi067 ceph-mon[32814]: scrub summary: idle+waiting paths [/] 2024-07-27T03:17:54.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:54 smithi149 ceph-mon[37173]: pgmap v470: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:54.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:54 smithi149 ceph-mon[37173]: scrub queued for path: / 2024-07-27T03:17:54.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:54 smithi149 ceph-mon[37173]: scrub summary: idle+waiting paths [/] 2024-07-27T03:17:54.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:54 smithi149 ceph-mon[37173]: scrub summary: active paths [/] 2024-07-27T03:17:54.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:54 smithi149 ceph-mon[37173]: scrub summary: idle+waiting paths [/] 2024-07-27T03:17:54.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:54 smithi175 ceph-mon[37125]: pgmap v470: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:17:54.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:54 smithi175 ceph-mon[37125]: scrub queued for path: / 2024-07-27T03:17:54.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:54 smithi175 ceph-mon[37125]: scrub summary: idle+waiting paths [/] 2024-07-27T03:17:54.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:54 smithi175 ceph-mon[37125]: scrub summary: active paths [/] 2024-07-27T03:17:54.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:54 smithi175 ceph-mon[37125]: scrub summary: idle+waiting paths [/] 2024-07-27T03:17:56.277 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-07-27T03:17:56.347 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-07-27T03:17:56.388 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-07-27T03:17:56.390 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-07-27T03:17:56.390 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-07-27T03:17:56.432 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-07-27T03:17:56.435 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-27T03:17:56.436 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-07-27T03:17:56.452 INFO:tasks.workunit:Running workunits matching suites/fsync-tester.sh on client.0... 2024-07-27T03:17:56.453 INFO:tasks.workunit:Running workunit suites/fsync-tester.sh... 2024-07-27T03:17:56.454 DEBUG:teuthology.orchestra.run.smithi067:workunit test suites/fsync-tester.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=af8fdbf079811fe158ce62927bd145c35b74d4b8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-07-27T03:17:56.514 INFO:tasks.workunit.client.0.smithi067.stderr:+ wget http://download.ceph.com/qa/fsync-tester.c 2024-07-27T03:17:56.542 INFO:tasks.workunit.client.0.smithi067.stderr:--2024-07-27 03:17:56-- http://download.ceph.com/qa/fsync-tester.c 2024-07-27T03:17:56.542 INFO:tasks.workunit.client.0.smithi067.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-07-27T03:17:56.563 INFO:tasks.workunit.client.0.smithi067.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-07-27T03:17:56.584 INFO:tasks.workunit.client.0.smithi067.stderr:HTTP request sent, awaiting response... 200 OK 2024-07-27T03:17:56.584 INFO:tasks.workunit.client.0.smithi067.stderr:Length: 3024 (3.0K) [application/octet-stream] 2024-07-27T03:17:56.605 INFO:tasks.workunit.client.0.smithi067.stderr:Saving to: ‘fsync-tester.c’ 2024-07-27T03:17:56.609 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-07-27T03:17:56.609 INFO:tasks.workunit.client.0.smithi067.stderr: 0K .. 100% 524M=0s 2024-07-27T03:17:56.609 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-07-27T03:17:56.609 INFO:tasks.workunit.client.0.smithi067.stderr:2024-07-27 03:17:56 (524 MB/s) - ‘fsync-tester.c’ saved [3024/3024] 2024-07-27T03:17:56.609 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-07-27T03:17:56.609 INFO:tasks.workunit.client.0.smithi067.stderr:+ gcc -D_GNU_SOURCE fsync-tester.c -o fsync-tester 2024-07-27T03:17:56.699 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:56 smithi067 ceph-mon[32814]: pgmap v471: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-07-27T03:17:56.699 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:56 smithi067 ceph-mon[32814]: scrub summary: idle 2024-07-27T03:17:56.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:56 smithi149 ceph-mon[37173]: pgmap v471: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-07-27T03:17:56.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:56 smithi149 ceph-mon[37173]: scrub summary: idle 2024-07-27T03:17:56.809 INFO:tasks.workunit.client.0.smithi067.stderr:+ ./fsync-tester 2024-07-27T03:17:56.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.804+0000 7f2f4ac24640 1 -- 172.21.15.67:0/429607189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f44107e10 msgr2=0x7f2f44108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:56.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.804+0000 7f2f4ac24640 1 --2- 172.21.15.67:0/429607189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f44107e10 0x7f2f44108220 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009900 tx=0x7f2f3402a410 comp rx=0 tx=0).stop 2024-07-27T03:17:56.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.804+0000 7f2f4ac24640 1 -- 172.21.15.67:0/429607189 shutdown_connections 2024-07-27T03:17:56.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.804+0000 7f2f4ac24640 1 --2- 172.21.15.67:0/429607189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f44108760 0x7f2f4410ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.804+0000 7f2f4ac24640 1 --2- 172.21.15.67:0/429607189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f44107e10 0x7f2f44108220 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.804+0000 7f2f4ac24640 1 --2- 172.21.15.67:0/429607189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f441024d0 0x7f2f441028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.804+0000 7f2f4ac24640 1 -- 172.21.15.67:0/429607189 >> 172.21.15.67:0/429607189 conn(0x7f2f440791a0 msgr2=0x7f2f440795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:56.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.805+0000 7f2f4ac24640 1 -- 172.21.15.67:0/429607189 shutdown_connections 2024-07-27T03:17:56.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.805+0000 7f2f4ac24640 1 -- 172.21.15.67:0/429607189 wait complete. 2024-07-27T03:17:56.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.805+0000 7f2f4ac24640 1 Processor -- start 2024-07-27T03:17:56.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.805+0000 7f2f4ac24640 1 -- start start 2024-07-27T03:17:56.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.806+0000 7f2f4ac24640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f441024d0 0x7f2f44199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:56.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.806+0000 7f2f4ac24640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f44107e10 0x7f2f4419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:56.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.806+0000 7f2f4ac24640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f44108760 0x7f2f4419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:56.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.806+0000 7f2f4ac24640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441134f0 con 0x7f2f44107e10 2024-07-27T03:17:56.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.806+0000 7f2f4ac24640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2f44113370 con 0x7f2f44108760 2024-07-27T03:17:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.806+0000 7f2f4ac24640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2f44113670 con 0x7f2f441024d0 2024-07-27T03:17:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.806+0000 7f2f43fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f44107e10 0x7f2f4419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.806+0000 7f2f43fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f44107e10 0x7f2f4419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60034/0 (socket says 172.21.15.67:60034) 2024-07-27T03:17:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.806+0000 7f2f43fff640 1 -- 172.21.15.67:0/1371766055 learned_addr learned my addr 172.21.15.67:0/1371766055 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.806+0000 7f2f489c0640 1 --2- 172.21.15.67:0/1371766055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f441024d0 0x7f2f44199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.806+0000 7f2f491c1640 1 --2- 172.21.15.67:0/1371766055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f44108760 0x7f2f4419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.806+0000 7f2f43fff640 1 -- 172.21.15.67:0/1371766055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f441024d0 msgr2=0x7f2f44199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.806+0000 7f2f43fff640 1 --2- 172.21.15.67:0/1371766055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f441024d0 0x7f2f44199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.806+0000 7f2f43fff640 1 -- 172.21.15.67:0/1371766055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f44108760 msgr2=0x7f2f4419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.806+0000 7f2f43fff640 1 --2- 172.21.15.67:0/1371766055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f44108760 0x7f2f4419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.806+0000 7f2f43fff640 1 -- 172.21.15.67:0/1371766055 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4419fe50 con 0x7f2f44107e10 2024-07-27T03:17:56.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.806+0000 7f2f491c1640 1 --2- 172.21.15.67:0/1371766055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f44108760 0x7f2f4419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:17:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.806+0000 7f2f43fff640 1 --2- 172.21.15.67:0/1371766055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f44107e10 0x7f2f4419a540 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2f34031a90 tx=0x7f2f34004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.807+0000 7f2f41ffb640 1 -- 172.21.15.67:0/1371766055 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3403f030 con 0x7f2f44107e10 2024-07-27T03:17:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.807+0000 7f2f4ac24640 1 -- 172.21.15.67:0/1371766055 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f441a00e0 con 0x7f2f44107e10 2024-07-27T03:17:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.807+0000 7f2f41ffb640 1 -- 172.21.15.67:0/1371766055 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f34003cb0 con 0x7f2f44107e10 2024-07-27T03:17:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.807+0000 7f2f41ffb640 1 -- 172.21.15.67:0/1371766055 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34035070 con 0x7f2f44107e10 2024-07-27T03:17:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.807+0000 7f2f4ac24640 1 -- 172.21.15.67:0/1371766055 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f441a05c0 con 0x7f2f44107e10 2024-07-27T03:17:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.810+0000 7f2f511a9740 1 Processor -- start 2024-07-27T03:17:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.810+0000 7f2f511a9740 1 -- start start 2024-07-27T03:17:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.810+0000 7f2f41ffb640 1 -- 172.21.15.67:0/1371766055 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f2f34042030 con 0x7f2f44107e10 2024-07-27T03:17:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.810+0000 7f2f511a9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80bc0070 0x55ac80f64de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.810+0000 7f2f511a9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80b971b0 0x55ac80f65320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.810+0000 7f2f511a9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac80b9f390 0x55ac80f65860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.810+0000 7f2f511a9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ac80e4d3b0 con 0x55ac80bc0070 2024-07-27T03:17:56.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.810+0000 7f2f511a9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ac80edbad0 con 0x55ac80b971b0 2024-07-27T03:17:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.810+0000 7f2f511a9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ac80ecafd0 con 0x55ac80b9f390 2024-07-27T03:17:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.810+0000 7f2f491c1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80bc0070 0x55ac80f64de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.810+0000 7f2f491c1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80bc0070 0x55ac80f64de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60048/0 (socket says 172.21.15.67:60048) 2024-07-27T03:17:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.810+0000 7f2f491c1640 1 -- 172.21.15.67:0/3818307934 learned_addr learned my addr 172.21.15.67:0/3818307934 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.810+0000 7f2f489c0640 1 --2- 172.21.15.67:0/3818307934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80b971b0 0x55ac80f65320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.810+0000 7f2f43fff640 1 --2- 172.21.15.67:0/3818307934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac80b9f390 0x55ac80f65860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.811+0000 7f2f491c1640 1 -- 172.21.15.67:0/3818307934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac80b9f390 msgr2=0x55ac80f65860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.811+0000 7f2f491c1640 1 --2- 172.21.15.67:0/3818307934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac80b9f390 0x55ac80f65860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.811+0000 7f2f491c1640 1 -- 172.21.15.67:0/3818307934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80b971b0 msgr2=0x55ac80f65320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.811+0000 7f2f491c1640 1 --2- 172.21.15.67:0/3818307934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80b971b0 0x55ac80f65320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.811+0000 7f2f491c1640 1 -- 172.21.15.67:0/3818307934 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac80e60720 con 0x55ac80bc0070 2024-07-27T03:17:56.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.811+0000 7f2f491c1640 1 --2- 172.21.15.67:0/3818307934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80bc0070 0x55ac80f64de0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00cbf0 tx=0x7f2f2c00e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb87ae3983638cda server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:56.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.811+0000 7f2f3affd640 1 -- 172.21.15.67:0/3818307934 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c005020 con 0x55ac80bc0070 2024-07-27T03:17:56.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.811+0000 7f2f41ffb640 1 --2- 172.21.15.67:0/1371766055 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f2f300b3c00 0x7f2f300b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:56.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.811+0000 7f2f3affd640 1 -- 172.21.15.67:0/3818307934 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f2c016070 con 0x55ac80bc0070 2024-07-27T03:17:56.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.811+0000 7f2f3affd640 1 -- 172.21.15.67:0/3818307934 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c010690 con 0x55ac80bc0070 2024-07-27T03:17:56.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.811+0000 7f2f489c0640 1 --2- 172.21.15.67:0/1371766055 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f2f300b3c00 0x7f2f300b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:56.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.811+0000 7f2f41ffb640 1 -- 172.21.15.67:0/1371766055 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f2f341045b0 con 0x7f2f44107e10 2024-07-27T03:17:56.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.812+0000 7f2f511a9740 1 -- 172.21.15.67:0/3818307934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80bc0070 msgr2=0x55ac80f64de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:56.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.812+0000 7f2f489c0640 1 --2- 172.21.15.67:0/1371766055 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f2f300b3c00 0x7f2f300b6110 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55ac80f63a60 tx=0x7f2f3c00a570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:56.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.812+0000 7f2f511a9740 1 --2- 172.21.15.67:0/3818307934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80bc0070 0x55ac80f64de0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00cbf0 tx=0x7f2f2c00e350 comp rx=0 tx=0).stop 2024-07-27T03:17:56.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.812+0000 7f2f511a9740 1 -- 172.21.15.67:0/3818307934 shutdown_connections 2024-07-27T03:17:56.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.812+0000 7f2f511a9740 1 --2- 172.21.15.67:0/3818307934 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac80b9f390 0x55ac80f65860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.812+0000 7f2f511a9740 1 --2- 172.21.15.67:0/3818307934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80b971b0 0x55ac80f65320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.812+0000 7f2f511a9740 1 --2- 172.21.15.67:0/3818307934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80bc0070 0x55ac80f64de0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.812+0000 7f2f511a9740 1 -- 172.21.15.67:0/3818307934 >> 172.21.15.67:0/3818307934 conn(0x55ac80bb4460 msgr2=0x55ac80f62cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:56.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.812+0000 7f2f511a9740 1 -- 172.21.15.67:0/3818307934 shutdown_connections 2024-07-27T03:17:56.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.812+0000 7f2f511a9740 1 -- 172.21.15.67:0/3818307934 wait complete. 2024-07-27T03:17:56.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.813+0000 7f2f511a9740 1 -- start start 2024-07-27T03:17:56.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.813+0000 7f2f511a9740 1 Processor -- start 2024-07-27T03:17:56.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.813+0000 7f2f511a9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80b971b0 0x55ac80f70ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:56.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.813+0000 7f2f511a9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac80b9f390 0x55ac80f710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:56.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.813+0000 7f2f511a9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80bc0070 0x55ac80f80b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:56.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.813+0000 7f2f511a9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55ac80e6d8b0 con 0x55ac80b971b0 2024-07-27T03:17:56.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.813+0000 7f2f511a9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55ac80ecf010 con 0x55ac80bc0070 2024-07-27T03:17:56.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.813+0000 7f2f511a9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55ac80d2e700 con 0x55ac80b9f390 2024-07-27T03:17:56.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.813+0000 7f2f511a9740 10 client.0 _ll_register_callbacks cb 0x55ac80e75d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:17:56.834 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.813+0000 7f2f489c0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80b971b0 0x55ac80f70ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:56.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.813+0000 7f2f489c0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80b971b0 0x55ac80f70ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60056/0 (socket says 172.21.15.67:60056) 2024-07-27T03:17:56.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.813+0000 7f2f489c0640 1 -- 172.21.15.67:0/855263556 learned_addr learned my addr 172.21.15.67:0/855263556 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:56.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.814+0000 7f2f43fff640 1 --2- 172.21.15.67:0/855263556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac80b9f390 0x55ac80f710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:56.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.814+0000 7f2f491c1640 1 --2- 172.21.15.67:0/855263556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80bc0070 0x55ac80f80b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:56.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.814+0000 7f2f489c0640 1 -- 172.21.15.67:0/855263556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac80b9f390 msgr2=0x55ac80f710a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:56.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.814+0000 7f2f489c0640 1 --2- 172.21.15.67:0/855263556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac80b9f390 0x55ac80f710a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.814+0000 7f2f489c0640 1 -- 172.21.15.67:0/855263556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80bc0070 msgr2=0x55ac80f80b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:56.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.814+0000 7f2f489c0640 1 --2- 172.21.15.67:0/855263556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80bc0070 0x55ac80f80b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.814+0000 7f2f489c0640 1 -- 172.21.15.67:0/855263556 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac80e71ba0 con 0x55ac80b971b0 2024-07-27T03:17:56.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.814+0000 7f2f491c1640 1 --2- 172.21.15.67:0/855263556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80bc0070 0x55ac80f80b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:17:56.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.814+0000 7f2f489c0640 1 --2- 172.21.15.67:0/855263556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80b971b0 0x55ac80f70ac0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c01cbe0 tx=0x7f2f3c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba1662a93efe76d server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:56.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.814+0000 7f2f23fff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:17:56.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.814+0000 7f2f23fff640 1 -- 172.21.15.67:0/855263556 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c02c060 con 0x55ac80b971b0 2024-07-27T03:17:56.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.814+0000 7f2f23fff640 1 -- 172.21.15.67:0/855263556 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f2f3c016030 con 0x55ac80b971b0 2024-07-27T03:17:56.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.814+0000 7f2f511a9740 1 -- 172.21.15.67:0/855263556 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f2c016070 con 0x55ac80b971b0 2024-07-27T03:17:56.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.814+0000 7f2f23fff640 1 -- 172.21.15.67:0/855263556 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2f3c033050 con 0x55ac80b971b0 2024-07-27T03:17:56.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.814+0000 7f2f511a9740 10 client.15675 fetch_fsmap learned FSMap version 30 2024-07-27T03:17:56.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.814+0000 7f2f511a9740 1 -- 172.21.15.67:0/855263556 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac80c2b650 con 0x55ac80b971b0 2024-07-27T03:17:56.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.815+0000 7f2f23fff640 1 -- 172.21.15.67:0/855263556 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f2c016070 con 0x55ac80b971b0 2024-07-27T03:17:56.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.815+0000 7f2f23fff640 1 -- 172.21.15.67:0/855263556 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2f3c01a760 con 0x55ac80b971b0 2024-07-27T03:17:56.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.815+0000 7f2f511a9740 10 client.15675 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:17:56.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.815+0000 7f2f511a9740 10 client.15675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:17:56.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.815+0000 7f2f511a9740 20 client.15675 populate_metadata read hostname 'smithi067' 2024-07-27T03:17:56.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.815+0000 7f2f511a9740 1 --2- 172.21.15.67:0/855263556 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55ac80b7e9f0 0x55ac80f86cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:56.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.815+0000 7f2f511a9740 4 client.15675 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:17:56.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.815+0000 7f2f511a9740 1 -- 172.21.15.67:0/855263556 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac80edda20 con 0x55ac80b7e9f0 2024-07-27T03:17:56.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.815+0000 7f2f43fff640 1 --2- 172.21.15.67:0/855263556 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55ac80b7e9f0 0x55ac80f86cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.816+0000 7f2f23fff640 10 client.15675 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:17:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.816+0000 7f2f43fff640 1 --2- 172.21.15.67:0/855263556 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55ac80b7e9f0 0x55ac80f86cd0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17aa462bf92126a5 server_cookie=f010a4a669149403 in_seq=0 out_seq=0 2024-07-27T03:17:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.817+0000 7f2f23fff640 1 -- 172.21.15.67:0/855263556 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac80edda20 con 0x55ac80b7e9f0 2024-07-27T03:17:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.817+0000 7f2f23fff640 10 client.15675 handle_command_reply: tid=0 2024-07-27T03:17:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.817+0000 7f2f511a9740 1 client.15675 shutdown 2024-07-27T03:17:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.817+0000 7f2f511a9740 1 -- 172.21.15.67:0/855263556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80b971b0 msgr2=0x55ac80f70ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.817+0000 7f2f511a9740 1 --2- 172.21.15.67:0/855263556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80b971b0 0x55ac80f70ac0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c01cbe0 tx=0x7f2f3c004ab0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.817+0000 7f2f511a9740 1 -- 172.21.15.67:0/855263556 shutdown_connections 2024-07-27T03:17:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.817+0000 7f2f511a9740 1 --2- 172.21.15.67:0/855263556 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55ac80b7e9f0 0x55ac80f86cd0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.817+0000 7f2f511a9740 1 --2- 172.21.15.67:0/855263556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80bc0070 0x55ac80f80b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.817+0000 7f2f511a9740 1 --2- 172.21.15.67:0/855263556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac80b9f390 0x55ac80f710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.817+0000 7f2f511a9740 1 --2- 172.21.15.67:0/855263556 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80b971b0 0x55ac80f70ac0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.817+0000 7f2f511a9740 1 -- 172.21.15.67:0/855263556 >> 172.21.15.67:0/855263556 conn(0x55ac80bb4460 msgr2=0x55ac80f67fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:56.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.817+0000 7f2f23fff640 0 client.15675 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:17:56.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.817+0000 7f2f511a9740 1 -- 172.21.15.67:0/855263556 shutdown_connections 2024-07-27T03:17:56.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.817+0000 7f2f511a9740 1 -- 172.21.15.67:0/855263556 wait complete. 2024-07-27T03:17:56.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.817+0000 7f2f511a9740 20 client.15675 trim_cache size 0 max 16384 2024-07-27T03:17:56.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.835+0000 7f2f511a9740 1 Processor -- start 2024-07-27T03:17:56.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.835+0000 7f2f511a9740 1 -- start start 2024-07-27T03:17:56.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.835+0000 7f2f511a9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80b971b0 0x55ac80f6a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:56.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.835+0000 7f2f511a9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80b9f390 0x55ac80f8c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:56.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.835+0000 7f2f511a9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac80bc0070 0x55ac80f8ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:56.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.835+0000 7f2f511a9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441ae390 con 0x55ac80b9f390 2024-07-27T03:17:56.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.835+0000 7f2f511a9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441afed0 con 0x55ac80b971b0 2024-07-27T03:17:56.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.835+0000 7f2f511a9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441aea60 con 0x55ac80bc0070 2024-07-27T03:17:56.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.835+0000 7f2f43fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80b9f390 0x55ac80f8c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:56.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.835+0000 7f2f43fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80b9f390 0x55ac80f8c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60072/0 (socket says 172.21.15.67:60072) 2024-07-27T03:17:56.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.835+0000 7f2f491c1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac80bc0070 0x55ac80f8ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:56.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.835+0000 7f2f43fff640 1 -- 172.21.15.67:0/2453879434 learned_addr learned my addr 172.21.15.67:0/2453879434 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:56.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.835+0000 7f2f489c0640 1 --2- 172.21.15.67:0/2453879434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80b971b0 0x55ac80f6a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:56.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.836+0000 7f2f43fff640 1 -- 172.21.15.67:0/2453879434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac80bc0070 msgr2=0x55ac80f8ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:56.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.836+0000 7f2f43fff640 1 --2- 172.21.15.67:0/2453879434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac80bc0070 0x55ac80f8ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.836+0000 7f2f43fff640 1 -- 172.21.15.67:0/2453879434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80b971b0 msgr2=0x55ac80f6a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:56.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.836+0000 7f2f43fff640 1 --2- 172.21.15.67:0/2453879434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80b971b0 0x55ac80f6a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.836+0000 7f2f43fff640 1 -- 172.21.15.67:0/2453879434 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac80f6f820 con 0x55ac80b9f390 2024-07-27T03:17:56.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.836+0000 7f2f43fff640 1 --2- 172.21.15.67:0/2453879434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80b9f390 0x55ac80f8c4e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2f34003fd0 tx=0x7f2f3402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5023b8b971ddf9d server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:56.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.836+0000 7f2f3a7fc640 1 -- 172.21.15.67:0/2453879434 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34049040 con 0x55ac80b9f390 2024-07-27T03:17:56.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.836+0000 7f2f3a7fc640 1 -- 172.21.15.67:0/2453879434 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f340cac60 con 0x55ac80b9f390 2024-07-27T03:17:56.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.836+0000 7f2f3a7fc640 1 -- 172.21.15.67:0/2453879434 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3403b460 con 0x55ac80b9f390 2024-07-27T03:17:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.837+0000 7f2f511a9740 1 -- 172.21.15.67:0/2453879434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80b9f390 msgr2=0x55ac80f8c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.837+0000 7f2f511a9740 1 --2- 172.21.15.67:0/2453879434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80b9f390 0x55ac80f8c4e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2f34003fd0 tx=0x7f2f3402a410 comp rx=0 tx=0).stop 2024-07-27T03:17:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.837+0000 7f2f511a9740 1 -- 172.21.15.67:0/2453879434 shutdown_connections 2024-07-27T03:17:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.837+0000 7f2f511a9740 1 --2- 172.21.15.67:0/2453879434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac80bc0070 0x55ac80f8ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.837+0000 7f2f511a9740 1 --2- 172.21.15.67:0/2453879434 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80b9f390 0x55ac80f8c4e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.837+0000 7f2f511a9740 1 --2- 172.21.15.67:0/2453879434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80b971b0 0x55ac80f6a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.844 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.837+0000 7f2f511a9740 1 -- 172.21.15.67:0/2453879434 >> 172.21.15.67:0/2453879434 conn(0x55ac80bb4460 msgr2=0x55ac80f69ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.837+0000 7f2f511a9740 1 -- 172.21.15.67:0/2453879434 shutdown_connections 2024-07-27T03:17:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.837+0000 7f2f511a9740 1 -- 172.21.15.67:0/2453879434 wait complete. 2024-07-27T03:17:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.838+0000 7f2f511a9740 1 -- start start 2024-07-27T03:17:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.838+0000 7f2f511a9740 1 Processor -- start 2024-07-27T03:17:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.838+0000 7f2f511a9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80b971b0 0x55ac80f7d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.838+0000 7f2f511a9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80b9f390 0x55ac80f78530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.838+0000 7f2f511a9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac80bc0070 0x55ac80f78a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.838+0000 7f2f511a9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441af800 con 0x55ac80b971b0 2024-07-27T03:17:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.838+0000 7f2f511a9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441af130 con 0x55ac80b9f390 2024-07-27T03:17:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.838+0000 7f2f511a9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441ac370 con 0x55ac80bc0070 2024-07-27T03:17:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.838+0000 7f2f511a9740 10 client.0 _ll_register_callbacks cb 0x7f2f3c01c890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:17:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.838+0000 7f2f489c0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80b971b0 0x55ac80f7d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.838+0000 7f2f43fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80b9f390 0x55ac80f78530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.839+0000 7f2f43fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80b9f390 0x55ac80f78530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.67:43406/0 (socket says 172.21.15.67:43406) 2024-07-27T03:17:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.839+0000 7f2f489c0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80b971b0 0x55ac80f7d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60086/0 (socket says 172.21.15.67:60086) 2024-07-27T03:17:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.839+0000 7f2f43fff640 1 -- 172.21.15.67:0/1716446214 learned_addr learned my addr 172.21.15.67:0/1716446214 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.839+0000 7f2f491c1640 1 --2- 172.21.15.67:0/1716446214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac80bc0070 0x55ac80f78a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.839+0000 7f2f489c0640 1 -- 172.21.15.67:0/1716446214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac80bc0070 msgr2=0x55ac80f78a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.839+0000 7f2f489c0640 1 --2- 172.21.15.67:0/1716446214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac80bc0070 0x55ac80f78a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.839+0000 7f2f489c0640 1 -- 172.21.15.67:0/1716446214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80b9f390 msgr2=0x55ac80f78530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:17:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.839+0000 7f2f489c0640 1 --2- 172.21.15.67:0/1716446214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80b9f390 0x55ac80f78530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.839+0000 7f2f489c0640 1 -- 172.21.15.67:0/1716446214 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac80f79110 con 0x55ac80b971b0 2024-07-27T03:17:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.839+0000 7f2f43fff640 1 --2- 172.21.15.67:0/1716446214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80b9f390 0x55ac80f78530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:17:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.839+0000 7f2f489c0640 1 --2- 172.21.15.67:0/1716446214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80b971b0 0x55ac80f7d080 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c011010 tx=0x7f2f3c01a3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f791e3773797411 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:56.846 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.839+0000 7f2f23fff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:17:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.839+0000 7f2f23fff640 1 -- 172.21.15.67:0/1716446214 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c03b070 con 0x55ac80b971b0 2024-07-27T03:17:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.839+0000 7f2f23fff640 1 -- 172.21.15.67:0/1716446214 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f2f3c01a7a0 con 0x55ac80b971b0 2024-07-27T03:17:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.839+0000 7f2f511a9740 1 -- 172.21.15.67:0/1716446214 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f340cac60 con 0x55ac80b971b0 2024-07-27T03:17:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.840+0000 7f2f23fff640 1 -- 172.21.15.67:0/1716446214 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2f3c047050 con 0x55ac80b971b0 2024-07-27T03:17:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.840+0000 7f2f511a9740 10 client.15681 fetch_fsmap learned FSMap version 30 2024-07-27T03:17:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.840+0000 7f2f511a9740 1 -- 172.21.15.67:0/1716446214 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ac80fef310 con 0x55ac80b971b0 2024-07-27T03:17:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.840+0000 7f2f23fff640 1 -- 172.21.15.67:0/1716446214 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f340cac60 con 0x55ac80b971b0 2024-07-27T03:17:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.840+0000 7f2f23fff640 1 -- 172.21.15.67:0/1716446214 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2f3c040640 con 0x55ac80b971b0 2024-07-27T03:17:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.840+0000 7f2f511a9740 10 client.15681 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:17:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.840+0000 7f2f511a9740 10 client.15681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:17:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.840+0000 7f2f511a9740 20 client.15681 populate_metadata read hostname 'smithi067' 2024-07-27T03:17:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.840+0000 7f2f511a9740 1 --2- 172.21.15.67:0/1716446214 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55ac80b53da0 0x55ac80ff1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:56.847 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.840+0000 7f2f511a9740 4 client.15681 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-27T03:17:56.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.840+0000 7f2f511a9740 1 -- 172.21.15.67:0/1716446214 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ac80ed94c0 con 0x55ac80b53da0 2024-07-27T03:17:56.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.840+0000 7f2f43fff640 1 --2- 172.21.15.67:0/1716446214 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55ac80b53da0 0x55ac80ff1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:56.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.841+0000 7f2f23fff640 10 client.15681 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:17:56.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.841+0000 7f2f43fff640 1 --2- 172.21.15.67:0/1716446214 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55ac80b53da0 0x55ac80ff1b00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbf17745a3d8be55 server_cookie=19181d4697f3e545 in_seq=0 out_seq=0 2024-07-27T03:17:56.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.841+0000 7f2f23fff640 1 -- 172.21.15.67:0/1716446214 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ac80ed94c0 con 0x55ac80b53da0 2024-07-27T03:17:56.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.841+0000 7f2f23fff640 10 client.15681 handle_command_reply: tid=0 2024-07-27T03:17:56.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.841+0000 7f2f511a9740 1 client.15681 shutdown 2024-07-27T03:17:56.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.841+0000 7f2f511a9740 1 -- 172.21.15.67:0/1716446214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80b971b0 msgr2=0x55ac80f7d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:56.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.841+0000 7f2f511a9740 1 --2- 172.21.15.67:0/1716446214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80b971b0 0x55ac80f7d080 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c011010 tx=0x7f2f3c01a3a0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.841+0000 7f2f511a9740 1 -- 172.21.15.67:0/1716446214 shutdown_connections 2024-07-27T03:17:56.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.841+0000 7f2f511a9740 1 --2- 172.21.15.67:0/1716446214 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55ac80b53da0 0x55ac80ff1b00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.841+0000 7f2f511a9740 1 --2- 172.21.15.67:0/1716446214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55ac80bc0070 0x55ac80f78a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.841+0000 7f2f511a9740 1 --2- 172.21.15.67:0/1716446214 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac80b9f390 0x55ac80f78530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.848 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.841+0000 7f2f511a9740 1 --2- 172.21.15.67:0/1716446214 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55ac80b971b0 0x55ac80f7d080 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.841+0000 7f2f23fff640 0 client.15681 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:17:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.841+0000 7f2f511a9740 1 -- 172.21.15.67:0/1716446214 >> 172.21.15.67:0/1716446214 conn(0x55ac80bb4460 msgr2=0x55ac80f65e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.841+0000 7f2f511a9740 1 -- 172.21.15.67:0/1716446214 shutdown_connections 2024-07-27T03:17:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.841+0000 7f2f511a9740 1 -- 172.21.15.67:0/1716446214 wait complete. 2024-07-27T03:17:56.849 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.842+0000 7f2f511a9740 20 client.15681 trim_cache size 0 max 16384 2024-07-27T03:17:56.849 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:17:56.849 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-27T03:17:56.849 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-27T03:17:56.849 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:17:56.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.842+0000 7f2f4ac24640 1 -- 172.21.15.67:0/1371766055 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f2f300b3c00 msgr2=0x7f2f300b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:56.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.842+0000 7f2f4ac24640 1 --2- 172.21.15.67:0/1371766055 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f2f300b3c00 0x7f2f300b6110 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55ac80f63a60 tx=0x7f2f3c00a570 comp rx=0 tx=0).stop 2024-07-27T03:17:56.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.842+0000 7f2f4ac24640 1 -- 172.21.15.67:0/1371766055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f44107e10 msgr2=0x7f2f4419a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:56.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.842+0000 7f2f4ac24640 1 --2- 172.21.15.67:0/1371766055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f44107e10 0x7f2f4419a540 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2f34031a90 tx=0x7f2f34004270 comp rx=0 tx=0).stop 2024-07-27T03:17:56.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.842+0000 7f2f4ac24640 1 -- 172.21.15.67:0/1371766055 shutdown_connections 2024-07-27T03:17:56.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.842+0000 7f2f4ac24640 1 --2- 172.21.15.67:0/1371766055 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f2f300b3c00 0x7f2f300b6110 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.850 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.842+0000 7f2f4ac24640 1 --2- 172.21.15.67:0/1371766055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f44108760 0x7f2f4419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.842+0000 7f2f4ac24640 1 --2- 172.21.15.67:0/1371766055 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2f44107e10 0x7f2f4419a540 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.842+0000 7f2f4ac24640 1 --2- 172.21.15.67:0/1371766055 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f441024d0 0x7f2f44199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:56.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.842+0000 7f2f4ac24640 1 -- 172.21.15.67:0/1371766055 >> 172.21.15.67:0/1371766055 conn(0x7f2f440791a0 msgr2=0x7f2f44103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:56.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.842+0000 7f2f4ac24640 1 -- 172.21.15.67:0/1371766055 shutdown_connections 2024-07-27T03:17:56.851 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:56.842+0000 7f2f4ac24640 1 -- 172.21.15.67:0/1371766055 wait complete. 2024-07-27T03:17:56.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:56 smithi175 ceph-mon[37125]: pgmap v471: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-07-27T03:17:56.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:56 smithi175 ceph-mon[37125]: scrub summary: idle 2024-07-27T03:17:57.622 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-27T03:17:57.622 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:17:57.943 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:17:58.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:17:58 smithi067 ceph-mon[32814]: pgmap v472: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-07-27T03:17:58.749 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:17:58 smithi149 ceph-mon[37173]: pgmap v472: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-07-27T03:17:58.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:17:58 smithi175 ceph-mon[37125]: pgmap v472: 129 pgs: 129 active+clean; 662 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-07-27T03:17:59.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.671+0000 7f125f210640 1 -- 172.21.15.67:0/2559593444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12581087a0 msgr2=0x7f125810efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:59.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.671+0000 7f125f210640 1 --2- 172.21.15.67:0/2559593444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12581087a0 0x7f125810efd0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1250009ca0 tx=0x7f125002a8c0 comp rx=0 tx=0).stop 2024-07-27T03:17:59.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.671+0000 7f125f210640 1 -- 172.21.15.67:0/2559593444 shutdown_connections 2024-07-27T03:17:59.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.671+0000 7f125f210640 1 --2- 172.21.15.67:0/2559593444 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12581087a0 0x7f125810efd0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:59.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.671+0000 7f125f210640 1 --2- 172.21.15.67:0/2559593444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1258107e50 0x7f1258108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:59.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.671+0000 7f125f210640 1 --2- 172.21.15.67:0/2559593444 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1258102510 0x7f1258102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:59.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.671+0000 7f125f210640 1 -- 172.21.15.67:0/2559593444 >> 172.21.15.67:0/2559593444 conn(0x7f1258079250 msgr2=0x7f1258079660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:59.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.671+0000 7f125f210640 1 -- 172.21.15.67:0/2559593444 shutdown_connections 2024-07-27T03:17:59.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.671+0000 7f125f210640 1 -- 172.21.15.67:0/2559593444 wait complete. 2024-07-27T03:17:59.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.672+0000 7f125f210640 1 Processor -- start 2024-07-27T03:17:59.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.672+0000 7f125f210640 1 -- start start 2024-07-27T03:17:59.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.672+0000 7f125f210640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1258102510 0x7f125819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:59.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.672+0000 7f125f210640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1258107e50 0x7f125819b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:59.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.672+0000 7f125f210640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12581087a0 0x7f125819b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:59.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.672+0000 7f125f210640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1258112420 con 0x7f12581087a0 2024-07-27T03:17:59.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.672+0000 7f125f210640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f12581122a0 con 0x7f1258102510 2024-07-27T03:17:59.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.672+0000 7f125f210640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f12581125a0 con 0x7f1258107e50 2024-07-27T03:17:59.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.673+0000 7f125d7ad640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12581087a0 0x7f125819b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:59.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.673+0000 7f125d7ad640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12581087a0 0x7f125819b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60090/0 (socket says 172.21.15.67:60090) 2024-07-27T03:17:59.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.673+0000 7f125cfac640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1258102510 0x7f125819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:59.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.673+0000 7f125d7ad640 1 -- 172.21.15.67:0/3549097882 learned_addr learned my addr 172.21.15.67:0/3549097882 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:17:59.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.673+0000 7f1257fff640 1 --2- 172.21.15.67:0/3549097882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1258107e50 0x7f125819b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:59.678 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.673+0000 7f125cfac640 1 -- 172.21.15.67:0/3549097882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1258107e50 msgr2=0x7f125819b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.673+0000 7f125cfac640 1 --2- 172.21.15.67:0/3549097882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1258107e50 0x7f125819b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.673+0000 7f125cfac640 1 -- 172.21.15.67:0/3549097882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12581087a0 msgr2=0x7f125819b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.673+0000 7f125cfac640 1 --2- 172.21.15.67:0/3549097882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12581087a0 0x7f125819b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.673+0000 7f125cfac640 1 -- 172.21.15.67:0/3549097882 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12581a5f40 con 0x7f1258102510 2024-07-27T03:17:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.673+0000 7f125d7ad640 1 --2- 172.21.15.67:0/3549097882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12581087a0 0x7f125819b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:17:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.673+0000 7f1257fff640 1 --2- 172.21.15.67:0/3549097882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1258107e50 0x7f125819b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:17:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.673+0000 7f125cfac640 1 --2- 172.21.15.67:0/3549097882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1258102510 0x7f125819acb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f124c00e9f0 tx=0x7f124c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.673+0000 7f1255ffb640 1 -- 172.21.15.67:0/3549097882 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c00ccc0 con 0x7f1258102510 2024-07-27T03:17:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.673+0000 7f125f210640 1 -- 172.21.15.67:0/3549097882 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12581a6230 con 0x7f1258102510 2024-07-27T03:17:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.673+0000 7f1255ffb640 1 -- 172.21.15.67:0/3549097882 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f124c005020 con 0x7f1258102510 2024-07-27T03:17:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.674+0000 7f1255ffb640 1 -- 172.21.15.67:0/3549097882 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c004a00 con 0x7f1258102510 2024-07-27T03:17:59.679 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.674+0000 7f125f210640 1 -- 172.21.15.67:0/3549097882 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12581a6770 con 0x7f1258102510 2024-07-27T03:17:59.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.675+0000 7f125f210640 1 -- 172.21.15.67:0/3549097882 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1258102920 con 0x7f1258102510 2024-07-27T03:17:59.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.676+0000 7f1255ffb640 1 -- 172.21.15.67:0/3549097882 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f124c004ba0 con 0x7f1258102510 2024-07-27T03:17:59.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.677+0000 7f1255ffb640 1 --2- 172.21.15.67:0/3549097882 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f12480b3c00 0x7f12480b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:17:59.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.677+0000 7f1257fff640 1 --2- 172.21.15.67:0/3549097882 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f12480b3c00 0x7f12480b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:17:59.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.677+0000 7f1255ffb640 1 -- 172.21.15.67:0/3549097882 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f124c0e4160 con 0x7f1258102510 2024-07-27T03:17:59.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.678+0000 7f1257fff640 1 --2- 172.21.15.67:0/3549097882 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f12480b3c00 0x7f12480b6110 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1258108700 tx=0x7f1244005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:17:59.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.680+0000 7f1255ffb640 1 -- 172.21.15.67:0/3549097882 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f124c0ab3a0 con 0x7f1258102510 2024-07-27T03:17:59.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.904+0000 7f125f210640 1 -- 172.21.15.67:0/3549097882 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f125806b6a0 con 0x7f1258102510 2024-07-27T03:17:59.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.905+0000 7f1255ffb640 1 -- 172.21.15.67:0/3549097882 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11983 (secure 0 0 0) 0x7f125806b6a0 con 0x7f1258102510 2024-07-27T03:17:59.908 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:17:59.908 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"btime":"2024-07-27T03:16:40:350577+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14679,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/252686369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":252686369},{"type":"v1","addr":"172.21.15.67:6835","nonce":252686369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/3358832762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3358832762},{"type":"v1","addr":"172.21.15.67:6837","nonce":3358832762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-27T03:12:34.320208+0000","modified":"2024-07-27T03:16:39.741149+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24620,"mds_1":24587,"mds_2":24583,"mds_3":24581,"mds_4":24563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14736":{"gid":14736,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/2162708317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2162708317},{"type":"v1","addr":"172.21.15.67:6839","nonce":2162708317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14748":{"gid":14748,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/795704244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":795704244},{"type":"v1","addr":"172.21.15.67:6841","nonce":795704244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/3868325215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3868325215},{"type":"v1","addr":"172.21.15.149:6833","nonce":3868325215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2462749027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2462749027},{"type":"v1","addr":"172.21.15.175:6835","nonce":2462749027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.149:6835/3730964806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3730964806},{"type":"v1","addr":"172.21.15.149:6835","nonce":3730964806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.175:6837/1304803502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1304803502},{"type":"v1","addr":"172.21.15.175:6837","nonce":1304803502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.175:6839/1670568853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1670568853},{"type":"v1","addr":"172.21.15.175:6839","nonce":1670568853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24587":{"gid":24587,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.149:6837/2932951845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2932951845},{"type":"v1","addr":"172.21.15.149:6837","nonce":2932951845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.149:6839/281746498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":281746498},{"type":"v1","addr":"172.21.15.149:6839","nonce":281746498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24620,"qdb_cluster":[24587,24583,24620,24581,24563]},"id":1}]} 2024-07-27T03:17:59.908 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 30 2024-07-27T03:17:59.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.907+0000 7f125f210640 1 -- 172.21.15.67:0/3549097882 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f12480b3c00 msgr2=0x7f12480b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:59.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.907+0000 7f125f210640 1 --2- 172.21.15.67:0/3549097882 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f12480b3c00 0x7f12480b6110 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1258108700 tx=0x7f1244005f90 comp rx=0 tx=0).stop 2024-07-27T03:17:59.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.907+0000 7f125f210640 1 -- 172.21.15.67:0/3549097882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1258102510 msgr2=0x7f125819acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:17:59.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.907+0000 7f125f210640 1 --2- 172.21.15.67:0/3549097882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1258102510 0x7f125819acb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f124c00e9f0 tx=0x7f124c00eeb0 comp rx=0 tx=0).stop 2024-07-27T03:17:59.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.907+0000 7f125f210640 1 -- 172.21.15.67:0/3549097882 shutdown_connections 2024-07-27T03:17:59.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.907+0000 7f125f210640 1 --2- 172.21.15.67:0/3549097882 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f12480b3c00 0x7f12480b6110 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:59.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.907+0000 7f125f210640 1 --2- 172.21.15.67:0/3549097882 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12581087a0 0x7f125819b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:59.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.907+0000 7f125f210640 1 --2- 172.21.15.67:0/3549097882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1258107e50 0x7f125819b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:59.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.907+0000 7f125f210640 1 --2- 172.21.15.67:0/3549097882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1258102510 0x7f125819acb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:17:59.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.907+0000 7f125f210640 1 -- 172.21.15.67:0/3549097882 >> 172.21.15.67:0/3549097882 conn(0x7f1258079250 msgr2=0x7f1258103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:17:59.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.908+0000 7f125f210640 1 -- 172.21.15.67:0/3549097882 shutdown_connections 2024-07-27T03:17:59.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:17:59.908+0000 7f125f210640 1 -- 172.21.15.67:0/3549097882 wait complete. 2024-07-27T03:18:00.774 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:00 smithi067 ceph-mon[32814]: pgmap v473: 129 pgs: 129 active+clean; 1.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 171 KiB/s wr, 9 op/s 2024-07-27T03:18:00.774 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:00 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3549097882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:18:00.807 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-27T03:18:00.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:00 smithi175 ceph-mon[37125]: pgmap v473: 129 pgs: 129 active+clean; 1.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 171 KiB/s wr, 9 op/s 2024-07-27T03:18:00.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:00 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3549097882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:18:01.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:00 smithi149 ceph-mon[37173]: pgmap v473: 129 pgs: 129 active+clean; 1.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 171 KiB/s wr, 9 op/s 2024-07-27T03:18:01.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:00 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3549097882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:18:01.101 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:18:02.764 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:02 smithi067 ceph-mon[32814]: pgmap v474: 129 pgs: 129 active+clean; 1.7 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 176 KiB/s wr, 13 op/s 2024-07-27T03:18:02.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:02 smithi175 ceph-mon[37125]: pgmap v474: 129 pgs: 129 active+clean; 1.7 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 176 KiB/s wr, 13 op/s 2024-07-27T03:18:02.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.976+0000 7fcc79df3640 1 -- 172.21.15.67:0/402678555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc741024d0 msgr2=0x7fcc741028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:02.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.976+0000 7fcc79df3640 1 --2- 172.21.15.67:0/402678555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc741024d0 0x7fcc741028e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009900 tx=0x7fcc6402a410 comp rx=0 tx=0).stop 2024-07-27T03:18:02.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.976+0000 7fcc79df3640 1 -- 172.21.15.67:0/402678555 shutdown_connections 2024-07-27T03:18:02.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.976+0000 7fcc79df3640 1 --2- 172.21.15.67:0/402678555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc74108760 0x7fcc7410ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:02.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.976+0000 7fcc79df3640 1 --2- 172.21.15.67:0/402678555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc74107e10 0x7fcc74108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:02.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.976+0000 7fcc79df3640 1 --2- 172.21.15.67:0/402678555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc741024d0 0x7fcc741028e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:02.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.976+0000 7fcc79df3640 1 -- 172.21.15.67:0/402678555 >> 172.21.15.67:0/402678555 conn(0x7fcc740791a0 msgr2=0x7fcc740795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:02.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.976+0000 7fcc79df3640 1 -- 172.21.15.67:0/402678555 shutdown_connections 2024-07-27T03:18:02.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.976+0000 7fcc79df3640 1 -- 172.21.15.67:0/402678555 wait complete. 2024-07-27T03:18:02.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.977+0000 7fcc79df3640 1 Processor -- start 2024-07-27T03:18:02.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.977+0000 7fcc79df3640 1 -- start start 2024-07-27T03:18:02.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.977+0000 7fcc79df3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc741024d0 0x7fcc74103cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:02.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.977+0000 7fcc79df3640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc74107e10 0x7fcc74104200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:02.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.977+0000 7fcc79df3640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc74108760 0x7fcc74104740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:02.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.977+0000 7fcc79df3640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcc741123c0 con 0x7fcc74108760 2024-07-27T03:18:02.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.977+0000 7fcc79df3640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcc74112240 con 0x7fcc741024d0 2024-07-27T03:18:02.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.977+0000 7fcc79df3640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcc74112540 con 0x7fcc74107e10 2024-07-27T03:18:02.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.977+0000 7fcc73fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc74108760 0x7fcc74104740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:02.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.977+0000 7fcc73fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc74108760 0x7fcc74104740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60116/0 (socket says 172.21.15.67:60116) 2024-07-27T03:18:02.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.977+0000 7fcc73fff640 1 -- 172.21.15.67:0/3202665871 learned_addr learned my addr 172.21.15.67:0/3202665871 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:02.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.978+0000 7fcc737fe640 1 --2- 172.21.15.67:0/3202665871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc741024d0 0x7fcc74103cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:02.982 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.978+0000 7fcc72ffd640 1 --2- 172.21.15.67:0/3202665871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc74107e10 0x7fcc74104200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:02.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.978+0000 7fcc73fff640 1 -- 172.21.15.67:0/3202665871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc74107e10 msgr2=0x7fcc74104200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:02.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.978+0000 7fcc73fff640 1 --2- 172.21.15.67:0/3202665871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc74107e10 0x7fcc74104200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:02.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.978+0000 7fcc73fff640 1 -- 172.21.15.67:0/3202665871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc741024d0 msgr2=0x7fcc74103cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:02.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.978+0000 7fcc73fff640 1 --2- 172.21.15.67:0/3202665871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc741024d0 0x7fcc74103cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:02.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.978+0000 7fcc73fff640 1 -- 172.21.15.67:0/3202665871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc741a81a0 con 0x7fcc74108760 2024-07-27T03:18:02.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.978+0000 7fcc73fff640 1 --2- 172.21.15.67:0/3202665871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc74108760 0x7fcc74104740 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009fd0 tx=0x7fcc6c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:02.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.978+0000 7fcc70ff9640 1 -- 172.21.15.67:0/3202665871 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c017070 con 0x7fcc74108760 2024-07-27T03:18:02.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.978+0000 7fcc79df3640 1 -- 172.21.15.67:0/3202665871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc741a8490 con 0x7fcc74108760 2024-07-27T03:18:02.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.978+0000 7fcc70ff9640 1 -- 172.21.15.67:0/3202665871 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc6c003e20 con 0x7fcc74108760 2024-07-27T03:18:02.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.978+0000 7fcc70ff9640 1 -- 172.21.15.67:0/3202665871 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c010460 con 0x7fcc74108760 2024-07-27T03:18:02.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.978+0000 7fcc79df3640 1 -- 172.21.15.67:0/3202665871 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc741a89d0 con 0x7fcc74108760 2024-07-27T03:18:02.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.981+0000 7fcc80378740 1 Processor -- start 2024-07-27T03:18:02.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.981+0000 7fcc70ff9640 1 -- 172.21.15.67:0/3202665871 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7fcc6c003970 con 0x7fcc74108760 2024-07-27T03:18:02.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.982+0000 7fcc80378740 1 -- start start 2024-07-27T03:18:02.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.982+0000 7fcc80378740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55790eca6070 0x55790f04ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:02.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.982+0000 7fcc80378740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55790ec7d1b0 0x55790f04b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:02.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.982+0000 7fcc80378740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790ec85390 0x55790f04b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:02.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.982+0000 7fcc80378740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55790ef33400 con 0x55790ec85390 2024-07-27T03:18:02.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.982+0000 7fcc80378740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55790efc1b20 con 0x55790eca6070 2024-07-27T03:18:02.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.982+0000 7fcc80378740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55790efb1020 con 0x55790ec7d1b0 2024-07-27T03:18:02.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.982+0000 7fcc72ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790ec85390 0x55790f04b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:02.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.982+0000 7fcc72ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790ec85390 0x55790f04b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60118/0 (socket says 172.21.15.67:60118) 2024-07-27T03:18:02.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.982+0000 7fcc737fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55790ec7d1b0 0x55790f04b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:02.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.982+0000 7fcc72ffd640 1 -- 172.21.15.67:0/760215713 learned_addr learned my addr 172.21.15.67:0/760215713 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:02.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.982+0000 7fcc70ff9640 1 --2- 172.21.15.67:0/3202665871 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcc5c0b3bd0 0x7fcc5c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:02.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.982+0000 7fcc72ffd640 1 --2- 172.21.15.67:0/760215713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55790eca6070 0x55790f04ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:02.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.983+0000 7fcc73fff640 1 --2- 172.21.15.67:0/3202665871 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcc5c0b3bd0 0x7fcc5c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:02.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.983+0000 7fcc737fe640 1 -- 172.21.15.67:0/760215713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55790eca6070 msgr2=0x55790f04ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:02.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.983+0000 7fcc737fe640 1 --2- 172.21.15.67:0/760215713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55790eca6070 0x55790f04ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:02.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.983+0000 7fcc737fe640 1 -- 172.21.15.67:0/760215713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790ec85390 msgr2=0x55790f04b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:02.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.983+0000 7fcc737fe640 1 --2- 172.21.15.67:0/760215713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790ec85390 0x55790f04b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:02.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.983+0000 7fcc70ff9640 1 -- 172.21.15.67:0/3202665871 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fcc6c0e3c90 con 0x7fcc74108760 2024-07-27T03:18:02.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.983+0000 7fcc737fe640 1 -- 172.21.15.67:0/760215713 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55790ef46770 con 0x55790ec7d1b0 2024-07-27T03:18:02.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.983+0000 7fcc72ffd640 1 --2- 172.21.15.67:0/760215713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790ec85390 0x55790f04b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:18:02.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.983+0000 7fcc73fff640 1 --2- 172.21.15.67:0/3202665871 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcc5c0b3bd0 0x7fcc5c0b60e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c003fd0 tx=0x7fcc6c004480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:02.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.983+0000 7fcc737fe640 1 --2- 172.21.15.67:0/760215713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55790ec7d1b0 0x55790f04b280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcc74105710 tx=0x7fcc64032c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b812357f06a63db6 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:02.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.983+0000 7fcc59ffb640 1 -- 172.21.15.67:0/760215713 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc64040070 con 0x55790ec7d1b0 2024-07-27T03:18:02.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.984+0000 7fcc59ffb640 1 -- 172.21.15.67:0/760215713 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc64032e50 con 0x55790ec7d1b0 2024-07-27T03:18:02.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.984+0000 7fcc59ffb640 1 -- 172.21.15.67:0/760215713 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc64035040 con 0x55790ec7d1b0 2024-07-27T03:18:02.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.984+0000 7fcc80378740 1 -- 172.21.15.67:0/760215713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55790ec7d1b0 msgr2=0x55790f04b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:02.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.984+0000 7fcc80378740 1 --2- 172.21.15.67:0/760215713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55790ec7d1b0 0x55790f04b280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcc74105710 tx=0x7fcc64032c60 comp rx=0 tx=0).stop 2024-07-27T03:18:02.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.985+0000 7fcc80378740 1 -- 172.21.15.67:0/760215713 shutdown_connections 2024-07-27T03:18:02.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.985+0000 7fcc80378740 1 --2- 172.21.15.67:0/760215713 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790ec85390 0x55790f04b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:02.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.985+0000 7fcc80378740 1 --2- 172.21.15.67:0/760215713 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55790ec7d1b0 0x55790f04b280 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:02.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.985+0000 7fcc80378740 1 --2- 172.21.15.67:0/760215713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55790eca6070 0x55790f04ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:02.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.985+0000 7fcc80378740 1 -- 172.21.15.67:0/760215713 >> 172.21.15.67:0/760215713 conn(0x55790ec9a460 msgr2=0x55790f048c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:02.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.985+0000 7fcc80378740 1 -- 172.21.15.67:0/760215713 shutdown_connections 2024-07-27T03:18:02.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.985+0000 7fcc80378740 1 -- 172.21.15.67:0/760215713 wait complete. 2024-07-27T03:18:02.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.985+0000 7fcc80378740 1 -- start start 2024-07-27T03:18:02.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.985+0000 7fcc80378740 1 Processor -- start 2024-07-27T03:18:02.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.986+0000 7fcc80378740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55790ec7d1b0 0x55790f056930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:02.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.986+0000 7fcc80378740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790ec85390 0x55790f056eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:02.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.986+0000 7fcc80378740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55790eca6070 0x55790f062630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:02.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.986+0000 7fcc80378740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55790ef53900 con 0x55790ec85390 2024-07-27T03:18:02.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.986+0000 7fcc80378740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55790efb5060 con 0x55790ec7d1b0 2024-07-27T03:18:02.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.986+0000 7fcc80378740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55790ee14700 con 0x55790eca6070 2024-07-27T03:18:02.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.986+0000 7fcc80378740 10 client.0 _ll_register_callbacks cb 0x55790ef5bd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:02.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.986+0000 7fcc737fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790ec85390 0x55790f056eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:02.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.986+0000 7fcc737fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790ec85390 0x55790f056eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60122/0 (socket says 172.21.15.67:60122) 2024-07-27T03:18:02.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.986+0000 7fcc737fe640 1 -- 172.21.15.67:0/4158006643 learned_addr learned my addr 172.21.15.67:0/4158006643 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:02.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.986+0000 7fcc72ffd640 1 --2- 172.21.15.67:0/4158006643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55790ec7d1b0 0x55790f056930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:02.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.986+0000 7fcc72ffd640 1 --2- 172.21.15.67:0/4158006643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55790eca6070 0x55790f062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:02.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.986+0000 7fcc737fe640 1 -- 172.21.15.67:0/4158006643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55790eca6070 msgr2=0x55790f062630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:02.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.986+0000 7fcc737fe640 1 --2- 172.21.15.67:0/4158006643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55790eca6070 0x55790f062630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:02.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.986+0000 7fcc737fe640 1 -- 172.21.15.67:0/4158006643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55790ec7d1b0 msgr2=0x55790f056930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:02.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.986+0000 7fcc737fe640 1 --2- 172.21.15.67:0/4158006643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55790ec7d1b0 0x55790f056930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:02.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.986+0000 7fcc737fe640 1 -- 172.21.15.67:0/4158006643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55790ef57bf0 con 0x55790ec85390 2024-07-27T03:18:02.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.986+0000 7fcc737fe640 1 --2- 172.21.15.67:0/4158006643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790ec85390 0x55790f056eb0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fcc64005510 tx=0x7fcc64005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2b337f69da844c server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:02.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.986+0000 7fcc52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:02.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.987+0000 7fcc52ffd640 1 -- 172.21.15.67:0/4158006643 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6404b020 con 0x55790ec85390 2024-07-27T03:18:02.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.987+0000 7fcc52ffd640 1 -- 172.21.15.67:0/4158006643 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fcc64004040 con 0x55790ec85390 2024-07-27T03:18:02.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.987+0000 7fcc80378740 1 -- 172.21.15.67:0/4158006643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc64032e50 con 0x55790ec85390 2024-07-27T03:18:02.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.987+0000 7fcc52ffd640 1 -- 172.21.15.67:0/4158006643 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcc64056050 con 0x55790ec85390 2024-07-27T03:18:02.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.987+0000 7fcc80378740 10 client.15699 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:02.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.987+0000 7fcc80378740 1 -- 172.21.15.67:0/4158006643 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55790ed11650 con 0x55790ec85390 2024-07-27T03:18:02.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.987+0000 7fcc52ffd640 1 -- 172.21.15.67:0/4158006643 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc64032e50 con 0x55790ec85390 2024-07-27T03:18:02.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.987+0000 7fcc52ffd640 1 -- 172.21.15.67:0/4158006643 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcc64004c00 con 0x55790ec85390 2024-07-27T03:18:02.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.988+0000 7fcc80378740 10 client.15699 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:02.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.988+0000 7fcc80378740 10 client.15699 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-07-27T03:18:02.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.988+0000 7fcc80378740 20 client.15699 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:02.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.988+0000 7fcc80378740 1 --2- 172.21.15.67:0/4158006643 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55790ec649f0 0x55790f068530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:02.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.988+0000 7fcc80378740 4 client.15699 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:18:02.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.988+0000 7fcc80378740 1 -- 172.21.15.67:0/4158006643 --> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55790efc3a70 con 0x55790ec649f0 2024-07-27T03:18:02.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.988+0000 7fcc72ffd640 1 --2- 172.21.15.67:0/4158006643 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55790ec649f0 0x55790f068530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:02.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.988+0000 7fcc72ffd640 1 --2- 172.21.15.67:0/4158006643 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55790ec649f0 0x55790f068530 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5c21a193d768d5e2 server_cookie=5c17f570e1872aba in_seq=0 out_seq=0 2024-07-27T03:18:02.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.988+0000 7fcc52ffd640 10 client.15699 ms_handle_connect on v2:172.21.15.149:6834/3730964806 2024-07-27T03:18:02.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.989+0000 7fcc52ffd640 1 -- 172.21.15.67:0/4158006643 <== mds.4 v2:172.21.15.149:6834/3730964806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55790efc3a70 con 0x55790ec649f0 2024-07-27T03:18:02.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.989+0000 7fcc52ffd640 10 client.15699 handle_command_reply: tid=0 2024-07-27T03:18:02.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.989+0000 7fcc80378740 1 client.15699 shutdown 2024-07-27T03:18:02.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.990+0000 7fcc80378740 1 -- 172.21.15.67:0/4158006643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790ec85390 msgr2=0x55790f056eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:02.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.990+0000 7fcc80378740 1 --2- 172.21.15.67:0/4158006643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790ec85390 0x55790f056eb0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fcc64005510 tx=0x7fcc64005540 comp rx=0 tx=0).stop 2024-07-27T03:18:02.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.990+0000 7fcc80378740 1 -- 172.21.15.67:0/4158006643 shutdown_connections 2024-07-27T03:18:02.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.990+0000 7fcc80378740 1 --2- 172.21.15.67:0/4158006643 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55790ec649f0 0x55790f068530 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:02.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.990+0000 7fcc80378740 1 --2- 172.21.15.67:0/4158006643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55790eca6070 0x55790f062630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:02.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.990+0000 7fcc80378740 1 --2- 172.21.15.67:0/4158006643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790ec85390 0x55790f056eb0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:02.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.990+0000 7fcc80378740 1 --2- 172.21.15.67:0/4158006643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55790ec7d1b0 0x55790f056930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:02.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.990+0000 7fcc80378740 1 -- 172.21.15.67:0/4158006643 >> 172.21.15.67:0/4158006643 conn(0x55790ec9a460 msgr2=0x55790f04df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:02.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.990+0000 7fcc52ffd640 0 client.15699 ms_handle_reset on v2:172.21.15.149:6834/3730964806 2024-07-27T03:18:02.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.990+0000 7fcc80378740 1 -- 172.21.15.67:0/4158006643 shutdown_connections 2024-07-27T03:18:02.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.990+0000 7fcc80378740 1 -- 172.21.15.67:0/4158006643 wait complete. 2024-07-27T03:18:02.999 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:02.990+0000 7fcc80378740 20 client.15699 trim_cache size 0 max 16384 2024-07-27T03:18:03.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:02 smithi149 ceph-mon[37173]: pgmap v474: 129 pgs: 129 active+clean; 1.7 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 176 KiB/s wr, 13 op/s 2024-07-27T03:18:03.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.003+0000 7fcc80378740 1 Processor -- start 2024-07-27T03:18:03.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.003+0000 7fcc80378740 1 -- start start 2024-07-27T03:18:03.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.004+0000 7fcc80378740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55790ec7d1b0 0x55790f04e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:03.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.004+0000 7fcc80378740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55790ec85390 0x55790f0625a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:03.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.004+0000 7fcc80378740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790eca6070 0x55790f062ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:03.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.004+0000 7fcc80378740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcc741b27b0 con 0x55790eca6070 2024-07-27T03:18:03.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.004+0000 7fcc80378740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcc741b42f0 con 0x55790ec7d1b0 2024-07-27T03:18:03.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.004+0000 7fcc80378740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcc741b2e80 con 0x55790ec85390 2024-07-27T03:18:03.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.004+0000 7fcc72ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790eca6070 0x55790f062ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:03.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.004+0000 7fcc72ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790eca6070 0x55790f062ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60130/0 (socket says 172.21.15.67:60130) 2024-07-27T03:18:03.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.004+0000 7fcc72ffd640 1 -- 172.21.15.67:0/227288885 learned_addr learned my addr 172.21.15.67:0/227288885 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:03.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.004+0000 7fcc72ffd640 1 --2- 172.21.15.67:0/227288885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55790ec7d1b0 0x55790f04e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:03.007 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.004+0000 7fcc737fe640 1 --2- 172.21.15.67:0/227288885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55790ec85390 0x55790f0625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:03.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.004+0000 7fcc72ffd640 1 -- 172.21.15.67:0/227288885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55790ec85390 msgr2=0x55790f0625a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:03.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.004+0000 7fcc72ffd640 1 --2- 172.21.15.67:0/227288885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55790ec85390 0x55790f0625a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:03.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.004+0000 7fcc72ffd640 1 -- 172.21.15.67:0/227288885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55790ec7d1b0 msgr2=0x55790f04e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:03.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.004+0000 7fcc72ffd640 1 --2- 172.21.15.67:0/227288885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55790ec7d1b0 0x55790f04e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:03.008 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.004+0000 7fcc72ffd640 1 -- 172.21.15.67:0/227288885 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55790f055f50 con 0x55790eca6070 2024-07-27T03:18:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.005+0000 7fcc72ffd640 1 --2- 172.21.15.67:0/227288885 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790eca6070 0x55790f062ae0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800bfd0 tx=0x7fcc6800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f76afebf4ae4f server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.005+0000 7fcc597fa640 1 -- 172.21.15.67:0/227288885 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc680040f0 con 0x55790eca6070 2024-07-27T03:18:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.005+0000 7fcc597fa640 1 -- 172.21.15.67:0/227288885 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc68010da0 con 0x55790eca6070 2024-07-27T03:18:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.005+0000 7fcc597fa640 1 -- 172.21.15.67:0/227288885 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc680037b0 con 0x55790eca6070 2024-07-27T03:18:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.006+0000 7fcc80378740 1 -- 172.21.15.67:0/227288885 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790eca6070 msgr2=0x55790f062ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.006+0000 7fcc80378740 1 --2- 172.21.15.67:0/227288885 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790eca6070 0x55790f062ae0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800bfd0 tx=0x7fcc6800b390 comp rx=0 tx=0).stop 2024-07-27T03:18:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.006+0000 7fcc80378740 1 -- 172.21.15.67:0/227288885 shutdown_connections 2024-07-27T03:18:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.006+0000 7fcc80378740 1 --2- 172.21.15.67:0/227288885 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790eca6070 0x55790f062ae0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.006+0000 7fcc80378740 1 --2- 172.21.15.67:0/227288885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55790ec85390 0x55790f0625a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.006+0000 7fcc80378740 1 --2- 172.21.15.67:0/227288885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55790ec7d1b0 0x55790f04e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.006+0000 7fcc80378740 1 -- 172.21.15.67:0/227288885 >> 172.21.15.67:0/227288885 conn(0x55790ec9a460 msgr2=0x55790f04de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.006+0000 7fcc80378740 1 -- 172.21.15.67:0/227288885 shutdown_connections 2024-07-27T03:18:03.009 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.006+0000 7fcc80378740 1 -- 172.21.15.67:0/227288885 wait complete. 2024-07-27T03:18:03.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.007+0000 7fcc80378740 1 -- start start 2024-07-27T03:18:03.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.007+0000 7fcc80378740 1 Processor -- start 2024-07-27T03:18:03.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.007+0000 7fcc80378740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790ec7d1b0 0x55790f068b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:03.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.007+0000 7fcc80378740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55790ec85390 0x55790f0690c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:03.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.007+0000 7fcc80378740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55790eca6070 0x55790f0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:03.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.007+0000 7fcc80378740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fcc741b3c20 con 0x55790ec7d1b0 2024-07-27T03:18:03.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.007+0000 7fcc80378740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcc741b3550 con 0x55790ec85390 2024-07-27T03:18:03.011 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.007+0000 7fcc80378740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcc741b0790 con 0x55790eca6070 2024-07-27T03:18:03.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.008+0000 7fcc80378740 10 client.0 _ll_register_callbacks cb 0x7fcc6800baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:03.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.008+0000 7fcc72ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790ec7d1b0 0x55790f068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:03.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.008+0000 7fcc737fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55790ec85390 0x55790f0690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:03.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.008+0000 7fcc72ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790ec7d1b0 0x55790f068b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60138/0 (socket says 172.21.15.67:60138) 2024-07-27T03:18:03.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.008+0000 7fcc72ffd640 1 -- 172.21.15.67:0/581010807 learned_addr learned my addr 172.21.15.67:0/581010807 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:03.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.008+0000 7fcc737fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55790ec85390 0x55790f0690c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.67:43494/0 (socket says 172.21.15.67:43494) 2024-07-27T03:18:03.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.008+0000 7fcc737fe640 1 -- 172.21.15.67:0/581010807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55790eca6070 msgr2=0x55790f0cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:03.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.008+0000 7fcc72ffd640 1 --2- 172.21.15.67:0/581010807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55790eca6070 0x55790f0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:03.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.008+0000 7fcc737fe640 1 --2- 172.21.15.67:0/581010807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55790eca6070 0x55790f0cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:03.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.008+0000 7fcc737fe640 1 -- 172.21.15.67:0/581010807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790ec7d1b0 msgr2=0x55790f068b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:03.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.008+0000 7fcc737fe640 1 --2- 172.21.15.67:0/581010807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790ec7d1b0 0x55790f068b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:03.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.008+0000 7fcc737fe640 1 -- 172.21.15.67:0/581010807 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55790f0cefb0 con 0x55790ec85390 2024-07-27T03:18:03.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.008+0000 7fcc72ffd640 1 --2- 172.21.15.67:0/581010807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55790eca6070 0x55790f0cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:18:03.012 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.009+0000 7fcc737fe640 1 --2- 172.21.15.67:0/581010807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55790ec85390 0x55790f0690c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcc64005050 tx=0x7fcc6402ab10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bdef33d64eb88fd server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:03.013 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.009+0000 7fcc52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-07-27T03:18:03.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.009+0000 7fcc52ffd640 1 -- 172.21.15.67:0/581010807 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6405f060 con 0x55790ec85390 2024-07-27T03:18:03.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.009+0000 7fcc52ffd640 1 -- 172.21.15.67:0/581010807 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fcc640046c0 con 0x55790ec85390 2024-07-27T03:18:03.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.009+0000 7fcc80378740 1 -- 172.21.15.67:0/581010807 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc68010da0 con 0x55790ec85390 2024-07-27T03:18:03.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.010+0000 7fcc52ffd640 1 -- 172.21.15.67:0/581010807 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcc64066050 con 0x55790ec85390 2024-07-27T03:18:03.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.010+0000 7fcc80378740 10 client.25346 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:03.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.010+0000 7fcc80378740 1 -- 172.21.15.67:0/581010807 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55790f0d0230 con 0x55790ec85390 2024-07-27T03:18:03.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.010+0000 7fcc52ffd640 1 -- 172.21.15.67:0/581010807 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc68010da0 con 0x55790ec85390 2024-07-27T03:18:03.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.010+0000 7fcc52ffd640 1 -- 172.21.15.67:0/581010807 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcc64032040 con 0x55790ec85390 2024-07-27T03:18:03.015 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.010+0000 7fcc80378740 10 client.25346 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:03.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.010+0000 7fcc80378740 10 client.25346 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-07-27T03:18:03.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.010+0000 7fcc80378740 20 client.25346 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:03.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.010+0000 7fcc80378740 1 --2- 172.21.15.67:0/581010807 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55790ec39da0 0x55790f0d2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:03.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.010+0000 7fcc80378740 4 client.25346 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-27T03:18:03.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.010+0000 7fcc80378740 1 -- 172.21.15.67:0/581010807 --> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55790efbf510 con 0x55790ec39da0 2024-07-27T03:18:03.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.011+0000 7fcc72ffd640 1 --2- 172.21.15.67:0/581010807 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55790ec39da0 0x55790f0d2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:03.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.011+0000 7fcc52ffd640 10 client.25346 ms_handle_connect on v2:172.21.15.149:6834/3730964806 2024-07-27T03:18:03.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.011+0000 7fcc72ffd640 1 --2- 172.21.15.67:0/581010807 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55790ec39da0 0x55790f0d2960 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=833b2106b1003188 server_cookie=403712eebe9c8b5 in_seq=0 out_seq=0 2024-07-27T03:18:03.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.012+0000 7fcc52ffd640 1 -- 172.21.15.67:0/581010807 <== mds.4 v2:172.21.15.149:6834/3730964806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55790efbf510 con 0x55790ec39da0 2024-07-27T03:18:03.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.012+0000 7fcc52ffd640 10 client.25346 handle_command_reply: tid=0 2024-07-27T03:18:03.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.012+0000 7fcc80378740 1 client.25346 shutdown 2024-07-27T03:18:03.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.012+0000 7fcc80378740 1 -- 172.21.15.67:0/581010807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55790ec85390 msgr2=0x55790f0690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:03.016 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.012+0000 7fcc80378740 1 --2- 172.21.15.67:0/581010807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55790ec85390 0x55790f0690c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcc64005050 tx=0x7fcc6402ab10 comp rx=0 tx=0).stop 2024-07-27T03:18:03.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.012+0000 7fcc80378740 1 -- 172.21.15.67:0/581010807 shutdown_connections 2024-07-27T03:18:03.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.012+0000 7fcc80378740 1 --2- 172.21.15.67:0/581010807 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55790ec39da0 0x55790f0d2960 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:03.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.012+0000 7fcc80378740 1 --2- 172.21.15.67:0/581010807 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55790eca6070 0x55790f0cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:03.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.012+0000 7fcc80378740 1 --2- 172.21.15.67:0/581010807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55790ec85390 0x55790f0690c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:03.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.012+0000 7fcc80378740 1 --2- 172.21.15.67:0/581010807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55790ec7d1b0 0x55790f068b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:03.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.012+0000 7fcc80378740 1 -- 172.21.15.67:0/581010807 >> 172.21.15.67:0/581010807 conn(0x55790ec9a460 msgr2=0x55790f057be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:03.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.012+0000 7fcc52ffd640 0 client.25346 ms_handle_reset on v2:172.21.15.149:6834/3730964806 2024-07-27T03:18:03.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.013+0000 7fcc80378740 1 -- 172.21.15.67:0/581010807 shutdown_connections 2024-07-27T03:18:03.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.013+0000 7fcc80378740 1 -- 172.21.15.67:0/581010807 wait complete. 2024-07-27T03:18:03.017 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.013+0000 7fcc80378740 20 client.25346 trim_cache size 0 max 16384 2024-07-27T03:18:03.017 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-27T03:18:03.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.013+0000 7fcc79df3640 1 -- 172.21.15.67:0/3202665871 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcc5c0b3bd0 msgr2=0x7fcc5c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:03.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.013+0000 7fcc79df3640 1 --2- 172.21.15.67:0/3202665871 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcc5c0b3bd0 0x7fcc5c0b60e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c003fd0 tx=0x7fcc6c004480 comp rx=0 tx=0).stop 2024-07-27T03:18:03.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.013+0000 7fcc79df3640 1 -- 172.21.15.67:0/3202665871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc74108760 msgr2=0x7fcc74104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:03.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.013+0000 7fcc79df3640 1 --2- 172.21.15.67:0/3202665871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc74108760 0x7fcc74104740 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009fd0 tx=0x7fcc6c00b260 comp rx=0 tx=0).stop 2024-07-27T03:18:03.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.014+0000 7fcc79df3640 1 -- 172.21.15.67:0/3202665871 shutdown_connections 2024-07-27T03:18:03.018 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.014+0000 7fcc79df3640 1 --2- 172.21.15.67:0/3202665871 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fcc5c0b3bd0 0x7fcc5c0b60e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:03.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.014+0000 7fcc79df3640 1 --2- 172.21.15.67:0/3202665871 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fcc74108760 0x7fcc74104740 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:03.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.014+0000 7fcc79df3640 1 --2- 172.21.15.67:0/3202665871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc74107e10 0x7fcc74104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:03.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.014+0000 7fcc79df3640 1 --2- 172.21.15.67:0/3202665871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc741024d0 0x7fcc74103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:03.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.014+0000 7fcc79df3640 1 -- 172.21.15.67:0/3202665871 >> 172.21.15.67:0/3202665871 conn(0x7fcc740791a0 msgr2=0x7fcc74101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:03.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.014+0000 7fcc79df3640 1 -- 172.21.15.67:0/3202665871 shutdown_connections 2024-07-27T03:18:03.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:03.014+0000 7fcc79df3640 1 -- 172.21.15.67:0/3202665871 wait complete. 2024-07-27T03:18:03.786 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-27T03:18:04.069 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:18:04.802 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:04 smithi149 ceph-mon[37173]: pgmap v475: 129 pgs: 129 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 176 KiB/s wr, 10 op/s 2024-07-27T03:18:04.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:04 smithi175 ceph-mon[37125]: pgmap v475: 129 pgs: 129 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 176 KiB/s wr, 10 op/s 2024-07-27T03:18:04.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:04 smithi067 ceph-mon[32814]: pgmap v475: 129 pgs: 129 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 176 KiB/s wr, 10 op/s 2024-07-27T03:18:05.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.916+0000 7f7470223640 1 -- 172.21.15.67:0/3640406963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7468107e40 msgr2=0x7f7468108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:05.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.916+0000 7f7470223640 1 --2- 172.21.15.67:0/3640406963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7468107e40 0x7f7468108250 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7458009900 tx=0x7f745802a410 comp rx=0 tx=0).stop 2024-07-27T03:18:05.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.917+0000 7f7470223640 1 -- 172.21.15.67:0/3640406963 shutdown_connections 2024-07-27T03:18:05.919 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.917+0000 7f7470223640 1 --2- 172.21.15.67:0/3640406963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7468108790 0x7f746810efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.917+0000 7f7470223640 1 --2- 172.21.15.67:0/3640406963 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7468107e40 0x7f7468108250 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.917+0000 7f7470223640 1 --2- 172.21.15.67:0/3640406963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7468102550 0x7f7468102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.917+0000 7f7470223640 1 -- 172.21.15.67:0/3640406963 >> 172.21.15.67:0/3640406963 conn(0x7f7468079220 msgr2=0x7f7468079630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:05.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.917+0000 7f7470223640 1 -- 172.21.15.67:0/3640406963 shutdown_connections 2024-07-27T03:18:05.920 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.917+0000 7f7470223640 1 -- 172.21.15.67:0/3640406963 wait complete. 2024-07-27T03:18:05.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.917+0000 7f7470223640 1 Processor -- start 2024-07-27T03:18:05.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.917+0000 7f7470223640 1 -- start start 2024-07-27T03:18:05.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.918+0000 7f7470223640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7468102550 0x7f74680ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:05.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.918+0000 7f7470223640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7468107e40 0x7f74680ff620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:05.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.918+0000 7f7470223640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7468108790 0x7f74680ffb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:05.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.918+0000 7f7470223640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7468112420 con 0x7f7468108790 2024-07-27T03:18:05.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.918+0000 7f7470223640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f74681122a0 con 0x7f7468102550 2024-07-27T03:18:05.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.918+0000 7f7470223640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74681125a0 con 0x7f7468107e40 2024-07-27T03:18:05.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.918+0000 7f746e7c0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7468108790 0x7f74680ffb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:05.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.918+0000 7f746e7c0640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7468108790 0x7f74680ffb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57426/0 (socket says 172.21.15.67:57426) 2024-07-27T03:18:05.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.918+0000 7f746e7c0640 1 -- 172.21.15.67:0/8942706 learned_addr learned my addr 172.21.15.67:0/8942706 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:05.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.918+0000 7f746dfbf640 1 --2- 172.21.15.67:0/8942706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7468102550 0x7f74680ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:05.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.918+0000 7f746d7be640 1 --2- 172.21.15.67:0/8942706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7468107e40 0x7f74680ff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:05.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.918+0000 7f746e7c0640 1 -- 172.21.15.67:0/8942706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7468107e40 msgr2=0x7f74680ff620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:05.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.918+0000 7f746e7c0640 1 --2- 172.21.15.67:0/8942706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7468107e40 0x7f74680ff620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.918+0000 7f746e7c0640 1 -- 172.21.15.67:0/8942706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7468102550 msgr2=0x7f74680ff0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:05.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.918+0000 7f746e7c0640 1 --2- 172.21.15.67:0/8942706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7468102550 0x7f74680ff0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.918+0000 7f746e7c0640 1 -- 172.21.15.67:0/8942706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74681a8200 con 0x7f7468108790 2024-07-27T03:18:05.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.919+0000 7f746e7c0640 1 --2- 172.21.15.67:0/8942706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7468108790 0x7f74680ffb60 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f746000eb10 tx=0x7f746000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:05.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.919+0000 7f745effd640 1 -- 172.21.15.67:0/8942706 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460004b20 con 0x7f7468108790 2024-07-27T03:18:05.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.919+0000 7f7470223640 1 -- 172.21.15.67:0/8942706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74681a84f0 con 0x7f7468108790 2024-07-27T03:18:05.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.919+0000 7f7470223640 1 -- 172.21.15.67:0/8942706 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74681a8a50 con 0x7f7468108790 2024-07-27T03:18:05.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.919+0000 7f745effd640 1 -- 172.21.15.67:0/8942706 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7460004cc0 con 0x7f7468108790 2024-07-27T03:18:05.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.919+0000 7f745effd640 1 -- 172.21.15.67:0/8942706 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460010690 con 0x7f7468108790 2024-07-27T03:18:05.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.922+0000 7f745effd640 1 -- 172.21.15.67:0/8942706 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f74600108f0 con 0x7f7468108790 2024-07-27T03:18:05.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.922+0000 7f74767a8740 1 Processor -- start 2024-07-27T03:18:05.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.922+0000 7f74767a8740 1 -- start start 2024-07-27T03:18:05.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.922+0000 7f74767a8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfe2070 0x55829c386d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:05.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.922+0000 7f74767a8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfb91b0 0x55829c387280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:05.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.922+0000 7f74767a8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55829bfc1390 0x55829c3877c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:05.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.922+0000 7f74767a8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55829c26f400 con 0x55829bfb91b0 2024-07-27T03:18:05.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.922+0000 7f74767a8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55829c2fdb20 con 0x55829bfc1390 2024-07-27T03:18:05.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.922+0000 7f74767a8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55829c2ed020 con 0x55829bfe2070 2024-07-27T03:18:05.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.923+0000 7f746d7be640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfe2070 0x55829c386d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:05.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.923+0000 7f746d7be640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfe2070 0x55829c386d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.67:49682/0 (socket says 172.21.15.67:49682) 2024-07-27T03:18:05.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.923+0000 7f746d7be640 1 -- 172.21.15.67:0/884021009 learned_addr learned my addr 172.21.15.67:0/884021009 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:05.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.923+0000 7f746dfbf640 1 --2- 172.21.15.67:0/884021009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfb91b0 0x55829c387280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:05.926 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.923+0000 7f746d7be640 1 --2- 172.21.15.67:0/884021009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55829bfc1390 0x55829c3877c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:05.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.923+0000 7f746d7be640 1 -- 172.21.15.67:0/884021009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55829bfc1390 msgr2=0x55829c3877c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:05.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.923+0000 7f746d7be640 1 --2- 172.21.15.67:0/884021009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55829bfc1390 0x55829c3877c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.923+0000 7f745effd640 1 --2- 172.21.15.67:0/8942706 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f74540b3c50 0x7f74540b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:05.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.923+0000 7f746d7be640 1 -- 172.21.15.67:0/884021009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfb91b0 msgr2=0x55829c387280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:05.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.923+0000 7f746d7be640 1 --2- 172.21.15.67:0/884021009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfb91b0 0x55829c387280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.923+0000 7f746d7be640 1 -- 172.21.15.67:0/884021009 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55829c282770 con 0x55829bfe2070 2024-07-27T03:18:05.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.924+0000 7f746d7be640 1 --2- 172.21.15.67:0/884021009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfe2070 0x55829c386d40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f74580057c0 tx=0x7f7458000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61033d4bb34f0614 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:05.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.924+0000 7f745effd640 1 -- 172.21.15.67:0/8942706 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f74600e4490 con 0x7f7468108790 2024-07-27T03:18:05.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.924+0000 7f746e7c0640 1 --2- 172.21.15.67:0/8942706 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f74540b3c50 0x7f74540b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:05.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.924+0000 7f7453fff640 1 -- 172.21.15.67:0/884021009 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458035070 con 0x55829bfe2070 2024-07-27T03:18:05.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.924+0000 7f7453fff640 1 -- 172.21.15.67:0/884021009 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7458004030 con 0x55829bfe2070 2024-07-27T03:18:05.927 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.924+0000 7f7453fff640 1 -- 172.21.15.67:0/884021009 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74580053b0 con 0x55829bfe2070 2024-07-27T03:18:05.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.925+0000 7f74767a8740 1 -- 172.21.15.67:0/884021009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfe2070 msgr2=0x55829c386d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:05.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.925+0000 7f746e7c0640 1 --2- 172.21.15.67:0/8942706 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f74540b3c50 0x7f74540b6160 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f74600aad30 tx=0x7f746000cf70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:05.928 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.925+0000 7f74767a8740 1 --2- 172.21.15.67:0/884021009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfe2070 0x55829c386d40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f74580057c0 tx=0x7f7458000bc0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.925+0000 7f74767a8740 1 -- 172.21.15.67:0/884021009 shutdown_connections 2024-07-27T03:18:05.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.925+0000 7f74767a8740 1 --2- 172.21.15.67:0/884021009 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55829bfc1390 0x55829c3877c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.925+0000 7f74767a8740 1 --2- 172.21.15.67:0/884021009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfb91b0 0x55829c387280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.925+0000 7f74767a8740 1 --2- 172.21.15.67:0/884021009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfe2070 0x55829c386d40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.925+0000 7f74767a8740 1 -- 172.21.15.67:0/884021009 >> 172.21.15.67:0/884021009 conn(0x55829bfd6460 msgr2=0x55829c384c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:05.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.925+0000 7f74767a8740 1 -- 172.21.15.67:0/884021009 shutdown_connections 2024-07-27T03:18:05.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.925+0000 7f74767a8740 1 -- 172.21.15.67:0/884021009 wait complete. 2024-07-27T03:18:05.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.926+0000 7f74767a8740 1 -- start start 2024-07-27T03:18:05.929 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.926+0000 7f74767a8740 1 Processor -- start 2024-07-27T03:18:05.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.926+0000 7f74767a8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfb91b0 0x55829c392960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:05.931 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.926+0000 7f74767a8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfc1390 0x55829c392ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.926+0000 7f74767a8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55829bfe2070 0x55829c39e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.926+0000 7f74767a8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55829c28f900 con 0x55829bfb91b0 2024-07-27T03:18:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.926+0000 7f74767a8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55829c2f1060 con 0x55829bfe2070 2024-07-27T03:18:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.926+0000 7f74767a8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55829c150700 con 0x55829bfc1390 2024-07-27T03:18:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.927+0000 7f74767a8740 10 client.0 _ll_register_callbacks cb 0x55829c297d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.927+0000 7f746d7be640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfb91b0 0x55829c392960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.927+0000 7f746dfbf640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfc1390 0x55829c392ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.927+0000 7f746dfbf640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfc1390 0x55829c392ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.67:49684/0 (socket says 172.21.15.67:49684) 2024-07-27T03:18:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.927+0000 7f746d7be640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfb91b0 0x55829c392960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57446/0 (socket says 172.21.15.67:57446) 2024-07-27T03:18:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.927+0000 7f746dfbf640 1 -- 172.21.15.67:0/1667671693 learned_addr learned my addr 172.21.15.67:0/1667671693 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.927+0000 7f746d7be640 1 --2- 172.21.15.67:0/1667671693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55829bfe2070 0x55829c39e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.927+0000 7f746dfbf640 1 -- 172.21.15.67:0/1667671693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55829bfe2070 msgr2=0x55829c39e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:05.932 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.927+0000 7f746dfbf640 1 --2- 172.21.15.67:0/1667671693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55829bfe2070 0x55829c39e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.927+0000 7f746dfbf640 1 -- 172.21.15.67:0/1667671693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfb91b0 msgr2=0x55829c392960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:05.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.927+0000 7f746dfbf640 1 --2- 172.21.15.67:0/1667671693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfb91b0 0x55829c392960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.927+0000 7f746dfbf640 1 -- 172.21.15.67:0/1667671693 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55829c293bf0 con 0x55829bfc1390 2024-07-27T03:18:05.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.927+0000 7f746dfbf640 1 --2- 172.21.15.67:0/1667671693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfc1390 0x55829c392ea0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7464012720 tx=0x7f7464012be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8582439e406f10c server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:05.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.927+0000 7f74517fa640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-27T03:18:05.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.928+0000 7f74517fa640 1 -- 172.21.15.67:0/1667671693 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464004030 con 0x55829bfc1390 2024-07-27T03:18:05.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.928+0000 7f74517fa640 1 -- 172.21.15.67:0/1667671693 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f74640041d0 con 0x55829bfc1390 2024-07-27T03:18:05.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.928+0000 7f74767a8740 1 -- 172.21.15.67:0/1667671693 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7458004030 con 0x55829bfc1390 2024-07-27T03:18:05.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.928+0000 7f74517fa640 1 -- 172.21.15.67:0/1667671693 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7464004410 con 0x55829bfc1390 2024-07-27T03:18:05.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.928+0000 7f74767a8740 10 client.25327 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:05.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.928+0000 7f74767a8740 1 -- 172.21.15.67:0/1667671693 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55829c04d650 con 0x55829bfc1390 2024-07-27T03:18:05.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.928+0000 7f74517fa640 1 -- 172.21.15.67:0/1667671693 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7458004030 con 0x55829bfc1390 2024-07-27T03:18:05.933 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.928+0000 7f74517fa640 1 -- 172.21.15.67:0/1667671693 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7464003a90 con 0x55829bfc1390 2024-07-27T03:18:05.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.928+0000 7f74767a8740 10 client.25327 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:05.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.929+0000 7f74767a8740 10 client.25327 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-27T03:18:05.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.929+0000 7f74767a8740 20 client.25327 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:05.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.929+0000 7f74767a8740 1 --2- 172.21.15.67:0/1667671693 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55829bfa09f0 0x55829c3a4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:05.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.929+0000 7f74767a8740 4 client.25327 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:18:05.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.929+0000 7f74767a8740 1 -- 172.21.15.67:0/1667671693 --> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55829c2ffa70 con 0x55829bfa09f0 2024-07-27T03:18:05.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.929+0000 7f746d7be640 1 --2- 172.21.15.67:0/1667671693 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55829bfa09f0 0x55829c3a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:05.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.929+0000 7f74517fa640 10 client.25327 ms_handle_connect on v2:172.21.15.175:6836/1304803502 2024-07-27T03:18:05.934 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.929+0000 7f746d7be640 1 --2- 172.21.15.67:0/1667671693 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55829bfa09f0 0x55829c3a4810 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eb0055ec1a145d68 server_cookie=6d4dbd59605f40bc in_seq=0 out_seq=0 2024-07-27T03:18:05.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.930+0000 7f74517fa640 1 -- 172.21.15.67:0/1667671693 <== mds.3 v2:172.21.15.175:6836/1304803502 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55829c2ffa70 con 0x55829bfa09f0 2024-07-27T03:18:05.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.930+0000 7f74517fa640 10 client.25327 handle_command_reply: tid=0 2024-07-27T03:18:05.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.930+0000 7f74767a8740 1 client.25327 shutdown 2024-07-27T03:18:05.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.930+0000 7f74767a8740 1 -- 172.21.15.67:0/1667671693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfc1390 msgr2=0x55829c392ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:05.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.930+0000 7f74767a8740 1 --2- 172.21.15.67:0/1667671693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfc1390 0x55829c392ea0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7464012720 tx=0x7f7464012be0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.930+0000 7f74767a8740 1 -- 172.21.15.67:0/1667671693 shutdown_connections 2024-07-27T03:18:05.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.931+0000 7f74767a8740 1 --2- 172.21.15.67:0/1667671693 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55829bfa09f0 0x55829c3a4810 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.931+0000 7f74767a8740 1 --2- 172.21.15.67:0/1667671693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55829bfe2070 0x55829c39e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.931+0000 7f74767a8740 1 --2- 172.21.15.67:0/1667671693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfc1390 0x55829c392ea0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.931+0000 7f74767a8740 1 --2- 172.21.15.67:0/1667671693 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfb91b0 0x55829c392960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.931+0000 7f74517fa640 0 client.25327 ms_handle_reset on v2:172.21.15.175:6836/1304803502 2024-07-27T03:18:05.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.931+0000 7f74767a8740 1 -- 172.21.15.67:0/1667671693 >> 172.21.15.67:0/1667671693 conn(0x55829bfd6460 msgr2=0x55829c389ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:05.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.931+0000 7f74767a8740 1 -- 172.21.15.67:0/1667671693 shutdown_connections 2024-07-27T03:18:05.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.931+0000 7f74767a8740 1 -- 172.21.15.67:0/1667671693 wait complete. 2024-07-27T03:18:05.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.931+0000 7f74767a8740 20 client.25327 trim_cache size 0 max 16384 2024-07-27T03:18:05.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.944+0000 7f74767a8740 1 Processor -- start 2024-07-27T03:18:05.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.944+0000 7f74767a8740 1 -- start start 2024-07-27T03:18:05.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.945+0000 7f74767a8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfb91b0 0x55829c38c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:05.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.945+0000 7f74767a8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55829bfc1390 0x55829c39e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:05.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.945+0000 7f74767a8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfe2070 0x55829c39e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:05.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.945+0000 7f74767a8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f74681b28b0 con 0x55829bfe2070 2024-07-27T03:18:05.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.945+0000 7f74767a8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f74681b43f0 con 0x55829bfc1390 2024-07-27T03:18:05.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.945+0000 7f74767a8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74681b2f80 con 0x55829bfb91b0 2024-07-27T03:18:05.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.945+0000 7f746d7be640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfe2070 0x55829c39e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:05.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.945+0000 7f746d7be640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfe2070 0x55829c39e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57450/0 (socket says 172.21.15.67:57450) 2024-07-27T03:18:05.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.945+0000 7f746d7be640 1 -- 172.21.15.67:0/3517547232 learned_addr learned my addr 172.21.15.67:0/3517547232 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:05.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.945+0000 7f746dfbf640 1 --2- 172.21.15.67:0/3517547232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55829bfc1390 0x55829c39e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:05.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.945+0000 7f746d7be640 1 --2- 172.21.15.67:0/3517547232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfb91b0 0x55829c38c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:05.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.945+0000 7f746d7be640 1 -- 172.21.15.67:0/3517547232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfb91b0 msgr2=0x55829c38c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:05.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.945+0000 7f746d7be640 1 --2- 172.21.15.67:0/3517547232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfb91b0 0x55829c38c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.945+0000 7f746d7be640 1 -- 172.21.15.67:0/3517547232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55829bfc1390 msgr2=0x55829c39e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:05.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.945+0000 7f746d7be640 1 --2- 172.21.15.67:0/3517547232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55829bfc1390 0x55829c39e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.945+0000 7f746d7be640 1 -- 172.21.15.67:0/3517547232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55829c390c60 con 0x55829bfe2070 2024-07-27T03:18:05.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.946+0000 7f746d7be640 1 --2- 172.21.15.67:0/3517547232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfe2070 0x55829c39e9c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7458031600 tx=0x7f745802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65bd07d923aa0054 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:05.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.946+0000 7f74537fe640 1 -- 172.21.15.67:0/3517547232 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458035040 con 0x55829bfe2070 2024-07-27T03:18:05.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.946+0000 7f74537fe640 1 -- 172.21.15.67:0/3517547232 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74580054d0 con 0x55829bfe2070 2024-07-27T03:18:05.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.946+0000 7f74537fe640 1 -- 172.21.15.67:0/3517547232 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458004cd0 con 0x55829bfe2070 2024-07-27T03:18:05.951 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.947+0000 7f74767a8740 1 -- 172.21.15.67:0/3517547232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfe2070 msgr2=0x55829c39e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:05.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.947+0000 7f74767a8740 1 --2- 172.21.15.67:0/3517547232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfe2070 0x55829c39e9c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7458031600 tx=0x7f745802a410 comp rx=0 tx=0).stop 2024-07-27T03:18:05.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.947+0000 7f74767a8740 1 -- 172.21.15.67:0/3517547232 shutdown_connections 2024-07-27T03:18:05.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.947+0000 7f74767a8740 1 --2- 172.21.15.67:0/3517547232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfe2070 0x55829c39e9c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.947+0000 7f74767a8740 1 --2- 172.21.15.67:0/3517547232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55829bfc1390 0x55829c39e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.947+0000 7f74767a8740 1 --2- 172.21.15.67:0/3517547232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfb91b0 0x55829c38c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.947+0000 7f74767a8740 1 -- 172.21.15.67:0/3517547232 >> 172.21.15.67:0/3517547232 conn(0x55829bfd6460 msgr2=0x55829c38be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:05.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.947+0000 7f74767a8740 1 -- 172.21.15.67:0/3517547232 shutdown_connections 2024-07-27T03:18:05.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.947+0000 7f74767a8740 1 -- 172.21.15.67:0/3517547232 wait complete. 2024-07-27T03:18:05.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.947+0000 7f74767a8740 1 -- start start 2024-07-27T03:18:05.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.947+0000 7f74767a8740 1 Processor -- start 2024-07-27T03:18:05.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.948+0000 7f74767a8740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfb91b0 0x55829c39ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:05.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.948+0000 7f74767a8740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55829bfc1390 0x55829c396140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:05.952 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.948+0000 7f74767a8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfe2070 0x55829c396680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:05.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.948+0000 7f74767a8740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f74681b3d20 con 0x55829bfb91b0 2024-07-27T03:18:05.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.948+0000 7f74767a8740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f74681b3650 con 0x55829bfc1390 2024-07-27T03:18:05.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.948+0000 7f74767a8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74681b0890 con 0x55829bfe2070 2024-07-27T03:18:05.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.948+0000 7f74767a8740 10 client.0 _ll_register_callbacks cb 0x7f7464009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:05.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.948+0000 7f746d7be640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfb91b0 0x55829c39ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:05.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.948+0000 7f746d7be640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfb91b0 0x55829c39ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57462/0 (socket says 172.21.15.67:57462) 2024-07-27T03:18:05.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.948+0000 7f746d7be640 1 -- 172.21.15.67:0/2802018042 learned_addr learned my addr 172.21.15.67:0/2802018042 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:05.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.948+0000 7f746dfbf640 1 --2- 172.21.15.67:0/2802018042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55829bfc1390 0x55829c396140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:05.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.948+0000 7f746d7be640 1 --2- 172.21.15.67:0/2802018042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfe2070 0x55829c396680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:05.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.949+0000 7f746d7be640 1 -- 172.21.15.67:0/2802018042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfe2070 msgr2=0x55829c396680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:05.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.949+0000 7f746d7be640 1 --2- 172.21.15.67:0/2802018042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfe2070 0x55829c396680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.949+0000 7f746d7be640 1 -- 172.21.15.67:0/2802018042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55829bfc1390 msgr2=0x55829c396140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:05.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.949+0000 7f746d7be640 1 --2- 172.21.15.67:0/2802018042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55829bfc1390 0x55829c396140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.949+0000 7f746d7be640 1 -- 172.21.15.67:0/2802018042 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55829c396df0 con 0x55829bfb91b0 2024-07-27T03:18:05.953 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.949+0000 7f746d7be640 1 --2- 172.21.15.67:0/2802018042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfb91b0 0x55829c39ac90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7458031440 tx=0x7f74580048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bcadaf196aee72e server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:05.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.949+0000 7f74517fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:05.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.949+0000 7f74517fa640 1 -- 172.21.15.67:0/2802018042 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745805e030 con 0x55829bfb91b0 2024-07-27T03:18:05.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.949+0000 7f74517fa640 1 -- 172.21.15.67:0/2802018042 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f7458031ad0 con 0x55829bfb91b0 2024-07-27T03:18:05.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.949+0000 7f74767a8740 1 -- 172.21.15.67:0/2802018042 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74580054d0 con 0x55829bfb91b0 2024-07-27T03:18:05.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.949+0000 7f74517fa640 1 -- 172.21.15.67:0/2802018042 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7458035040 con 0x55829bfb91b0 2024-07-27T03:18:05.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.949+0000 7f74767a8740 10 client.15723 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:05.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.949+0000 7f74767a8740 1 -- 172.21.15.67:0/2802018042 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55829c40ce20 con 0x55829bfb91b0 2024-07-27T03:18:05.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.950+0000 7f74517fa640 1 -- 172.21.15.67:0/2802018042 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74580054d0 con 0x55829bfb91b0 2024-07-27T03:18:05.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.950+0000 7f74517fa640 1 -- 172.21.15.67:0/2802018042 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f745804a040 con 0x55829bfb91b0 2024-07-27T03:18:05.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.950+0000 7f74767a8740 10 client.15723 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:05.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.950+0000 7f74767a8740 10 client.15723 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-27T03:18:05.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.950+0000 7f74767a8740 20 client.15723 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:05.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.950+0000 7f74767a8740 1 --2- 172.21.15.67:0/2802018042 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55829bf75da0 0x55829c40f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:05.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.950+0000 7f74767a8740 4 client.15723 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-27T03:18:05.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.950+0000 7f74767a8740 1 -- 172.21.15.67:0/2802018042 --> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55829c2fb510 con 0x55829bf75da0 2024-07-27T03:18:05.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.950+0000 7f746dfbf640 1 --2- 172.21.15.67:0/2802018042 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55829bf75da0 0x55829c40f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:05.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.951+0000 7f74517fa640 10 client.15723 ms_handle_connect on v2:172.21.15.175:6836/1304803502 2024-07-27T03:18:05.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.951+0000 7f746dfbf640 1 --2- 172.21.15.67:0/2802018042 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55829bf75da0 0x55829c40f050 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dd8a438eca8d2812 server_cookie=917a79bbb9b48377 in_seq=0 out_seq=0 2024-07-27T03:18:05.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.951+0000 7f74517fa640 1 -- 172.21.15.67:0/2802018042 <== mds.3 v2:172.21.15.175:6836/1304803502 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55829c2fb510 con 0x55829bf75da0 2024-07-27T03:18:05.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.951+0000 7f74517fa640 10 client.15723 handle_command_reply: tid=0 2024-07-27T03:18:05.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.951+0000 7f74767a8740 1 client.15723 shutdown 2024-07-27T03:18:05.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.951+0000 7f74767a8740 1 -- 172.21.15.67:0/2802018042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfb91b0 msgr2=0x55829c39ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:05.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.951+0000 7f74767a8740 1 --2- 172.21.15.67:0/2802018042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfb91b0 0x55829c39ac90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7458031440 tx=0x7f74580048d0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.951+0000 7f74767a8740 1 -- 172.21.15.67:0/2802018042 shutdown_connections 2024-07-27T03:18:05.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.951+0000 7f74767a8740 1 --2- 172.21.15.67:0/2802018042 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55829bf75da0 0x55829c40f050 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.951+0000 7f74767a8740 1 --2- 172.21.15.67:0/2802018042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55829bfe2070 0x55829c396680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.951+0000 7f74767a8740 1 --2- 172.21.15.67:0/2802018042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55829bfc1390 0x55829c396140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.951+0000 7f74517fa640 0 client.15723 ms_handle_reset on v2:172.21.15.175:6836/1304803502 2024-07-27T03:18:05.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.951+0000 7f74767a8740 1 --2- 172.21.15.67:0/2802018042 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55829bfb91b0 0x55829c39ac90 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.951+0000 7f74767a8740 1 -- 172.21.15.67:0/2802018042 >> 172.21.15.67:0/2802018042 conn(0x55829bfd6460 msgr2=0x55829c387d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:05.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.951+0000 7f74767a8740 1 -- 172.21.15.67:0/2802018042 shutdown_connections 2024-07-27T03:18:05.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.952+0000 7f74767a8740 1 -- 172.21.15.67:0/2802018042 wait complete. 2024-07-27T03:18:05.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.952+0000 7f74767a8740 20 client.15723 trim_cache size 0 max 16384 2024-07-27T03:18:05.958 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-27T03:18:05.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.952+0000 7f7470223640 1 -- 172.21.15.67:0/8942706 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f74540b3c50 msgr2=0x7f74540b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:05.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.952+0000 7f7470223640 1 --2- 172.21.15.67:0/8942706 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f74540b3c50 0x7f74540b6160 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f74600aad30 tx=0x7f746000cf70 comp rx=0 tx=0).stop 2024-07-27T03:18:05.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.952+0000 7f7470223640 1 -- 172.21.15.67:0/8942706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7468108790 msgr2=0x7f74680ffb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:05.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.952+0000 7f7470223640 1 --2- 172.21.15.67:0/8942706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7468108790 0x7f74680ffb60 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f746000eb10 tx=0x7f746000c2d0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.952+0000 7f7470223640 1 -- 172.21.15.67:0/8942706 shutdown_connections 2024-07-27T03:18:05.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.952+0000 7f7470223640 1 --2- 172.21.15.67:0/8942706 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f74540b3c50 0x7f74540b6160 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.952+0000 7f7470223640 1 --2- 172.21.15.67:0/8942706 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7468108790 0x7f74680ffb60 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.952+0000 7f7470223640 1 --2- 172.21.15.67:0/8942706 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7468107e40 0x7f74680ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.952+0000 7f7470223640 1 --2- 172.21.15.67:0/8942706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7468102550 0x7f74680ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:05.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.952+0000 7f7470223640 1 -- 172.21.15.67:0/8942706 >> 172.21.15.67:0/8942706 conn(0x7f7468079220 msgr2=0x7f7468101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:05.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.952+0000 7f7470223640 1 -- 172.21.15.67:0/8942706 shutdown_connections 2024-07-27T03:18:05.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:05.952+0000 7f7470223640 1 -- 172.21.15.67:0/8942706 wait complete. 2024-07-27T03:18:06.699 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-27T03:18:06.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:06 smithi175 ceph-mon[37125]: pgmap v476: 129 pgs: 129 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 606 KiB/s wr, 14 op/s 2024-07-27T03:18:06.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:06 smithi067 ceph-mon[32814]: pgmap v476: 129 pgs: 129 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 606 KiB/s wr, 14 op/s 2024-07-27T03:18:06.977 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:18:07.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:06 smithi149 ceph-mon[37173]: pgmap v476: 129 pgs: 129 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 606 KiB/s wr, 14 op/s 2024-07-27T03:18:08.760 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.757+0000 7f3238781640 1 -- 172.21.15.67:0/3105263025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323010d880 msgr2=0x7f323010fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.757+0000 7f3238781640 1 --2- 172.21.15.67:0/3105263025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323010d880 0x7f323010fd60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3228009ca0 tx=0x7f322802a8c0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.757+0000 7f3238781640 1 -- 172.21.15.67:0/3105263025 shutdown_connections 2024-07-27T03:18:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.757+0000 7f3238781640 1 --2- 172.21.15.67:0/3105263025 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323010d880 0x7f323010fd60 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.757+0000 7f3238781640 1 --2- 172.21.15.67:0/3105263025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3230104d00 0x7f323010d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.757+0000 7f3238781640 1 --2- 172.21.15.67:0/3105263025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32301042d0 0x7f32301046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.757+0000 7f3238781640 1 -- 172.21.15.67:0/3105263025 >> 172.21.15.67:0/3105263025 conn(0x7f32300791a0 msgr2=0x7f32300795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.757+0000 7f3238781640 1 -- 172.21.15.67:0/3105263025 shutdown_connections 2024-07-27T03:18:08.761 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.757+0000 7f3238781640 1 -- 172.21.15.67:0/3105263025 wait complete. 2024-07-27T03:18:08.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:08 smithi067 ceph-mon[32814]: pgmap v477: 129 pgs: 129 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 606 KiB/s wr, 12 op/s 2024-07-27T03:18:08.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.758+0000 7f3238781640 1 Processor -- start 2024-07-27T03:18:08.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.758+0000 7f3238781640 1 -- start start 2024-07-27T03:18:08.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.759+0000 7f3238781640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32301042d0 0x7f3230199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:08.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.759+0000 7f3238781640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3230104d00 0x7f323019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:08.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.759+0000 7f3238781640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323010d880 0x7f323019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:08.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.759+0000 7f3238781640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f32301134d0 con 0x7f323010d880 2024-07-27T03:18:08.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.759+0000 7f3238781640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3230113350 con 0x7f3230104d00 2024-07-27T03:18:08.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.759+0000 7f3238781640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3230113650 con 0x7f32301042d0 2024-07-27T03:18:08.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.759+0000 7f3236d1e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323010d880 0x7f323019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:08.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.759+0000 7f3236d1e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323010d880 0x7f323019f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57478/0 (socket says 172.21.15.67:57478) 2024-07-27T03:18:08.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.759+0000 7f3236d1e640 1 -- 172.21.15.67:0/3884077324 learned_addr learned my addr 172.21.15.67:0/3884077324 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:08.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.759+0000 7f3235d1c640 1 --2- 172.21.15.67:0/3884077324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3230104d00 0x7f323019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:08.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.759+0000 7f323651d640 1 --2- 172.21.15.67:0/3884077324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32301042d0 0x7f3230199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:08.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.759+0000 7f3236d1e640 1 -- 172.21.15.67:0/3884077324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32301042d0 msgr2=0x7f3230199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:08.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.760+0000 7f3236d1e640 1 --2- 172.21.15.67:0/3884077324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32301042d0 0x7f3230199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.760+0000 7f3236d1e640 1 -- 172.21.15.67:0/3884077324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3230104d00 msgr2=0x7f323019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:08.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.760+0000 7f3236d1e640 1 --2- 172.21.15.67:0/3884077324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3230104d00 0x7f323019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.760+0000 7f3236d1e640 1 -- 172.21.15.67:0/3884077324 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323019fdf0 con 0x7f323010d880 2024-07-27T03:18:08.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.760+0000 7f3235d1c640 1 --2- 172.21.15.67:0/3884077324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3230104d00 0x7f323019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:18:08.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.760+0000 7f323651d640 1 --2- 172.21.15.67:0/3884077324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32301042d0 0x7f3230199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:18:08.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.760+0000 7f3236d1e640 1 --2- 172.21.15.67:0/3884077324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323010d880 0x7f323019f6c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3228005cc0 tx=0x7f322802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.760+0000 7f32277fe640 1 -- 172.21.15.67:0/3884077324 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3228003bf0 con 0x7f323010d880 2024-07-27T03:18:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.760+0000 7f3238781640 1 -- 172.21.15.67:0/3884077324 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32301a0080 con 0x7f323010d880 2024-07-27T03:18:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.760+0000 7f3238781640 1 -- 172.21.15.67:0/3884077324 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32301a6840 con 0x7f323010d880 2024-07-27T03:18:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.760+0000 7f32277fe640 1 -- 172.21.15.67:0/3884077324 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3228003d90 con 0x7f323010d880 2024-07-27T03:18:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.760+0000 7f32277fe640 1 -- 172.21.15.67:0/3884077324 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322802fbc0 con 0x7f323010d880 2024-07-27T03:18:08.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.763+0000 7f323ed06740 1 Processor -- start 2024-07-27T03:18:08.766 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.764+0000 7f32277fe640 1 -- 172.21.15.67:0/3884077324 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f322802fd60 con 0x7f323010d880 2024-07-27T03:18:08.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.764+0000 7f323ed06740 1 -- start start 2024-07-27T03:18:08.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.764+0000 7f323ed06740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4b7292070 0x55b4b7636d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:08.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.764+0000 7f323ed06740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b72691b0 0x55b4b7637280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:08.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.764+0000 7f323ed06740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b7271390 0x55b4b76377c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:08.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.764+0000 7f323ed06740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b4b751f400 con 0x55b4b7271390 2024-07-27T03:18:08.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.764+0000 7f323ed06740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b4b75adb20 con 0x55b4b72691b0 2024-07-27T03:18:08.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.764+0000 7f323ed06740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b4b759d020 con 0x55b4b7292070 2024-07-27T03:18:08.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.765+0000 7f3235d1c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b7271390 0x55b4b76377c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:08.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.765+0000 7f323651d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b72691b0 0x55b4b7637280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:08.767 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.765+0000 7f323651d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b72691b0 0x55b4b7637280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.67:53580/0 (socket says 172.21.15.67:53580) 2024-07-27T03:18:08.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.765+0000 7f3235d1c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b7271390 0x55b4b76377c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57490/0 (socket says 172.21.15.67:57490) 2024-07-27T03:18:08.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.765+0000 7f323651d640 1 -- 172.21.15.67:0/316267722 learned_addr learned my addr 172.21.15.67:0/316267722 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:08.768 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.765+0000 7f3235d1c640 1 --2- 172.21.15.67:0/316267722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4b7292070 0x55b4b7636d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:08.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.765+0000 7f3235d1c640 1 -- 172.21.15.67:0/316267722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4b7292070 msgr2=0x55b4b7636d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:08.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.765+0000 7f3235d1c640 1 --2- 172.21.15.67:0/316267722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4b7292070 0x55b4b7636d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.769 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.765+0000 7f3235d1c640 1 -- 172.21.15.67:0/316267722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b72691b0 msgr2=0x55b4b7637280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:08.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.765+0000 7f3235d1c640 1 --2- 172.21.15.67:0/316267722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b72691b0 0x55b4b7637280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.765+0000 7f3235d1c640 1 -- 172.21.15.67:0/316267722 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4b7532770 con 0x55b4b7271390 2024-07-27T03:18:08.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.765+0000 7f32277fe640 1 --2- 172.21.15.67:0/3884077324 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f321c0b3bd0 0x7f321c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:08.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.765+0000 7f3235d1c640 1 --2- 172.21.15.67:0/316267722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b7271390 0x55b4b76377c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3220005fd0 tx=0x7f322000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9326d61268c7bb55 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:08.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.765+0000 7f323651d640 1 --2- 172.21.15.67:0/3884077324 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f321c0b3bd0 0x7f321c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:08.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.765+0000 7f3224ff9640 1 -- 172.21.15.67:0/316267722 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32200142d0 con 0x55b4b7271390 2024-07-27T03:18:08.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.766+0000 7f32277fe640 1 -- 172.21.15.67:0/3884077324 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f32281056d0 con 0x7f323010d880 2024-07-27T03:18:08.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.766+0000 7f3224ff9640 1 -- 172.21.15.67:0/316267722 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3220005020 con 0x55b4b7271390 2024-07-27T03:18:08.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.766+0000 7f3224ff9640 1 -- 172.21.15.67:0/316267722 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220004a00 con 0x55b4b7271390 2024-07-27T03:18:08.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.767+0000 7f323ed06740 1 -- 172.21.15.67:0/316267722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b7271390 msgr2=0x55b4b76377c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:08.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.767+0000 7f323651d640 1 --2- 172.21.15.67:0/3884077324 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f321c0b3bd0 0x7f321c0b60e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55b4b76359c0 tx=0x7f322c00b660 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:08.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.767+0000 7f323ed06740 1 --2- 172.21.15.67:0/316267722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b7271390 0x55b4b76377c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3220005fd0 tx=0x7f322000b4a0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.767+0000 7f323ed06740 1 -- 172.21.15.67:0/316267722 shutdown_connections 2024-07-27T03:18:08.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.767+0000 7f323ed06740 1 --2- 172.21.15.67:0/316267722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b7271390 0x55b4b76377c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.767+0000 7f323ed06740 1 --2- 172.21.15.67:0/316267722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b72691b0 0x55b4b7637280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.767+0000 7f323ed06740 1 --2- 172.21.15.67:0/316267722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4b7292070 0x55b4b7636d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.767+0000 7f323ed06740 1 -- 172.21.15.67:0/316267722 >> 172.21.15.67:0/316267722 conn(0x55b4b7286460 msgr2=0x55b4b7634c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:08.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.767+0000 7f323ed06740 1 -- 172.21.15.67:0/316267722 shutdown_connections 2024-07-27T03:18:08.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.767+0000 7f323ed06740 1 -- 172.21.15.67:0/316267722 wait complete. 2024-07-27T03:18:08.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.767+0000 7f323ed06740 1 -- start start 2024-07-27T03:18:08.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.767+0000 7f323ed06740 1 Processor -- start 2024-07-27T03:18:08.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.768+0000 7f323ed06740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4b72691b0 0x55b4b7646c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:08.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.768+0000 7f323ed06740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b7271390 0x55b4b76471d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:08.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.768+0000 7f323ed06740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b7292070 0x55b4b7652a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:08.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.768+0000 7f323ed06740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55b4b753f900 con 0x55b4b7292070 2024-07-27T03:18:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.768+0000 7f323ed06740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55b4b75a1060 con 0x55b4b7271390 2024-07-27T03:18:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.768+0000 7f323ed06740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55b4b7400700 con 0x55b4b72691b0 2024-07-27T03:18:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.768+0000 7f3236d1e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b7292070 0x55b4b7652a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.768+0000 7f323ed06740 10 client.0 _ll_register_callbacks cb 0x55b4b7547d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.768+0000 7f3236d1e640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b7292070 0x55b4b7652a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57494/0 (socket says 172.21.15.67:57494) 2024-07-27T03:18:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.768+0000 7f3236d1e640 1 -- 172.21.15.67:0/3109700774 learned_addr learned my addr 172.21.15.67:0/3109700774 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.768+0000 7f3235d1c640 1 --2- 172.21.15.67:0/3109700774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b7271390 0x55b4b76471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.768+0000 7f323651d640 1 --2- 172.21.15.67:0/3109700774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4b72691b0 0x55b4b7646c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.768+0000 7f3236d1e640 1 -- 172.21.15.67:0/3109700774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4b72691b0 msgr2=0x55b4b7646c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.768+0000 7f3236d1e640 1 --2- 172.21.15.67:0/3109700774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4b72691b0 0x55b4b7646c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.768+0000 7f3236d1e640 1 -- 172.21.15.67:0/3109700774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b7271390 msgr2=0x55b4b76471d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.768+0000 7f3236d1e640 1 --2- 172.21.15.67:0/3109700774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b7271390 0x55b4b76471d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.768+0000 7f3236d1e640 1 -- 172.21.15.67:0/3109700774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4b7543bf0 con 0x55b4b7292070 2024-07-27T03:18:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.769+0000 7f3236d1e640 1 --2- 172.21.15.67:0/3109700774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b7292070 0x55b4b7652a60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f32280049d0 tx=0x7f32280cbe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edde82ca1f42f285 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.769+0000 7f3219ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.769+0000 7f3219ffb640 1 -- 172.21.15.67:0/3109700774 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32280cc270 con 0x55b4b7292070 2024-07-27T03:18:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.769+0000 7f3219ffb640 1 -- 172.21.15.67:0/3109700774 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f32280c8860 con 0x55b4b7292070 2024-07-27T03:18:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.769+0000 7f323ed06740 1 -- 172.21.15.67:0/3109700774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3220005020 con 0x55b4b7292070 2024-07-27T03:18:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.769+0000 7f3219ffb640 1 -- 172.21.15.67:0/3109700774 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f32280d0870 con 0x55b4b7292070 2024-07-27T03:18:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.769+0000 7f323ed06740 10 client.15735 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.769+0000 7f323ed06740 1 -- 172.21.15.67:0/3109700774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b4b72fd650 con 0x55b4b7292070 2024-07-27T03:18:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.769+0000 7f3219ffb640 1 -- 172.21.15.67:0/3109700774 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3220005020 con 0x55b4b7292070 2024-07-27T03:18:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.769+0000 7f3219ffb640 1 -- 172.21.15.67:0/3109700774 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f32280cb4d0 con 0x55b4b7292070 2024-07-27T03:18:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.769+0000 7f323ed06740 10 client.15735 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:08.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.769+0000 7f323ed06740 10 client.15735 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-27T03:18:08.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.770+0000 7f323ed06740 20 client.15735 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:08.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.770+0000 7f323ed06740 1 --2- 172.21.15.67:0/3109700774 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55b4b72509f0 0x55b4b7658aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:08.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.770+0000 7f323ed06740 4 client.15735 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:18:08.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.770+0000 7f323ed06740 1 -- 172.21.15.67:0/3109700774 --> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4b75afa70 con 0x55b4b72509f0 2024-07-27T03:18:08.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.770+0000 7f323651d640 1 --2- 172.21.15.67:0/3109700774 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55b4b72509f0 0x55b4b7658aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:08.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.770+0000 7f323651d640 1 --2- 172.21.15.67:0/3109700774 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55b4b72509f0 0x55b4b7658aa0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5366faa94e0a8da5 server_cookie=382abcdd9b00ad70 in_seq=0 out_seq=0 2024-07-27T03:18:08.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.770+0000 7f3219ffb640 10 client.15735 ms_handle_connect on v2:172.21.15.175:6838/1670568853 2024-07-27T03:18:08.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.771+0000 7f3219ffb640 1 -- 172.21.15.67:0/3109700774 <== mds.2 v2:172.21.15.175:6838/1670568853 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b4b75afa70 con 0x55b4b72509f0 2024-07-27T03:18:08.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.771+0000 7f3219ffb640 10 client.15735 handle_command_reply: tid=0 2024-07-27T03:18:08.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.771+0000 7f323ed06740 1 client.15735 shutdown 2024-07-27T03:18:08.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.771+0000 7f323ed06740 1 -- 172.21.15.67:0/3109700774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b7292070 msgr2=0x55b4b7652a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:08.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.771+0000 7f323ed06740 1 --2- 172.21.15.67:0/3109700774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b7292070 0x55b4b7652a60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f32280049d0 tx=0x7f32280cbe40 comp rx=0 tx=0).stop 2024-07-27T03:18:08.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.771+0000 7f323ed06740 1 -- 172.21.15.67:0/3109700774 shutdown_connections 2024-07-27T03:18:08.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.771+0000 7f323ed06740 1 --2- 172.21.15.67:0/3109700774 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55b4b72509f0 0x55b4b7658aa0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.771+0000 7f3219ffb640 0 client.15735 ms_handle_reset on v2:172.21.15.175:6838/1670568853 2024-07-27T03:18:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.771+0000 7f323ed06740 1 --2- 172.21.15.67:0/3109700774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b7292070 0x55b4b7652a60 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.771+0000 7f323ed06740 1 --2- 172.21.15.67:0/3109700774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b7271390 0x55b4b76471d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.771+0000 7f323ed06740 1 --2- 172.21.15.67:0/3109700774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4b72691b0 0x55b4b7646c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.771+0000 7f323ed06740 1 -- 172.21.15.67:0/3109700774 >> 172.21.15.67:0/3109700774 conn(0x55b4b7286460 msgr2=0x55b4b76331d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.772+0000 7f323ed06740 1 -- 172.21.15.67:0/3109700774 shutdown_connections 2024-07-27T03:18:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.772+0000 7f323ed06740 1 -- 172.21.15.67:0/3109700774 wait complete. 2024-07-27T03:18:08.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.772+0000 7f323ed06740 20 client.15735 trim_cache size 0 max 16384 2024-07-27T03:18:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.785+0000 7f323ed06740 1 Processor -- start 2024-07-27T03:18:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.785+0000 7f323ed06740 1 -- start start 2024-07-27T03:18:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.785+0000 7f323ed06740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b72509f0 0x55b4b7652850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.785+0000 7f323ed06740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b72691b0 0x55b4b7652d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:08.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.785+0000 7f323ed06740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4b7271390 0x55b4b76331d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.786+0000 7f323ed06740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f321400fb60 con 0x55b4b72691b0 2024-07-27T03:18:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.786+0000 7f323ed06740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f32140116a0 con 0x55b4b72509f0 2024-07-27T03:18:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.786+0000 7f323ed06740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3214010230 con 0x55b4b7271390 2024-07-27T03:18:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.786+0000 7f3235d1c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b72691b0 0x55b4b7652d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.786+0000 7f3235d1c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b72691b0 0x55b4b7652d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57500/0 (socket says 172.21.15.67:57500) 2024-07-27T03:18:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.786+0000 7f3235d1c640 1 -- 172.21.15.67:0/1626082272 learned_addr learned my addr 172.21.15.67:0/1626082272 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.786+0000 7f323651d640 1 --2- 172.21.15.67:0/1626082272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b72509f0 0x55b4b7652850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.786+0000 7f3236d1e640 1 --2- 172.21.15.67:0/1626082272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4b7271390 0x55b4b76331d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.786+0000 7f3235d1c640 1 -- 172.21.15.67:0/1626082272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4b7271390 msgr2=0x55b4b76331d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.786+0000 7f3235d1c640 1 --2- 172.21.15.67:0/1626082272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4b7271390 0x55b4b76331d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.786+0000 7f3235d1c640 1 -- 172.21.15.67:0/1626082272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b72509f0 msgr2=0x55b4b7652850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:08.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.786+0000 7f3235d1c640 1 --2- 172.21.15.67:0/1626082272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b72509f0 0x55b4b7652850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.786+0000 7f3235d1c640 1 -- 172.21.15.67:0/1626082272 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4b76491d0 con 0x55b4b72691b0 2024-07-27T03:18:08.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.786+0000 7f3235d1c640 1 --2- 172.21.15.67:0/1626082272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b72691b0 0x55b4b7652d90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f32200146d0 tx=0x7f322000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f051e385618e2 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:08.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.787+0000 7f321bfff640 1 -- 172.21.15.67:0/1626082272 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220005050 con 0x55b4b72691b0 2024-07-27T03:18:08.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.787+0000 7f321bfff640 1 -- 172.21.15.67:0/1626082272 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f322000f730 con 0x55b4b72691b0 2024-07-27T03:18:08.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.787+0000 7f321bfff640 1 -- 172.21.15.67:0/1626082272 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220003850 con 0x55b4b72691b0 2024-07-27T03:18:08.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.787+0000 7f323ed06740 1 -- 172.21.15.67:0/1626082272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b72691b0 msgr2=0x55b4b7652d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:08.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.788+0000 7f323ed06740 1 --2- 172.21.15.67:0/1626082272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b72691b0 0x55b4b7652d90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f32200146d0 tx=0x7f322000bd30 comp rx=0 tx=0).stop 2024-07-27T03:18:08.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.788+0000 7f323ed06740 1 -- 172.21.15.67:0/1626082272 shutdown_connections 2024-07-27T03:18:08.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.788+0000 7f323ed06740 1 --2- 172.21.15.67:0/1626082272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4b7271390 0x55b4b76331d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.788+0000 7f323ed06740 1 --2- 172.21.15.67:0/1626082272 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b72691b0 0x55b4b7652d90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.788+0000 7f323ed06740 1 --2- 172.21.15.67:0/1626082272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b72509f0 0x55b4b7652850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.788+0000 7f323ed06740 1 -- 172.21.15.67:0/1626082272 >> 172.21.15.67:0/1626082272 conn(0x55b4b7286460 msgr2=0x55b4b7645310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:08.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.788+0000 7f323ed06740 1 -- 172.21.15.67:0/1626082272 shutdown_connections 2024-07-27T03:18:08.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.788+0000 7f323ed06740 1 -- 172.21.15.67:0/1626082272 wait complete. 2024-07-27T03:18:08.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.788+0000 7f323ed06740 1 -- start start 2024-07-27T03:18:08.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.788+0000 7f323ed06740 1 Processor -- start 2024-07-27T03:18:08.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.789+0000 7f323ed06740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b72509f0 0x55b4b764ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:08.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.789+0000 7f323ed06740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b72691b0 0x55b4b764a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:08.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.789+0000 7f323ed06740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4b7271390 0x55b4b764a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:08.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.789+0000 7f323ed06740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3214010fd0 con 0x55b4b72691b0 2024-07-27T03:18:08.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.789+0000 7f323ed06740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3214010900 con 0x55b4b72509f0 2024-07-27T03:18:08.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.789+0000 7f323ed06740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f321400db40 con 0x55b4b7271390 2024-07-27T03:18:08.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.789+0000 7f323ed06740 10 client.0 _ll_register_callbacks cb 0x7f32280095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:08.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.789+0000 7f3235d1c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b72691b0 0x55b4b764a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:08.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.789+0000 7f3235d1c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b72691b0 0x55b4b764a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57502/0 (socket says 172.21.15.67:57502) 2024-07-27T03:18:08.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.789+0000 7f3235d1c640 1 -- 172.21.15.67:0/2424450771 learned_addr learned my addr 172.21.15.67:0/2424450771 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:08.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.789+0000 7f323651d640 1 --2- 172.21.15.67:0/2424450771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b72509f0 0x55b4b764ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:08.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.789+0000 7f3236d1e640 1 --2- 172.21.15.67:0/2424450771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4b7271390 0x55b4b764a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:08.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.789+0000 7f3235d1c640 1 -- 172.21.15.67:0/2424450771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4b7271390 msgr2=0x55b4b764a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:08.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.789+0000 7f3235d1c640 1 --2- 172.21.15.67:0/2424450771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4b7271390 0x55b4b764a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.789+0000 7f3235d1c640 1 -- 172.21.15.67:0/2424450771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b72509f0 msgr2=0x55b4b764ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:08.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.789+0000 7f3235d1c640 1 --2- 172.21.15.67:0/2424450771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b72509f0 0x55b4b764ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.795 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.789+0000 7f3235d1c640 1 -- 172.21.15.67:0/2424450771 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4b764ac80 con 0x55b4b72691b0 2024-07-27T03:18:08.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.790+0000 7f3235d1c640 1 --2- 172.21.15.67:0/2424450771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b72691b0 0x55b4b764a200 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3220005fd0 tx=0x7f322000b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2dae424760808d3 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:08.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.790+0000 7f3219ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:08.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.790+0000 7f3219ffb640 1 -- 172.21.15.67:0/2424450771 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3220030030 con 0x55b4b72691b0 2024-07-27T03:18:08.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.790+0000 7f3219ffb640 1 -- 172.21.15.67:0/2424450771 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f322000f8e0 con 0x55b4b72691b0 2024-07-27T03:18:08.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.790+0000 7f323ed06740 1 -- 172.21.15.67:0/2424450771 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f322000f730 con 0x55b4b72691b0 2024-07-27T03:18:08.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.790+0000 7f3219ffb640 1 -- 172.21.15.67:0/2424450771 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f322003d050 con 0x55b4b72691b0 2024-07-27T03:18:08.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.790+0000 7f323ed06740 10 client.15741 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:08.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.790+0000 7f323ed06740 1 -- 172.21.15.67:0/2424450771 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b4b76c0e30 con 0x55b4b72691b0 2024-07-27T03:18:08.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.790+0000 7f3219ffb640 1 -- 172.21.15.67:0/2424450771 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f322000f730 con 0x55b4b72691b0 2024-07-27T03:18:08.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.790+0000 7f3219ffb640 1 -- 172.21.15.67:0/2424450771 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3220027070 con 0x55b4b72691b0 2024-07-27T03:18:08.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.791+0000 7f323ed06740 10 client.15741 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:08.796 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.791+0000 7f323ed06740 10 client.15741 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-27T03:18:08.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.791+0000 7f323ed06740 20 client.15741 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:08.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.791+0000 7f323ed06740 1 --2- 172.21.15.67:0/2424450771 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55b4b7292070 0x55b4b76c33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:08.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.791+0000 7f323ed06740 4 client.15741 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-27T03:18:08.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.791+0000 7f323ed06740 1 -- 172.21.15.67:0/2424450771 --> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b4b75ab510 con 0x55b4b7292070 2024-07-27T03:18:08.797 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.791+0000 7f323651d640 1 --2- 172.21.15.67:0/2424450771 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55b4b7292070 0x55b4b76c33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:08.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.791+0000 7f323651d640 1 --2- 172.21.15.67:0/2424450771 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55b4b7292070 0x55b4b76c33c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c26b3c1312c06e68 server_cookie=b09dbd2ec56a7ef1 in_seq=0 out_seq=0 2024-07-27T03:18:08.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.791+0000 7f3219ffb640 10 client.15741 ms_handle_connect on v2:172.21.15.175:6838/1670568853 2024-07-27T03:18:08.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.792+0000 7f3219ffb640 1 -- 172.21.15.67:0/2424450771 <== mds.2 v2:172.21.15.175:6838/1670568853 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b4b75ab510 con 0x55b4b7292070 2024-07-27T03:18:08.798 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.792+0000 7f3219ffb640 10 client.15741 handle_command_reply: tid=0 2024-07-27T03:18:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.792+0000 7f323ed06740 1 client.15741 shutdown 2024-07-27T03:18:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.792+0000 7f323ed06740 1 -- 172.21.15.67:0/2424450771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b72691b0 msgr2=0x55b4b764a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.792+0000 7f323ed06740 1 --2- 172.21.15.67:0/2424450771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b72691b0 0x55b4b764a200 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3220005fd0 tx=0x7f322000b2e0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.792+0000 7f323ed06740 1 -- 172.21.15.67:0/2424450771 shutdown_connections 2024-07-27T03:18:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.792+0000 7f323ed06740 1 --2- 172.21.15.67:0/2424450771 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55b4b7292070 0x55b4b76c33c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.792+0000 7f323ed06740 1 --2- 172.21.15.67:0/2424450771 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55b4b7271390 0x55b4b764a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.792+0000 7f323ed06740 1 --2- 172.21.15.67:0/2424450771 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55b4b72691b0 0x55b4b764a200 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.792+0000 7f323ed06740 1 --2- 172.21.15.67:0/2424450771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b4b72509f0 0x55b4b764ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.792+0000 7f323ed06740 1 -- 172.21.15.67:0/2424450771 >> 172.21.15.67:0/2424450771 conn(0x55b4b7286460 msgr2=0x55b4b76331d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.792+0000 7f3219ffb640 0 client.15741 ms_handle_reset on v2:172.21.15.175:6838/1670568853 2024-07-27T03:18:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.792+0000 7f323ed06740 1 -- 172.21.15.67:0/2424450771 shutdown_connections 2024-07-27T03:18:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.792+0000 7f323ed06740 1 -- 172.21.15.67:0/2424450771 wait complete. 2024-07-27T03:18:08.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.792+0000 7f323ed06740 20 client.15741 trim_cache size 0 max 16384 2024-07-27T03:18:08.800 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-27T03:18:08.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.793+0000 7f3238781640 1 -- 172.21.15.67:0/3884077324 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f321c0b3bd0 msgr2=0x7f321c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:08.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.793+0000 7f3238781640 1 --2- 172.21.15.67:0/3884077324 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f321c0b3bd0 0x7f321c0b60e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55b4b76359c0 tx=0x7f322c00b660 comp rx=0 tx=0).stop 2024-07-27T03:18:08.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.793+0000 7f3238781640 1 -- 172.21.15.67:0/3884077324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323010d880 msgr2=0x7f323019f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:08.800 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.793+0000 7f3238781640 1 --2- 172.21.15.67:0/3884077324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323010d880 0x7f323019f6c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3228005cc0 tx=0x7f322802f380 comp rx=0 tx=0).stop 2024-07-27T03:18:08.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.793+0000 7f3238781640 1 -- 172.21.15.67:0/3884077324 shutdown_connections 2024-07-27T03:18:08.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.793+0000 7f3238781640 1 --2- 172.21.15.67:0/3884077324 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f321c0b3bd0 0x7f321c0b60e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.793+0000 7f3238781640 1 --2- 172.21.15.67:0/3884077324 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f323010d880 0x7f323019f6c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.793+0000 7f3238781640 1 --2- 172.21.15.67:0/3884077324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3230104d00 0x7f323019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.793+0000 7f3238781640 1 --2- 172.21.15.67:0/3884077324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32301042d0 0x7f3230199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:08.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.793+0000 7f3238781640 1 -- 172.21.15.67:0/3884077324 >> 172.21.15.67:0/3884077324 conn(0x7f32300791a0 msgr2=0x7f32301000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:08.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.793+0000 7f3238781640 1 -- 172.21.15.67:0/3884077324 shutdown_connections 2024-07-27T03:18:08.801 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:08.793+0000 7f3238781640 1 -- 172.21.15.67:0/3884077324 wait complete. 2024-07-27T03:18:08.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:08 smithi175 ceph-mon[37125]: pgmap v477: 129 pgs: 129 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 606 KiB/s wr, 12 op/s 2024-07-27T03:18:09.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:08 smithi149 ceph-mon[37173]: pgmap v477: 129 pgs: 129 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 606 KiB/s wr, 12 op/s 2024-07-27T03:18:09.477 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-27T03:18:09.763 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:18:10.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:10 smithi175 ceph-mon[37125]: pgmap v478: 129 pgs: 129 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.0 MiB/s wr, 12 op/s 2024-07-27T03:18:10.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:10 smithi067 ceph-mon[32814]: pgmap v478: 129 pgs: 129 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.0 MiB/s wr, 12 op/s 2024-07-27T03:18:11.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:10 smithi149 ceph-mon[37173]: pgmap v478: 129 pgs: 129 active+clean; 1.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.0 MiB/s wr, 12 op/s 2024-07-27T03:18:11.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.450+0000 7f971db90640 1 -- 172.21.15.67:0/3522996438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9718107e40 msgr2=0x7f9718108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:11.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.450+0000 7f971db90640 1 --2- 172.21.15.67:0/3522996438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9718107e40 0x7f9718108250 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9700009900 tx=0x7f970002a410 comp rx=0 tx=0).stop 2024-07-27T03:18:11.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.450+0000 7f971db90640 1 -- 172.21.15.67:0/3522996438 shutdown_connections 2024-07-27T03:18:11.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.450+0000 7f971db90640 1 --2- 172.21.15.67:0/3522996438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9718108790 0x7f971810efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.450+0000 7f971db90640 1 --2- 172.21.15.67:0/3522996438 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9718107e40 0x7f9718108250 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.450+0000 7f971db90640 1 --2- 172.21.15.67:0/3522996438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9718102550 0x7f9718102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.450+0000 7f971db90640 1 -- 172.21.15.67:0/3522996438 >> 172.21.15.67:0/3522996438 conn(0x7f9718079220 msgr2=0x7f9718079630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:11.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.450+0000 7f971db90640 1 -- 172.21.15.67:0/3522996438 shutdown_connections 2024-07-27T03:18:11.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.450+0000 7f971db90640 1 -- 172.21.15.67:0/3522996438 wait complete. 2024-07-27T03:18:11.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.451+0000 7f971db90640 1 Processor -- start 2024-07-27T03:18:11.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.451+0000 7f971db90640 1 -- start start 2024-07-27T03:18:11.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.452+0000 7f971db90640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9718102550 0x7f971819ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:11.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.452+0000 7f971db90640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9718107e40 0x7f971819b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:11.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.452+0000 7f971db90640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9718108790 0x7f971819b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:11.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.452+0000 7f971db90640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f9718112420 con 0x7f9718102550 2024-07-27T03:18:11.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.452+0000 7f971db90640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f97181122a0 con 0x7f9718108790 2024-07-27T03:18:11.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.452+0000 7f971db90640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f97181125a0 con 0x7f9718107e40 2024-07-27T03:18:11.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.452+0000 7f970f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9718102550 0x7f971819ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:11.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.452+0000 7f970f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9718102550 0x7f971819ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57532/0 (socket says 172.21.15.67:57532) 2024-07-27T03:18:11.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.452+0000 7f970f7fe640 1 -- 172.21.15.67:0/2824554879 learned_addr learned my addr 172.21.15.67:0/2824554879 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:11.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.452+0000 7f97177fe640 1 --2- 172.21.15.67:0/2824554879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9718107e40 0x7f971819b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:11.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.452+0000 7f9717fff640 1 --2- 172.21.15.67:0/2824554879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9718108790 0x7f971819b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:11.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.452+0000 7f970f7fe640 1 -- 172.21.15.67:0/2824554879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9718107e40 msgr2=0x7f971819b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:11.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.452+0000 7f970f7fe640 1 --2- 172.21.15.67:0/2824554879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9718107e40 0x7f971819b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.452+0000 7f970f7fe640 1 -- 172.21.15.67:0/2824554879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9718108790 msgr2=0x7f971819b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:11.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.452+0000 7f970f7fe640 1 --2- 172.21.15.67:0/2824554879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9718108790 0x7f971819b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.452+0000 7f970f7fe640 1 -- 172.21.15.67:0/2824554879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9718198b60 con 0x7f9718102550 2024-07-27T03:18:11.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.452+0000 7f9717fff640 1 --2- 172.21.15.67:0/2824554879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9718108790 0x7f971819b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:18:11.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.453+0000 7f970f7fe640 1 --2- 172.21.15.67:0/2824554879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9718102550 0x7f971819ad70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f970800d770 tx=0x7f970800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:11.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.453+0000 7f97157fa640 1 -- 172.21.15.67:0/2824554879 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708017070 con 0x7f9718102550 2024-07-27T03:18:11.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.453+0000 7f97157fa640 1 -- 172.21.15.67:0/2824554879 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97080054d0 con 0x7f9718102550 2024-07-27T03:18:11.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.453+0000 7f97157fa640 1 -- 172.21.15.67:0/2824554879 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708004c30 con 0x7f9718102550 2024-07-27T03:18:11.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.453+0000 7f971db90640 1 -- 172.21.15.67:0/2824554879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9718198e50 con 0x7f9718102550 2024-07-27T03:18:11.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.453+0000 7f971db90640 1 -- 172.21.15.67:0/2824554879 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9718199360 con 0x7f9718102550 2024-07-27T03:18:11.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.456+0000 7f9724115740 1 Processor -- start 2024-07-27T03:18:11.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.456+0000 7f9724115740 1 -- start start 2024-07-27T03:18:11.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.456+0000 7f9724115740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c37371e070 0x55c373ac2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:11.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.456+0000 7f9724115740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3736f51b0 0x55c373ac3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:11.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.456+0000 7f9724115740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c3736fd390 0x55c373ac37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:11.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.456+0000 7f9724115740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c3739ab400 con 0x55c37371e070 2024-07-27T03:18:11.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.456+0000 7f9724115740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c373a39b20 con 0x55c3736f51b0 2024-07-27T03:18:11.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.456+0000 7f9724115740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c373a29020 con 0x55c3736fd390 2024-07-27T03:18:11.462 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.457+0000 7f9717fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c37371e070 0x55c373ac2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:11.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.457+0000 7f9717fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c37371e070 0x55c373ac2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57538/0 (socket says 172.21.15.67:57538) 2024-07-27T03:18:11.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.457+0000 7f9717fff640 1 -- 172.21.15.67:0/378592075 learned_addr learned my addr 172.21.15.67:0/378592075 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:11.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.457+0000 7f97177fe640 1 --2- 172.21.15.67:0/378592075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c3736fd390 0x55c373ac37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:11.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.457+0000 7f9717fff640 1 -- 172.21.15.67:0/378592075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c3736fd390 msgr2=0x55c373ac37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:11.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.457+0000 7f9717fff640 1 --2- 172.21.15.67:0/378592075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c3736fd390 0x55c373ac37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.457+0000 7f9717fff640 1 -- 172.21.15.67:0/378592075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3736f51b0 msgr2=0x55c373ac3280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-27T03:18:11.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.457+0000 7f9717fff640 1 --2- 172.21.15.67:0/378592075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3736f51b0 0x55c373ac3280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.457+0000 7f9717fff640 1 -- 172.21.15.67:0/378592075 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3739be770 con 0x55c37371e070 2024-07-27T03:18:11.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.457+0000 7f97157fa640 1 -- 172.21.15.67:0/2824554879 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f9708004e10 con 0x7f9718102550 2024-07-27T03:18:11.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.457+0000 7f9717fff640 1 --2- 172.21.15.67:0/378592075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c37371e070 0x55c373ac2d40 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f971000f790 tx=0x7f971000fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2029cc4aafc91ac server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:11.463 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.457+0000 7f970dffb640 1 -- 172.21.15.67:0/378592075 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710004d20 con 0x55c37371e070 2024-07-27T03:18:11.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.457+0000 7f970dffb640 1 -- 172.21.15.67:0/378592075 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f971000ddd0 con 0x55c37371e070 2024-07-27T03:18:11.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.457+0000 7f970dffb640 1 -- 172.21.15.67:0/378592075 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710005970 con 0x55c37371e070 2024-07-27T03:18:11.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.458+0000 7f97157fa640 1 --2- 172.21.15.67:0/2824554879 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f97040b3c50 0x7f97040b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:11.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.458+0000 7f970f7fe640 1 --2- 172.21.15.67:0/2824554879 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f97040b3c50 0x7f97040b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:11.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.458+0000 7f97157fa640 1 -- 172.21.15.67:0/2824554879 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f97080e3950 con 0x7f9718102550 2024-07-27T03:18:11.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.458+0000 7f9724115740 1 -- 172.21.15.67:0/378592075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c37371e070 msgr2=0x55c373ac2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:11.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.458+0000 7f9724115740 1 --2- 172.21.15.67:0/378592075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c37371e070 0x55c373ac2d40 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f971000f790 tx=0x7f971000fc50 comp rx=0 tx=0).stop 2024-07-27T03:18:11.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.459+0000 7f9724115740 1 -- 172.21.15.67:0/378592075 shutdown_connections 2024-07-27T03:18:11.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.459+0000 7f9724115740 1 --2- 172.21.15.67:0/378592075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c3736fd390 0x55c373ac37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.459+0000 7f9724115740 1 --2- 172.21.15.67:0/378592075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3736f51b0 0x55c373ac3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.464 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.459+0000 7f9724115740 1 --2- 172.21.15.67:0/378592075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c37371e070 0x55c373ac2d40 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.459+0000 7f9724115740 1 -- 172.21.15.67:0/378592075 >> 172.21.15.67:0/378592075 conn(0x55c373712460 msgr2=0x55c373ac0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:11.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.459+0000 7f9724115740 1 -- 172.21.15.67:0/378592075 shutdown_connections 2024-07-27T03:18:11.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.459+0000 7f9724115740 1 -- 172.21.15.67:0/378592075 wait complete. 2024-07-27T03:18:11.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.459+0000 7f970f7fe640 1 --2- 172.21.15.67:0/2824554879 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f97040b3c50 0x7f97040b6160 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f97080a6620 tx=0x7f97080cd040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:11.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.459+0000 7f9724115740 1 -- start start 2024-07-27T03:18:11.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.459+0000 7f9724115740 1 Processor -- start 2024-07-27T03:18:11.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.460+0000 7f9724115740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c3736f51b0 0x55c373ace930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:11.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.460+0000 7f9724115740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3736fd390 0x55c373acee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:11.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.460+0000 7f9724115740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c37371e070 0x55c373ada6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:11.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.460+0000 7f9717fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c3736f51b0 0x55c373ace930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:11.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.460+0000 7f9717fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c3736f51b0 0x55c373ace930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57544/0 (socket says 172.21.15.67:57544) 2024-07-27T03:18:11.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.460+0000 7f9717fff640 1 -- 172.21.15.67:0/4269180312 learned_addr learned my addr 172.21.15.67:0/4269180312 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:11.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.460+0000 7f9724115740 1 -- 172.21.15.67:0/4269180312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55c3739cb900 con 0x55c3736f51b0 2024-07-27T03:18:11.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.460+0000 7f9724115740 1 -- 172.21.15.67:0/4269180312 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55c373a2d060 con 0x55c3736fd390 2024-07-27T03:18:11.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.460+0000 7f9724115740 1 -- 172.21.15.67:0/4269180312 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c37388c700 con 0x55c37371e070 2024-07-27T03:18:11.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.460+0000 7f97177fe640 1 --2- 172.21.15.67:0/4269180312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3736fd390 0x55c373acee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:11.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.460+0000 7f9724115740 10 client.0 _ll_register_callbacks cb 0x55c3739d3d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:11.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.461+0000 7f9717fff640 1 -- 172.21.15.67:0/4269180312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c37371e070 msgr2=0x55c373ada6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:11.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.461+0000 7f9717fff640 1 --2- 172.21.15.67:0/4269180312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c37371e070 0x55c373ada6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.461+0000 7f9717fff640 1 -- 172.21.15.67:0/4269180312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3736fd390 msgr2=0x55c373acee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:11.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.461+0000 7f9717fff640 1 --2- 172.21.15.67:0/4269180312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3736fd390 0x55c373acee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.461+0000 7f9717fff640 1 -- 172.21.15.67:0/4269180312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3739cfbf0 con 0x55c3736f51b0 2024-07-27T03:18:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.461+0000 7f9724115740 1 -- 172.21.15.67:0/4269180312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f971000ddd0 con 0x55c3736f51b0 2024-07-27T03:18:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.461+0000 7f9717fff640 1 --2- 172.21.15.67:0/4269180312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c3736f51b0 0x55c373ace930 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f971000ffd0 tx=0x7f97100058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de70c211ffeac3dc server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.461+0000 7f96f6ffd640 10 client.15753 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.461+0000 7f96f6ffd640 1 -- 172.21.15.67:0/4269180312 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971001d040 con 0x55c3736f51b0 2024-07-27T03:18:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.461+0000 7f96f6ffd640 1 -- 172.21.15.67:0/4269180312 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f9710003b70 con 0x55c3736f51b0 2024-07-27T03:18:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.461+0000 7f96f6ffd640 1 -- 172.21.15.67:0/4269180312 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9710003dd0 con 0x55c3736f51b0 2024-07-27T03:18:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.461+0000 7f9724115740 10 client.15753 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.461+0000 7f9724115740 1 -- 172.21.15.67:0/4269180312 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55c373789650 con 0x55c3736f51b0 2024-07-27T03:18:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.462+0000 7f96f6ffd640 1 -- 172.21.15.67:0/4269180312 <== mon.0 v2:172.21.15.67:3300/0 4 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9710004a40 con 0x55c3736f51b0 2024-07-27T03:18:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.462+0000 7f9724115740 10 client.15753 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.462+0000 7f9724115740 10 client.15753 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-07-27T03:18:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.462+0000 7f9724115740 20 client.15753 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:11.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.462+0000 7f9724115740 1 --2- 172.21.15.67:0/4269180312 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55c3736dc9f0 0x55c373ae04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:11.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.462+0000 7f9724115740 4 client.15753 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:18:11.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.462+0000 7f9724115740 1 -- 172.21.15.67:0/4269180312 --> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c373a3ba70 con 0x55c3736dc9f0 2024-07-27T03:18:11.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.462+0000 7f97177fe640 1 --2- 172.21.15.67:0/4269180312 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55c3736dc9f0 0x55c373ae04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:11.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.463+0000 7f96f6ffd640 10 client.15753 ms_handle_connect on v2:172.21.15.149:6836/2932951845 2024-07-27T03:18:11.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.463+0000 7f97177fe640 1 --2- 172.21.15.67:0/4269180312 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55c3736dc9f0 0x55c373ae04a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=940df035edf8ca8 server_cookie=cb9238d92b5edb07 in_seq=0 out_seq=0 2024-07-27T03:18:11.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.464+0000 7f96f6ffd640 1 -- 172.21.15.67:0/4269180312 <== mds.1 v2:172.21.15.149:6836/2932951845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c373a3ba70 con 0x55c3736dc9f0 2024-07-27T03:18:11.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.464+0000 7f96f6ffd640 10 client.15753 handle_command_reply: tid=0 2024-07-27T03:18:11.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.464+0000 7f9724115740 1 client.15753 shutdown 2024-07-27T03:18:11.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.464+0000 7f9724115740 1 -- 172.21.15.67:0/4269180312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c3736f51b0 msgr2=0x55c373ace930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:11.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.464+0000 7f9724115740 1 --2- 172.21.15.67:0/4269180312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c3736f51b0 0x55c373ace930 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f971000ffd0 tx=0x7f97100058e0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.464+0000 7f9724115740 1 -- 172.21.15.67:0/4269180312 shutdown_connections 2024-07-27T03:18:11.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.464+0000 7f9724115740 1 --2- 172.21.15.67:0/4269180312 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55c3736dc9f0 0x55c373ae04a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.464+0000 7f9724115740 1 --2- 172.21.15.67:0/4269180312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c37371e070 0x55c373ada6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.464+0000 7f9724115740 1 --2- 172.21.15.67:0/4269180312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3736fd390 0x55c373acee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.464+0000 7f96f6ffd640 0 client.15753 ms_handle_reset on v2:172.21.15.149:6836/2932951845 2024-07-27T03:18:11.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.464+0000 7f9724115740 1 --2- 172.21.15.67:0/4269180312 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c3736f51b0 0x55c373ace930 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.464+0000 7f9724115740 1 -- 172.21.15.67:0/4269180312 >> 172.21.15.67:0/4269180312 conn(0x55c373712460 msgr2=0x55c373ac5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:11.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.464+0000 7f9724115740 1 -- 172.21.15.67:0/4269180312 shutdown_connections 2024-07-27T03:18:11.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.464+0000 7f9724115740 1 -- 172.21.15.67:0/4269180312 wait complete. 2024-07-27T03:18:11.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.464+0000 7f9724115740 20 client.15753 trim_cache size 0 max 16384 2024-07-27T03:18:11.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.478+0000 7f9724115740 1 Processor -- start 2024-07-27T03:18:11.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.478+0000 7f9724115740 1 -- start start 2024-07-27T03:18:11.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.478+0000 7f9724115740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3736f51b0 0x55c373ad0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:11.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.478+0000 7f9724115740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c3736fd390 0x55c373ac01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:11.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.478+0000 7f9724115740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c37371e070 0x55c373ac0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:11.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.478+0000 7f9724115740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f97181b2700 con 0x55c37371e070 2024-07-27T03:18:11.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.478+0000 7f9724115740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f97181b4240 con 0x55c3736f51b0 2024-07-27T03:18:11.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.478+0000 7f9724115740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f97181b2dd0 con 0x55c3736fd390 2024-07-27T03:18:11.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.478+0000 7f9717fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c37371e070 0x55c373ac0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:11.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.478+0000 7f9717fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c37371e070 0x55c373ac0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57560/0 (socket says 172.21.15.67:57560) 2024-07-27T03:18:11.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.478+0000 7f9717fff640 1 -- 172.21.15.67:0/1816328648 learned_addr learned my addr 172.21.15.67:0/1816328648 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:11.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.478+0000 7f97177fe640 1 --2- 172.21.15.67:0/1816328648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c3736fd390 0x55c373ac01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:11.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.478+0000 7f9717fff640 1 --2- 172.21.15.67:0/1816328648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3736f51b0 0x55c373ad0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:11.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.479+0000 7f9717fff640 1 -- 172.21.15.67:0/1816328648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c3736fd390 msgr2=0x55c373ac01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:11.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.479+0000 7f9717fff640 1 --2- 172.21.15.67:0/1816328648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c3736fd390 0x55c373ac01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.479+0000 7f9717fff640 1 -- 172.21.15.67:0/1816328648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3736f51b0 msgr2=0x55c373ad0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:11.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.479+0000 7f9717fff640 1 --2- 172.21.15.67:0/1816328648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3736f51b0 0x55c373ad0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.479+0000 7f9717fff640 1 -- 172.21.15.67:0/1816328648 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c373abc280 con 0x55c37371e070 2024-07-27T03:18:11.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.479+0000 7f9717fff640 1 --2- 172.21.15.67:0/1816328648 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c37371e070 0x55c373ac0720 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9710005680 tx=0x7f971001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9dcbe978f5bea4 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:11.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.479+0000 7f970d7fa640 1 -- 172.21.15.67:0/1816328648 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710004ef0 con 0x55c37371e070 2024-07-27T03:18:11.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.479+0000 7f970d7fa640 1 -- 172.21.15.67:0/1816328648 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f971000ddd0 con 0x55c37371e070 2024-07-27T03:18:11.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.479+0000 7f970d7fa640 1 -- 172.21.15.67:0/1816328648 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710010bd0 con 0x55c37371e070 2024-07-27T03:18:11.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.480+0000 7f9724115740 1 -- 172.21.15.67:0/1816328648 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c37371e070 msgr2=0x55c373ac0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:11.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.480+0000 7f9724115740 1 --2- 172.21.15.67:0/1816328648 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c37371e070 0x55c373ac0720 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9710005680 tx=0x7f971001e000 comp rx=0 tx=0).stop 2024-07-27T03:18:11.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.480+0000 7f9724115740 1 -- 172.21.15.67:0/1816328648 shutdown_connections 2024-07-27T03:18:11.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.480+0000 7f9724115740 1 --2- 172.21.15.67:0/1816328648 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c37371e070 0x55c373ac0720 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.480+0000 7f9724115740 1 --2- 172.21.15.67:0/1816328648 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c3736fd390 0x55c373ac01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.480+0000 7f9724115740 1 --2- 172.21.15.67:0/1816328648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3736f51b0 0x55c373ad0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.480+0000 7f9724115740 1 -- 172.21.15.67:0/1816328648 >> 172.21.15.67:0/1816328648 conn(0x55c373712460 msgr2=0x55c373ac4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:11.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.480+0000 7f9724115740 1 -- 172.21.15.67:0/1816328648 shutdown_connections 2024-07-27T03:18:11.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.480+0000 7f9724115740 1 -- 172.21.15.67:0/1816328648 wait complete. 2024-07-27T03:18:11.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.481+0000 7f9724115740 1 -- start start 2024-07-27T03:18:11.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.481+0000 7f9724115740 1 Processor -- start 2024-07-27T03:18:11.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.481+0000 7f9724115740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c3736f51b0 0x55c373ae3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:11.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.481+0000 7f9724115740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c3736fd390 0x55c373ae0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:11.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.481+0000 7f9724115740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c37371e070 0x55c373ae1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:11.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.481+0000 7f9724115740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f97181b3b70 con 0x55c3736fd390 2024-07-27T03:18:11.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.481+0000 7f9724115740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f97181b34a0 con 0x55c37371e070 2024-07-27T03:18:11.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.481+0000 7f9724115740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f97181b06e0 con 0x55c3736f51b0 2024-07-27T03:18:11.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.481+0000 7f9724115740 10 client.0 _ll_register_callbacks cb 0x7f9700009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:11.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.481+0000 7f97177fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c3736fd390 0x55c373ae0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:11.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.481+0000 7f97177fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c3736fd390 0x55c373ae0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57564/0 (socket says 172.21.15.67:57564) 2024-07-27T03:18:11.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.481+0000 7f97177fe640 1 -- 172.21.15.67:0/2136471011 learned_addr learned my addr 172.21.15.67:0/2136471011 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:11.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.482+0000 7f9717fff640 1 --2- 172.21.15.67:0/2136471011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c3736f51b0 0x55c373ae3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:11.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.482+0000 7f9717fff640 1 --2- 172.21.15.67:0/2136471011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c37371e070 0x55c373ae1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:11.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.482+0000 7f97177fe640 1 -- 172.21.15.67:0/2136471011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c3736f51b0 msgr2=0x55c373ae3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:11.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.482+0000 7f97177fe640 1 --2- 172.21.15.67:0/2136471011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c3736f51b0 0x55c373ae3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.482+0000 7f97177fe640 1 -- 172.21.15.67:0/2136471011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c37371e070 msgr2=0x55c373ae1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:11.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.482+0000 7f97177fe640 1 --2- 172.21.15.67:0/2136471011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c37371e070 0x55c373ae1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.482+0000 7f97177fe640 1 -- 172.21.15.67:0/2136471011 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c373ae18b0 con 0x55c3736fd390 2024-07-27T03:18:11.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.482+0000 7f9717fff640 1 --2- 172.21.15.67:0/2136471011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c37371e070 0x55c373ae1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:18:11.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.482+0000 7f97177fe640 1 --2- 172.21.15.67:0/2136471011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c3736fd390 0x55c373ae0e30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9700009230 tx=0x7f97000058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640144718e658c22 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:11.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.482+0000 7f96f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:11.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.482+0000 7f96f6ffd640 1 -- 172.21.15.67:0/2136471011 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700035040 con 0x55c3736fd390 2024-07-27T03:18:11.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.482+0000 7f9724115740 1 -- 172.21.15.67:0/2136471011 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f971000ddd0 con 0x55c3736fd390 2024-07-27T03:18:11.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.482+0000 7f96f6ffd640 1 -- 172.21.15.67:0/2136471011 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f9700005040 con 0x55c3736fd390 2024-07-27T03:18:11.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.483+0000 7f96f6ffd640 1 -- 172.21.15.67:0/2136471011 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9700055050 con 0x55c3736fd390 2024-07-27T03:18:11.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.483+0000 7f9724115740 10 client.15759 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:11.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.483+0000 7f9724115740 1 -- 172.21.15.67:0/2136471011 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c373b48770 con 0x55c3736fd390 2024-07-27T03:18:11.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.483+0000 7f96f6ffd640 1 -- 172.21.15.67:0/2136471011 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f971000ddd0 con 0x55c3736fd390 2024-07-27T03:18:11.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.483+0000 7f96f6ffd640 1 -- 172.21.15.67:0/2136471011 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f97000035f0 con 0x55c3736fd390 2024-07-27T03:18:11.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.483+0000 7f9724115740 10 client.15759 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:11.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.483+0000 7f9724115740 10 client.15759 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-07-27T03:18:11.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.483+0000 7f9724115740 20 client.15759 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:11.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.483+0000 7f9724115740 1 --2- 172.21.15.67:0/2136471011 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55c3736b1da0 0x55c373b4af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:11.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.483+0000 7f9724115740 4 client.15759 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-27T03:18:11.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.483+0000 7f9724115740 1 -- 172.21.15.67:0/2136471011 --> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c373a37510 con 0x55c3736b1da0 2024-07-27T03:18:11.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.483+0000 7f9717fff640 1 --2- 172.21.15.67:0/2136471011 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55c3736b1da0 0x55c373b4af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:11.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.484+0000 7f9717fff640 1 --2- 172.21.15.67:0/2136471011 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55c3736b1da0 0x55c373b4af60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b814639a24dfa42c server_cookie=dab24c8074b497c in_seq=0 out_seq=0 2024-07-27T03:18:11.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.484+0000 7f96f6ffd640 10 client.15759 ms_handle_connect on v2:172.21.15.149:6836/2932951845 2024-07-27T03:18:11.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.484+0000 7f96f6ffd640 1 -- 172.21.15.67:0/2136471011 <== mds.1 v2:172.21.15.149:6836/2932951845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c373a37510 con 0x55c3736b1da0 2024-07-27T03:18:11.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.484+0000 7f96f6ffd640 10 client.15759 handle_command_reply: tid=0 2024-07-27T03:18:11.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.484+0000 7f9724115740 1 client.15759 shutdown 2024-07-27T03:18:11.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.485+0000 7f9724115740 1 -- 172.21.15.67:0/2136471011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c3736fd390 msgr2=0x55c373ae0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:11.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.485+0000 7f9724115740 1 --2- 172.21.15.67:0/2136471011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c3736fd390 0x55c373ae0e30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9700009230 tx=0x7f97000058e0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.485+0000 7f9724115740 1 -- 172.21.15.67:0/2136471011 shutdown_connections 2024-07-27T03:18:11.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.485+0000 7f9724115740 1 --2- 172.21.15.67:0/2136471011 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x55c3736b1da0 0x55c373b4af60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.485+0000 7f9724115740 1 --2- 172.21.15.67:0/2136471011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c37371e070 0x55c373ae1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.485+0000 7f9724115740 1 --2- 172.21.15.67:0/2136471011 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55c3736fd390 0x55c373ae0e30 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.485+0000 7f9724115740 1 --2- 172.21.15.67:0/2136471011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c3736f51b0 0x55c373ae3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.485+0000 7f9724115740 1 -- 172.21.15.67:0/2136471011 >> 172.21.15.67:0/2136471011 conn(0x55c373712460 msgr2=0x55c373ac4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:11.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.485+0000 7f96f6ffd640 0 client.15759 ms_handle_reset on v2:172.21.15.149:6836/2932951845 2024-07-27T03:18:11.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.485+0000 7f9724115740 1 -- 172.21.15.67:0/2136471011 shutdown_connections 2024-07-27T03:18:11.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.485+0000 7f9724115740 1 -- 172.21.15.67:0/2136471011 wait complete. 2024-07-27T03:18:11.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.485+0000 7f9724115740 20 client.15759 trim_cache size 0 max 16384 2024-07-27T03:18:11.492 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-27T03:18:11.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.485+0000 7f971db90640 1 -- 172.21.15.67:0/2824554879 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f97040b3c50 msgr2=0x7f97040b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:11.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.485+0000 7f971db90640 1 --2- 172.21.15.67:0/2824554879 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f97040b3c50 0x7f97040b6160 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f97080a6620 tx=0x7f97080cd040 comp rx=0 tx=0).stop 2024-07-27T03:18:11.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.485+0000 7f971db90640 1 -- 172.21.15.67:0/2824554879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9718102550 msgr2=0x7f971819ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:11.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.485+0000 7f971db90640 1 --2- 172.21.15.67:0/2824554879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9718102550 0x7f971819ad70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f970800d770 tx=0x7f970800dc30 comp rx=0 tx=0).stop 2024-07-27T03:18:11.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.486+0000 7f971db90640 1 -- 172.21.15.67:0/2824554879 shutdown_connections 2024-07-27T03:18:11.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.486+0000 7f971db90640 1 --2- 172.21.15.67:0/2824554879 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f97040b3c50 0x7f97040b6160 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.486+0000 7f971db90640 1 --2- 172.21.15.67:0/2824554879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9718108790 0x7f971819b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.486+0000 7f971db90640 1 --2- 172.21.15.67:0/2824554879 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9718107e40 0x7f971819b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.486+0000 7f971db90640 1 --2- 172.21.15.67:0/2824554879 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9718102550 0x7f971819ad70 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:11.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.486+0000 7f971db90640 1 -- 172.21.15.67:0/2824554879 >> 172.21.15.67:0/2824554879 conn(0x7f9718079220 msgr2=0x7f9718103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:11.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.486+0000 7f971db90640 1 -- 172.21.15.67:0/2824554879 shutdown_connections 2024-07-27T03:18:11.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:11.486+0000 7f971db90640 1 -- 172.21.15.67:0/2824554879 wait complete. 2024-07-27T03:18:12.152 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-27T03:18:12.435 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:18:12.814 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:12 smithi067 ceph-mon[32814]: pgmap v479: 129 pgs: 129 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 864 KiB/s wr, 16 op/s 2024-07-27T03:18:12.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:12 smithi175 ceph-mon[37125]: pgmap v479: 129 pgs: 129 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 864 KiB/s wr, 16 op/s 2024-07-27T03:18:13.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:12 smithi149 ceph-mon[37173]: pgmap v479: 129 pgs: 129 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 864 KiB/s wr, 16 op/s 2024-07-27T03:18:14.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.179+0000 7f7a4ee20640 1 -- 172.21.15.67:0/2179329676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a48107e40 msgr2=0x7f7a48108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:14.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.179+0000 7f7a4ee20640 1 --2- 172.21.15.67:0/2179329676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a48107e40 0x7f7a48108250 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009900 tx=0x7f7a3002a410 comp rx=0 tx=0).stop 2024-07-27T03:18:14.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.180+0000 7f7a4ee20640 1 -- 172.21.15.67:0/2179329676 shutdown_connections 2024-07-27T03:18:14.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.180+0000 7f7a4ee20640 1 --2- 172.21.15.67:0/2179329676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a48108790 0x7f7a4810efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.180+0000 7f7a4ee20640 1 --2- 172.21.15.67:0/2179329676 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a48107e40 0x7f7a48108250 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.180+0000 7f7a4ee20640 1 --2- 172.21.15.67:0/2179329676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a48102550 0x7f7a48102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.180+0000 7f7a4ee20640 1 -- 172.21.15.67:0/2179329676 >> 172.21.15.67:0/2179329676 conn(0x7f7a48079220 msgr2=0x7f7a48079630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:14.228 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.180+0000 7f7a4ee20640 1 -- 172.21.15.67:0/2179329676 shutdown_connections 2024-07-27T03:18:14.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.180+0000 7f7a4ee20640 1 -- 172.21.15.67:0/2179329676 wait complete. 2024-07-27T03:18:14.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.181+0000 7f7a4ee20640 1 Processor -- start 2024-07-27T03:18:14.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.181+0000 7f7a4ee20640 1 -- start start 2024-07-27T03:18:14.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.181+0000 7f7a4ee20640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a48102550 0x7f7a48199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:14.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.181+0000 7f7a4ee20640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a48107e40 0x7f7a4819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:14.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.181+0000 7f7a4ee20640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a48108790 0x7f7a4819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:14.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.181+0000 7f7a4ee20640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7a48112420 con 0x7f7a48108790 2024-07-27T03:18:14.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.181+0000 7f7a4ee20640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7a481122a0 con 0x7f7a48107e40 2024-07-27T03:18:14.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.181+0000 7f7a4ee20640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7a481125a0 con 0x7f7a48102550 2024-07-27T03:18:14.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.181+0000 7f7a4d3bd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a48108790 0x7f7a4819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:14.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.181+0000 7f7a4d3bd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a48108790 0x7f7a4819f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57582/0 (socket says 172.21.15.67:57582) 2024-07-27T03:18:14.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.181+0000 7f7a4d3bd640 1 -- 172.21.15.67:0/4144627774 learned_addr learned my addr 172.21.15.67:0/4144627774 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.181+0000 7f7a47fff640 1 --2- 172.21.15.67:0/4144627774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a48107e40 0x7f7a4819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.181+0000 7f7a4cbbc640 1 --2- 172.21.15.67:0/4144627774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a48102550 0x7f7a48199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.182+0000 7f7a4d3bd640 1 -- 172.21.15.67:0/4144627774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a48102550 msgr2=0x7f7a48199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.182+0000 7f7a4d3bd640 1 --2- 172.21.15.67:0/4144627774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a48102550 0x7f7a48199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.182+0000 7f7a4d3bd640 1 -- 172.21.15.67:0/4144627774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a48107e40 msgr2=0x7f7a4819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.182+0000 7f7a4d3bd640 1 --2- 172.21.15.67:0/4144627774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a48107e40 0x7f7a4819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.182+0000 7f7a4d3bd640 1 -- 172.21.15.67:0/4144627774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4819feb0 con 0x7f7a48108790 2024-07-27T03:18:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.182+0000 7f7a4d3bd640 1 --2- 172.21.15.67:0/4144627774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a48108790 0x7f7a4819f780 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000d750 tx=0x7f7a4000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.182+0000 7f7a45ffb640 1 -- 172.21.15.67:0/4144627774 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40017070 con 0x7f7a48108790 2024-07-27T03:18:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.182+0000 7f7a4ee20640 1 -- 172.21.15.67:0/4144627774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a481a01a0 con 0x7f7a48108790 2024-07-27T03:18:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.182+0000 7f7a45ffb640 1 -- 172.21.15.67:0/4144627774 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a400054d0 con 0x7f7a48108790 2024-07-27T03:18:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.182+0000 7f7a45ffb640 1 -- 172.21.15.67:0/4144627774 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40004c60 con 0x7f7a48108790 2024-07-27T03:18:14.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.182+0000 7f7a4ee20640 1 -- 172.21.15.67:0/4144627774 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a481a6900 con 0x7f7a48108790 2024-07-27T03:18:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.185+0000 7f7a45ffb640 1 -- 172.21.15.67:0/4144627774 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f7a40009bd0 con 0x7f7a48108790 2024-07-27T03:18:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.186+0000 7f7a45ffb640 1 --2- 172.21.15.67:0/4144627774 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7a340b3bd0 0x7f7a340b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.186+0000 7f7a4cbbc640 1 --2- 172.21.15.67:0/4144627774 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7a340b3bd0 0x7f7a340b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.186+0000 7f7a45ffb640 1 -- 172.21.15.67:0/4144627774 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f7a400e38d0 con 0x7f7a48108790 2024-07-27T03:18:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.187+0000 7f7a553a5740 1 Processor -- start 2024-07-27T03:18:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.187+0000 7f7a553a5740 1 -- start start 2024-07-27T03:18:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.187+0000 7f7a4cbbc640 1 --2- 172.21.15.67:0/4144627774 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7a340b3bd0 0x7f7a340b60e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7a38005fd0 tx=0x7f7a38009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.187+0000 7f7a553a5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedc0a070 0x562aedfac890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.187+0000 7f7a553a5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedbe11b0 0x562aedfacdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.187+0000 7f7a553a5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562aedbe9390 0x562aedfbf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.187+0000 7f7a553a5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562aede97400 con 0x562aedbe11b0 2024-07-27T03:18:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.187+0000 7f7a553a5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562aedf25b20 con 0x562aedbe9390 2024-07-27T03:18:14.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.187+0000 7f7a553a5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562aedf15020 con 0x562aedc0a070 2024-07-27T03:18:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.187+0000 7f7a47fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedbe11b0 0x562aedfacdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.187+0000 7f7a47fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedbe11b0 0x562aedfacdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57586/0 (socket says 172.21.15.67:57586) 2024-07-27T03:18:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.187+0000 7f7a47fff640 1 -- 172.21.15.67:0/3164972629 learned_addr learned my addr 172.21.15.67:0/3164972629 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.187+0000 7f7a4d3bd640 1 --2- 172.21.15.67:0/3164972629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562aedbe9390 0x562aedfbf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.187+0000 7f7a4cbbc640 1 --2- 172.21.15.67:0/3164972629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedc0a070 0x562aedfac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.188+0000 7f7a47fff640 1 -- 172.21.15.67:0/3164972629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedc0a070 msgr2=0x562aedfac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.188+0000 7f7a47fff640 1 --2- 172.21.15.67:0/3164972629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedc0a070 0x562aedfac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.188+0000 7f7a47fff640 1 -- 172.21.15.67:0/3164972629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562aedbe9390 msgr2=0x562aedfbf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.188+0000 7f7a47fff640 1 --2- 172.21.15.67:0/3164972629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562aedbe9390 0x562aedfbf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.188+0000 7f7a47fff640 1 -- 172.21.15.67:0/3164972629 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aedeaa770 con 0x562aedbe11b0 2024-07-27T03:18:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.188+0000 7f7a4cbbc640 1 --2- 172.21.15.67:0/3164972629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedc0a070 0x562aedfac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:18:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.188+0000 7f7a47fff640 1 --2- 172.21.15.67:0/3164972629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedbe11b0 0x562aedfacdf0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009900 tx=0x7f7a30000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba6663f446c7b48 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.188+0000 7f7a3effd640 1 -- 172.21.15.67:0/3164972629 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a30040070 con 0x562aedbe11b0 2024-07-27T03:18:14.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.188+0000 7f7a3effd640 1 -- 172.21.15.67:0/3164972629 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a30004030 con 0x562aedbe11b0 2024-07-27T03:18:14.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.188+0000 7f7a3effd640 1 -- 172.21.15.67:0/3164972629 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a30035040 con 0x562aedbe11b0 2024-07-27T03:18:14.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.189+0000 7f7a553a5740 1 -- 172.21.15.67:0/3164972629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedbe11b0 msgr2=0x562aedfacdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:14.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.189+0000 7f7a553a5740 1 --2- 172.21.15.67:0/3164972629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedbe11b0 0x562aedfacdf0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009900 tx=0x7f7a30000bc0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.189+0000 7f7a553a5740 1 -- 172.21.15.67:0/3164972629 shutdown_connections 2024-07-27T03:18:14.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.189+0000 7f7a553a5740 1 --2- 172.21.15.67:0/3164972629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562aedbe9390 0x562aedfbf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.189+0000 7f7a553a5740 1 --2- 172.21.15.67:0/3164972629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedbe11b0 0x562aedfacdf0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.189+0000 7f7a553a5740 1 --2- 172.21.15.67:0/3164972629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedc0a070 0x562aedfac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.189+0000 7f7a553a5740 1 -- 172.21.15.67:0/3164972629 >> 172.21.15.67:0/3164972629 conn(0x562aedbfe460 msgr2=0x562aedfaedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:14.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.189+0000 7f7a553a5740 1 -- 172.21.15.67:0/3164972629 shutdown_connections 2024-07-27T03:18:14.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.189+0000 7f7a553a5740 1 -- 172.21.15.67:0/3164972629 wait complete. 2024-07-27T03:18:14.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.189+0000 7f7a553a5740 1 -- start start 2024-07-27T03:18:14.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.190+0000 7f7a553a5740 1 Processor -- start 2024-07-27T03:18:14.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.190+0000 7f7a553a5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562aedbe11b0 0x562aedfc36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.190+0000 7f7a553a5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedbe9390 0x562aedfc3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.190+0000 7f7a553a5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedc0a070 0x562aedfc8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.190+0000 7f7a553a5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562aedeb7900 con 0x562aedc0a070 2024-07-27T03:18:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.190+0000 7f7a553a5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562aedf19060 con 0x562aedbe11b0 2024-07-27T03:18:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.190+0000 7f7a553a5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562aedd78700 con 0x562aedbe9390 2024-07-27T03:18:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.190+0000 7f7a553a5740 10 client.0 _ll_register_callbacks cb 0x562aedebfd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.190+0000 7f7a4d3bd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedc0a070 0x562aedfc8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.190+0000 7f7a4d3bd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedc0a070 0x562aedfc8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57590/0 (socket says 172.21.15.67:57590) 2024-07-27T03:18:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.190+0000 7f7a4d3bd640 1 -- 172.21.15.67:0/2246164429 learned_addr learned my addr 172.21.15.67:0/2246164429 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.190+0000 7f7a4cbbc640 1 --2- 172.21.15.67:0/2246164429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562aedbe11b0 0x562aedfc36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.191+0000 7f7a47fff640 1 --2- 172.21.15.67:0/2246164429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedbe9390 0x562aedfc3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.191+0000 7f7a4d3bd640 1 -- 172.21.15.67:0/2246164429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedbe9390 msgr2=0x562aedfc3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:14.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.191+0000 7f7a4d3bd640 1 --2- 172.21.15.67:0/2246164429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedbe9390 0x562aedfc3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.191+0000 7f7a4d3bd640 1 -- 172.21.15.67:0/2246164429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562aedbe11b0 msgr2=0x562aedfc36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:14.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.191+0000 7f7a4d3bd640 1 --2- 172.21.15.67:0/2246164429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562aedbe11b0 0x562aedfc36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.191+0000 7f7a4d3bd640 1 -- 172.21.15.67:0/2246164429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aedebbbf0 con 0x562aedc0a070 2024-07-27T03:18:14.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.191+0000 7f7a47fff640 1 --2- 172.21.15.67:0/2246164429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedbe9390 0x562aedfc3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:18:14.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.191+0000 7f7a4d3bd640 1 --2- 172.21.15.67:0/2246164429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedc0a070 0x562aedfc8cb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7a400aa220 tx=0x7f7a400aa170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a2e035edd441c2 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:14.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.191+0000 7f7a27fff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:14.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.191+0000 7f7a27fff640 1 -- 172.21.15.67:0/2246164429 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40020070 con 0x562aedc0a070 2024-07-27T03:18:14.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.191+0000 7f7a27fff640 1 -- 172.21.15.67:0/2246164429 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f7a400a9780 con 0x562aedc0a070 2024-07-27T03:18:14.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.191+0000 7f7a553a5740 1 -- 172.21.15.67:0/2246164429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a30004030 con 0x562aedc0a070 2024-07-27T03:18:14.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.191+0000 7f7a27fff640 1 -- 172.21.15.67:0/2246164429 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7a400b3520 con 0x562aedc0a070 2024-07-27T03:18:14.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.191+0000 7f7a553a5740 10 client.15771 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:14.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.191+0000 7f7a553a5740 1 -- 172.21.15.67:0/2246164429 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562aedc75650 con 0x562aedc0a070 2024-07-27T03:18:14.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.192+0000 7f7a27fff640 1 -- 172.21.15.67:0/2246164429 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a30004030 con 0x562aedc0a070 2024-07-27T03:18:14.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.192+0000 7f7a27fff640 1 -- 172.21.15.67:0/2246164429 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7a400a91f0 con 0x562aedc0a070 2024-07-27T03:18:14.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.192+0000 7f7a553a5740 10 client.15771 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:14.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.192+0000 7f7a553a5740 10 client.15771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:18:14.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.192+0000 7f7a553a5740 20 client.15771 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:14.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.192+0000 7f7a553a5740 1 --2- 172.21.15.67:0/2246164429 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x562aedbc89f0 0x562aedfd5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:14.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.192+0000 7f7a553a5740 4 client.15771 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:18:14.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.192+0000 7f7a553a5740 1 -- 172.21.15.67:0/2246164429 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562aedf27a70 con 0x562aedbc89f0 2024-07-27T03:18:14.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.192+0000 7f7a4cbbc640 1 --2- 172.21.15.67:0/2246164429 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x562aedbc89f0 0x562aedfd5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:14.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.193+0000 7f7a4cbbc640 1 --2- 172.21.15.67:0/2246164429 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x562aedbc89f0 0x562aedfd5050 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3953875d506ab7a1 server_cookie=19d39d8040c4a4ca in_seq=0 out_seq=0 2024-07-27T03:18:14.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.193+0000 7f7a27fff640 10 client.15771 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:14.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.194+0000 7f7a27fff640 1 -- 172.21.15.67:0/2246164429 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562aedf27a70 con 0x562aedbc89f0 2024-07-27T03:18:14.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.194+0000 7f7a27fff640 10 client.15771 handle_command_reply: tid=0 2024-07-27T03:18:14.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.194+0000 7f7a553a5740 1 client.15771 shutdown 2024-07-27T03:18:14.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.194+0000 7f7a553a5740 1 -- 172.21.15.67:0/2246164429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedc0a070 msgr2=0x562aedfc8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:14.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.194+0000 7f7a553a5740 1 --2- 172.21.15.67:0/2246164429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedc0a070 0x562aedfc8cb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7a400aa220 tx=0x7f7a400aa170 comp rx=0 tx=0).stop 2024-07-27T03:18:14.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.194+0000 7f7a553a5740 1 -- 172.21.15.67:0/2246164429 shutdown_connections 2024-07-27T03:18:14.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.194+0000 7f7a553a5740 1 --2- 172.21.15.67:0/2246164429 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x562aedbc89f0 0x562aedfd5050 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.194+0000 7f7a553a5740 1 --2- 172.21.15.67:0/2246164429 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedc0a070 0x562aedfc8cb0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.194+0000 7f7a27fff640 0 client.15771 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:14.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.194+0000 7f7a553a5740 1 --2- 172.21.15.67:0/2246164429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedbe9390 0x562aedfc3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.194+0000 7f7a553a5740 1 --2- 172.21.15.67:0/2246164429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562aedbe11b0 0x562aedfc36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.194+0000 7f7a553a5740 1 -- 172.21.15.67:0/2246164429 >> 172.21.15.67:0/2246164429 conn(0x562aedbfe460 msgr2=0x562aedfabcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:14.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.194+0000 7f7a553a5740 1 -- 172.21.15.67:0/2246164429 shutdown_connections 2024-07-27T03:18:14.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.194+0000 7f7a553a5740 1 -- 172.21.15.67:0/2246164429 wait complete. 2024-07-27T03:18:14.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.194+0000 7f7a553a5740 20 client.15771 trim_cache size 0 max 16384 2024-07-27T03:18:14.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.222+0000 7f7a553a5740 1 Processor -- start 2024-07-27T03:18:14.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.223+0000 7f7a553a5740 1 -- start start 2024-07-27T03:18:14.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.223+0000 7f7a553a5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedbc89f0 0x562aedfc0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:14.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.223+0000 7f7a553a5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedbe11b0 0x562aedfc2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:14.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.223+0000 7f7a553a5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562aedbe9390 0x562aedfda7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:14.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.223+0000 7f7a553a5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7a481ae380 con 0x562aedbe11b0 2024-07-27T03:18:14.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.223+0000 7f7a553a5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7a481afec0 con 0x562aedbe9390 2024-07-27T03:18:14.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.223+0000 7f7a553a5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7a481aea50 con 0x562aedbc89f0 2024-07-27T03:18:14.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.223+0000 7f7a47fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedbe11b0 0x562aedfc2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:14.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.223+0000 7f7a47fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedbe11b0 0x562aedfc2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57596/0 (socket says 172.21.15.67:57596) 2024-07-27T03:18:14.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.223+0000 7f7a47fff640 1 -- 172.21.15.67:0/1071794759 learned_addr learned my addr 172.21.15.67:0/1071794759 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:14.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.223+0000 7f7a4cbbc640 1 --2- 172.21.15.67:0/1071794759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedbc89f0 0x562aedfc0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:14.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.223+0000 7f7a4d3bd640 1 --2- 172.21.15.67:0/1071794759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562aedbe9390 0x562aedfda7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:14.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.223+0000 7f7a47fff640 1 -- 172.21.15.67:0/1071794759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedbc89f0 msgr2=0x562aedfc0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:14.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.223+0000 7f7a47fff640 1 --2- 172.21.15.67:0/1071794759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedbc89f0 0x562aedfc0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.223+0000 7f7a47fff640 1 -- 172.21.15.67:0/1071794759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562aedbe9390 msgr2=0x562aedfda7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:14.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.223+0000 7f7a47fff640 1 --2- 172.21.15.67:0/1071794759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562aedbe9390 0x562aedfda7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.223+0000 7f7a47fff640 1 -- 172.21.15.67:0/1071794759 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aedfad640 con 0x562aedbe11b0 2024-07-27T03:18:14.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.223+0000 7f7a4d3bd640 1 --2- 172.21.15.67:0/1071794759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562aedbe9390 0x562aedfda7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:18:14.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.224+0000 7f7a47fff640 1 --2- 172.21.15.67:0/1071794759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedbe11b0 0x562aedfc2460 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7a30005f40 tx=0x7f7a30005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c211ec4d45274ec server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:14.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.224+0000 7f7a3e7fc640 1 -- 172.21.15.67:0/1071794759 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a30035040 con 0x562aedbe11b0 2024-07-27T03:18:14.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.224+0000 7f7a3e7fc640 1 -- 172.21.15.67:0/1071794759 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a30032de0 con 0x562aedbe11b0 2024-07-27T03:18:14.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.224+0000 7f7a3e7fc640 1 -- 172.21.15.67:0/1071794759 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a30040040 con 0x562aedbe11b0 2024-07-27T03:18:14.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.225+0000 7f7a553a5740 1 -- 172.21.15.67:0/1071794759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedbe11b0 msgr2=0x562aedfc2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:14.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.225+0000 7f7a553a5740 1 --2- 172.21.15.67:0/1071794759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedbe11b0 0x562aedfc2460 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7a30005f40 tx=0x7f7a30005fd0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.225+0000 7f7a553a5740 1 -- 172.21.15.67:0/1071794759 shutdown_connections 2024-07-27T03:18:14.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.225+0000 7f7a553a5740 1 --2- 172.21.15.67:0/1071794759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562aedbe9390 0x562aedfda7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.225+0000 7f7a553a5740 1 --2- 172.21.15.67:0/1071794759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedbe11b0 0x562aedfc2460 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.225+0000 7f7a553a5740 1 --2- 172.21.15.67:0/1071794759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedbc89f0 0x562aedfc0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.225+0000 7f7a553a5740 1 -- 172.21.15.67:0/1071794759 >> 172.21.15.67:0/1071794759 conn(0x562aedbfe460 msgr2=0x562aedfc1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:14.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.225+0000 7f7a553a5740 1 -- 172.21.15.67:0/1071794759 shutdown_connections 2024-07-27T03:18:14.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.225+0000 7f7a553a5740 1 -- 172.21.15.67:0/1071794759 wait complete. 2024-07-27T03:18:14.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.226+0000 7f7a553a5740 1 -- start start 2024-07-27T03:18:14.240 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.226+0000 7f7a553a5740 1 Processor -- start 2024-07-27T03:18:14.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.226+0000 7f7a553a5740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedbc89f0 0x562aedfcd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:14.242 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.226+0000 7f7a553a5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562aedbe11b0 0x562aedfcd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:14.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.226+0000 7f7a553a5740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedbe9390 0x562aee03c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:14.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.226+0000 7f7a553a5740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7a481af7f0 con 0x562aedbe9390 2024-07-27T03:18:14.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.226+0000 7f7a553a5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7a481af120 con 0x562aedbe11b0 2024-07-27T03:18:14.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.226+0000 7f7a553a5740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7a481ac360 con 0x562aedbc89f0 2024-07-27T03:18:14.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.226+0000 7f7a553a5740 10 client.0 _ll_register_callbacks cb 0x7f7a400cd0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:14.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.226+0000 7f7a4d3bd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedbe9390 0x562aee03c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:14.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.227+0000 7f7a4d3bd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedbe9390 0x562aee03c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57602/0 (socket says 172.21.15.67:57602) 2024-07-27T03:18:14.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.227+0000 7f7a4d3bd640 1 -- 172.21.15.67:0/1433469085 learned_addr learned my addr 172.21.15.67:0/1433469085 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:14.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.227+0000 7f7a47fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562aedbe11b0 0x562aedfcd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:14.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.227+0000 7f7a4cbbc640 1 --2- 172.21.15.67:0/1433469085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedbc89f0 0x562aedfcd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:14.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.227+0000 7f7a47fff640 1 -- 172.21.15.67:0/1433469085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedbc89f0 msgr2=0x562aedfcd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:14.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.227+0000 7f7a47fff640 1 --2- 172.21.15.67:0/1433469085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedbc89f0 0x562aedfcd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.227+0000 7f7a47fff640 1 -- 172.21.15.67:0/1433469085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedbe9390 msgr2=0x562aee03c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:14.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.227+0000 7f7a47fff640 1 --2- 172.21.15.67:0/1433469085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedbe9390 0x562aee03c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.227+0000 7f7a47fff640 1 -- 172.21.15.67:0/1433469085 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aedfce0c0 con 0x562aedbe11b0 2024-07-27T03:18:14.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.227+0000 7f7a4cbbc640 1 --2- 172.21.15.67:0/1433469085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedbc89f0 0x562aedfcd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:18:14.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.227+0000 7f7a47fff640 1 --2- 172.21.15.67:0/1433469085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562aedbe11b0 0x562aedfcd840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7a30004650 tx=0x7f7a30032f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86e1570fccb26ee0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:14.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.227+0000 7f7a27fff640 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2024-07-27T03:18:14.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.227+0000 7f7a27fff640 1 -- 172.21.15.67:0/1433469085 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3005a060 con 0x562aedbe11b0 2024-07-27T03:18:14.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.227+0000 7f7a553a5740 1 -- 172.21.15.67:0/1433469085 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a30032de0 con 0x562aedbe11b0 2024-07-27T03:18:14.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.228+0000 7f7a27fff640 1 -- 172.21.15.67:0/1433469085 <== mon.2 v2:172.21.15.149:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f7a30049040 con 0x562aedbe11b0 2024-07-27T03:18:14.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.228+0000 7f7a27fff640 1 -- 172.21.15.67:0/1433469085 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7a30035030 con 0x562aedbe11b0 2024-07-27T03:18:14.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.228+0000 7f7a553a5740 10 client.25409 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:14.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.228+0000 7f7a553a5740 1 -- 172.21.15.67:0/1433469085 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562aee03dd30 con 0x562aedbe11b0 2024-07-27T03:18:14.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.228+0000 7f7a27fff640 1 -- 172.21.15.67:0/1433469085 <== mon.2 v2:172.21.15.149:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a30032de0 con 0x562aedbe11b0 2024-07-27T03:18:14.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.228+0000 7f7a27fff640 1 -- 172.21.15.67:0/1433469085 <== mon.2 v2:172.21.15.149:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7a3003c440 con 0x562aedbe11b0 2024-07-27T03:18:14.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.228+0000 7f7a553a5740 10 client.25409 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:14.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.228+0000 7f7a553a5740 10 client.25409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:18:14.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.228+0000 7f7a553a5740 20 client.25409 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:14.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.228+0000 7f7a553a5740 1 --2- 172.21.15.67:0/1433469085 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x562aedc0a070 0x562aee03ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:14.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.228+0000 7f7a553a5740 4 client.25409 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-27T03:18:14.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.228+0000 7f7a553a5740 1 -- 172.21.15.67:0/1433469085 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562aedf23510 con 0x562aedc0a070 2024-07-27T03:18:14.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.229+0000 7f7a4cbbc640 1 --2- 172.21.15.67:0/1433469085 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x562aedc0a070 0x562aee03ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:14.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.229+0000 7f7a27fff640 10 client.25409 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:14.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.229+0000 7f7a4cbbc640 1 --2- 172.21.15.67:0/1433469085 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x562aedc0a070 0x562aee03ffb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33d06f8fa357666d server_cookie=1a80a636fe3e8dbc in_seq=0 out_seq=0 2024-07-27T03:18:14.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.229+0000 7f7a27fff640 1 -- 172.21.15.67:0/1433469085 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562aedf23510 con 0x562aedc0a070 2024-07-27T03:18:14.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.229+0000 7f7a27fff640 10 client.25409 handle_command_reply: tid=0 2024-07-27T03:18:14.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.229+0000 7f7a553a5740 1 client.25409 shutdown 2024-07-27T03:18:14.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.230+0000 7f7a553a5740 1 -- 172.21.15.67:0/1433469085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562aedbe11b0 msgr2=0x562aedfcd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:14.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.230+0000 7f7a553a5740 1 --2- 172.21.15.67:0/1433469085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562aedbe11b0 0x562aedfcd840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7a30004650 tx=0x7f7a30032f90 comp rx=0 tx=0).stop 2024-07-27T03:18:14.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.230+0000 7f7a553a5740 1 -- 172.21.15.67:0/1433469085 shutdown_connections 2024-07-27T03:18:14.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.230+0000 7f7a553a5740 1 --2- 172.21.15.67:0/1433469085 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x562aedc0a070 0x562aee03ffb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.230+0000 7f7a553a5740 1 --2- 172.21.15.67:0/1433469085 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x562aedbe9390 0x562aee03c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.230+0000 7f7a553a5740 1 --2- 172.21.15.67:0/1433469085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562aedbe11b0 0x562aedfcd840 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.230+0000 7f7a553a5740 1 --2- 172.21.15.67:0/1433469085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562aedbc89f0 0x562aedfcd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.230+0000 7f7a553a5740 1 -- 172.21.15.67:0/1433469085 >> 172.21.15.67:0/1433469085 conn(0x562aedbfe460 msgr2=0x562aedfc2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:14.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.230+0000 7f7a27fff640 0 client.25409 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:14.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.230+0000 7f7a553a5740 1 -- 172.21.15.67:0/1433469085 shutdown_connections 2024-07-27T03:18:14.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.230+0000 7f7a553a5740 1 -- 172.21.15.67:0/1433469085 wait complete. 2024-07-27T03:18:14.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.230+0000 7f7a553a5740 20 client.25409 trim_cache size 0 max 16384 2024-07-27T03:18:14.246 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-27T03:18:14.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.231+0000 7f7a4ee20640 1 -- 172.21.15.67:0/4144627774 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7a340b3bd0 msgr2=0x7f7a340b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:14.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.231+0000 7f7a4ee20640 1 --2- 172.21.15.67:0/4144627774 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7a340b3bd0 0x7f7a340b60e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7a38005fd0 tx=0x7f7a38009680 comp rx=0 tx=0).stop 2024-07-27T03:18:14.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.231+0000 7f7a4ee20640 1 -- 172.21.15.67:0/4144627774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a48108790 msgr2=0x7f7a4819f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:14.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.231+0000 7f7a4ee20640 1 --2- 172.21.15.67:0/4144627774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a48108790 0x7f7a4819f780 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000d750 tx=0x7f7a4000dc10 comp rx=0 tx=0).stop 2024-07-27T03:18:14.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.231+0000 7f7a4ee20640 1 -- 172.21.15.67:0/4144627774 shutdown_connections 2024-07-27T03:18:14.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.231+0000 7f7a4ee20640 1 --2- 172.21.15.67:0/4144627774 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7a340b3bd0 0x7f7a340b60e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.231+0000 7f7a4ee20640 1 --2- 172.21.15.67:0/4144627774 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a48108790 0x7f7a4819f780 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.231+0000 7f7a4ee20640 1 --2- 172.21.15.67:0/4144627774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a48107e40 0x7f7a4819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.231+0000 7f7a4ee20640 1 --2- 172.21.15.67:0/4144627774 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a48102550 0x7f7a48199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:14.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.231+0000 7f7a4ee20640 1 -- 172.21.15.67:0/4144627774 >> 172.21.15.67:0/4144627774 conn(0x7f7a48079220 msgr2=0x7f7a48101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:14.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.231+0000 7f7a4ee20640 1 -- 172.21.15.67:0/4144627774 shutdown_connections 2024-07-27T03:18:14.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:14.231+0000 7f7a4ee20640 1 -- 172.21.15.67:0/4144627774 wait complete. 2024-07-27T03:18:14.782 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:14 smithi067 ceph-mon[32814]: pgmap v480: 129 pgs: 129 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 859 KiB/s wr, 11 op/s 2024-07-27T03:18:14.782 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:14 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:18:14.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:14 smithi175 ceph-mon[37125]: pgmap v480: 129 pgs: 129 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 859 KiB/s wr, 11 op/s 2024-07-27T03:18:14.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:14 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:18:15.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:14 smithi149 ceph-mon[37173]: pgmap v480: 129 pgs: 129 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 859 KiB/s wr, 11 op/s 2024-07-27T03:18:15.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:14 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:18:15.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:15 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:18:15.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:15 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:18:15.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:15 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:18:15.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:15 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:18:15.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:15 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:18:15.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:15 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:18:15.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:15 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:18:16.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:15 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:18:16.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:15 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:18:16.000 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-27T03:18:16.000 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-27T03:18:16.281 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:18:16.666 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:16 smithi067 ceph-mon[32814]: pgmap v481: 129 pgs: 129 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-07-27T03:18:16.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:16 smithi175 ceph-mon[37125]: pgmap v481: 129 pgs: 129 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-07-27T03:18:17.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:16 smithi149 ceph-mon[37173]: pgmap v481: 129 pgs: 129 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-07-27T03:18:18.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.136+0000 7fb18cc2d640 1 -- 172.21.15.67:0/2634702402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb188108f20 msgr2=0x7fb188109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:18.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.136+0000 7fb18cc2d640 1 --2- 172.21.15.67:0/2634702402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb188108f20 0x7fb188109330 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb17c009900 tx=0x7fb17c02a410 comp rx=0 tx=0).stop 2024-07-27T03:18:18.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.136+0000 7fb18cc2d640 1 -- 172.21.15.67:0/2634702402 shutdown_connections 2024-07-27T03:18:18.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.136+0000 7fb18cc2d640 1 --2- 172.21.15.67:0/2634702402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb188109870 0x7fb1881100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.136+0000 7fb18cc2d640 1 --2- 172.21.15.67:0/2634702402 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb188108f20 0x7fb188109330 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.136+0000 7fb18cc2d640 1 --2- 172.21.15.67:0/2634702402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb188103630 0x7fb188103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.139 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.136+0000 7fb18cc2d640 1 -- 172.21.15.67:0/2634702402 >> 172.21.15.67:0/2634702402 conn(0x7fb18807b380 msgr2=0x7fb18807b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:18.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.136+0000 7fb18cc2d640 1 -- 172.21.15.67:0/2634702402 shutdown_connections 2024-07-27T03:18:18.140 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.136+0000 7fb18cc2d640 1 -- 172.21.15.67:0/2634702402 wait complete. 2024-07-27T03:18:18.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.137+0000 7fb18cc2d640 1 Processor -- start 2024-07-27T03:18:18.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.137+0000 7fb18cc2d640 1 -- start start 2024-07-27T03:18:18.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.138+0000 7fb18cc2d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb188103630 0x7fb188075180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:18.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.138+0000 7fb18cc2d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb188108f20 0x7fb1880756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:18.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.138+0000 7fb18cc2d640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb188109870 0x7fb188075c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:18.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.138+0000 7fb18cc2d640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb188113500 con 0x7fb188103630 2024-07-27T03:18:18.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.138+0000 7fb18cc2d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb188113380 con 0x7fb188108f20 2024-07-27T03:18:18.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.138+0000 7fb18cc2d640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb188113680 con 0x7fb188109870 2024-07-27T03:18:18.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.138+0000 7fb18659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb188103630 0x7fb188075180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:18.142 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.138+0000 7fb18659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb188103630 0x7fb188075180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48240/0 (socket says 172.21.15.67:48240) 2024-07-27T03:18:18.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.138+0000 7fb18659c640 1 -- 172.21.15.67:0/2328778496 learned_addr learned my addr 172.21.15.67:0/2328778496 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:18.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.138+0000 7fb185d9b640 1 --2- 172.21.15.67:0/2328778496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb188108f20 0x7fb1880756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:18.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.138+0000 7fb186d9d640 1 --2- 172.21.15.67:0/2328778496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb188109870 0x7fb188075c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:18.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.138+0000 7fb18659c640 1 -- 172.21.15.67:0/2328778496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb188109870 msgr2=0x7fb188075c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:18.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.138+0000 7fb18659c640 1 --2- 172.21.15.67:0/2328778496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb188109870 0x7fb188075c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.138+0000 7fb18659c640 1 -- 172.21.15.67:0/2328778496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb188108f20 msgr2=0x7fb1880756c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:18.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.138+0000 7fb18659c640 1 --2- 172.21.15.67:0/2328778496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb188108f20 0x7fb1880756c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.138+0000 7fb18659c640 1 -- 172.21.15.67:0/2328778496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1881a8230 con 0x7fb188103630 2024-07-27T03:18:18.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.139+0000 7fb18659c640 1 --2- 172.21.15.67:0/2328778496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb188103630 0x7fb188075180 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb17800d770 tx=0x7fb17800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:18.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.139+0000 7fb1777fe640 1 -- 172.21.15.67:0/2328778496 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb178017070 con 0x7fb188103630 2024-07-27T03:18:18.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.139+0000 7fb1777fe640 1 -- 172.21.15.67:0/2328778496 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1780054d0 con 0x7fb188103630 2024-07-27T03:18:18.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.139+0000 7fb18cc2d640 1 -- 172.21.15.67:0/2328778496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1881a8520 con 0x7fb188103630 2024-07-27T03:18:18.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.139+0000 7fb1777fe640 1 -- 172.21.15.67:0/2328778496 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb178004c30 con 0x7fb188103630 2024-07-27T03:18:18.143 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.139+0000 7fb18cc2d640 1 -- 172.21.15.67:0/2328778496 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1881a8a30 con 0x7fb188103630 2024-07-27T03:18:18.144 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.142+0000 7fb1777fe640 1 -- 172.21.15.67:0/2328778496 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7fb178009c30 con 0x7fb188103630 2024-07-27T03:18:18.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.143+0000 7fb1777fe640 1 --2- 172.21.15.67:0/2328778496 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb1700b3c50 0x7fb1700b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:18.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.143+0000 7fb185d9b640 1 --2- 172.21.15.67:0/2328778496 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb1700b3c50 0x7fb1700b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:18.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.143+0000 7fb1777fe640 1 -- 172.21.15.67:0/2328778496 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fb1780e37d0 con 0x7fb188103630 2024-07-27T03:18:18.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.143+0000 7fb1931b2740 1 Processor -- start 2024-07-27T03:18:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.143+0000 7fb1931b2740 1 -- start start 2024-07-27T03:18:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.144+0000 7fb185d9b640 1 --2- 172.21.15.67:0/2328778496 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb1700b3c50 0x7fb1700b6160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb17c009230 tx=0x7fb17c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.144+0000 7fb1931b2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e3f350 0x55d4211e3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.144+0000 7fb1931b2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d420e16490 0x55d4211e4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.144+0000 7fb1931b2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d420e1e670 0x55d4211e4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.144+0000 7fb1931b2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d42115aa60 con 0x55d420e3f350 2024-07-27T03:18:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.144+0000 7fb1931b2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d421149f60 con 0x55d420e16490 2024-07-27T03:18:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.144+0000 7fb1931b2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d42114dfa0 con 0x55d420e1e670 2024-07-27T03:18:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.144+0000 7fb18659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e3f350 0x55d4211e3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.144+0000 7fb18659c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e3f350 0x55d4211e3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48248/0 (socket says 172.21.15.67:48248) 2024-07-27T03:18:18.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.144+0000 7fb18659c640 1 -- 172.21.15.67:0/3839739235 learned_addr learned my addr 172.21.15.67:0/3839739235 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.144+0000 7fb185d9b640 1 --2- 172.21.15.67:0/3839739235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d420e16490 0x55d4211e4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.144+0000 7fb186d9d640 1 --2- 172.21.15.67:0/3839739235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d420e1e670 0x55d4211e4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.144+0000 7fb18659c640 1 -- 172.21.15.67:0/3839739235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d420e1e670 msgr2=0x55d4211e4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.144+0000 7fb18659c640 1 --2- 172.21.15.67:0/3839739235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d420e1e670 0x55d4211e4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.144+0000 7fb18659c640 1 -- 172.21.15.67:0/3839739235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d420e16490 msgr2=0x55d4211e4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.144+0000 7fb18659c640 1 --2- 172.21.15.67:0/3839739235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d420e16490 0x55d4211e4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.144+0000 7fb18659c640 1 -- 172.21.15.67:0/3839739235 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d420eaa930 con 0x55d420e3f350 2024-07-27T03:18:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.145+0000 7fb18659c640 1 --2- 172.21.15.67:0/3839739235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e3f350 0x55d4211e3e10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb1780aa010 tx=0x7fb1780aaa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76f6f58ec347a42 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.145+0000 7fb174ff9640 1 -- 172.21.15.67:0/3839739235 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb178020040 con 0x55d420e3f350 2024-07-27T03:18:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.145+0000 7fb174ff9640 1 -- 172.21.15.67:0/3839739235 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1780ae910 con 0x55d420e3f350 2024-07-27T03:18:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.145+0000 7fb174ff9640 1 -- 172.21.15.67:0/3839739235 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1780a6000 con 0x55d420e3f350 2024-07-27T03:18:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.146+0000 7fb1931b2740 1 -- 172.21.15.67:0/3839739235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e3f350 msgr2=0x55d4211e3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.146+0000 7fb1931b2740 1 --2- 172.21.15.67:0/3839739235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e3f350 0x55d4211e3e10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb1780aa010 tx=0x7fb1780aaa10 comp rx=0 tx=0).stop 2024-07-27T03:18:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.146+0000 7fb1931b2740 1 -- 172.21.15.67:0/3839739235 shutdown_connections 2024-07-27T03:18:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.146+0000 7fb1931b2740 1 --2- 172.21.15.67:0/3839739235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d420e1e670 0x55d4211e4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.146+0000 7fb1931b2740 1 --2- 172.21.15.67:0/3839739235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d420e16490 0x55d4211e4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.146+0000 7fb1931b2740 1 --2- 172.21.15.67:0/3839739235 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e3f350 0x55d4211e3e10 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.146+0000 7fb1931b2740 1 -- 172.21.15.67:0/3839739235 >> 172.21.15.67:0/3839739235 conn(0x55d420e33740 msgr2=0x55d4211e1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.146+0000 7fb1931b2740 1 -- 172.21.15.67:0/3839739235 shutdown_connections 2024-07-27T03:18:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.146+0000 7fb1931b2740 1 -- 172.21.15.67:0/3839739235 wait complete. 2024-07-27T03:18:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.147+0000 7fb1931b2740 1 -- start start 2024-07-27T03:18:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.147+0000 7fb1931b2740 1 Processor -- start 2024-07-27T03:18:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.147+0000 7fb1931b2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d420e16490 0x55d4211efa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.147+0000 7fb1931b2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e1e670 0x55d4211eff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.147+0000 7fb1931b2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d420e3f350 0x55d4211fb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.147+0000 7fb1931b2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d420fad890 con 0x55d420e1e670 2024-07-27T03:18:18.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.147+0000 7fb1931b2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d4210ec580 con 0x55d420e16490 2024-07-27T03:18:18.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.147+0000 7fb1931b2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d4211f2590 con 0x55d420e3f350 2024-07-27T03:18:18.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.147+0000 7fb1931b2740 10 client.0 _ll_register_callbacks cb 0x55d421143f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:18.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.148+0000 7fb185d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e1e670 0x55d4211eff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:18.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.148+0000 7fb18659c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d420e16490 0x55d4211efa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:18.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.148+0000 7fb185d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e1e670 0x55d4211eff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48254/0 (socket says 172.21.15.67:48254) 2024-07-27T03:18:18.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.148+0000 7fb185d9b640 1 -- 172.21.15.67:0/1540605315 learned_addr learned my addr 172.21.15.67:0/1540605315 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:18.154 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.148+0000 7fb186d9d640 1 --2- 172.21.15.67:0/1540605315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d420e3f350 0x55d4211fb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:18.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.148+0000 7fb185d9b640 1 -- 172.21.15.67:0/1540605315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d420e3f350 msgr2=0x55d4211fb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:18.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.148+0000 7fb185d9b640 1 --2- 172.21.15.67:0/1540605315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d420e3f350 0x55d4211fb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.148+0000 7fb185d9b640 1 -- 172.21.15.67:0/1540605315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d420e16490 msgr2=0x55d4211efa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:18.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.148+0000 7fb185d9b640 1 --2- 172.21.15.67:0/1540605315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d420e16490 0x55d4211efa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.148+0000 7fb185d9b640 1 -- 172.21.15.67:0/1540605315 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4211fbea0 con 0x55d420e1e670 2024-07-27T03:18:18.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.148+0000 7fb185d9b640 1 --2- 172.21.15.67:0/1540605315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e1e670 0x55d4211eff40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb17c009900 tx=0x7fb17c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c07d0c91a1be257 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:18.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.148+0000 7fb16dffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:18.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.148+0000 7fb16dffb640 1 -- 172.21.15.67:0/1540605315 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c04d070 con 0x55d420e1e670 2024-07-27T03:18:18.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.148+0000 7fb16dffb640 1 -- 172.21.15.67:0/1540605315 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fb17c004250 con 0x55d420e1e670 2024-07-27T03:18:18.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.148+0000 7fb1931b2740 1 -- 172.21.15.67:0/1540605315 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1780ae910 con 0x55d420e1e670 2024-07-27T03:18:18.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.149+0000 7fb16dffb640 1 -- 172.21.15.67:0/1540605315 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb17c059050 con 0x55d420e1e670 2024-07-27T03:18:18.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.149+0000 7fb1931b2740 10 client.15789 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:18.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.149+0000 7fb1931b2740 1 -- 172.21.15.67:0/1540605315 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d4211fe7b0 con 0x55d420e1e670 2024-07-27T03:18:18.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.149+0000 7fb16dffb640 1 -- 172.21.15.67:0/1540605315 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1780ae910 con 0x55d420e1e670 2024-07-27T03:18:18.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.149+0000 7fb16dffb640 1 -- 172.21.15.67:0/1540605315 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb17c004b70 con 0x55d420e1e670 2024-07-27T03:18:18.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.149+0000 7fb1931b2740 10 client.15789 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:18.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.149+0000 7fb1931b2740 10 client.15789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:18:18.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.149+0000 7fb1931b2740 20 client.15789 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:18.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.149+0000 7fb1931b2740 1 --2- 172.21.15.67:0/1540605315 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55d420dfdcd0 0x55d421201d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:18.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.149+0000 7fb1931b2740 4 client.15789 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:18:18.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.149+0000 7fb1931b2740 1 -- 172.21.15.67:0/1540605315 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d42115c9b0 con 0x55d420dfdcd0 2024-07-27T03:18:18.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.150+0000 7fb18659c640 1 --2- 172.21.15.67:0/1540605315 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55d420dfdcd0 0x55d421201d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:18.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.150+0000 7fb18659c640 1 --2- 172.21.15.67:0/1540605315 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55d420dfdcd0 0x55d421201d00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7205085988be00d server_cookie=1c6d336b98e18008 in_seq=0 out_seq=0 2024-07-27T03:18:18.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.150+0000 7fb16dffb640 10 client.15789 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:18.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.151+0000 7fb16dffb640 1 -- 172.21.15.67:0/1540605315 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d42115c9b0 con 0x55d420dfdcd0 2024-07-27T03:18:18.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.151+0000 7fb16dffb640 10 client.15789 handle_command_reply: tid=0 2024-07-27T03:18:18.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.151+0000 7fb1931b2740 1 client.15789 shutdown 2024-07-27T03:18:18.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.151+0000 7fb1931b2740 1 -- 172.21.15.67:0/1540605315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e1e670 msgr2=0x55d4211eff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:18.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.151+0000 7fb1931b2740 1 --2- 172.21.15.67:0/1540605315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e1e670 0x55d4211eff40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb17c009900 tx=0x7fb17c009260 comp rx=0 tx=0).stop 2024-07-27T03:18:18.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.151+0000 7fb1931b2740 1 -- 172.21.15.67:0/1540605315 shutdown_connections 2024-07-27T03:18:18.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.151+0000 7fb1931b2740 1 --2- 172.21.15.67:0/1540605315 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55d420dfdcd0 0x55d421201d00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.151+0000 7fb1931b2740 1 --2- 172.21.15.67:0/1540605315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d420e3f350 0x55d4211fb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.151+0000 7fb1931b2740 1 --2- 172.21.15.67:0/1540605315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e1e670 0x55d4211eff40 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.151+0000 7fb1931b2740 1 --2- 172.21.15.67:0/1540605315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d420e16490 0x55d4211efa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.151+0000 7fb16dffb640 0 client.15789 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:18.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.151+0000 7fb1931b2740 1 -- 172.21.15.67:0/1540605315 >> 172.21.15.67:0/1540605315 conn(0x55d420e33740 msgr2=0x55d4211e6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:18.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.151+0000 7fb1931b2740 1 -- 172.21.15.67:0/1540605315 shutdown_connections 2024-07-27T03:18:18.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.151+0000 7fb1931b2740 1 -- 172.21.15.67:0/1540605315 wait complete. 2024-07-27T03:18:18.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.151+0000 7fb1931b2740 20 client.15789 trim_cache size 0 max 16384 2024-07-27T03:18:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.169+0000 7fb1931b2740 1 Processor -- start 2024-07-27T03:18:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.169+0000 7fb1931b2740 1 -- start start 2024-07-27T03:18:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.169+0000 7fb1931b2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d420e16490 0x55d4211e9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.169+0000 7fb1931b2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e1e670 0x55d4211e1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.169+0000 7fb1931b2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d420e3f350 0x55d4211fb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.169+0000 7fb1931b2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb16800fc20 con 0x55d420e1e670 2024-07-27T03:18:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.169+0000 7fb1931b2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb168011760 con 0x55d420e16490 2024-07-27T03:18:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.169+0000 7fb1931b2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb1680102f0 con 0x55d420e3f350 2024-07-27T03:18:18.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.169+0000 7fb185d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e1e670 0x55d4211e1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:18.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.170+0000 7fb185d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e1e670 0x55d4211e1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48260/0 (socket says 172.21.15.67:48260) 2024-07-27T03:18:18.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.170+0000 7fb185d9b640 1 -- 172.21.15.67:0/223976352 learned_addr learned my addr 172.21.15.67:0/223976352 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:18.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.170+0000 7fb18659c640 1 --2- 172.21.15.67:0/223976352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d420e16490 0x55d4211e9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:18.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.170+0000 7fb186d9d640 1 --2- 172.21.15.67:0/223976352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d420e3f350 0x55d4211fb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:18.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.170+0000 7fb185d9b640 1 -- 172.21.15.67:0/223976352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d420e3f350 msgr2=0x55d4211fb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:18.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.170+0000 7fb185d9b640 1 --2- 172.21.15.67:0/223976352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d420e3f350 0x55d4211fb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.170+0000 7fb185d9b640 1 -- 172.21.15.67:0/223976352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d420e16490 msgr2=0x55d4211e9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:18.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.170+0000 7fb185d9b640 1 --2- 172.21.15.67:0/223976352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d420e16490 0x55d4211e9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.170+0000 7fb185d9b640 1 -- 172.21.15.67:0/223976352 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4211e6130 con 0x55d420e1e670 2024-07-27T03:18:18.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.170+0000 7fb185d9b640 1 --2- 172.21.15.67:0/223976352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e1e670 0x55d4211e1c40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb17c02a930 tx=0x7fb17c032f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e1c2e1c7fd02d5 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:18.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.170+0000 7fb16ffff640 1 -- 172.21.15.67:0/223976352 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c061020 con 0x55d420e1e670 2024-07-27T03:18:18.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.170+0000 7fb16ffff640 1 -- 172.21.15.67:0/223976352 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb17c004450 con 0x55d420e1e670 2024-07-27T03:18:18.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.170+0000 7fb16ffff640 1 -- 172.21.15.67:0/223976352 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c04d040 con 0x55d420e1e670 2024-07-27T03:18:18.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.172+0000 7fb1931b2740 1 -- 172.21.15.67:0/223976352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e1e670 msgr2=0x55d4211e1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:18.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.172+0000 7fb1931b2740 1 --2- 172.21.15.67:0/223976352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e1e670 0x55d4211e1c40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb17c02a930 tx=0x7fb17c032f70 comp rx=0 tx=0).stop 2024-07-27T03:18:18.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.172+0000 7fb1931b2740 1 -- 172.21.15.67:0/223976352 shutdown_connections 2024-07-27T03:18:18.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.172+0000 7fb1931b2740 1 --2- 172.21.15.67:0/223976352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d420e3f350 0x55d4211fb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.172+0000 7fb1931b2740 1 --2- 172.21.15.67:0/223976352 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e1e670 0x55d4211e1c40 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.172+0000 7fb1931b2740 1 --2- 172.21.15.67:0/223976352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d420e16490 0x55d4211e9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.172+0000 7fb1931b2740 1 -- 172.21.15.67:0/223976352 >> 172.21.15.67:0/223976352 conn(0x55d420e33740 msgr2=0x55d4211e1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:18.177 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.172+0000 7fb1931b2740 1 -- 172.21.15.67:0/223976352 shutdown_connections 2024-07-27T03:18:18.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.172+0000 7fb1931b2740 1 -- 172.21.15.67:0/223976352 wait complete. 2024-07-27T03:18:18.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.173+0000 7fb1931b2740 1 -- start start 2024-07-27T03:18:18.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.173+0000 7fb1931b2740 1 Processor -- start 2024-07-27T03:18:18.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.173+0000 7fb1931b2740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d420e16490 0x55d4211f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:18.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.173+0000 7fb1931b2740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e1e670 0x55d4211f4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:18.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.173+0000 7fb1931b2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d420e3f350 0x55d421263560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:18.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.173+0000 7fb1931b2740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb168011090 con 0x55d420e1e670 2024-07-27T03:18:18.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.173+0000 7fb1931b2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb1680109c0 con 0x55d420e3f350 2024-07-27T03:18:18.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.173+0000 7fb1931b2740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb16800dc00 con 0x55d420e16490 2024-07-27T03:18:18.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.173+0000 7fb185d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e1e670 0x55d4211f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:18.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.173+0000 7fb185d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e1e670 0x55d4211f4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48262/0 (socket says 172.21.15.67:48262) 2024-07-27T03:18:18.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.173+0000 7fb185d9b640 1 -- 172.21.15.67:0/3901076436 learned_addr learned my addr 172.21.15.67:0/3901076436 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:18.178 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.174+0000 7fb186d9d640 1 --2- 172.21.15.67:0/3901076436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d420e3f350 0x55d421263560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:18.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.174+0000 7fb1931b2740 10 client.0 _ll_register_callbacks cb 0x7fb17c040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:18.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.174+0000 7fb18659c640 1 --2- 172.21.15.67:0/3901076436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d420e16490 0x55d4211f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:18.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.174+0000 7fb185d9b640 1 -- 172.21.15.67:0/3901076436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d420e16490 msgr2=0x55d4211f41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:18.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.174+0000 7fb185d9b640 1 --2- 172.21.15.67:0/3901076436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d420e16490 0x55d4211f41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.174+0000 7fb185d9b640 1 -- 172.21.15.67:0/3901076436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d420e3f350 msgr2=0x55d421263560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:18.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.174+0000 7fb185d9b640 1 --2- 172.21.15.67:0/3901076436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d420e3f350 0x55d421263560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.174+0000 7fb185d9b640 1 -- 172.21.15.67:0/3901076436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d421263bd0 con 0x55d420e1e670 2024-07-27T03:18:18.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.174+0000 7fb18659c640 1 --2- 172.21.15.67:0/3901076436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d420e16490 0x55d4211f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:18:18.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.174+0000 7fb185d9b640 1 --2- 172.21.15.67:0/3901076436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e1e670 0x55d4211f4790 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0092a0 tx=0x7fb17c0048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1dfd8d176abab4c server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:18.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.174+0000 7fb16dffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:18.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.174+0000 7fb16dffb640 1 -- 172.21.15.67:0/3901076436 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c06a070 con 0x55d420e1e670 2024-07-27T03:18:18.179 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.174+0000 7fb16dffb640 1 -- 172.21.15.67:0/3901076436 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fb17c003a30 con 0x55d420e1e670 2024-07-27T03:18:18.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.174+0000 7fb1931b2740 1 -- 172.21.15.67:0/3901076436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb17c004450 con 0x55d420e1e670 2024-07-27T03:18:18.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.175+0000 7fb16dffb640 1 -- 172.21.15.67:0/3901076436 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb17c076050 con 0x55d420e1e670 2024-07-27T03:18:18.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.175+0000 7fb1931b2740 10 client.15795 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:18.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.175+0000 7fb1931b2740 1 -- 172.21.15.67:0/3901076436 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d42126afc0 con 0x55d420e1e670 2024-07-27T03:18:18.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.175+0000 7fb16dffb640 1 -- 172.21.15.67:0/3901076436 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb17c004450 con 0x55d420e1e670 2024-07-27T03:18:18.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.175+0000 7fb16dffb640 1 -- 172.21.15.67:0/3901076436 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb17c049450 con 0x55d420e1e670 2024-07-27T03:18:18.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.175+0000 7fb1931b2740 10 client.15795 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:18.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.175+0000 7fb1931b2740 10 client.15795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:18:18.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.175+0000 7fb1931b2740 20 client.15795 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:18.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.175+0000 7fb1931b2740 1 --2- 172.21.15.67:0/3901076436 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55d420dd3080 0x55d42126d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:18.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.175+0000 7fb1931b2740 4 client.15795 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-27T03:18:18.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.175+0000 7fb1931b2740 1 -- 172.21.15.67:0/3901076436 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d421158450 con 0x55d420dd3080 2024-07-27T03:18:18.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.176+0000 7fb18659c640 1 --2- 172.21.15.67:0/3901076436 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55d420dd3080 0x55d42126d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:18.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.176+0000 7fb16dffb640 10 client.15795 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:18.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.176+0000 7fb18659c640 1 --2- 172.21.15.67:0/3901076436 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55d420dd3080 0x55d42126d5e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7b7ccecc900da5 server_cookie=3f27ae0a8f88b443 in_seq=0 out_seq=0 2024-07-27T03:18:18.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.177+0000 7fb16dffb640 1 -- 172.21.15.67:0/3901076436 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d421158450 con 0x55d420dd3080 2024-07-27T03:18:18.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.177+0000 7fb16dffb640 10 client.15795 handle_command_reply: tid=0 2024-07-27T03:18:18.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.177+0000 7fb1931b2740 1 client.15795 shutdown 2024-07-27T03:18:18.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.177+0000 7fb1931b2740 1 -- 172.21.15.67:0/3901076436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e1e670 msgr2=0x55d4211f4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:18.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.177+0000 7fb1931b2740 1 --2- 172.21.15.67:0/3901076436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e1e670 0x55d4211f4790 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0092a0 tx=0x7fb17c0048d0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.178+0000 7fb1931b2740 1 -- 172.21.15.67:0/3901076436 shutdown_connections 2024-07-27T03:18:18.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.178+0000 7fb1931b2740 1 --2- 172.21.15.67:0/3901076436 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55d420dd3080 0x55d42126d5e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.178+0000 7fb1931b2740 1 --2- 172.21.15.67:0/3901076436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d420e3f350 0x55d421263560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.178+0000 7fb16dffb640 0 client.15795 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:18.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.178+0000 7fb1931b2740 1 --2- 172.21.15.67:0/3901076436 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d420e1e670 0x55d4211f4790 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.178+0000 7fb1931b2740 1 --2- 172.21.15.67:0/3901076436 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d420e16490 0x55d4211f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.178+0000 7fb1931b2740 1 -- 172.21.15.67:0/3901076436 >> 172.21.15.67:0/3901076436 conn(0x55d420e33740 msgr2=0x55d4211e02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:18.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.178+0000 7fb1931b2740 1 -- 172.21.15.67:0/3901076436 shutdown_connections 2024-07-27T03:18:18.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.178+0000 7fb1931b2740 1 -- 172.21.15.67:0/3901076436 wait complete. 2024-07-27T03:18:18.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.178+0000 7fb1931b2740 20 client.15795 trim_cache size 0 max 16384 2024-07-27T03:18:18.184 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:18:18.184 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-27T03:18:18.184 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "0fea49e7-6c4a-4150-ae8f-78825d83ddcb", 2024-07-27T03:18:18.184 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-27T03:18:18.184 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:18:18.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.179+0000 7fb18cc2d640 1 -- 172.21.15.67:0/2328778496 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb1700b3c50 msgr2=0x7fb1700b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:18.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.179+0000 7fb18cc2d640 1 --2- 172.21.15.67:0/2328778496 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb1700b3c50 0x7fb1700b6160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb17c009230 tx=0x7fb17c031400 comp rx=0 tx=0).stop 2024-07-27T03:18:18.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.179+0000 7fb18cc2d640 1 -- 172.21.15.67:0/2328778496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb188103630 msgr2=0x7fb188075180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:18.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.179+0000 7fb18cc2d640 1 --2- 172.21.15.67:0/2328778496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb188103630 0x7fb188075180 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb17800d770 tx=0x7fb17800dc30 comp rx=0 tx=0).stop 2024-07-27T03:18:18.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.179+0000 7fb18cc2d640 1 -- 172.21.15.67:0/2328778496 shutdown_connections 2024-07-27T03:18:18.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.179+0000 7fb18cc2d640 1 --2- 172.21.15.67:0/2328778496 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb1700b3c50 0x7fb1700b6160 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.179+0000 7fb18cc2d640 1 --2- 172.21.15.67:0/2328778496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb188109870 0x7fb188075c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.179+0000 7fb18cc2d640 1 --2- 172.21.15.67:0/2328778496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb188108f20 0x7fb1880756c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.179+0000 7fb18cc2d640 1 --2- 172.21.15.67:0/2328778496 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb188103630 0x7fb188075180 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:18.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.179+0000 7fb18cc2d640 1 -- 172.21.15.67:0/2328778496 >> 172.21.15.67:0/2328778496 conn(0x7fb18807b380 msgr2=0x7fb188107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:18.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.179+0000 7fb18cc2d640 1 -- 172.21.15.67:0/2328778496 shutdown_connections 2024-07-27T03:18:18.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:18.179+0000 7fb18cc2d640 1 -- 172.21.15.67:0/2328778496 wait complete. 2024-07-27T03:18:18.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:18 smithi175 ceph-mon[37125]: pgmap v482: 129 pgs: 129 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 859 KiB/s wr, 12 op/s 2024-07-27T03:18:18.927 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:18 smithi067 ceph-mon[32814]: pgmap v482: 129 pgs: 129 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 859 KiB/s wr, 12 op/s 2024-07-27T03:18:18.932 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-27T03:18:18.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:18 smithi149 ceph-mon[37173]: pgmap v482: 129 pgs: 129 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 859 KiB/s wr, 12 op/s 2024-07-27T03:18:19.216 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:18:19.603 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:19 smithi067 ceph-mon[32814]: scrub queued for path: / 2024-07-27T03:18:19.603 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:19 smithi067 ceph-mon[32814]: scrub summary: idle+waiting paths [/] 2024-07-27T03:18:19.604 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:19 smithi067 ceph-mon[32814]: scrub summary: active paths [/] 2024-07-27T03:18:19.604 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:19 smithi067 ceph-mon[32814]: scrub summary: idle+waiting paths [/] 2024-07-27T03:18:19.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:19 smithi175 ceph-mon[37125]: scrub queued for path: / 2024-07-27T03:18:19.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:19 smithi175 ceph-mon[37125]: scrub summary: idle+waiting paths [/] 2024-07-27T03:18:19.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:19 smithi175 ceph-mon[37125]: scrub summary: active paths [/] 2024-07-27T03:18:19.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:19 smithi175 ceph-mon[37125]: scrub summary: idle+waiting paths [/] 2024-07-27T03:18:20.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:19 smithi149 ceph-mon[37173]: scrub queued for path: / 2024-07-27T03:18:20.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:19 smithi149 ceph-mon[37173]: scrub summary: idle+waiting paths [/] 2024-07-27T03:18:20.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:19 smithi149 ceph-mon[37173]: scrub summary: active paths [/] 2024-07-27T03:18:20.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:19 smithi149 ceph-mon[37173]: scrub summary: idle+waiting paths [/] 2024-07-27T03:18:20.647 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:20 smithi067 ceph-mon[32814]: pgmap v483: 129 pgs: 129 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 1.3 MiB/s wr, 11 op/s 2024-07-27T03:18:20.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:20 smithi175 ceph-mon[37125]: pgmap v483: 129 pgs: 129 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 1.3 MiB/s wr, 11 op/s 2024-07-27T03:18:20.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.901+0000 7f69a3764640 1 -- 172.21.15.67:0/2016222835 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f699c107e40 msgr2=0x7f699c108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:20.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.901+0000 7f69a3764640 1 --2- 172.21.15.67:0/2016222835 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f699c107e40 0x7f699c108250 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6998009900 tx=0x7f699802a410 comp rx=0 tx=0).stop 2024-07-27T03:18:20.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.901+0000 7f69a3764640 1 -- 172.21.15.67:0/2016222835 shutdown_connections 2024-07-27T03:18:20.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.901+0000 7f69a3764640 1 --2- 172.21.15.67:0/2016222835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f699c108790 0x7f699c10efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.901+0000 7f69a3764640 1 --2- 172.21.15.67:0/2016222835 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f699c107e40 0x7f699c108250 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.901+0000 7f69a3764640 1 --2- 172.21.15.67:0/2016222835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f699c102550 0x7f699c102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.901+0000 7f69a3764640 1 -- 172.21.15.67:0/2016222835 >> 172.21.15.67:0/2016222835 conn(0x7f699c079220 msgr2=0x7f699c079630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:20.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.901+0000 7f69a3764640 1 -- 172.21.15.67:0/2016222835 shutdown_connections 2024-07-27T03:18:20.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.901+0000 7f69a3764640 1 -- 172.21.15.67:0/2016222835 wait complete. 2024-07-27T03:18:20.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.902+0000 7f69a3764640 1 Processor -- start 2024-07-27T03:18:20.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.902+0000 7f69a3764640 1 -- start start 2024-07-27T03:18:20.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.903+0000 7f69a3764640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f699c102550 0x7f699c19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:20.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.903+0000 7f69a3764640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f699c107e40 0x7f699c19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:20.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.903+0000 7f69a3764640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f699c108790 0x7f699c19f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:20.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.903+0000 7f69a3764640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f699c112400 con 0x7f699c107e40 2024-07-27T03:18:20.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.903+0000 7f69a3764640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f699c112280 con 0x7f699c108790 2024-07-27T03:18:20.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.903+0000 7f69a3764640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f699c112580 con 0x7f699c102550 2024-07-27T03:18:20.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.903+0000 7f69a0cff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f699c107e40 0x7f699c19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:20.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.903+0000 7f69a0cff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f699c107e40 0x7f699c19a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48270/0 (socket says 172.21.15.67:48270) 2024-07-27T03:18:20.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.903+0000 7f69a0cff640 1 -- 172.21.15.67:0/2794101915 learned_addr learned my addr 172.21.15.67:0/2794101915 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:20.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.903+0000 7f69a1500640 1 --2- 172.21.15.67:0/2794101915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f699c102550 0x7f699c19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:20.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.903+0000 7f69a1d01640 1 --2- 172.21.15.67:0/2794101915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f699c108790 0x7f699c19f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:20.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.903+0000 7f69a0cff640 1 -- 172.21.15.67:0/2794101915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f699c102550 msgr2=0x7f699c19a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:20.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.903+0000 7f69a0cff640 1 --2- 172.21.15.67:0/2794101915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f699c102550 0x7f699c19a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.903+0000 7f69a0cff640 1 -- 172.21.15.67:0/2794101915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f699c108790 msgr2=0x7f699c19f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:20.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.903+0000 7f69a0cff640 1 --2- 172.21.15.67:0/2794101915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f699c108790 0x7f699c19f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.903+0000 7f69a0cff640 1 -- 172.21.15.67:0/2794101915 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699c1a0050 con 0x7f699c107e40 2024-07-27T03:18:20.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.903+0000 7f69a1500640 1 --2- 172.21.15.67:0/2794101915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f699c102550 0x7f699c19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:18:20.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.904+0000 7f69a0cff640 1 --2- 172.21.15.67:0/2794101915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f699c107e40 0x7f699c19a680 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f699802a930 tx=0x7f6998004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:20.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.904+0000 7f698e7fc640 1 -- 172.21.15.67:0/2794101915 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699803f030 con 0x7f699c107e40 2024-07-27T03:18:20.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.904+0000 7f698e7fc640 1 -- 172.21.15.67:0/2794101915 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6998003aa0 con 0x7f699c107e40 2024-07-27T03:18:20.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.904+0000 7f69a3764640 1 -- 172.21.15.67:0/2794101915 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699c1a02e0 con 0x7f699c107e40 2024-07-27T03:18:20.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.904+0000 7f698e7fc640 1 -- 172.21.15.67:0/2794101915 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6998035070 con 0x7f699c107e40 2024-07-27T03:18:20.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.904+0000 7f69a3764640 1 -- 172.21.15.67:0/2794101915 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f699c1a6aa0 con 0x7f699c107e40 2024-07-27T03:18:20.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.907+0000 7f69a9ce9740 1 Processor -- start 2024-07-27T03:18:20.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.907+0000 7f69a9ce9740 1 -- start start 2024-07-27T03:18:20.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.907+0000 7f698e7fc640 1 -- 172.21.15.67:0/2794101915 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f6998042030 con 0x7f699c107e40 2024-07-27T03:18:20.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.907+0000 7f69a9ce9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56118bf28070 0x56118c2ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:20.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.907+0000 7f69a9ce9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118beff1b0 0x56118c2cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:20.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.907+0000 7f69a9ce9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56118bf07390 0x56118c2cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:20.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.907+0000 7f69a9ce9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56118c1b53e0 con 0x56118beff1b0 2024-07-27T03:18:20.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.907+0000 7f69a9ce9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56118c243b00 con 0x56118bf07390 2024-07-27T03:18:20.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.907+0000 7f69a9ce9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56118c233000 con 0x56118bf28070 2024-07-27T03:18:20.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.907+0000 7f69a1500640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118beff1b0 0x56118c2cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:20.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.907+0000 7f69a1500640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118beff1b0 0x56118c2cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48272/0 (socket says 172.21.15.67:48272) 2024-07-27T03:18:20.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.907+0000 7f69a1500640 1 -- 172.21.15.67:0/2855569165 learned_addr learned my addr 172.21.15.67:0/2855569165 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:20.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.908+0000 7f69a0cff640 1 --2- 172.21.15.67:0/2855569165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56118bf07390 0x56118c2cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:20.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.908+0000 7f69a1d01640 1 --2- 172.21.15.67:0/2855569165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56118bf28070 0x56118c2ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:20.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.908+0000 7f69a1500640 1 -- 172.21.15.67:0/2855569165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56118bf28070 msgr2=0x56118c2ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:20.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.908+0000 7f69a1500640 1 --2- 172.21.15.67:0/2855569165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56118bf28070 0x56118c2ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.908+0000 7f69a1500640 1 -- 172.21.15.67:0/2855569165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56118bf07390 msgr2=0x56118c2cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:20.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.908+0000 7f69a1500640 1 --2- 172.21.15.67:0/2855569165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56118bf07390 0x56118c2cd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.916 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.908+0000 7f69a1500640 1 -- 172.21.15.67:0/2855569165 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56118c1c8750 con 0x56118beff1b0 2024-07-27T03:18:20.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.908+0000 7f69a1500640 1 --2- 172.21.15.67:0/2855569165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118beff1b0 0x56118c2cd260 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f699000cca0 tx=0x7f699000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3d34fee5d18702 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:20.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.908+0000 7f69877fe640 1 -- 172.21.15.67:0/2855569165 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990005220 con 0x56118beff1b0 2024-07-27T03:18:20.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.908+0000 7f69877fe640 1 -- 172.21.15.67:0/2855569165 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6990004c50 con 0x56118beff1b0 2024-07-27T03:18:20.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.908+0000 7f69877fe640 1 -- 172.21.15.67:0/2855569165 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69900107f0 con 0x56118beff1b0 2024-07-27T03:18:20.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.909+0000 7f698e7fc640 1 --2- 172.21.15.67:0/2794101915 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f69880b3bd0 0x7f69880b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:20.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.909+0000 7f69a1d01640 1 --2- 172.21.15.67:0/2794101915 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f69880b3bd0 0x7f69880b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:20.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.909+0000 7f698e7fc640 1 -- 172.21.15.67:0/2794101915 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f6998104570 con 0x7f699c107e40 2024-07-27T03:18:20.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.909+0000 7f69a1d01640 1 --2- 172.21.15.67:0/2794101915 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f69880b3bd0 0x7f69880b60e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6994005fd0 tx=0x7f69940096e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:20.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.909+0000 7f69a9ce9740 1 -- 172.21.15.67:0/2855569165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118beff1b0 msgr2=0x56118c2cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:20.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.909+0000 7f69a9ce9740 1 --2- 172.21.15.67:0/2855569165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118beff1b0 0x56118c2cd260 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f699000cca0 tx=0x7f699000e720 comp rx=0 tx=0).stop 2024-07-27T03:18:20.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.909+0000 7f69a9ce9740 1 -- 172.21.15.67:0/2855569165 shutdown_connections 2024-07-27T03:18:20.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.909+0000 7f69a9ce9740 1 --2- 172.21.15.67:0/2855569165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56118bf07390 0x56118c2cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.917 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.909+0000 7f69a9ce9740 1 --2- 172.21.15.67:0/2855569165 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118beff1b0 0x56118c2cd260 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.909+0000 7f69a9ce9740 1 --2- 172.21.15.67:0/2855569165 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56118bf28070 0x56118c2ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.910+0000 7f69a9ce9740 1 -- 172.21.15.67:0/2855569165 >> 172.21.15.67:0/2855569165 conn(0x56118bf1c460 msgr2=0x56118c2cac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:20.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.910+0000 7f69a9ce9740 1 -- 172.21.15.67:0/2855569165 shutdown_connections 2024-07-27T03:18:20.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.910+0000 7f69a9ce9740 1 -- 172.21.15.67:0/2855569165 wait complete. 2024-07-27T03:18:20.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.910+0000 7f69a9ce9740 1 -- start start 2024-07-27T03:18:20.918 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.910+0000 7f69a9ce9740 1 Processor -- start 2024-07-27T03:18:20.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.911+0000 7f69a9ce9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56118beff1b0 0x56118c2d8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:20.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.911+0000 7f69a9ce9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118bf07390 0x56118c2d8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:20.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.911+0000 7f69a9ce9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56118bf28070 0x56118c2e46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:20.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.911+0000 7f69a9ce9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56118c1d58e0 con 0x56118bf07390 2024-07-27T03:18:20.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.911+0000 7f69a9ce9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56118c237040 con 0x56118bf28070 2024-07-27T03:18:20.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.911+0000 7f69a9ce9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56118c096700 con 0x56118beff1b0 2024-07-27T03:18:20.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.911+0000 7f69a0cff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118bf07390 0x56118c2d8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:20.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.911+0000 7f69a0cff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118bf07390 0x56118c2d8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48274/0 (socket says 172.21.15.67:48274) 2024-07-27T03:18:20.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.911+0000 7f69a0cff640 1 -- 172.21.15.67:0/3561319150 learned_addr learned my addr 172.21.15.67:0/3561319150 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:20.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.911+0000 7f69a1500640 1 --2- 172.21.15.67:0/3561319150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56118beff1b0 0x56118c2d8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:20.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.911+0000 7f69a9ce9740 10 client.0 _ll_register_callbacks cb 0x56118c1ddd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:20.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.911+0000 7f69a1d01640 1 --2- 172.21.15.67:0/3561319150 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56118bf28070 0x56118c2e46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:20.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.911+0000 7f69a0cff640 1 -- 172.21.15.67:0/3561319150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56118beff1b0 msgr2=0x56118c2d8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:20.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.911+0000 7f69a0cff640 1 --2- 172.21.15.67:0/3561319150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56118beff1b0 0x56118c2d8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.911+0000 7f69a0cff640 1 -- 172.21.15.67:0/3561319150 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56118bf28070 msgr2=0x56118c2e46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:20.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.911+0000 7f69a0cff640 1 --2- 172.21.15.67:0/3561319150 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56118bf28070 0x56118c2e46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.911+0000 7f69a0cff640 1 -- 172.21.15.67:0/3561319150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56118c1d9bd0 con 0x56118bf07390 2024-07-27T03:18:20.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.912+0000 7f69a0cff640 1 --2- 172.21.15.67:0/3561319150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118bf07390 0x56118c2d8d90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6998031440 tx=0x7f6998009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67a4649b153ac4b server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:20.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.912+0000 7f6984ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:20.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.912+0000 7f6984ff9640 1 -- 172.21.15.67:0/3561319150 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6998049070 con 0x56118bf07390 2024-07-27T03:18:20.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.912+0000 7f6984ff9640 1 -- 172.21.15.67:0/3561319150 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f69980cf450 con 0x56118bf07390 2024-07-27T03:18:20.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.912+0000 7f69a9ce9740 1 -- 172.21.15.67:0/3561319150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6990004c50 con 0x56118bf07390 2024-07-27T03:18:20.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.912+0000 7f6984ff9640 1 -- 172.21.15.67:0/3561319150 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f69980caa60 con 0x56118bf07390 2024-07-27T03:18:20.923 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.912+0000 7f69a9ce9740 10 client.15807 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:20.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.912+0000 7f69a9ce9740 1 -- 172.21.15.67:0/3561319150 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56118bf93650 con 0x56118bf07390 2024-07-27T03:18:20.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.912+0000 7f6984ff9640 1 -- 172.21.15.67:0/3561319150 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6990004c50 con 0x56118bf07390 2024-07-27T03:18:20.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.913+0000 7f6984ff9640 1 -- 172.21.15.67:0/3561319150 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f69980c73a0 con 0x56118bf07390 2024-07-27T03:18:20.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.913+0000 7f69a9ce9740 10 client.15807 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:20.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.913+0000 7f69a9ce9740 10 client.15807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:18:20.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.913+0000 7f69a9ce9740 20 client.15807 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:20.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.913+0000 7f69a9ce9740 1 --2- 172.21.15.67:0/3561319150 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x56118bee69f0 0x56118c2ea6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:20.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.913+0000 7f69a9ce9740 4 client.15807 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:18:20.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.913+0000 7f69a9ce9740 1 -- 172.21.15.67:0/3561319150 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56118c245a50 con 0x56118bee69f0 2024-07-27T03:18:20.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.913+0000 7f69a1500640 1 --2- 172.21.15.67:0/3561319150 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x56118bee69f0 0x56118c2ea6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:20.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.914+0000 7f6984ff9640 10 client.15807 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:20.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.914+0000 7f69a1500640 1 --2- 172.21.15.67:0/3561319150 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x56118bee69f0 0x56118c2ea6d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddcdd8cc7374c49e server_cookie=91829245609a3964 in_seq=0 out_seq=0 2024-07-27T03:18:20.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.914+0000 7f6984ff9640 1 -- 172.21.15.67:0/3561319150 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56118c245a50 con 0x56118bee69f0 2024-07-27T03:18:20.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.914+0000 7f6984ff9640 10 client.15807 handle_command_reply: tid=0 2024-07-27T03:18:20.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.914+0000 7f69a9ce9740 1 client.15807 shutdown 2024-07-27T03:18:20.924 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.915+0000 7f69a9ce9740 1 -- 172.21.15.67:0/3561319150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118bf07390 msgr2=0x56118c2d8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:20.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.915+0000 7f69a9ce9740 1 --2- 172.21.15.67:0/3561319150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118bf07390 0x56118c2d8d90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6998031440 tx=0x7f6998009560 comp rx=0 tx=0).stop 2024-07-27T03:18:20.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.915+0000 7f69a9ce9740 1 -- 172.21.15.67:0/3561319150 shutdown_connections 2024-07-27T03:18:20.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.915+0000 7f69a9ce9740 1 --2- 172.21.15.67:0/3561319150 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x56118bee69f0 0x56118c2ea6d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.915+0000 7f69a9ce9740 1 --2- 172.21.15.67:0/3561319150 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56118bf28070 0x56118c2e46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.915+0000 7f69a9ce9740 1 --2- 172.21.15.67:0/3561319150 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118bf07390 0x56118c2d8d90 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.915+0000 7f69a9ce9740 1 --2- 172.21.15.67:0/3561319150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56118beff1b0 0x56118c2d8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.915+0000 7f69a9ce9740 1 -- 172.21.15.67:0/3561319150 >> 172.21.15.67:0/3561319150 conn(0x56118bf1c460 msgr2=0x56118c2cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:20.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.915+0000 7f6984ff9640 0 client.15807 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:20.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.915+0000 7f69a9ce9740 1 -- 172.21.15.67:0/3561319150 shutdown_connections 2024-07-27T03:18:20.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.915+0000 7f69a9ce9740 1 -- 172.21.15.67:0/3561319150 wait complete. 2024-07-27T03:18:20.925 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.915+0000 7f69a9ce9740 20 client.15807 trim_cache size 0 max 16384 2024-07-27T03:18:20.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.933+0000 7f69a9ce9740 1 Processor -- start 2024-07-27T03:18:20.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.933+0000 7f69a9ce9740 1 -- start start 2024-07-27T03:18:20.935 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.933+0000 7f69a9ce9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56118bf1c460 0x56118c2d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:20.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.933+0000 7f69a9ce9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118beff1b0 0x56118c2da280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:20.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.933+0000 7f69a9ce9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56118bf07390 0x56118c2e44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:20.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.933+0000 7f69a9ce9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1ae490 con 0x56118beff1b0 2024-07-27T03:18:20.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.933+0000 7f69a9ce9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1affd0 con 0x56118bf1c460 2024-07-27T03:18:20.936 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.933+0000 7f69a9ce9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1aeb60 con 0x56118bf07390 2024-07-27T03:18:20.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.933+0000 7f69a0cff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118beff1b0 0x56118c2da280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:20.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.934+0000 7f69a0cff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118beff1b0 0x56118c2da280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48280/0 (socket says 172.21.15.67:48280) 2024-07-27T03:18:20.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.934+0000 7f69a0cff640 1 -- 172.21.15.67:0/995215492 learned_addr learned my addr 172.21.15.67:0/995215492 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:20.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.934+0000 7f69a1500640 1 --2- 172.21.15.67:0/995215492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56118bf1c460 0x56118c2d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:20.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.934+0000 7f69a1d01640 1 --2- 172.21.15.67:0/995215492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56118bf07390 0x56118c2e44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:20.937 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.934+0000 7f69a0cff640 1 -- 172.21.15.67:0/995215492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56118bf07390 msgr2=0x56118c2e44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:20.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.934+0000 7f69a0cff640 1 --2- 172.21.15.67:0/995215492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56118bf07390 0x56118c2e44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.934+0000 7f69a0cff640 1 -- 172.21.15.67:0/995215492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56118bf1c460 msgr2=0x56118c2d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:20.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.934+0000 7f69a0cff640 1 --2- 172.21.15.67:0/995215492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56118bf1c460 0x56118c2d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.934+0000 7f69a0cff640 1 -- 172.21.15.67:0/995215492 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56118c2cf010 con 0x56118beff1b0 2024-07-27T03:18:20.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.934+0000 7f69a0cff640 1 --2- 172.21.15.67:0/995215492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118beff1b0 0x56118c2da280 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6998102030 tx=0x7f6998003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb74abfee3e98d4 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:20.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.934+0000 7f6986ffd640 1 -- 172.21.15.67:0/995215492 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6998049070 con 0x56118beff1b0 2024-07-27T03:18:20.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.934+0000 7f6986ffd640 1 -- 172.21.15.67:0/995215492 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6998103070 con 0x56118beff1b0 2024-07-27T03:18:20.938 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.934+0000 7f6986ffd640 1 -- 172.21.15.67:0/995215492 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69980ca880 con 0x56118beff1b0 2024-07-27T03:18:20.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.935+0000 7f69a9ce9740 1 -- 172.21.15.67:0/995215492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118beff1b0 msgr2=0x56118c2da280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:20.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.935+0000 7f69a9ce9740 1 --2- 172.21.15.67:0/995215492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118beff1b0 0x56118c2da280 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6998102030 tx=0x7f6998003f50 comp rx=0 tx=0).stop 2024-07-27T03:18:20.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.935+0000 7f69a9ce9740 1 -- 172.21.15.67:0/995215492 shutdown_connections 2024-07-27T03:18:20.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.935+0000 7f69a9ce9740 1 --2- 172.21.15.67:0/995215492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56118bf07390 0x56118c2e44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.935+0000 7f69a9ce9740 1 --2- 172.21.15.67:0/995215492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118beff1b0 0x56118c2da280 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.935+0000 7f69a9ce9740 1 --2- 172.21.15.67:0/995215492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56118bf1c460 0x56118c2d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.935+0000 7f69a9ce9740 1 -- 172.21.15.67:0/995215492 >> 172.21.15.67:0/995215492 conn(0x56118c2cabf0 msgr2=0x56118c2d9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:20.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.935+0000 7f69a9ce9740 1 -- 172.21.15.67:0/995215492 shutdown_connections 2024-07-27T03:18:20.941 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.936+0000 7f69a9ce9740 1 -- 172.21.15.67:0/995215492 wait complete. 2024-07-27T03:18:20.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.936+0000 7f69a9ce9740 1 -- start start 2024-07-27T03:18:20.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.936+0000 7f69a9ce9740 1 Processor -- start 2024-07-27T03:18:20.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.936+0000 7f69a9ce9740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118beff1b0 0x56118c2dc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:20.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.936+0000 7f69a9ce9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56118bf07390 0x56118c2dcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:20.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.937+0000 7f69a9ce9740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56118bf1c460 0x56118c2dd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:20.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.937+0000 7f69a9ce9740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1af900 con 0x56118beff1b0 2024-07-27T03:18:20.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.937+0000 7f69a9ce9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1af230 con 0x56118bf07390 2024-07-27T03:18:20.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.937+0000 7f69a9ce9740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1ac470 con 0x56118bf1c460 2024-07-27T03:18:20.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.937+0000 7f69a9ce9740 10 client.0 _ll_register_callbacks cb 0x7f6998005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:20.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.937+0000 7f69a1500640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118beff1b0 0x56118c2dc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:20.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.937+0000 7f69a1500640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118beff1b0 0x56118c2dc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48284/0 (socket says 172.21.15.67:48284) 2024-07-27T03:18:20.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.937+0000 7f69a1500640 1 -- 172.21.15.67:0/4144848091 learned_addr learned my addr 172.21.15.67:0/4144848091 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:20.942 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.937+0000 7f69a1d01640 1 --2- 172.21.15.67:0/4144848091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56118bf1c460 0x56118c2dd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:20.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.937+0000 7f69a0cff640 1 --2- 172.21.15.67:0/4144848091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56118bf07390 0x56118c2dcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:20.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.937+0000 7f69a1500640 1 -- 172.21.15.67:0/4144848091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56118bf1c460 msgr2=0x56118c2dd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:20.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.937+0000 7f69a1500640 1 --2- 172.21.15.67:0/4144848091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56118bf1c460 0x56118c2dd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.937+0000 7f69a1500640 1 -- 172.21.15.67:0/4144848091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56118bf07390 msgr2=0x56118c2dcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:20.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.937+0000 7f69a1500640 1 --2- 172.21.15.67:0/4144848091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56118bf07390 0x56118c2dcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.937+0000 7f69a1500640 1 -- 172.21.15.67:0/4144848091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56118c3520f0 con 0x56118beff1b0 2024-07-27T03:18:20.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.937+0000 7f69a1500640 1 --2- 172.21.15.67:0/4144848091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118beff1b0 0x56118c2dc730 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f699000ec20 tx=0x7f6990010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b861530519ab1e0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:20.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.937+0000 7f6984ff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:20.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.938+0000 7f6984ff9640 1 -- 172.21.15.67:0/4144848091 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699002b030 con 0x56118beff1b0 2024-07-27T03:18:20.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.938+0000 7f69a9ce9740 1 -- 172.21.15.67:0/4144848091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6998103070 con 0x56118beff1b0 2024-07-27T03:18:20.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.938+0000 7f6984ff9640 1 -- 172.21.15.67:0/4144848091 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f6990004020 con 0x56118beff1b0 2024-07-27T03:18:20.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.938+0000 7f6984ff9640 1 -- 172.21.15.67:0/4144848091 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6990004220 con 0x56118beff1b0 2024-07-27T03:18:20.943 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.938+0000 7f69a9ce9740 10 client.15813 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:20.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.938+0000 7f69a9ce9740 1 -- 172.21.15.67:0/4144848091 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56118c3533b0 con 0x56118beff1b0 2024-07-27T03:18:20.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.938+0000 7f6984ff9640 1 -- 172.21.15.67:0/4144848091 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6998103070 con 0x56118beff1b0 2024-07-27T03:18:20.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.938+0000 7f6984ff9640 1 -- 172.21.15.67:0/4144848091 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6990028400 con 0x56118beff1b0 2024-07-27T03:18:20.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.938+0000 7f69a9ce9740 10 client.15813 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:20.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.938+0000 7f69a9ce9740 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:18:20.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.938+0000 7f69a9ce9740 20 client.15813 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:20.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.938+0000 7f69a9ce9740 1 --2- 172.21.15.67:0/4144848091 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x56118bf28070 0x56118c355a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:20.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.939+0000 7f69a9ce9740 4 client.15813 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-27T03:18:20.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.939+0000 7f69a9ce9740 1 -- 172.21.15.67:0/4144848091 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56118c2414f0 con 0x56118bf28070 2024-07-27T03:18:20.944 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.939+0000 7f69a0cff640 1 --2- 172.21.15.67:0/4144848091 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x56118bf28070 0x56118c355a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:20.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.939+0000 7f6984ff9640 10 client.15813 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:20.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.939+0000 7f69a0cff640 1 --2- 172.21.15.67:0/4144848091 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x56118bf28070 0x56118c355a00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf637c023b39f1bb server_cookie=cb955be947382daa in_seq=0 out_seq=0 2024-07-27T03:18:20.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.939+0000 7f6984ff9640 1 -- 172.21.15.67:0/4144848091 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56118c2414f0 con 0x56118bf28070 2024-07-27T03:18:20.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.939+0000 7f6984ff9640 10 client.15813 handle_command_reply: tid=0 2024-07-27T03:18:20.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.939+0000 7f69a9ce9740 1 client.15813 shutdown 2024-07-27T03:18:20.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.940+0000 7f69a9ce9740 1 -- 172.21.15.67:0/4144848091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118beff1b0 msgr2=0x56118c2dc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:20.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.940+0000 7f69a9ce9740 1 --2- 172.21.15.67:0/4144848091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118beff1b0 0x56118c2dc730 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f699000ec20 tx=0x7f6990010430 comp rx=0 tx=0).stop 2024-07-27T03:18:20.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.940+0000 7f69a9ce9740 1 -- 172.21.15.67:0/4144848091 shutdown_connections 2024-07-27T03:18:20.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.940+0000 7f69a9ce9740 1 --2- 172.21.15.67:0/4144848091 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x56118bf28070 0x56118c355a00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.940+0000 7f69a9ce9740 1 --2- 172.21.15.67:0/4144848091 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56118bf1c460 0x56118c2dd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.940+0000 7f6984ff9640 0 client.15813 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:20.946 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.940+0000 7f69a9ce9740 1 --2- 172.21.15.67:0/4144848091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56118bf07390 0x56118c2dcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.940+0000 7f69a9ce9740 1 --2- 172.21.15.67:0/4144848091 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56118beff1b0 0x56118c2dc730 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.940+0000 7f69a9ce9740 1 -- 172.21.15.67:0/4144848091 >> 172.21.15.67:0/4144848091 conn(0x56118c2cabf0 msgr2=0x56118c2ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:20.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.940+0000 7f69a9ce9740 1 -- 172.21.15.67:0/4144848091 shutdown_connections 2024-07-27T03:18:20.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.940+0000 7f69a9ce9740 1 -- 172.21.15.67:0/4144848091 wait complete. 2024-07-27T03:18:20.947 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.940+0000 7f69a9ce9740 20 client.15813 trim_cache size 0 max 16384 2024-07-27T03:18:20.947 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:18:20.948 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-27T03:18:20.948 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-27T03:18:20.948 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:18:20.948 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.941+0000 7f69a3764640 1 -- 172.21.15.67:0/2794101915 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f69880b3bd0 msgr2=0x7f69880b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:20.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.941+0000 7f69a3764640 1 --2- 172.21.15.67:0/2794101915 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f69880b3bd0 0x7f69880b60e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6994005fd0 tx=0x7f69940096e0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.941+0000 7f69a3764640 1 -- 172.21.15.67:0/2794101915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f699c107e40 msgr2=0x7f699c19a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:20.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.941+0000 7f69a3764640 1 --2- 172.21.15.67:0/2794101915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f699c107e40 0x7f699c19a680 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f699802a930 tx=0x7f6998004270 comp rx=0 tx=0).stop 2024-07-27T03:18:20.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.941+0000 7f69a3764640 1 -- 172.21.15.67:0/2794101915 shutdown_connections 2024-07-27T03:18:20.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.941+0000 7f69a3764640 1 --2- 172.21.15.67:0/2794101915 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f69880b3bd0 0x7f69880b60e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.941+0000 7f69a3764640 1 --2- 172.21.15.67:0/2794101915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f699c108790 0x7f699c19f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.941+0000 7f69a3764640 1 --2- 172.21.15.67:0/2794101915 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f699c107e40 0x7f699c19a680 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.941+0000 7f69a3764640 1 --2- 172.21.15.67:0/2794101915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f699c102550 0x7f699c19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:20.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.941+0000 7f69a3764640 1 -- 172.21.15.67:0/2794101915 >> 172.21.15.67:0/2794101915 conn(0x7f699c079220 msgr2=0x7f699c06b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:20.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.941+0000 7f69a3764640 1 -- 172.21.15.67:0/2794101915 shutdown_connections 2024-07-27T03:18:20.949 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:20.941+0000 7f69a3764640 1 -- 172.21.15.67:0/2794101915 wait complete. 2024-07-27T03:18:21.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:20 smithi149 ceph-mon[37173]: pgmap v483: 129 pgs: 129 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 1.3 MiB/s wr, 11 op/s 2024-07-27T03:18:21.626 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-27T03:18:21.626 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:18:21.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:21 smithi175 ceph-mon[37125]: scrub summary: idle 2024-07-27T03:18:21.907 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:18:21.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:21 smithi067 ceph-mon[32814]: scrub summary: idle 2024-07-27T03:18:21.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:21 smithi149 ceph-mon[37173]: scrub summary: idle 2024-07-27T03:18:22.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:22 smithi067 ceph-mon[32814]: pgmap v484: 129 pgs: 129 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 862 KiB/s wr, 15 op/s 2024-07-27T03:18:22.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:22 smithi175 ceph-mon[37125]: pgmap v484: 129 pgs: 129 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 862 KiB/s wr, 15 op/s 2024-07-27T03:18:23.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:22 smithi149 ceph-mon[37173]: pgmap v484: 129 pgs: 129 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 862 KiB/s wr, 15 op/s 2024-07-27T03:18:23.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.639+0000 7f32e47fd640 1 -- 172.21.15.67:0/2608844163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32dc107e10 msgr2=0x7f32dc108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:23.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.639+0000 7f32e47fd640 1 --2- 172.21.15.67:0/2608844163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32dc107e10 0x7f32dc108220 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009900 tx=0x7f32d002a410 comp rx=0 tx=0).stop 2024-07-27T03:18:23.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.640+0000 7f32e47fd640 1 -- 172.21.15.67:0/2608844163 shutdown_connections 2024-07-27T03:18:23.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.640+0000 7f32e47fd640 1 --2- 172.21.15.67:0/2608844163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32dc108760 0x7f32dc10ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:23.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.640+0000 7f32e47fd640 1 --2- 172.21.15.67:0/2608844163 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32dc107e10 0x7f32dc108220 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:23.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.640+0000 7f32e47fd640 1 --2- 172.21.15.67:0/2608844163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32dc1024d0 0x7f32dc1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:23.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.640+0000 7f32e47fd640 1 -- 172.21.15.67:0/2608844163 >> 172.21.15.67:0/2608844163 conn(0x7f32dc0791a0 msgr2=0x7f32dc0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:23.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.640+0000 7f32e47fd640 1 -- 172.21.15.67:0/2608844163 shutdown_connections 2024-07-27T03:18:23.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.640+0000 7f32e47fd640 1 -- 172.21.15.67:0/2608844163 wait complete. 2024-07-27T03:18:23.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.641+0000 7f32e47fd640 1 Processor -- start 2024-07-27T03:18:23.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.641+0000 7f32e47fd640 1 -- start start 2024-07-27T03:18:23.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.641+0000 7f32e47fd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32dc1024d0 0x7f32dc199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:23.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.641+0000 7f32e47fd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32dc107e10 0x7f32dc19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:23.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.641+0000 7f32e47fd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32dc108760 0x7f32dc19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:23.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.641+0000 7f32e47fd640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc1123b0 con 0x7f32dc107e10 2024-07-27T03:18:23.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.641+0000 7f32e47fd640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc112230 con 0x7f32dc108760 2024-07-27T03:18:23.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.641+0000 7f32e47fd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32dc112530 con 0x7f32dc1024d0 2024-07-27T03:18:23.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.641+0000 7f32e1d98640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32dc107e10 0x7f32dc19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:23.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.641+0000 7f32e1d98640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32dc107e10 0x7f32dc19a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:48294/0 (socket says 172.21.15.67:48294) 2024-07-27T03:18:23.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.641+0000 7f32e1d98640 1 -- 172.21.15.67:0/2208876427 learned_addr learned my addr 172.21.15.67:0/2208876427 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:23.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.641+0000 7f32e2d9a640 1 --2- 172.21.15.67:0/2208876427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32dc108760 0x7f32dc19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:23.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.642+0000 7f32e2599640 1 --2- 172.21.15.67:0/2208876427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32dc1024d0 0x7f32dc199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:23.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.642+0000 7f32e1d98640 1 -- 172.21.15.67:0/2208876427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32dc1024d0 msgr2=0x7f32dc199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:23.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.642+0000 7f32e1d98640 1 --2- 172.21.15.67:0/2208876427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32dc1024d0 0x7f32dc199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:23.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.642+0000 7f32e1d98640 1 -- 172.21.15.67:0/2208876427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32dc108760 msgr2=0x7f32dc19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:23.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.642+0000 7f32e1d98640 1 --2- 172.21.15.67:0/2208876427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32dc108760 0x7f32dc19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:23.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.642+0000 7f32e1d98640 1 -- 172.21.15.67:0/2208876427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32dc19ff00 con 0x7f32dc107e10 2024-07-27T03:18:23.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.642+0000 7f32e1d98640 1 --2- 172.21.15.67:0/2208876427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32dc107e10 0x7f32dc19a530 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f32d0032fd0 tx=0x7f32d0031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:23.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.642+0000 7f32cf7fe640 1 -- 172.21.15.67:0/2208876427 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d003f030 con 0x7f32dc107e10 2024-07-27T03:18:23.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.642+0000 7f32e47fd640 1 -- 172.21.15.67:0/2208876427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32dc1a0190 con 0x7f32dc107e10 2024-07-27T03:18:23.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.642+0000 7f32e47fd640 1 -- 172.21.15.67:0/2208876427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32dc1a6950 con 0x7f32dc107e10 2024-07-27T03:18:23.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.642+0000 7f32cf7fe640 1 -- 172.21.15.67:0/2208876427 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32d0004490 con 0x7f32dc107e10 2024-07-27T03:18:23.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.642+0000 7f32cf7fe640 1 -- 172.21.15.67:0/2208876427 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0035070 con 0x7f32dc107e10 2024-07-27T03:18:23.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.644+0000 7f32e47fd640 1 -- 172.21.15.67:0/2208876427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32dc1028e0 con 0x7f32dc107e10 2024-07-27T03:18:23.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.645+0000 7f32cf7fe640 1 -- 172.21.15.67:0/2208876427 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f32d0042030 con 0x7f32dc107e10 2024-07-27T03:18:23.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.646+0000 7f32cf7fe640 1 --2- 172.21.15.67:0/2208876427 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f32c80b3bd0 0x7f32c80b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:23.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.647+0000 7f32e2599640 1 --2- 172.21.15.67:0/2208876427 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f32c80b3bd0 0x7f32c80b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:23.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.647+0000 7f32cf7fe640 1 -- 172.21.15.67:0/2208876427 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f32d0103fe0 con 0x7f32dc107e10 2024-07-27T03:18:23.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.647+0000 7f32e2599640 1 --2- 172.21.15.67:0/2208876427 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f32c80b3bd0 0x7f32c80b60e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f32d8009b40 tx=0x7f32d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:23.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.651+0000 7f32cf7fe640 1 -- 172.21.15.67:0/2208876427 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f32d00cb280 con 0x7f32dc107e10 2024-07-27T03:18:23.877 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.875+0000 7f32e47fd640 1 -- 172.21.15.67:0/2208876427 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32dc10b560 con 0x7f32dc107e10 2024-07-27T03:18:23.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.876+0000 7f32cf7fe640 1 -- 172.21.15.67:0/2208876427 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11983 (secure 0 0 0) 0x7f32d00cab80 con 0x7f32dc107e10 2024-07-27T03:18:23.879 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:18:23.879 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"btime":"2024-07-27T03:16:40:350577+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14679,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/252686369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":252686369},{"type":"v1","addr":"172.21.15.67:6835","nonce":252686369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/3358832762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3358832762},{"type":"v1","addr":"172.21.15.67:6837","nonce":3358832762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-27T03:12:34.320208+0000","modified":"2024-07-27T03:16:39.741149+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24620,"mds_1":24587,"mds_2":24583,"mds_3":24581,"mds_4":24563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14736":{"gid":14736,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/2162708317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2162708317},{"type":"v1","addr":"172.21.15.67:6839","nonce":2162708317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14748":{"gid":14748,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/795704244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":795704244},{"type":"v1","addr":"172.21.15.67:6841","nonce":795704244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/3868325215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3868325215},{"type":"v1","addr":"172.21.15.149:6833","nonce":3868325215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2462749027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2462749027},{"type":"v1","addr":"172.21.15.175:6835","nonce":2462749027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.149:6835/3730964806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3730964806},{"type":"v1","addr":"172.21.15.149:6835","nonce":3730964806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.175:6837/1304803502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1304803502},{"type":"v1","addr":"172.21.15.175:6837","nonce":1304803502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.175:6839/1670568853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1670568853},{"type":"v1","addr":"172.21.15.175:6839","nonce":1670568853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24587":{"gid":24587,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.149:6837/2932951845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2932951845},{"type":"v1","addr":"172.21.15.149:6837","nonce":2932951845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.149:6839/281746498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":281746498},{"type":"v1","addr":"172.21.15.149:6839","nonce":281746498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24620,"qdb_cluster":[24587,24583,24620,24581,24563]},"id":1}]} 2024-07-27T03:18:23.879 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 30 2024-07-27T03:18:23.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.878+0000 7f32e47fd640 1 -- 172.21.15.67:0/2208876427 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f32c80b3bd0 msgr2=0x7f32c80b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:23.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.878+0000 7f32e47fd640 1 --2- 172.21.15.67:0/2208876427 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f32c80b3bd0 0x7f32c80b60e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f32d8009b40 tx=0x7f32d8005f90 comp rx=0 tx=0).stop 2024-07-27T03:18:23.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.878+0000 7f32e47fd640 1 -- 172.21.15.67:0/2208876427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32dc107e10 msgr2=0x7f32dc19a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:23.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.878+0000 7f32e47fd640 1 --2- 172.21.15.67:0/2208876427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32dc107e10 0x7f32dc19a530 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f32d0032fd0 tx=0x7f32d0031a20 comp rx=0 tx=0).stop 2024-07-27T03:18:23.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.878+0000 7f32e47fd640 1 -- 172.21.15.67:0/2208876427 shutdown_connections 2024-07-27T03:18:23.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.878+0000 7f32e47fd640 1 --2- 172.21.15.67:0/2208876427 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f32c80b3bd0 0x7f32c80b60e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:23.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.878+0000 7f32e47fd640 1 --2- 172.21.15.67:0/2208876427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32dc108760 0x7f32dc19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:23.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.878+0000 7f32e47fd640 1 --2- 172.21.15.67:0/2208876427 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f32dc107e10 0x7f32dc19a530 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:23.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.878+0000 7f32e47fd640 1 --2- 172.21.15.67:0/2208876427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f32dc1024d0 0x7f32dc199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:23.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.878+0000 7f32e47fd640 1 -- 172.21.15.67:0/2208876427 >> 172.21.15.67:0/2208876427 conn(0x7f32dc0791a0 msgr2=0x7f32dc06ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:23.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.878+0000 7f32e47fd640 1 -- 172.21.15.67:0/2208876427 shutdown_connections 2024-07-27T03:18:23.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:23.878+0000 7f32e47fd640 1 -- 172.21.15.67:0/2208876427 wait complete. 2024-07-27T03:18:24.590 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-27T03:18:24.867 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:18:24.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:24 smithi175 ceph-mon[37125]: pgmap v485: 129 pgs: 129 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 859 KiB/s wr, 11 op/s 2024-07-27T03:18:24.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:24 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2208876427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:18:24.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:24 smithi067 ceph-mon[32814]: pgmap v485: 129 pgs: 129 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 859 KiB/s wr, 11 op/s 2024-07-27T03:18:24.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:24 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2208876427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:18:24.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:24 smithi149 ceph-mon[37173]: pgmap v485: 129 pgs: 129 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 859 KiB/s wr, 11 op/s 2024-07-27T03:18:25.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:24 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2208876427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:18:26.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:26 smithi175 ceph-mon[37125]: pgmap v486: 129 pgs: 129 active+clean; 1.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-07-27T03:18:26.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:26 smithi067 ceph-mon[32814]: pgmap v486: 129 pgs: 129 active+clean; 1.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-07-27T03:18:27.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:26 smithi149 ceph-mon[37173]: pgmap v486: 129 pgs: 129 active+clean; 1.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-07-27T03:18:28.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.691+0000 7f4d7464a640 1 -- 172.21.15.67:0/1837014287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d6c102540 msgr2=0x7f4d6c102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:28.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.691+0000 7f4d7464a640 1 --2- 172.21.15.67:0/1837014287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d6c102540 0x7f4d6c102950 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4d68009900 tx=0x7f4d6802a410 comp rx=0 tx=0).stop 2024-07-27T03:18:28.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.691+0000 7f4d7464a640 1 -- 172.21.15.67:0/1837014287 shutdown_connections 2024-07-27T03:18:28.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.691+0000 7f4d7464a640 1 --2- 172.21.15.67:0/1837014287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d6c108780 0x7f4d6c10efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.691+0000 7f4d7464a640 1 --2- 172.21.15.67:0/1837014287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d6c107e30 0x7f4d6c108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.691+0000 7f4d7464a640 1 --2- 172.21.15.67:0/1837014287 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d6c102540 0x7f4d6c102950 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.691+0000 7f4d7464a640 1 -- 172.21.15.67:0/1837014287 >> 172.21.15.67:0/1837014287 conn(0x7f4d6c079140 msgr2=0x7f4d6c079550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:28.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.691+0000 7f4d7464a640 1 -- 172.21.15.67:0/1837014287 shutdown_connections 2024-07-27T03:18:28.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.691+0000 7f4d7464a640 1 -- 172.21.15.67:0/1837014287 wait complete. 2024-07-27T03:18:28.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.692+0000 7f4d7464a640 1 Processor -- start 2024-07-27T03:18:28.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.692+0000 7f4d7464a640 1 -- start start 2024-07-27T03:18:28.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.692+0000 7f4d7464a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d6c102540 0x7f4d6c0ff110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:28.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.692+0000 7f4d7464a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d6c107e30 0x7f4d6c0ff650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:28.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.692+0000 7f4d7464a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d6c108780 0x7f4d6c0ffb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:28.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.692+0000 7f4d7464a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c112450 con 0x7f4d6c102540 2024-07-27T03:18:28.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.692+0000 7f4d7464a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1122d0 con 0x7f4d6c107e30 2024-07-27T03:18:28.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.692+0000 7f4d7464a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1125d0 con 0x7f4d6c108780 2024-07-27T03:18:28.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.692+0000 7f4d723e6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d6c102540 0x7f4d6c0ff110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:28.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.692+0000 7f4d723e6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d6c102540 0x7f4d6c0ff110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49372/0 (socket says 172.21.15.67:49372) 2024-07-27T03:18:28.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.692+0000 7f4d723e6640 1 -- 172.21.15.67:0/1346557576 learned_addr learned my addr 172.21.15.67:0/1346557576 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:28.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.692+0000 7f4d71be5640 1 --2- 172.21.15.67:0/1346557576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d6c107e30 0x7f4d6c0ff650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:28.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.692+0000 7f4d72be7640 1 --2- 172.21.15.67:0/1346557576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d6c108780 0x7f4d6c0ffb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:28.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.693+0000 7f4d723e6640 1 -- 172.21.15.67:0/1346557576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d6c108780 msgr2=0x7f4d6c0ffb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:28.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.693+0000 7f4d723e6640 1 --2- 172.21.15.67:0/1346557576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d6c108780 0x7f4d6c0ffb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.693+0000 7f4d723e6640 1 -- 172.21.15.67:0/1346557576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d6c107e30 msgr2=0x7f4d6c0ff650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:28.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.693+0000 7f4d723e6640 1 --2- 172.21.15.67:0/1346557576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d6c107e30 0x7f4d6c0ff650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.693+0000 7f4d723e6640 1 -- 172.21.15.67:0/1346557576 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6c1a8310 con 0x7f4d6c102540 2024-07-27T03:18:28.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.693+0000 7f4d72be7640 1 --2- 172.21.15.67:0/1346557576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d6c108780 0x7f4d6c0ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:18:28.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.693+0000 7f4d71be5640 1 --2- 172.21.15.67:0/1346557576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d6c107e30 0x7f4d6c0ff650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:18:28.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.693+0000 7f4d723e6640 1 --2- 172.21.15.67:0/1346557576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d6c102540 0x7f4d6c0ff110 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4d68005940 tx=0x7f4d680040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:28.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.693+0000 7f4d637fe640 1 -- 172.21.15.67:0/1346557576 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6803f030 con 0x7f4d6c102540 2024-07-27T03:18:28.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.693+0000 7f4d7464a640 1 -- 172.21.15.67:0/1346557576 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6c1a85a0 con 0x7f4d6c102540 2024-07-27T03:18:28.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.693+0000 7f4d637fe640 1 -- 172.21.15.67:0/1346557576 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d680044c0 con 0x7f4d6c102540 2024-07-27T03:18:28.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.693+0000 7f4d637fe640 1 -- 172.21.15.67:0/1346557576 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d68035070 con 0x7f4d6c102540 2024-07-27T03:18:28.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.693+0000 7f4d7464a640 1 -- 172.21.15.67:0/1346557576 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d6c1a8a80 con 0x7f4d6c102540 2024-07-27T03:18:28.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.696+0000 7f4d7abcf740 1 Processor -- start 2024-07-27T03:18:28.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.696+0000 7f4d7abcf740 1 -- start start 2024-07-27T03:18:28.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.697+0000 7f4d637fe640 1 -- 172.21.15.67:0/1346557576 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f4d68042030 con 0x7f4d6c102540 2024-07-27T03:18:28.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.697+0000 7f4d7abcf740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98eb070 0x55e3f9c8fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:28.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.697+0000 7f4d7abcf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3f98c21b0 0x55e3f9c90280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:28.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.697+0000 7f4d7abcf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f98ca390 0x55e3f9c907c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:28.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.697+0000 7f4d7abcf740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55e3f9b78400 con 0x55e3f98eb070 2024-07-27T03:18:28.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.697+0000 7f4d7abcf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e3f9c06b20 con 0x55e3f98ca390 2024-07-27T03:18:28.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.697+0000 7f4d7abcf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e3f9bf6020 con 0x55e3f98c21b0 2024-07-27T03:18:28.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.697+0000 7f4d72be7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98eb070 0x55e3f9c8fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:28.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.697+0000 7f4d723e6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3f98c21b0 0x55e3f9c90280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:28.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.697+0000 7f4d71be5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f98ca390 0x55e3f9c907c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:28.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.697+0000 7f4d72be7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98eb070 0x55e3f9c8fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49374/0 (socket says 172.21.15.67:49374) 2024-07-27T03:18:28.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.697+0000 7f4d72be7640 1 -- 172.21.15.67:0/2114298415 learned_addr learned my addr 172.21.15.67:0/2114298415 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:28.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.697+0000 7f4d72be7640 1 -- 172.21.15.67:0/2114298415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3f98c21b0 msgr2=0x55e3f9c90280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.698+0000 7f4d72be7640 1 --2- 172.21.15.67:0/2114298415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3f98c21b0 0x55e3f9c90280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.698+0000 7f4d72be7640 1 -- 172.21.15.67:0/2114298415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f98ca390 msgr2=0x55e3f9c907c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.698+0000 7f4d72be7640 1 --2- 172.21.15.67:0/2114298415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f98ca390 0x55e3f9c907c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.698+0000 7f4d72be7640 1 -- 172.21.15.67:0/2114298415 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3f9b8b770 con 0x55e3f98eb070 2024-07-27T03:18:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.698+0000 7f4d71be5640 1 --2- 172.21.15.67:0/2114298415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f98ca390 0x55e3f9c907c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:18:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.698+0000 7f4d72be7640 1 --2- 172.21.15.67:0/2114298415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98eb070 0x55e3f9c8fd40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4d64011790 tx=0x7f4d64011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c4566fb6ff6821 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.698+0000 7f4d60ff9640 1 -- 172.21.15.67:0/2114298415 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d64003c30 con 0x55e3f98eb070 2024-07-27T03:18:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.698+0000 7f4d637fe640 1 --2- 172.21.15.67:0/1346557576 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4d580b3bd0 0x7f4d580b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.698+0000 7f4d60ff9640 1 -- 172.21.15.67:0/2114298415 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d6400fe50 con 0x55e3f98eb070 2024-07-27T03:18:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.698+0000 7f4d723e6640 1 --2- 172.21.15.67:0/1346557576 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4d580b3bd0 0x7f4d580b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.698+0000 7f4d637fe640 1 -- 172.21.15.67:0/1346557576 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f4d68103f80 con 0x7f4d6c102540 2024-07-27T03:18:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.699+0000 7f4d60ff9640 1 -- 172.21.15.67:0/2114298415 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d640193e0 con 0x55e3f98eb070 2024-07-27T03:18:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.699+0000 7f4d7abcf740 1 -- 172.21.15.67:0/2114298415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98eb070 msgr2=0x55e3f9c8fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:28.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.700+0000 7f4d7abcf740 1 --2- 172.21.15.67:0/2114298415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98eb070 0x55e3f9c8fd40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4d64011790 tx=0x7f4d64011c50 comp rx=0 tx=0).stop 2024-07-27T03:18:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.699+0000 7f4d723e6640 1 --2- 172.21.15.67:0/1346557576 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4d580b3bd0 0x7f4d580b60e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55e3f9c8e9c0 tx=0x7f4d68107a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.700+0000 7f4d7abcf740 1 -- 172.21.15.67:0/2114298415 shutdown_connections 2024-07-27T03:18:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.700+0000 7f4d7abcf740 1 --2- 172.21.15.67:0/2114298415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f98ca390 0x55e3f9c907c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.700+0000 7f4d7abcf740 1 --2- 172.21.15.67:0/2114298415 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3f98c21b0 0x55e3f9c90280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.700+0000 7f4d7abcf740 1 --2- 172.21.15.67:0/2114298415 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98eb070 0x55e3f9c8fd40 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.700+0000 7f4d7abcf740 1 -- 172.21.15.67:0/2114298415 >> 172.21.15.67:0/2114298415 conn(0x55e3f98df460 msgr2=0x55e3f9c8dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.700+0000 7f4d7abcf740 1 -- 172.21.15.67:0/2114298415 shutdown_connections 2024-07-27T03:18:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.700+0000 7f4d7abcf740 1 -- 172.21.15.67:0/2114298415 wait complete. 2024-07-27T03:18:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.700+0000 7f4d7abcf740 1 -- start start 2024-07-27T03:18:28.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.701+0000 7f4d7abcf740 1 Processor -- start 2024-07-27T03:18:28.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.701+0000 7f4d7abcf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f98c21b0 0x55e3f9c9b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:28.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.701+0000 7f4d7abcf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3f98ca390 0x55e3f9c9bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:28.713 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.701+0000 7f4d7abcf740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98eb070 0x55e3f9ca7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:28.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.701+0000 7f4d7abcf740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55e3f9b98900 con 0x55e3f98eb070 2024-07-27T03:18:28.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.701+0000 7f4d7abcf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e3f9bfa060 con 0x55e3f98c21b0 2024-07-27T03:18:28.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.701+0000 7f4d7abcf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e3f9a59700 con 0x55e3f98ca390 2024-07-27T03:18:28.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.701+0000 7f4d7abcf740 10 client.0 _ll_register_callbacks cb 0x55e3f9ba0d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:28.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.701+0000 7f4d72be7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98eb070 0x55e3f9ca7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:28.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.701+0000 7f4d72be7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98eb070 0x55e3f9ca7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49376/0 (socket says 172.21.15.67:49376) 2024-07-27T03:18:28.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.701+0000 7f4d72be7640 1 -- 172.21.15.67:0/1178734748 learned_addr learned my addr 172.21.15.67:0/1178734748 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:28.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.701+0000 7f4d72be7640 1 --2- 172.21.15.67:0/1178734748 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f98c21b0 0x55e3f9c9b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:28.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.702+0000 7f4d71be5640 1 --2- 172.21.15.67:0/1178734748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3f98ca390 0x55e3f9c9bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:28.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.702+0000 7f4d72be7640 1 -- 172.21.15.67:0/1178734748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3f98ca390 msgr2=0x55e3f9c9bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:28.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.702+0000 7f4d72be7640 1 --2- 172.21.15.67:0/1178734748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3f98ca390 0x55e3f9c9bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.702+0000 7f4d72be7640 1 -- 172.21.15.67:0/1178734748 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f98c21b0 msgr2=0x55e3f9c9b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:28.714 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.702+0000 7f4d72be7640 1 --2- 172.21.15.67:0/1178734748 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f98c21b0 0x55e3f9c9b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.702+0000 7f4d72be7640 1 -- 172.21.15.67:0/1178734748 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3f9b9cbf0 con 0x55e3f98eb070 2024-07-27T03:18:28.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.702+0000 7f4d72be7640 1 --2- 172.21.15.67:0/1178734748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98eb070 0x55e3f9ca7740 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4d640115a0 tx=0x7f4d64005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f707ee22f72a8ea5 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:28.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.702+0000 7f4d55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:28.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.702+0000 7f4d55ffb640 1 -- 172.21.15.67:0/1178734748 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d64011260 con 0x55e3f98eb070 2024-07-27T03:18:28.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.702+0000 7f4d55ffb640 1 -- 172.21.15.67:0/1178734748 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f4d64003a30 con 0x55e3f98eb070 2024-07-27T03:18:28.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.702+0000 7f4d7abcf740 1 -- 172.21.15.67:0/1178734748 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d6400fe50 con 0x55e3f98eb070 2024-07-27T03:18:28.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.703+0000 7f4d55ffb640 1 -- 172.21.15.67:0/1178734748 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4d64003c30 con 0x55e3f98eb070 2024-07-27T03:18:28.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.703+0000 7f4d7abcf740 10 client.15831 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:28.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.703+0000 7f4d7abcf740 1 -- 172.21.15.67:0/1178734748 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e3f9956650 con 0x55e3f98eb070 2024-07-27T03:18:28.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.703+0000 7f4d55ffb640 1 -- 172.21.15.67:0/1178734748 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d6400fe50 con 0x55e3f98eb070 2024-07-27T03:18:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.703+0000 7f4d55ffb640 1 -- 172.21.15.67:0/1178734748 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4d64018070 con 0x55e3f98eb070 2024-07-27T03:18:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.703+0000 7f4d7abcf740 10 client.15831 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.703+0000 7f4d7abcf740 10 client.15831 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-07-27T03:18:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.703+0000 7f4d7abcf740 20 client.15831 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.703+0000 7f4d7abcf740 1 --2- 172.21.15.67:0/1178734748 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55e3f98a99f0 0x55e3f9cad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.703+0000 7f4d7abcf740 4 client.15831 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:18:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.703+0000 7f4d7abcf740 1 -- 172.21.15.67:0/1178734748 --> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3f9c08a70 con 0x55e3f98a99f0 2024-07-27T03:18:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.704+0000 7f4d71be5640 1 --2- 172.21.15.67:0/1178734748 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55e3f98a99f0 0x55e3f9cad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.704+0000 7f4d55ffb640 10 client.15831 ms_handle_connect on v2:172.21.15.149:6834/3730964806 2024-07-27T03:18:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.704+0000 7f4d71be5640 1 --2- 172.21.15.67:0/1178734748 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55e3f98a99f0 0x55e3f9cad520 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=37bf03395bde179a server_cookie=8297dc0d75829441 in_seq=0 out_seq=0 2024-07-27T03:18:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.705+0000 7f4d55ffb640 1 -- 172.21.15.67:0/1178734748 <== mds.4 v2:172.21.15.149:6834/3730964806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3f9c08a70 con 0x55e3f98a99f0 2024-07-27T03:18:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.705+0000 7f4d55ffb640 10 client.15831 handle_command_reply: tid=0 2024-07-27T03:18:28.716 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.705+0000 7f4d7abcf740 1 client.15831 shutdown 2024-07-27T03:18:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.705+0000 7f4d7abcf740 1 -- 172.21.15.67:0/1178734748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98eb070 msgr2=0x55e3f9ca7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.705+0000 7f4d7abcf740 1 --2- 172.21.15.67:0/1178734748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98eb070 0x55e3f9ca7740 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4d640115a0 tx=0x7f4d64005bd0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.705+0000 7f4d7abcf740 1 -- 172.21.15.67:0/1178734748 shutdown_connections 2024-07-27T03:18:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.705+0000 7f4d7abcf740 1 --2- 172.21.15.67:0/1178734748 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55e3f98a99f0 0x55e3f9cad520 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.705+0000 7f4d7abcf740 1 --2- 172.21.15.67:0/1178734748 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98eb070 0x55e3f9ca7740 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.705+0000 7f4d7abcf740 1 --2- 172.21.15.67:0/1178734748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3f98ca390 0x55e3f9c9bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.705+0000 7f4d55ffb640 0 client.15831 ms_handle_reset on v2:172.21.15.149:6834/3730964806 2024-07-27T03:18:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.705+0000 7f4d7abcf740 1 --2- 172.21.15.67:0/1178734748 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f98c21b0 0x55e3f9c9b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.705+0000 7f4d7abcf740 1 -- 172.21.15.67:0/1178734748 >> 172.21.15.67:0/1178734748 conn(0x55e3f98df460 msgr2=0x55e3f9c90d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.706+0000 7f4d7abcf740 1 -- 172.21.15.67:0/1178734748 shutdown_connections 2024-07-27T03:18:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.706+0000 7f4d7abcf740 1 -- 172.21.15.67:0/1178734748 wait complete. 2024-07-27T03:18:28.717 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.706+0000 7f4d7abcf740 20 client.15831 trim_cache size 0 max 16384 2024-07-27T03:18:28.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.734+0000 7f4d7abcf740 1 Processor -- start 2024-07-27T03:18:28.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.734+0000 7f4d7abcf740 1 -- start start 2024-07-27T03:18:28.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.735+0000 7f4d7abcf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f98a99f0 0x55e3f9c9d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:28.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.735+0000 7f4d7abcf740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98c21b0 0x55e3f9ca7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:28.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.735+0000 7f4d7abcf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3f98ca390 0x55e3f9ca7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:28.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.735+0000 7f4d7abcf740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1b28e0 con 0x55e3f98c21b0 2024-07-27T03:18:28.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.735+0000 7f4d7abcf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1b4420 con 0x55e3f98a99f0 2024-07-27T03:18:28.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.735+0000 7f4d7abcf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1b2fb0 con 0x55e3f98ca390 2024-07-27T03:18:28.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.735+0000 7f4d71be5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98c21b0 0x55e3f9ca7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:28.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.735+0000 7f4d71be5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98c21b0 0x55e3f9ca7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49380/0 (socket says 172.21.15.67:49380) 2024-07-27T03:18:28.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.735+0000 7f4d71be5640 1 -- 172.21.15.67:0/621516077 learned_addr learned my addr 172.21.15.67:0/621516077 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:28.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.735+0000 7f4d72be7640 1 --2- 172.21.15.67:0/621516077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f98a99f0 0x55e3f9c9d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:28.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.735+0000 7f4d72be7640 1 --2- 172.21.15.67:0/621516077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3f98ca390 0x55e3f9ca7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:28.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.735+0000 7f4d72be7640 1 -- 172.21.15.67:0/621516077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3f98ca390 msgr2=0x55e3f9ca7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:28.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.735+0000 7f4d72be7640 1 --2- 172.21.15.67:0/621516077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3f98ca390 0x55e3f9ca7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.735+0000 7f4d72be7640 1 -- 172.21.15.67:0/621516077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98c21b0 msgr2=0x55e3f9ca7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:28.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.735+0000 7f4d72be7640 1 --2- 172.21.15.67:0/621516077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98c21b0 0x55e3f9ca7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.735+0000 7f4d72be7640 1 -- 172.21.15.67:0/621516077 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3f9c9e400 con 0x55e3f98a99f0 2024-07-27T03:18:28.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.735+0000 7f4d71be5640 1 --2- 172.21.15.67:0/621516077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98c21b0 0x55e3f9ca7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:18:28.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.735+0000 7f4d72be7640 1 --2- 172.21.15.67:0/621516077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f98a99f0 0x55e3f9c9d530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4d64009fd0 tx=0x7f4d64011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9732e088bdd79f7 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:28.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.736+0000 7f4d57fff640 1 -- 172.21.15.67:0/621516077 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6400fe50 con 0x55e3f98a99f0 2024-07-27T03:18:28.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.736+0000 7f4d57fff640 1 -- 172.21.15.67:0/621516077 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d6400f790 con 0x55e3f98a99f0 2024-07-27T03:18:28.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.736+0000 7f4d57fff640 1 -- 172.21.15.67:0/621516077 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d64005610 con 0x55e3f98a99f0 2024-07-27T03:18:28.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.737+0000 7f4d7abcf740 1 -- 172.21.15.67:0/621516077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f98a99f0 msgr2=0x55e3f9c9d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:28.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.737+0000 7f4d7abcf740 1 --2- 172.21.15.67:0/621516077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f98a99f0 0x55e3f9c9d530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4d64009fd0 tx=0x7f4d64011c50 comp rx=0 tx=0).stop 2024-07-27T03:18:28.745 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.737+0000 7f4d7abcf740 1 -- 172.21.15.67:0/621516077 shutdown_connections 2024-07-27T03:18:28.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.737+0000 7f4d7abcf740 1 --2- 172.21.15.67:0/621516077 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3f98ca390 0x55e3f9ca7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.737+0000 7f4d7abcf740 1 --2- 172.21.15.67:0/621516077 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98c21b0 0x55e3f9ca7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.737+0000 7f4d7abcf740 1 --2- 172.21.15.67:0/621516077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f98a99f0 0x55e3f9c9d530 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.737+0000 7f4d7abcf740 1 -- 172.21.15.67:0/621516077 >> 172.21.15.67:0/621516077 conn(0x55e3f98df460 msgr2=0x55e3f9c94e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:28.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.737+0000 7f4d7abcf740 1 -- 172.21.15.67:0/621516077 shutdown_connections 2024-07-27T03:18:28.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.737+0000 7f4d7abcf740 1 -- 172.21.15.67:0/621516077 wait complete. 2024-07-27T03:18:28.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.737+0000 7f4d7abcf740 1 -- start start 2024-07-27T03:18:28.746 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.737+0000 7f4d7abcf740 1 Processor -- start 2024-07-27T03:18:28.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.738+0000 7f4d7abcf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f98a99f0 0x55e3f9ca3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:28.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.738+0000 7f4d7abcf740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98c21b0 0x55e3f9c9f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:28.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.738+0000 7f4d7abcf740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3f98ca390 0x55e3f9c9f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:28.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.738+0000 7f4d7abcf740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1b3d50 con 0x55e3f98c21b0 2024-07-27T03:18:28.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.738+0000 7f4d7abcf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1b3680 con 0x55e3f98a99f0 2024-07-27T03:18:28.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.738+0000 7f4d7abcf740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1b08c0 con 0x55e3f98ca390 2024-07-27T03:18:28.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.738+0000 7f4d71be5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98c21b0 0x55e3f9c9f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:28.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.738+0000 7f4d7abcf740 10 client.0 _ll_register_callbacks cb 0x7f4d5c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:28.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.738+0000 7f4d71be5640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98c21b0 0x55e3f9c9f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49384/0 (socket says 172.21.15.67:49384) 2024-07-27T03:18:28.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.738+0000 7f4d71be5640 1 -- 172.21.15.67:0/2618875167 learned_addr learned my addr 172.21.15.67:0/2618875167 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:28.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.738+0000 7f4d72be7640 1 --2- 172.21.15.67:0/2618875167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f98a99f0 0x55e3f9ca3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:28.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.739+0000 7f4d72be7640 1 --2- 172.21.15.67:0/2618875167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3f98ca390 0x55e3f9c9f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:28.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.739+0000 7f4d71be5640 1 -- 172.21.15.67:0/2618875167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3f98ca390 msgr2=0x55e3f9c9f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:28.747 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.739+0000 7f4d71be5640 1 --2- 172.21.15.67:0/2618875167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3f98ca390 0x55e3f9c9f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.739+0000 7f4d71be5640 1 -- 172.21.15.67:0/2618875167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f98a99f0 msgr2=0x55e3f9ca3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:28.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.739+0000 7f4d71be5640 1 --2- 172.21.15.67:0/2618875167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f98a99f0 0x55e3f9ca3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.739+0000 7f4d71be5640 1 -- 172.21.15.67:0/2618875167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3f9ca0040 con 0x55e3f98c21b0 2024-07-27T03:18:28.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.739+0000 7f4d71be5640 1 --2- 172.21.15.67:0/2618875167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98c21b0 0x55e3f9c9f2d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c06b130 tx=0x7f4d5c01e7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e43188554f22a41 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:28.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.739+0000 7f4d55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:28.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.739+0000 7f4d55ffb640 1 -- 172.21.15.67:0/2618875167 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c02b060 con 0x55e3f98c21b0 2024-07-27T03:18:28.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.739+0000 7f4d55ffb640 1 -- 172.21.15.67:0/2618875167 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f4d5c010430 con 0x55e3f98c21b0 2024-07-27T03:18:28.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.739+0000 7f4d7abcf740 1 -- 172.21.15.67:0/2618875167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d6400f790 con 0x55e3f98c21b0 2024-07-27T03:18:28.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.739+0000 7f4d55ffb640 1 -- 172.21.15.67:0/2618875167 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4d5c032050 con 0x55e3f98c21b0 2024-07-27T03:18:28.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.739+0000 7f4d7abcf740 10 client.15837 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:28.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.739+0000 7f4d7abcf740 1 -- 172.21.15.67:0/2618875167 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e3f9d15b90 con 0x55e3f98c21b0 2024-07-27T03:18:28.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.740+0000 7f4d55ffb640 1 -- 172.21.15.67:0/2618875167 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d6400f790 con 0x55e3f98c21b0 2024-07-27T03:18:28.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.740+0000 7f4d55ffb640 1 -- 172.21.15.67:0/2618875167 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f4d5c025030 con 0x55e3f98c21b0 2024-07-27T03:18:28.748 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.740+0000 7f4d7abcf740 10 client.15837 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:28.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.740+0000 7f4d7abcf740 10 client.15837 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-07-27T03:18:28.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.740+0000 7f4d7abcf740 20 client.15837 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:28.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.740+0000 7f4d7abcf740 1 --2- 172.21.15.67:0/2618875167 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55e3f98eb070 0x55e3f9d183b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:28.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.740+0000 7f4d7abcf740 4 client.15837 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-27T03:18:28.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.740+0000 7f4d7abcf740 1 -- 172.21.15.67:0/2618875167 --> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e3f9c04510 con 0x55e3f98eb070 2024-07-27T03:18:28.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.740+0000 7f4d72be7640 1 --2- 172.21.15.67:0/2618875167 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55e3f98eb070 0x55e3f9d183b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:28.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.741+0000 7f4d55ffb640 10 client.15837 ms_handle_connect on v2:172.21.15.149:6834/3730964806 2024-07-27T03:18:28.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.741+0000 7f4d72be7640 1 --2- 172.21.15.67:0/2618875167 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55e3f98eb070 0x55e3f9d183b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4a2674c51cdfe29a server_cookie=dd0d6a78c3602536 in_seq=0 out_seq=0 2024-07-27T03:18:28.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.741+0000 7f4d55ffb640 1 -- 172.21.15.67:0/2618875167 <== mds.4 v2:172.21.15.149:6834/3730964806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e3f9c04510 con 0x55e3f98eb070 2024-07-27T03:18:28.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.741+0000 7f4d55ffb640 10 client.15837 handle_command_reply: tid=0 2024-07-27T03:18:28.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.741+0000 7f4d7abcf740 1 client.15837 shutdown 2024-07-27T03:18:28.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.742+0000 7f4d7abcf740 1 -- 172.21.15.67:0/2618875167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98c21b0 msgr2=0x55e3f9c9f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:28.749 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.742+0000 7f4d7abcf740 1 --2- 172.21.15.67:0/2618875167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98c21b0 0x55e3f9c9f2d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c06b130 tx=0x7f4d5c01e7c0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.742+0000 7f4d7abcf740 1 -- 172.21.15.67:0/2618875167 shutdown_connections 2024-07-27T03:18:28.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.742+0000 7f4d7abcf740 1 --2- 172.21.15.67:0/2618875167 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x55e3f98eb070 0x55e3f9d183b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.742+0000 7f4d7abcf740 1 --2- 172.21.15.67:0/2618875167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e3f98ca390 0x55e3f9c9f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.742+0000 7f4d7abcf740 1 --2- 172.21.15.67:0/2618875167 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e3f98c21b0 0x55e3f9c9f2d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.742+0000 7f4d55ffb640 0 client.15837 ms_handle_reset on v2:172.21.15.149:6834/3730964806 2024-07-27T03:18:28.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.742+0000 7f4d7abcf740 1 --2- 172.21.15.67:0/2618875167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e3f98a99f0 0x55e3f9ca3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.742+0000 7f4d7abcf740 1 -- 172.21.15.67:0/2618875167 >> 172.21.15.67:0/2618875167 conn(0x55e3f98df460 msgr2=0x55e3f9c94e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:28.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.742+0000 7f4d7abcf740 1 -- 172.21.15.67:0/2618875167 shutdown_connections 2024-07-27T03:18:28.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.742+0000 7f4d7abcf740 1 -- 172.21.15.67:0/2618875167 wait complete. 2024-07-27T03:18:28.750 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.742+0000 7f4d7abcf740 20 client.15837 trim_cache size 0 max 16384 2024-07-27T03:18:28.751 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-27T03:18:28.751 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.743+0000 7f4d7464a640 1 -- 172.21.15.67:0/1346557576 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4d580b3bd0 msgr2=0x7f4d580b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:28.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.743+0000 7f4d7464a640 1 --2- 172.21.15.67:0/1346557576 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4d580b3bd0 0x7f4d580b60e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55e3f9c8e9c0 tx=0x7f4d68107a60 comp rx=0 tx=0).stop 2024-07-27T03:18:28.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.743+0000 7f4d7464a640 1 -- 172.21.15.67:0/1346557576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d6c102540 msgr2=0x7f4d6c0ff110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:28.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.743+0000 7f4d7464a640 1 --2- 172.21.15.67:0/1346557576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d6c102540 0x7f4d6c0ff110 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4d68005940 tx=0x7f4d680040c0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.743+0000 7f4d7464a640 1 -- 172.21.15.67:0/1346557576 shutdown_connections 2024-07-27T03:18:28.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.743+0000 7f4d7464a640 1 --2- 172.21.15.67:0/1346557576 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f4d580b3bd0 0x7f4d580b60e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.743+0000 7f4d7464a640 1 --2- 172.21.15.67:0/1346557576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d6c108780 0x7f4d6c0ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.743+0000 7f4d7464a640 1 --2- 172.21.15.67:0/1346557576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d6c107e30 0x7f4d6c0ff650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.743+0000 7f4d7464a640 1 --2- 172.21.15.67:0/1346557576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4d6c102540 0x7f4d6c0ff110 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:28.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.743+0000 7f4d7464a640 1 -- 172.21.15.67:0/1346557576 >> 172.21.15.67:0/1346557576 conn(0x7f4d6c079140 msgr2=0x7f4d6c101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:28.752 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.743+0000 7f4d7464a640 1 -- 172.21.15.67:0/1346557576 shutdown_connections 2024-07-27T03:18:28.753 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:28.743+0000 7f4d7464a640 1 -- 172.21.15.67:0/1346557576 wait complete. 2024-07-27T03:18:28.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:28 smithi175 ceph-mon[37125]: pgmap v487: 129 pgs: 129 active+clean; 1.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 859 KiB/s wr, 12 op/s 2024-07-27T03:18:28.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:28 smithi067 ceph-mon[32814]: pgmap v487: 129 pgs: 129 active+clean; 1.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 859 KiB/s wr, 12 op/s 2024-07-27T03:18:29.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:28 smithi149 ceph-mon[37173]: pgmap v487: 129 pgs: 129 active+clean; 1.9 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 859 KiB/s wr, 12 op/s 2024-07-27T03:18:29.544 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-27T03:18:29.830 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:18:30.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:30 smithi175 ceph-mon[37125]: pgmap v488: 129 pgs: 129 active+clean; 50 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 MiB/s wr, 13 op/s 2024-07-27T03:18:30.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:30 smithi067 ceph-mon[32814]: pgmap v488: 129 pgs: 129 active+clean; 50 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 MiB/s wr, 13 op/s 2024-07-27T03:18:30.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:30 smithi149 ceph-mon[37173]: pgmap v488: 129 pgs: 129 active+clean; 50 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 MiB/s wr, 13 op/s 2024-07-27T03:18:31.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.635+0000 7f278b7d8640 1 -- 172.21.15.67:0/3344998555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2784107e10 msgr2=0x7f2784108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:31.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.635+0000 7f278b7d8640 1 --2- 172.21.15.67:0/3344998555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2784107e10 0x7f2784108220 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2778009900 tx=0x7f277802a410 comp rx=0 tx=0).stop 2024-07-27T03:18:31.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.635+0000 7f278b7d8640 1 -- 172.21.15.67:0/3344998555 shutdown_connections 2024-07-27T03:18:31.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.635+0000 7f278b7d8640 1 --2- 172.21.15.67:0/3344998555 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2784108760 0x7f278410ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.635+0000 7f278b7d8640 1 --2- 172.21.15.67:0/3344998555 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f2784107e10 0x7f2784108220 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.635+0000 7f278b7d8640 1 --2- 172.21.15.67:0/3344998555 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27841024d0 0x7f27841028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.635+0000 7f278b7d8640 1 -- 172.21.15.67:0/3344998555 >> 172.21.15.67:0/3344998555 conn(0x7f27840791a0 msgr2=0x7f27840795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:31.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.635+0000 7f278b7d8640 1 -- 172.21.15.67:0/3344998555 shutdown_connections 2024-07-27T03:18:31.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.635+0000 7f278b7d8640 1 -- 172.21.15.67:0/3344998555 wait complete. 2024-07-27T03:18:31.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.636+0000 7f278b7d8640 1 Processor -- start 2024-07-27T03:18:31.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.636+0000 7f278b7d8640 1 -- start start 2024-07-27T03:18:31.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.637+0000 7f278b7d8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27841024d0 0x7f27841a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.637+0000 7f278b7d8640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2784107e10 0x7f27841a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.637+0000 7f278b7d8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2784108760 0x7f27841a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.637+0000 7f278b7d8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f27841123b0 con 0x7f27841024d0 2024-07-27T03:18:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.637+0000 7f278b7d8640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2784112230 con 0x7f2784107e10 2024-07-27T03:18:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.637+0000 7f278b7d8640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2784112530 con 0x7f2784108760 2024-07-27T03:18:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.637+0000 7f2789574640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27841024d0 0x7f27841a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.637+0000 7f2789574640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27841024d0 0x7f27841a04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49406/0 (socket says 172.21.15.67:49406) 2024-07-27T03:18:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.637+0000 7f2789574640 1 -- 172.21.15.67:0/3443400524 learned_addr learned my addr 172.21.15.67:0/3443400524 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.637+0000 7f2789d75640 1 --2- 172.21.15.67:0/3443400524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2784108760 0x7f27841a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.637+0000 7f2788d73640 1 --2- 172.21.15.67:0/3443400524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2784107e10 0x7f27841a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.637+0000 7f2789574640 1 -- 172.21.15.67:0/3443400524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2784108760 msgr2=0x7f27841a5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:31.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.637+0000 7f2789574640 1 --2- 172.21.15.67:0/3443400524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2784108760 0x7f27841a5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.637+0000 7f2789574640 1 -- 172.21.15.67:0/3443400524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2784107e10 msgr2=0x7f27841a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:31.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.637+0000 7f2789574640 1 --2- 172.21.15.67:0/3443400524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2784107e10 0x7f27841a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.637+0000 7f2789574640 1 -- 172.21.15.67:0/3443400524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27841a6450 con 0x7f27841024d0 2024-07-27T03:18:31.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.638+0000 7f2789574640 1 --2- 172.21.15.67:0/3443400524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27841024d0 0x7f27841a04e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f278000d770 tx=0x7f278000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:31.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.638+0000 7f27767fc640 1 -- 172.21.15.67:0/3443400524 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780017070 con 0x7f27841024d0 2024-07-27T03:18:31.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.638+0000 7f278b7d8640 1 -- 172.21.15.67:0/3443400524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27841a6740 con 0x7f27841024d0 2024-07-27T03:18:31.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.638+0000 7f27767fc640 1 -- 172.21.15.67:0/3443400524 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27800054d0 con 0x7f27841024d0 2024-07-27T03:18:31.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.638+0000 7f27767fc640 1 -- 172.21.15.67:0/3443400524 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780004c30 con 0x7f27841024d0 2024-07-27T03:18:31.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.638+0000 7f278b7d8640 1 -- 172.21.15.67:0/3443400524 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27841acea0 con 0x7f27841024d0 2024-07-27T03:18:31.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.641+0000 7f2791d5d740 1 Processor -- start 2024-07-27T03:18:31.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.641+0000 7f2791d5d740 1 -- start start 2024-07-27T03:18:31.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.641+0000 7f27767fc640 1 -- 172.21.15.67:0/3443400524 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f2780004eb0 con 0x7f27841024d0 2024-07-27T03:18:31.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.642+0000 7f2791d5d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43ef26070 0x55a43f2cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:31.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.642+0000 7f2791d5d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a43eefd1b0 0x55a43f2cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:31.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.642+0000 7f2791d5d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a43ef05390 0x55a43f2cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:31.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.642+0000 7f2791d5d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a43f1b3400 con 0x55a43ef26070 2024-07-27T03:18:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.642+0000 7f2791d5d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a43f241b20 con 0x55a43eefd1b0 2024-07-27T03:18:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.642+0000 7f2791d5d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a43f231020 con 0x55a43ef05390 2024-07-27T03:18:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.642+0000 7f2789d75640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43ef26070 0x55a43f2cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.642+0000 7f2789d75640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43ef26070 0x55a43f2cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49410/0 (socket says 172.21.15.67:49410) 2024-07-27T03:18:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.642+0000 7f2789d75640 1 -- 172.21.15.67:0/2766447131 learned_addr learned my addr 172.21.15.67:0/2766447131 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.642+0000 7f2789574640 1 --2- 172.21.15.67:0/2766447131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a43eefd1b0 0x55a43f2cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.642+0000 7f2788d73640 1 --2- 172.21.15.67:0/2766447131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a43ef05390 0x55a43f2cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.642+0000 7f2789d75640 1 -- 172.21.15.67:0/2766447131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a43ef05390 msgr2=0x55a43f2cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.642+0000 7f2789d75640 1 --2- 172.21.15.67:0/2766447131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a43ef05390 0x55a43f2cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.642+0000 7f2789d75640 1 -- 172.21.15.67:0/2766447131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a43eefd1b0 msgr2=0x55a43f2cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:31.647 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.642+0000 7f2789d75640 1 --2- 172.21.15.67:0/2766447131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a43eefd1b0 0x55a43f2cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.642+0000 7f2789d75640 1 -- 172.21.15.67:0/2766447131 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a43f1c6770 con 0x55a43ef26070 2024-07-27T03:18:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.643+0000 7f2789d75640 1 --2- 172.21.15.67:0/2766447131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43ef26070 0x55a43f2cad40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f277c010370 tx=0x7f277c010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ddf16846b3065f server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.643+0000 7f276f7fe640 1 -- 172.21.15.67:0/2766447131 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c016070 con 0x55a43ef26070 2024-07-27T03:18:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.643+0000 7f276f7fe640 1 -- 172.21.15.67:0/2766447131 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f277c004d20 con 0x55a43ef26070 2024-07-27T03:18:31.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.643+0000 7f276f7fe640 1 -- 172.21.15.67:0/2766447131 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c005350 con 0x55a43ef26070 2024-07-27T03:18:31.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.643+0000 7f27767fc640 1 --2- 172.21.15.67:0/3443400524 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f27700b3c50 0x7f27700b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:31.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.643+0000 7f2789574640 1 --2- 172.21.15.67:0/3443400524 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f27700b3c50 0x7f27700b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:31.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.643+0000 7f27767fc640 1 -- 172.21.15.67:0/3443400524 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f27800e3980 con 0x7f27841024d0 2024-07-27T03:18:31.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.644+0000 7f2791d5d740 1 -- 172.21.15.67:0/2766447131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43ef26070 msgr2=0x55a43f2cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:31.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.644+0000 7f2789574640 1 --2- 172.21.15.67:0/3443400524 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f27700b3c50 0x7f27700b6160 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55a43f2c99c0 tx=0x7f27800e9550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:31.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.644+0000 7f2791d5d740 1 --2- 172.21.15.67:0/2766447131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43ef26070 0x55a43f2cad40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f277c010370 tx=0x7f277c010830 comp rx=0 tx=0).stop 2024-07-27T03:18:31.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.644+0000 7f2791d5d740 1 -- 172.21.15.67:0/2766447131 shutdown_connections 2024-07-27T03:18:31.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.644+0000 7f2791d5d740 1 --2- 172.21.15.67:0/2766447131 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a43ef05390 0x55a43f2cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.644+0000 7f2791d5d740 1 --2- 172.21.15.67:0/2766447131 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a43eefd1b0 0x55a43f2cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.644+0000 7f2791d5d740 1 --2- 172.21.15.67:0/2766447131 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43ef26070 0x55a43f2cad40 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.644+0000 7f2791d5d740 1 -- 172.21.15.67:0/2766447131 >> 172.21.15.67:0/2766447131 conn(0x55a43ef1a460 msgr2=0x55a43f2c8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:31.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.644+0000 7f2791d5d740 1 -- 172.21.15.67:0/2766447131 shutdown_connections 2024-07-27T03:18:31.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.644+0000 7f2791d5d740 1 -- 172.21.15.67:0/2766447131 wait complete. 2024-07-27T03:18:31.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.644+0000 7f2791d5d740 1 -- start start 2024-07-27T03:18:31.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.645+0000 7f2791d5d740 1 Processor -- start 2024-07-27T03:18:31.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.645+0000 7f2791d5d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43eefd1b0 0x55a43f2d6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:31.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.645+0000 7f2791d5d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a43ef05390 0x55a43f2d6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:31.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.645+0000 7f2791d5d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a43ef26070 0x55a43f2e26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:31.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.645+0000 7f2791d5d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55a43f1d3900 con 0x55a43eefd1b0 2024-07-27T03:18:31.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.645+0000 7f2791d5d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55a43f235060 con 0x55a43ef26070 2024-07-27T03:18:31.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.645+0000 7f2791d5d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a43f094700 con 0x55a43ef05390 2024-07-27T03:18:31.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.645+0000 7f2791d5d740 10 client.0 _ll_register_callbacks cb 0x55a43f1dbd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:31.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.645+0000 7f2789d75640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43eefd1b0 0x55a43f2d6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:31.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.645+0000 7f2789d75640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43eefd1b0 0x55a43f2d6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49426/0 (socket says 172.21.15.67:49426) 2024-07-27T03:18:31.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.645+0000 7f2789d75640 1 -- 172.21.15.67:0/1047895422 learned_addr learned my addr 172.21.15.67:0/1047895422 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:31.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.645+0000 7f2788d73640 1 --2- 172.21.15.67:0/1047895422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a43ef05390 0x55a43f2d6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:31.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.646+0000 7f2789d75640 1 --2- 172.21.15.67:0/1047895422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a43ef26070 0x55a43f2e26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:31.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.646+0000 7f2788d73640 1 -- 172.21.15.67:0/1047895422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a43ef26070 msgr2=0x55a43f2e26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:31.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.646+0000 7f2788d73640 1 --2- 172.21.15.67:0/1047895422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a43ef26070 0x55a43f2e26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.646+0000 7f2788d73640 1 -- 172.21.15.67:0/1047895422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43eefd1b0 msgr2=0x55a43f2d6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:31.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.646+0000 7f2788d73640 1 --2- 172.21.15.67:0/1047895422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43eefd1b0 0x55a43f2d6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.646+0000 7f2788d73640 1 -- 172.21.15.67:0/1047895422 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a43f1d7bf0 con 0x55a43ef05390 2024-07-27T03:18:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.646+0000 7f2789d75640 1 --2- 172.21.15.67:0/1047895422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43eefd1b0 0x55a43f2d6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:18:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.646+0000 7f2788d73640 1 --2- 172.21.15.67:0/1047895422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a43ef05390 0x55a43f2d6ea0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2778006010 tx=0x7f2778032f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=271e1d7137c60116 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.646+0000 7f276cff9640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-27T03:18:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.646+0000 7f276cff9640 1 -- 172.21.15.67:0/1047895422 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778035040 con 0x55a43ef05390 2024-07-27T03:18:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.646+0000 7f2791d5d740 1 -- 172.21.15.67:0/1047895422 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f277c004d20 con 0x55a43ef05390 2024-07-27T03:18:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.646+0000 7f276cff9640 1 -- 172.21.15.67:0/1047895422 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f2778031860 con 0x55a43ef05390 2024-07-27T03:18:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.647+0000 7f276cff9640 1 -- 172.21.15.67:0/1047895422 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f277804f050 con 0x55a43ef05390 2024-07-27T03:18:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.647+0000 7f2791d5d740 10 client.25450 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.647+0000 7f2791d5d740 1 -- 172.21.15.67:0/1047895422 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a43ef91650 con 0x55a43ef05390 2024-07-27T03:18:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.647+0000 7f276cff9640 1 -- 172.21.15.67:0/1047895422 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f277c004d20 con 0x55a43ef05390 2024-07-27T03:18:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.647+0000 7f276cff9640 1 -- 172.21.15.67:0/1047895422 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2778004750 con 0x55a43ef05390 2024-07-27T03:18:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.647+0000 7f2791d5d740 10 client.25450 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.647+0000 7f2791d5d740 10 client.25450 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-27T03:18:31.655 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.647+0000 7f2791d5d740 20 client.25450 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:31.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.647+0000 7f2791d5d740 1 --2- 172.21.15.67:0/1047895422 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55a43eee49f0 0x55a43f2e86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:31.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.647+0000 7f2791d5d740 4 client.25450 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:18:31.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.647+0000 7f2791d5d740 1 -- 172.21.15.67:0/1047895422 --> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a43f243a70 con 0x55a43eee49f0 2024-07-27T03:18:31.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.647+0000 7f2789d75640 1 --2- 172.21.15.67:0/1047895422 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55a43eee49f0 0x55a43f2e86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:31.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.648+0000 7f276cff9640 10 client.25450 ms_handle_connect on v2:172.21.15.175:6836/1304803502 2024-07-27T03:18:31.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.648+0000 7f2789d75640 1 --2- 172.21.15.67:0/1047895422 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55a43eee49f0 0x55a43f2e86d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=aab022af78ac9181 server_cookie=5071b9fc730d519a in_seq=0 out_seq=0 2024-07-27T03:18:31.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.649+0000 7f276cff9640 1 -- 172.21.15.67:0/1047895422 <== mds.3 v2:172.21.15.175:6836/1304803502 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a43f243a70 con 0x55a43eee49f0 2024-07-27T03:18:31.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.649+0000 7f276cff9640 10 client.25450 handle_command_reply: tid=0 2024-07-27T03:18:31.656 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.649+0000 7f2791d5d740 1 client.25450 shutdown 2024-07-27T03:18:31.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.649+0000 7f2791d5d740 1 -- 172.21.15.67:0/1047895422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a43ef05390 msgr2=0x55a43f2d6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:31.657 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.649+0000 7f2791d5d740 1 --2- 172.21.15.67:0/1047895422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a43ef05390 0x55a43f2d6ea0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2778006010 tx=0x7f2778032f60 comp rx=0 tx=0).stop 2024-07-27T03:18:31.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.649+0000 7f2791d5d740 1 -- 172.21.15.67:0/1047895422 shutdown_connections 2024-07-27T03:18:31.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.649+0000 7f2791d5d740 1 --2- 172.21.15.67:0/1047895422 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55a43eee49f0 0x55a43f2e86d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.649+0000 7f2791d5d740 1 --2- 172.21.15.67:0/1047895422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a43ef26070 0x55a43f2e26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.649+0000 7f2791d5d740 1 --2- 172.21.15.67:0/1047895422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a43ef05390 0x55a43f2d6ea0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.649+0000 7f276cff9640 0 client.25450 ms_handle_reset on v2:172.21.15.175:6836/1304803502 2024-07-27T03:18:31.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.649+0000 7f2791d5d740 1 --2- 172.21.15.67:0/1047895422 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43eefd1b0 0x55a43f2d6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.649+0000 7f2791d5d740 1 -- 172.21.15.67:0/1047895422 >> 172.21.15.67:0/1047895422 conn(0x55a43ef1a460 msgr2=0x55a43f2cdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:31.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.650+0000 7f2791d5d740 1 -- 172.21.15.67:0/1047895422 shutdown_connections 2024-07-27T03:18:31.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.650+0000 7f2791d5d740 1 -- 172.21.15.67:0/1047895422 wait complete. 2024-07-27T03:18:31.658 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.650+0000 7f2791d5d740 20 client.25450 trim_cache size 0 max 16384 2024-07-27T03:18:31.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.677+0000 7f2791d5d740 1 Processor -- start 2024-07-27T03:18:31.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.677+0000 7f2791d5d740 1 -- start start 2024-07-27T03:18:31.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.677+0000 7f2791d5d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a43eee49f0 0x55a43f2d8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:31.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.677+0000 7f2791d5d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a43eefd1b0 0x55a43f2cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:31.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.677+0000 7f2791d5d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43ef05390 0x55a43f2ede90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:31.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.677+0000 7f2791d5d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f27841b6c10 con 0x55a43ef05390 2024-07-27T03:18:31.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.677+0000 7f2791d5d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f27841b8750 con 0x55a43eee49f0 2024-07-27T03:18:31.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.677+0000 7f2791d5d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f27841b72e0 con 0x55a43eefd1b0 2024-07-27T03:18:31.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.677+0000 7f2789d75640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43ef05390 0x55a43f2ede90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:31.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.677+0000 7f2789d75640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43ef05390 0x55a43f2ede90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49440/0 (socket says 172.21.15.67:49440) 2024-07-27T03:18:31.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.677+0000 7f2789d75640 1 -- 172.21.15.67:0/2505173702 learned_addr learned my addr 172.21.15.67:0/2505173702 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:31.682 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.677+0000 7f2789d75640 1 --2- 172.21.15.67:0/2505173702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a43eee49f0 0x55a43f2d8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:31.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.678+0000 7f2788d73640 1 --2- 172.21.15.67:0/2505173702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a43eefd1b0 0x55a43f2cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:31.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.678+0000 7f2789d75640 1 -- 172.21.15.67:0/2505173702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a43eefd1b0 msgr2=0x55a43f2cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:31.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.678+0000 7f2789d75640 1 --2- 172.21.15.67:0/2505173702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a43eefd1b0 0x55a43f2cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.678+0000 7f2789d75640 1 -- 172.21.15.67:0/2505173702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a43eee49f0 msgr2=0x55a43f2d8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:31.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.678+0000 7f2789d75640 1 --2- 172.21.15.67:0/2505173702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a43eee49f0 0x55a43f2d8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.678+0000 7f2789d75640 1 -- 172.21.15.67:0/2505173702 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a43f2e3090 con 0x55a43ef05390 2024-07-27T03:18:31.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.678+0000 7f2789d75640 1 --2- 172.21.15.67:0/2505173702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43ef05390 0x55a43f2ede90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f277c009f40 tx=0x7f277c004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cd2a3075e6c28c server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:31.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.678+0000 7f276effd640 1 -- 172.21.15.67:0/2505173702 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c03a070 con 0x55a43ef05390 2024-07-27T03:18:31.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.678+0000 7f276effd640 1 -- 172.21.15.67:0/2505173702 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f277c005020 con 0x55a43ef05390 2024-07-27T03:18:31.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.678+0000 7f276effd640 1 -- 172.21.15.67:0/2505173702 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c016040 con 0x55a43ef05390 2024-07-27T03:18:31.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.679+0000 7f2791d5d740 1 -- 172.21.15.67:0/2505173702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43ef05390 msgr2=0x55a43f2ede90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:31.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.679+0000 7f2791d5d740 1 --2- 172.21.15.67:0/2505173702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43ef05390 0x55a43f2ede90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f277c009f40 tx=0x7f277c004ed0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.679+0000 7f2791d5d740 1 -- 172.21.15.67:0/2505173702 shutdown_connections 2024-07-27T03:18:31.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.679+0000 7f2791d5d740 1 --2- 172.21.15.67:0/2505173702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43ef05390 0x55a43f2ede90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.679+0000 7f2791d5d740 1 --2- 172.21.15.67:0/2505173702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a43eefd1b0 0x55a43f2cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.679+0000 7f2791d5d740 1 --2- 172.21.15.67:0/2505173702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a43eee49f0 0x55a43f2d8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.679+0000 7f2791d5d740 1 -- 172.21.15.67:0/2505173702 >> 172.21.15.67:0/2505173702 conn(0x55a43ef1a460 msgr2=0x55a43f2cbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:31.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.679+0000 7f2791d5d740 1 -- 172.21.15.67:0/2505173702 shutdown_connections 2024-07-27T03:18:31.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.679+0000 7f2791d5d740 1 -- 172.21.15.67:0/2505173702 wait complete. 2024-07-27T03:18:31.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.680+0000 7f2791d5d740 1 -- start start 2024-07-27T03:18:31.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.680+0000 7f2791d5d740 1 Processor -- start 2024-07-27T03:18:31.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.680+0000 7f2791d5d740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43eee49f0 0x55a43f2da640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:31.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.680+0000 7f2791d5d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a43eefd1b0 0x55a43f2dab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:31.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.680+0000 7f2791d5d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a43ef05390 0x55a43f2db0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:31.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.680+0000 7f2791d5d740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f27841b8080 con 0x55a43eee49f0 2024-07-27T03:18:31.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.680+0000 7f2791d5d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f27841b79b0 con 0x55a43ef05390 2024-07-27T03:18:31.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.680+0000 7f2791d5d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f27841b4bf0 con 0x55a43eefd1b0 2024-07-27T03:18:31.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.680+0000 7f2791d5d740 10 client.0 _ll_register_callbacks cb 0x7f2778005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:31.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.681+0000 7f2789d75640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43eee49f0 0x55a43f2da640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:31.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.681+0000 7f2789d75640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43eee49f0 0x55a43f2da640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49450/0 (socket says 172.21.15.67:49450) 2024-07-27T03:18:31.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.681+0000 7f2789d75640 1 -- 172.21.15.67:0/4267948068 learned_addr learned my addr 172.21.15.67:0/4267948068 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:31.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.681+0000 7f2788d73640 1 --2- 172.21.15.67:0/4267948068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a43eefd1b0 0x55a43f2dab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:31.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.681+0000 7f2789d75640 1 -- 172.21.15.67:0/4267948068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a43eefd1b0 msgr2=0x55a43f2dab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:31.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.681+0000 7f2789d75640 1 --2- 172.21.15.67:0/4267948068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a43eefd1b0 0x55a43f2dab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.681+0000 7f2789d75640 1 -- 172.21.15.67:0/4267948068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a43ef05390 msgr2=0x55a43f2db0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:31.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.681+0000 7f2789d75640 1 --2- 172.21.15.67:0/4267948068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a43ef05390 0x55a43f2db0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.681+0000 7f2789d75640 1 -- 172.21.15.67:0/4267948068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a43f34ffc0 con 0x55a43eee49f0 2024-07-27T03:18:31.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.681+0000 7f2788d73640 1 --2- 172.21.15.67:0/4267948068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a43eefd1b0 0x55a43f2dab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:18:31.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.681+0000 7f2789d75640 1 --2- 172.21.15.67:0/4267948068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43eee49f0 0x55a43f2da640 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f277c038040 tx=0x7f277c004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c57a9cb6d52a0a server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:31.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.681+0000 7f276cff9640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:31.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.682+0000 7f276cff9640 1 -- 172.21.15.67:0/4267948068 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c042020 con 0x55a43eee49f0 2024-07-27T03:18:31.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.682+0000 7f2791d5d740 1 -- 172.21.15.67:0/4267948068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f277c005020 con 0x55a43eee49f0 2024-07-27T03:18:31.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.682+0000 7f276cff9640 1 -- 172.21.15.67:0/4267948068 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f277c012690 con 0x55a43eee49f0 2024-07-27T03:18:31.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.682+0000 7f276cff9640 1 -- 172.21.15.67:0/4267948068 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f277c04d050 con 0x55a43eee49f0 2024-07-27T03:18:31.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.682+0000 7f2791d5d740 10 client.15855 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:31.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.682+0000 7f2791d5d740 1 -- 172.21.15.67:0/4267948068 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a43f351240 con 0x55a43eee49f0 2024-07-27T03:18:31.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.682+0000 7f276cff9640 1 -- 172.21.15.67:0/4267948068 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f277c005020 con 0x55a43eee49f0 2024-07-27T03:18:31.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.682+0000 7f276cff9640 1 -- 172.21.15.67:0/4267948068 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f277c03d050 con 0x55a43eee49f0 2024-07-27T03:18:31.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.682+0000 7f2791d5d740 10 client.15855 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:31.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.682+0000 7f2791d5d740 10 client.15855 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-27T03:18:31.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.682+0000 7f2791d5d740 20 client.15855 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:31.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.682+0000 7f2791d5d740 1 --2- 172.21.15.67:0/4267948068 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55a43ef26070 0x55a43f353750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:31.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.682+0000 7f2791d5d740 4 client.15855 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-27T03:18:31.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.682+0000 7f2791d5d740 1 -- 172.21.15.67:0/4267948068 --> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a43f23f510 con 0x55a43ef26070 2024-07-27T03:18:31.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.683+0000 7f2788d73640 1 --2- 172.21.15.67:0/4267948068 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55a43ef26070 0x55a43f353750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:31.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.683+0000 7f276cff9640 10 client.15855 ms_handle_connect on v2:172.21.15.175:6836/1304803502 2024-07-27T03:18:31.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.683+0000 7f2788d73640 1 --2- 172.21.15.67:0/4267948068 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55a43ef26070 0x55a43f353750 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9e1230377ff0c43a server_cookie=fb13e16b401cff41 in_seq=0 out_seq=0 2024-07-27T03:18:31.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.684+0000 7f276cff9640 1 -- 172.21.15.67:0/4267948068 <== mds.3 v2:172.21.15.175:6836/1304803502 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a43f23f510 con 0x55a43ef26070 2024-07-27T03:18:31.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.684+0000 7f276cff9640 10 client.15855 handle_command_reply: tid=0 2024-07-27T03:18:31.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.684+0000 7f2791d5d740 1 client.15855 shutdown 2024-07-27T03:18:31.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.684+0000 7f2791d5d740 1 -- 172.21.15.67:0/4267948068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43eee49f0 msgr2=0x55a43f2da640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:31.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.684+0000 7f2791d5d740 1 --2- 172.21.15.67:0/4267948068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43eee49f0 0x55a43f2da640 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f277c038040 tx=0x7f277c004f40 comp rx=0 tx=0).stop 2024-07-27T03:18:31.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.684+0000 7f2791d5d740 1 -- 172.21.15.67:0/4267948068 shutdown_connections 2024-07-27T03:18:31.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.684+0000 7f2791d5d740 1 --2- 172.21.15.67:0/4267948068 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x55a43ef26070 0x55a43f353750 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.684+0000 7f2791d5d740 1 --2- 172.21.15.67:0/4267948068 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a43ef05390 0x55a43f2db0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.684+0000 7f2791d5d740 1 --2- 172.21.15.67:0/4267948068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a43eefd1b0 0x55a43f2dab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.684+0000 7f2791d5d740 1 --2- 172.21.15.67:0/4267948068 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55a43eee49f0 0x55a43f2da640 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.684+0000 7f276cff9640 0 client.15855 ms_handle_reset on v2:172.21.15.175:6836/1304803502 2024-07-27T03:18:31.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.684+0000 7f2791d5d740 1 -- 172.21.15.67:0/4267948068 >> 172.21.15.67:0/4267948068 conn(0x55a43ef1a460 msgr2=0x55a43f2d83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:31.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.684+0000 7f2791d5d740 1 -- 172.21.15.67:0/4267948068 shutdown_connections 2024-07-27T03:18:31.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.684+0000 7f2791d5d740 1 -- 172.21.15.67:0/4267948068 wait complete. 2024-07-27T03:18:31.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.684+0000 7f2791d5d740 20 client.15855 trim_cache size 0 max 16384 2024-07-27T03:18:31.694 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-27T03:18:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.685+0000 7f278b7d8640 1 -- 172.21.15.67:0/3443400524 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f27700b3c50 msgr2=0x7f27700b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.685+0000 7f278b7d8640 1 --2- 172.21.15.67:0/3443400524 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f27700b3c50 0x7f27700b6160 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55a43f2c99c0 tx=0x7f27800e9550 comp rx=0 tx=0).stop 2024-07-27T03:18:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.685+0000 7f278b7d8640 1 -- 172.21.15.67:0/3443400524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27841024d0 msgr2=0x7f27841a04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.685+0000 7f278b7d8640 1 --2- 172.21.15.67:0/3443400524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27841024d0 0x7f27841a04e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f278000d770 tx=0x7f278000dc30 comp rx=0 tx=0).stop 2024-07-27T03:18:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.685+0000 7f278b7d8640 1 -- 172.21.15.67:0/3443400524 shutdown_connections 2024-07-27T03:18:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.685+0000 7f278b7d8640 1 --2- 172.21.15.67:0/3443400524 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f27700b3c50 0x7f27700b6160 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.685+0000 7f278b7d8640 1 --2- 172.21.15.67:0/3443400524 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2784108760 0x7f27841a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.685+0000 7f278b7d8640 1 --2- 172.21.15.67:0/3443400524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2784107e10 0x7f27841a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.685+0000 7f278b7d8640 1 --2- 172.21.15.67:0/3443400524 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f27841024d0 0x7f27841a04e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.685+0000 7f278b7d8640 1 -- 172.21.15.67:0/3443400524 >> 172.21.15.67:0/3443400524 conn(0x7f27840791a0 msgr2=0x7f278406aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:31.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.685+0000 7f278b7d8640 1 -- 172.21.15.67:0/3443400524 shutdown_connections 2024-07-27T03:18:31.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:31.685+0000 7f278b7d8640 1 -- 172.21.15.67:0/3443400524 wait complete. 2024-07-27T03:18:32.421 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-27T03:18:32.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:32 smithi067 ceph-mon[32814]: pgmap v489: 129 pgs: 129 active+clean; 182 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 19 op/s 2024-07-27T03:18:32.832 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:18:32.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:32 smithi175 ceph-mon[37125]: pgmap v489: 129 pgs: 129 active+clean; 182 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 19 op/s 2024-07-27T03:18:33.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:32 smithi149 ceph-mon[37173]: pgmap v489: 129 pgs: 129 active+clean; 182 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 19 op/s 2024-07-27T03:18:34.699 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:34 smithi067 ceph-mon[32814]: pgmap v490: 129 pgs: 129 active+clean; 202 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 16 op/s 2024-07-27T03:18:34.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:34 smithi175 ceph-mon[37125]: pgmap v490: 129 pgs: 129 active+clean; 202 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 16 op/s 2024-07-27T03:18:34.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.948+0000 7fe33d6c6640 1 -- 172.21.15.67:0/3438222512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe33810d8a0 msgr2=0x7fe33810fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:34.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.948+0000 7fe33d6c6640 1 --2- 172.21.15.67:0/3438222512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe33810d8a0 0x7fe33810fd90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe330009ca0 tx=0x7fe33002a8c0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.948+0000 7fe33d6c6640 1 -- 172.21.15.67:0/3438222512 shutdown_connections 2024-07-27T03:18:34.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.948+0000 7fe33d6c6640 1 --2- 172.21.15.67:0/3438222512 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe33810d8a0 0x7fe33810fd90 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.948+0000 7fe33d6c6640 1 --2- 172.21.15.67:0/3438222512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe338104d90 0x7fe33810d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.948+0000 7fe33d6c6640 1 --2- 172.21.15.67:0/3438222512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe338104440 0x7fe338104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.948+0000 7fe33d6c6640 1 -- 172.21.15.67:0/3438222512 >> 172.21.15.67:0/3438222512 conn(0x7fe3380791b0 msgr2=0x7fe3380795c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:34.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.948+0000 7fe33d6c6640 1 -- 172.21.15.67:0/3438222512 shutdown_connections 2024-07-27T03:18:34.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.948+0000 7fe33d6c6640 1 -- 172.21.15.67:0/3438222512 wait complete. 2024-07-27T03:18:34.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.949+0000 7fe33d6c6640 1 Processor -- start 2024-07-27T03:18:34.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.949+0000 7fe33d6c6640 1 -- start start 2024-07-27T03:18:34.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.949+0000 7fe33d6c6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe338104440 0x7fe338199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:34.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.949+0000 7fe33d6c6640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe338104d90 0x7fe33819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:34.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.949+0000 7fe33d6c6640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe33810d8a0 0x7fe33819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:34.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.949+0000 7fe33d6c6640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe338113500 con 0x7fe338104d90 2024-07-27T03:18:34.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.949+0000 7fe33d6c6640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe338113380 con 0x7fe338104440 2024-07-27T03:18:34.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.949+0000 7fe33d6c6640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe338113680 con 0x7fe33810d8a0 2024-07-27T03:18:34.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.949+0000 7fe32ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe338104d90 0x7fe33819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:34.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.949+0000 7fe32ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe338104d90 0x7fe33819a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56032/0 (socket says 172.21.15.67:56032) 2024-07-27T03:18:34.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.949+0000 7fe32ffff640 1 -- 172.21.15.67:0/4156573168 learned_addr learned my addr 172.21.15.67:0/4156573168 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:34.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.950+0000 7fe336ffd640 1 --2- 172.21.15.67:0/4156573168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe338104440 0x7fe338199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:34.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.950+0000 7fe3377fe640 1 --2- 172.21.15.67:0/4156573168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe33810d8a0 0x7fe33819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:34.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.950+0000 7fe32ffff640 1 -- 172.21.15.67:0/4156573168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe33810d8a0 msgr2=0x7fe33819f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:34.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.950+0000 7fe32ffff640 1 --2- 172.21.15.67:0/4156573168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe33810d8a0 0x7fe33819f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.950+0000 7fe32ffff640 1 -- 172.21.15.67:0/4156573168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe338104440 msgr2=0x7fe338199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:34.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.950+0000 7fe32ffff640 1 --2- 172.21.15.67:0/4156573168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe338104440 0x7fe338199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.950+0000 7fe32ffff640 1 -- 172.21.15.67:0/4156573168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33819fee0 con 0x7fe338104d90 2024-07-27T03:18:34.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.950+0000 7fe32ffff640 1 --2- 172.21.15.67:0/4156573168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe338104d90 0x7fe33819a510 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe32000d950 tx=0x7fe32000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:34.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.950+0000 7fe334ff9640 1 -- 172.21.15.67:0/4156573168 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320017070 con 0x7fe338104d90 2024-07-27T03:18:34.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.950+0000 7fe33d6c6640 1 -- 172.21.15.67:0/4156573168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3381a01d0 con 0x7fe338104d90 2024-07-27T03:18:34.962 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.950+0000 7fe334ff9640 1 -- 172.21.15.67:0/4156573168 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe320009c30 con 0x7fe338104d90 2024-07-27T03:18:34.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.950+0000 7fe334ff9640 1 -- 172.21.15.67:0/4156573168 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3200047a0 con 0x7fe338104d90 2024-07-27T03:18:34.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.950+0000 7fe33d6c6640 1 -- 172.21.15.67:0/4156573168 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3381a6930 con 0x7fe338104d90 2024-07-27T03:18:34.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.954+0000 7fe343c4b740 1 Processor -- start 2024-07-27T03:18:34.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.954+0000 7fe343c4b740 1 -- start start 2024-07-27T03:18:34.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.954+0000 7fe343c4b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0c8ce2070 0x55d0c9086d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:34.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.954+0000 7fe334ff9640 1 -- 172.21.15.67:0/4156573168 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7fe320004020 con 0x7fe338104d90 2024-07-27T03:18:34.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.954+0000 7fe343c4b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0c8cb91b0 0x55d0c9087280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:34.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.954+0000 7fe343c4b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cc1390 0x55d0c90877c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:34.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.954+0000 7fe343c4b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d0c8f6f400 con 0x55d0c8cc1390 2024-07-27T03:18:34.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.954+0000 7fe343c4b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d0c8ffdb20 con 0x55d0c8cb91b0 2024-07-27T03:18:34.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.954+0000 7fe343c4b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d0c8fed020 con 0x55d0c8ce2070 2024-07-27T03:18:34.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.955+0000 7fe32ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cc1390 0x55d0c90877c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:34.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.955+0000 7fe32ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cc1390 0x55d0c90877c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56042/0 (socket says 172.21.15.67:56042) 2024-07-27T03:18:34.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.955+0000 7fe32ffff640 1 -- 172.21.15.67:0/1957855845 learned_addr learned my addr 172.21.15.67:0/1957855845 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:34.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.955+0000 7fe3377fe640 1 --2- 172.21.15.67:0/1957855845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0c8ce2070 0x55d0c9086d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:34.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.955+0000 7fe336ffd640 1 --2- 172.21.15.67:0/1957855845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0c8cb91b0 0x55d0c9087280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:34.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.955+0000 7fe32ffff640 1 -- 172.21.15.67:0/1957855845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0c8ce2070 msgr2=0x55d0c9086d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:34.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.955+0000 7fe32ffff640 1 --2- 172.21.15.67:0/1957855845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0c8ce2070 0x55d0c9086d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.955+0000 7fe32ffff640 1 -- 172.21.15.67:0/1957855845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0c8cb91b0 msgr2=0x55d0c9087280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:34.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.955+0000 7fe32ffff640 1 --2- 172.21.15.67:0/1957855845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0c8cb91b0 0x55d0c9087280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.955+0000 7fe32ffff640 1 -- 172.21.15.67:0/1957855845 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0c8f82770 con 0x55d0c8cc1390 2024-07-27T03:18:34.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.955+0000 7fe3377fe640 1 --2- 172.21.15.67:0/1957855845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0c8ce2070 0x55d0c9086d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:18:34.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.955+0000 7fe336ffd640 1 --2- 172.21.15.67:0/1957855845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0c8cb91b0 0x55d0c9087280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:18:34.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.955+0000 7fe32ffff640 1 --2- 172.21.15.67:0/1957855845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cc1390 0x55d0c90877c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe3200e7af0 tx=0x7fe3200ec1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d0253b3b389f2a server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:34.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.955+0000 7fe334ff9640 1 --2- 172.21.15.67:0/4156573168 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe3240b3bd0 0x7fe3240b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:34.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.956+0000 7fe32d7fa640 1 -- 172.21.15.67:0/1957855845 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3200a6cd0 con 0x55d0c8cc1390 2024-07-27T03:18:34.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.956+0000 7fe3377fe640 1 --2- 172.21.15.67:0/4156573168 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe3240b3bd0 0x7fe3240b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:34.964 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.956+0000 7fe32d7fa640 1 -- 172.21.15.67:0/1957855845 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3200f5070 con 0x55d0c8cc1390 2024-07-27T03:18:34.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.956+0000 7fe334ff9640 1 -- 172.21.15.67:0/4156573168 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fe3200e3ae0 con 0x7fe338104d90 2024-07-27T03:18:34.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.956+0000 7fe32d7fa640 1 -- 172.21.15.67:0/1957855845 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3200a9fd0 con 0x55d0c8cc1390 2024-07-27T03:18:34.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.957+0000 7fe343c4b740 1 -- 172.21.15.67:0/1957855845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cc1390 msgr2=0x55d0c90877c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:34.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.957+0000 7fe3377fe640 1 --2- 172.21.15.67:0/4156573168 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe3240b3bd0 0x7fe3240b60e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe33002f500 tx=0x7fe330009820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:34.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.957+0000 7fe343c4b740 1 --2- 172.21.15.67:0/1957855845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cc1390 0x55d0c90877c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe3200e7af0 tx=0x7fe3200ec1e0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.957+0000 7fe343c4b740 1 -- 172.21.15.67:0/1957855845 shutdown_connections 2024-07-27T03:18:34.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.957+0000 7fe343c4b740 1 --2- 172.21.15.67:0/1957855845 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cc1390 0x55d0c90877c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.957+0000 7fe343c4b740 1 --2- 172.21.15.67:0/1957855845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0c8cb91b0 0x55d0c9087280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.957+0000 7fe343c4b740 1 --2- 172.21.15.67:0/1957855845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0c8ce2070 0x55d0c9086d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.957+0000 7fe343c4b740 1 -- 172.21.15.67:0/1957855845 >> 172.21.15.67:0/1957855845 conn(0x55d0c8cd6460 msgr2=0x55d0c9084c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:34.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.957+0000 7fe343c4b740 1 -- 172.21.15.67:0/1957855845 shutdown_connections 2024-07-27T03:18:34.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.957+0000 7fe343c4b740 1 -- 172.21.15.67:0/1957855845 wait complete. 2024-07-27T03:18:34.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.958+0000 7fe343c4b740 1 -- start start 2024-07-27T03:18:34.965 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.958+0000 7fe343c4b740 1 Processor -- start 2024-07-27T03:18:34.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.958+0000 7fe343c4b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0c8cb91b0 0x55d0c9092830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:34.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.958+0000 7fe343c4b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cc1390 0x55d0c9092d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:34.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.958+0000 7fe343c4b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0c8ce2070 0x55d0c909e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:34.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.958+0000 7fe343c4b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55d0c8f8f900 con 0x55d0c8cc1390 2024-07-27T03:18:34.968 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.958+0000 7fe343c4b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55d0c8ff1060 con 0x55d0c8cb91b0 2024-07-27T03:18:34.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.958+0000 7fe343c4b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d0c8e50700 con 0x55d0c8ce2070 2024-07-27T03:18:34.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.958+0000 7fe32ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cc1390 0x55d0c9092d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:34.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.958+0000 7fe343c4b740 10 client.0 _ll_register_callbacks cb 0x55d0c8f97d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:34.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.958+0000 7fe32ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cc1390 0x55d0c9092d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56056/0 (socket says 172.21.15.67:56056) 2024-07-27T03:18:34.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.958+0000 7fe32ffff640 1 -- 172.21.15.67:0/2756116559 learned_addr learned my addr 172.21.15.67:0/2756116559 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:34.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.958+0000 7fe336ffd640 1 --2- 172.21.15.67:0/2756116559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0c8cb91b0 0x55d0c9092830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:34.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.959+0000 7fe3377fe640 1 --2- 172.21.15.67:0/2756116559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0c8ce2070 0x55d0c909e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:34.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.959+0000 7fe32ffff640 1 -- 172.21.15.67:0/2756116559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0c8ce2070 msgr2=0x55d0c909e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:34.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.959+0000 7fe32ffff640 1 --2- 172.21.15.67:0/2756116559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0c8ce2070 0x55d0c909e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.959+0000 7fe32ffff640 1 -- 172.21.15.67:0/2756116559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0c8cb91b0 msgr2=0x55d0c9092830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:34.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.959+0000 7fe32ffff640 1 --2- 172.21.15.67:0/2756116559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0c8cb91b0 0x55d0c9092830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.959+0000 7fe32ffff640 1 -- 172.21.15.67:0/2756116559 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0c8f93bf0 con 0x55d0c8cc1390 2024-07-27T03:18:34.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.959+0000 7fe32ffff640 1 --2- 172.21.15.67:0/2756116559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cc1390 0x55d0c9092d70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe320021000 tx=0x7fe320004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a39cb1987a51b33 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:34.969 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.959+0000 7fe31e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:34.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.959+0000 7fe31e7fc640 1 -- 172.21.15.67:0/2756116559 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3200aa1c0 con 0x55d0c8cc1390 2024-07-27T03:18:34.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.959+0000 7fe343c4b740 1 -- 172.21.15.67:0/2756116559 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3200f5070 con 0x55d0c8cc1390 2024-07-27T03:18:34.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.959+0000 7fe31e7fc640 1 -- 172.21.15.67:0/2756116559 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fe3200cd0b0 con 0x55d0c8cc1390 2024-07-27T03:18:34.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.959+0000 7fe31e7fc640 1 -- 172.21.15.67:0/2756116559 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe3200c4050 con 0x55d0c8cc1390 2024-07-27T03:18:34.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.959+0000 7fe343c4b740 10 client.15867 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:34.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.959+0000 7fe343c4b740 1 -- 172.21.15.67:0/2756116559 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d0c8d4d650 con 0x55d0c8cc1390 2024-07-27T03:18:34.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.960+0000 7fe31e7fc640 1 -- 172.21.15.67:0/2756116559 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3200f5070 con 0x55d0c8cc1390 2024-07-27T03:18:34.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.960+0000 7fe31e7fc640 1 -- 172.21.15.67:0/2756116559 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe3200a6910 con 0x55d0c8cc1390 2024-07-27T03:18:34.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.960+0000 7fe343c4b740 10 client.15867 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:34.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.960+0000 7fe343c4b740 10 client.15867 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-27T03:18:34.970 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.960+0000 7fe343c4b740 20 client.15867 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:34.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.960+0000 7fe343c4b740 1 --2- 172.21.15.67:0/2756116559 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55d0c8ca09f0 0x55d0c90a4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:34.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.960+0000 7fe343c4b740 4 client.15867 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:18:34.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.960+0000 7fe343c4b740 1 -- 172.21.15.67:0/2756116559 --> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0c8fffa70 con 0x55d0c8ca09f0 2024-07-27T03:18:34.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.960+0000 7fe336ffd640 1 --2- 172.21.15.67:0/2756116559 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55d0c8ca09f0 0x55d0c90a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:34.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.961+0000 7fe336ffd640 1 --2- 172.21.15.67:0/2756116559 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55d0c8ca09f0 0x55d0c90a4460 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3250b6e0d8eab6f3 server_cookie=d5bc621f2eade756 in_seq=0 out_seq=0 2024-07-27T03:18:34.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.961+0000 7fe31e7fc640 10 client.15867 ms_handle_connect on v2:172.21.15.175:6838/1670568853 2024-07-27T03:18:34.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.962+0000 7fe31e7fc640 1 -- 172.21.15.67:0/2756116559 <== mds.2 v2:172.21.15.175:6838/1670568853 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0c8fffa70 con 0x55d0c8ca09f0 2024-07-27T03:18:34.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.962+0000 7fe31e7fc640 10 client.15867 handle_command_reply: tid=0 2024-07-27T03:18:34.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.962+0000 7fe343c4b740 1 client.15867 shutdown 2024-07-27T03:18:34.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.962+0000 7fe343c4b740 1 -- 172.21.15.67:0/2756116559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cc1390 msgr2=0x55d0c9092d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:34.971 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.962+0000 7fe343c4b740 1 --2- 172.21.15.67:0/2756116559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cc1390 0x55d0c9092d70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe320021000 tx=0x7fe320004560 comp rx=0 tx=0).stop 2024-07-27T03:18:34.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.962+0000 7fe343c4b740 1 -- 172.21.15.67:0/2756116559 shutdown_connections 2024-07-27T03:18:34.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.962+0000 7fe343c4b740 1 --2- 172.21.15.67:0/2756116559 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55d0c8ca09f0 0x55d0c90a4460 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.962+0000 7fe343c4b740 1 --2- 172.21.15.67:0/2756116559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0c8ce2070 0x55d0c909e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.962+0000 7fe343c4b740 1 --2- 172.21.15.67:0/2756116559 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cc1390 0x55d0c9092d70 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.962+0000 7fe343c4b740 1 --2- 172.21.15.67:0/2756116559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0c8cb91b0 0x55d0c9092830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.962+0000 7fe343c4b740 1 -- 172.21.15.67:0/2756116559 >> 172.21.15.67:0/2756116559 conn(0x55d0c8cd6460 msgr2=0x55d0c9088730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:34.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.962+0000 7fe31e7fc640 0 client.15867 ms_handle_reset on v2:172.21.15.175:6838/1670568853 2024-07-27T03:18:34.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.962+0000 7fe343c4b740 1 -- 172.21.15.67:0/2756116559 shutdown_connections 2024-07-27T03:18:34.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.962+0000 7fe343c4b740 1 -- 172.21.15.67:0/2756116559 wait complete. 2024-07-27T03:18:34.972 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.962+0000 7fe343c4b740 20 client.15867 trim_cache size 0 max 16384 2024-07-27T03:18:34.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.976+0000 7fe343c4b740 1 Processor -- start 2024-07-27T03:18:34.978 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.976+0000 7fe343c4b740 1 -- start start 2024-07-27T03:18:34.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.976+0000 7fe343c4b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0c8cb91b0 0x55d0c9094510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:34.979 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.976+0000 7fe343c4b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cc1390 0x55d0c9088480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.976+0000 7fe343c4b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0c8ce2070 0x55d0c90841e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.976+0000 7fe343c4b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe3381b0770 con 0x55d0c8cc1390 2024-07-27T03:18:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.976+0000 7fe343c4b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe3381b22b0 con 0x55d0c8ce2070 2024-07-27T03:18:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.976+0000 7fe343c4b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe3381b0e40 con 0x55d0c8cb91b0 2024-07-27T03:18:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.976+0000 7fe32ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cc1390 0x55d0c9088480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.976+0000 7fe32ffff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cc1390 0x55d0c9088480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56072/0 (socket says 172.21.15.67:56072) 2024-07-27T03:18:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.976+0000 7fe32ffff640 1 -- 172.21.15.67:0/2654679511 learned_addr learned my addr 172.21.15.67:0/2654679511 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.977+0000 7fe3377fe640 1 --2- 172.21.15.67:0/2654679511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0c8ce2070 0x55d0c90841e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.977+0000 7fe336ffd640 1 --2- 172.21.15.67:0/2654679511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0c8cb91b0 0x55d0c9094510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.977+0000 7fe32ffff640 1 -- 172.21.15.67:0/2654679511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0c8cb91b0 msgr2=0x55d0c9094510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.977+0000 7fe32ffff640 1 --2- 172.21.15.67:0/2654679511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0c8cb91b0 0x55d0c9094510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.977+0000 7fe32ffff640 1 -- 172.21.15.67:0/2654679511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0c8ce2070 msgr2=0x55d0c90841e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.977+0000 7fe32ffff640 1 --2- 172.21.15.67:0/2654679511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0c8ce2070 0x55d0c90841e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.980 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.977+0000 7fe32ffff640 1 -- 172.21.15.67:0/2654679511 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0c90938f0 con 0x55d0c8cc1390 2024-07-27T03:18:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.977+0000 7fe32ffff640 1 --2- 172.21.15.67:0/2654679511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cc1390 0x55d0c9088480 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe3200a78f0 tx=0x7fe320009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810f6c0a32c0a8d1 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.977+0000 7fe32cff9640 1 -- 172.21.15.67:0/2654679511 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3200aa450 con 0x55d0c8cc1390 2024-07-27T03:18:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.977+0000 7fe32cff9640 1 -- 172.21.15.67:0/2654679511 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3200f5070 con 0x55d0c8cc1390 2024-07-27T03:18:34.981 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.977+0000 7fe32cff9640 1 -- 172.21.15.67:0/2654679511 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3200a5e10 con 0x55d0c8cc1390 2024-07-27T03:18:34.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.978+0000 7fe343c4b740 1 -- 172.21.15.67:0/2654679511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cc1390 msgr2=0x55d0c9088480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:34.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.978+0000 7fe343c4b740 1 --2- 172.21.15.67:0/2654679511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cc1390 0x55d0c9088480 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe3200a78f0 tx=0x7fe320009740 comp rx=0 tx=0).stop 2024-07-27T03:18:34.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.978+0000 7fe343c4b740 1 -- 172.21.15.67:0/2654679511 shutdown_connections 2024-07-27T03:18:34.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.978+0000 7fe343c4b740 1 --2- 172.21.15.67:0/2654679511 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0c8ce2070 0x55d0c90841e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.978+0000 7fe343c4b740 1 --2- 172.21.15.67:0/2654679511 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cc1390 0x55d0c9088480 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.978+0000 7fe343c4b740 1 --2- 172.21.15.67:0/2654679511 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0c8cb91b0 0x55d0c9094510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.978+0000 7fe343c4b740 1 -- 172.21.15.67:0/2654679511 >> 172.21.15.67:0/2654679511 conn(0x55d0c8cd6460 msgr2=0x55d0c9087d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:34.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.978+0000 7fe343c4b740 1 -- 172.21.15.67:0/2654679511 shutdown_connections 2024-07-27T03:18:34.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.978+0000 7fe343c4b740 1 -- 172.21.15.67:0/2654679511 wait complete. 2024-07-27T03:18:34.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.979+0000 7fe343c4b740 1 -- start start 2024-07-27T03:18:34.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.979+0000 7fe343c4b740 1 Processor -- start 2024-07-27T03:18:34.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.979+0000 7fe343c4b740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cb91b0 0x55d0c90a7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:34.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.979+0000 7fe343c4b740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0c8cc1390 0x55d0c90a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:34.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.979+0000 7fe343c4b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0c8ce2070 0x55d0c90a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:34.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.979+0000 7fe343c4b740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe3381b1be0 con 0x55d0c8cb91b0 2024-07-27T03:18:34.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.979+0000 7fe343c4b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe3381b1510 con 0x55d0c8ce2070 2024-07-27T03:18:34.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.979+0000 7fe343c4b740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe3381ae750 con 0x55d0c8cc1390 2024-07-27T03:18:34.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.979+0000 7fe336ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cb91b0 0x55d0c90a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:34.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.979+0000 7fe336ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cb91b0 0x55d0c90a7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56088/0 (socket says 172.21.15.67:56088) 2024-07-27T03:18:34.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.979+0000 7fe336ffd640 1 -- 172.21.15.67:0/1931937333 learned_addr learned my addr 172.21.15.67:0/1931937333 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:34.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.979+0000 7fe343c4b740 10 client.0 _ll_register_callbacks cb 0x7fe328004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:34.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.980+0000 7fe336ffd640 1 -- 172.21.15.67:0/1931937333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0c8cc1390 msgr2=0x55d0c90a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:34.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.980+0000 7fe32ffff640 1 --2- 172.21.15.67:0/1931937333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0c8cc1390 0x55d0c90a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:34.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.980+0000 7fe336ffd640 1 --2- 172.21.15.67:0/1931937333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0c8cc1390 0x55d0c90a4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.980+0000 7fe336ffd640 1 -- 172.21.15.67:0/1931937333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0c8ce2070 msgr2=0x55d0c90a52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:34.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.980+0000 7fe336ffd640 1 --2- 172.21.15.67:0/1931937333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0c8ce2070 0x55d0c90a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.980+0000 7fe336ffd640 1 -- 172.21.15.67:0/1931937333 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0c90a5830 con 0x55d0c8cb91b0 2024-07-27T03:18:34.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.980+0000 7fe32ffff640 1 --2- 172.21.15.67:0/1931937333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0c8cc1390 0x55d0c90a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:18:34.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.980+0000 7fe336ffd640 1 --2- 172.21.15.67:0/1931937333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cb91b0 0x55d0c90a7890 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x55d0c90859c0 tx=0x7fe32800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3672dce084762f1 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:34.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.980+0000 7fe31e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:34.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.980+0000 7fe31e7fc640 1 -- 172.21.15.67:0/1931937333 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328003a00 con 0x55d0c8cb91b0 2024-07-27T03:18:34.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.980+0000 7fe31e7fc640 1 -- 172.21.15.67:0/1931937333 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fe328003ba0 con 0x55d0c8cb91b0 2024-07-27T03:18:34.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.980+0000 7fe343c4b740 1 -- 172.21.15.67:0/1931937333 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3200f5070 con 0x55d0c8cb91b0 2024-07-27T03:18:34.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.980+0000 7fe31e7fc640 1 -- 172.21.15.67:0/1931937333 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe328034050 con 0x55d0c8cb91b0 2024-07-27T03:18:34.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.981+0000 7fe343c4b740 10 client.15873 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:34.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.981+0000 7fe343c4b740 1 -- 172.21.15.67:0/1931937333 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d0c910c680 con 0x55d0c8cb91b0 2024-07-27T03:18:34.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.981+0000 7fe31e7fc640 1 -- 172.21.15.67:0/1931937333 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3200f5070 con 0x55d0c8cb91b0 2024-07-27T03:18:34.987 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.981+0000 7fe31e7fc640 1 -- 172.21.15.67:0/1931937333 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe32802f030 con 0x55d0c8cb91b0 2024-07-27T03:18:34.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.981+0000 7fe343c4b740 10 client.15873 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:34.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.981+0000 7fe343c4b740 10 client.15873 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-27T03:18:34.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.981+0000 7fe343c4b740 20 client.15873 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:34.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.981+0000 7fe343c4b740 1 --2- 172.21.15.67:0/1931937333 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55d0c8c75da0 0x55d0c910ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:34.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.981+0000 7fe343c4b740 4 client.15873 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-27T03:18:34.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.981+0000 7fe343c4b740 1 -- 172.21.15.67:0/1931937333 --> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d0c8ffb510 con 0x55d0c8c75da0 2024-07-27T03:18:34.988 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.982+0000 7fe32ffff640 1 --2- 172.21.15.67:0/1931937333 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55d0c8c75da0 0x55d0c910ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:34.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.983+0000 7fe32ffff640 1 --2- 172.21.15.67:0/1931937333 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55d0c8c75da0 0x55d0c910ee90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=596a57ae439d0642 server_cookie=7837347d7fd32855 in_seq=0 out_seq=0 2024-07-27T03:18:34.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.983+0000 7fe31e7fc640 10 client.15873 ms_handle_connect on v2:172.21.15.175:6838/1670568853 2024-07-27T03:18:34.989 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.983+0000 7fe31e7fc640 1 -- 172.21.15.67:0/1931937333 <== mds.2 v2:172.21.15.175:6838/1670568853 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d0c8ffb510 con 0x55d0c8c75da0 2024-07-27T03:18:34.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.983+0000 7fe31e7fc640 10 client.15873 handle_command_reply: tid=0 2024-07-27T03:18:34.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.983+0000 7fe343c4b740 1 client.15873 shutdown 2024-07-27T03:18:34.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.984+0000 7fe343c4b740 1 -- 172.21.15.67:0/1931937333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cb91b0 msgr2=0x55d0c90a7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:34.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.984+0000 7fe343c4b740 1 --2- 172.21.15.67:0/1931937333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cb91b0 0x55d0c90a7890 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x55d0c90859c0 tx=0x7fe32800be70 comp rx=0 tx=0).stop 2024-07-27T03:18:34.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.984+0000 7fe343c4b740 1 -- 172.21.15.67:0/1931937333 shutdown_connections 2024-07-27T03:18:34.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.984+0000 7fe343c4b740 1 --2- 172.21.15.67:0/1931937333 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55d0c8c75da0 0x55d0c910ee90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.984+0000 7fe343c4b740 1 --2- 172.21.15.67:0/1931937333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d0c8ce2070 0x55d0c90a52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.984+0000 7fe343c4b740 1 --2- 172.21.15.67:0/1931937333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d0c8cc1390 0x55d0c90a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.984+0000 7fe343c4b740 1 --2- 172.21.15.67:0/1931937333 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55d0c8cb91b0 0x55d0c90a7890 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.984+0000 7fe343c4b740 1 -- 172.21.15.67:0/1931937333 >> 172.21.15.67:0/1931937333 conn(0x55d0c8cd6460 msgr2=0x55d0c90886e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:34.990 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.984+0000 7fe31e7fc640 0 client.15873 ms_handle_reset on v2:172.21.15.175:6838/1670568853 2024-07-27T03:18:34.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.984+0000 7fe343c4b740 1 -- 172.21.15.67:0/1931937333 shutdown_connections 2024-07-27T03:18:34.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.984+0000 7fe343c4b740 1 -- 172.21.15.67:0/1931937333 wait complete. 2024-07-27T03:18:34.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.984+0000 7fe343c4b740 20 client.15873 trim_cache size 0 max 16384 2024-07-27T03:18:34.991 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-27T03:18:34.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.985+0000 7fe33d6c6640 1 -- 172.21.15.67:0/4156573168 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe3240b3bd0 msgr2=0x7fe3240b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:34.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.985+0000 7fe33d6c6640 1 --2- 172.21.15.67:0/4156573168 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe3240b3bd0 0x7fe3240b60e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe33002f500 tx=0x7fe330009820 comp rx=0 tx=0).stop 2024-07-27T03:18:34.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.985+0000 7fe33d6c6640 1 -- 172.21.15.67:0/4156573168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe338104d90 msgr2=0x7fe33819a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:34.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.985+0000 7fe33d6c6640 1 --2- 172.21.15.67:0/4156573168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe338104d90 0x7fe33819a510 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe32000d950 tx=0x7fe32000de10 comp rx=0 tx=0).stop 2024-07-27T03:18:34.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.985+0000 7fe33d6c6640 1 -- 172.21.15.67:0/4156573168 shutdown_connections 2024-07-27T03:18:34.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.985+0000 7fe33d6c6640 1 --2- 172.21.15.67:0/4156573168 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe3240b3bd0 0x7fe3240b60e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.985+0000 7fe33d6c6640 1 --2- 172.21.15.67:0/4156573168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe33810d8a0 0x7fe33819f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.985+0000 7fe33d6c6640 1 --2- 172.21.15.67:0/4156573168 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe338104d90 0x7fe33819a510 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.985+0000 7fe33d6c6640 1 --2- 172.21.15.67:0/4156573168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe338104440 0x7fe338199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:34.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.985+0000 7fe33d6c6640 1 -- 172.21.15.67:0/4156573168 >> 172.21.15.67:0/4156573168 conn(0x7fe3380791b0 msgr2=0x7fe338105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:34.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.985+0000 7fe33d6c6640 1 -- 172.21.15.67:0/4156573168 shutdown_connections 2024-07-27T03:18:34.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:34.985+0000 7fe33d6c6640 1 -- 172.21.15.67:0/4156573168 wait complete. 2024-07-27T03:18:35.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:34 smithi149 ceph-mon[37173]: pgmap v490: 129 pgs: 129 active+clean; 202 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 19 MiB/s wr, 16 op/s 2024-07-27T03:18:35.640 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-27T03:18:35.938 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:18:36.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:36 smithi175 ceph-mon[37125]: pgmap v491: 129 pgs: 129 active+clean; 230 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-07-27T03:18:36.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:36 smithi067 ceph-mon[32814]: pgmap v491: 129 pgs: 129 active+clean; 230 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-07-27T03:18:37.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:36 smithi149 ceph-mon[37173]: pgmap v491: 129 pgs: 129 active+clean; 230 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 MiB/s wr, 21 op/s 2024-07-27T03:18:37.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.733+0000 7f5f2b215640 1 -- 172.21.15.67:0/2370282596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f24108fe0 msgr2=0x7f5f241093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:37.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.733+0000 7f5f2b215640 1 --2- 172.21.15.67:0/2370282596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f24108fe0 0x7f5f241093f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009900 tx=0x7f5f1002a410 comp rx=0 tx=0).stop 2024-07-27T03:18:37.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.734+0000 7f5f2b215640 1 -- 172.21.15.67:0/2370282596 shutdown_connections 2024-07-27T03:18:37.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.734+0000 7f5f2b215640 1 --2- 172.21.15.67:0/2370282596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f24109930 0x7f5f24110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.770 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.734+0000 7f5f2b215640 1 --2- 172.21.15.67:0/2370282596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f24108fe0 0x7f5f241093f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.734+0000 7f5f2b215640 1 --2- 172.21.15.67:0/2370282596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f241036a0 0x7f5f24103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.734+0000 7f5f2b215640 1 -- 172.21.15.67:0/2370282596 >> 172.21.15.67:0/2370282596 conn(0x7f5f2407b340 msgr2=0x7f5f2407b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:37.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.734+0000 7f5f2b215640 1 -- 172.21.15.67:0/2370282596 shutdown_connections 2024-07-27T03:18:37.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.734+0000 7f5f2b215640 1 -- 172.21.15.67:0/2370282596 wait complete. 2024-07-27T03:18:37.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.734+0000 7f5f2b215640 1 Processor -- start 2024-07-27T03:18:37.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.734+0000 7f5f2b215640 1 -- start start 2024-07-27T03:18:37.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.735+0000 7f5f2b215640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f241036a0 0x7f5f24199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:37.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.735+0000 7f5f2b215640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f24108fe0 0x7f5f2419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:37.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.735+0000 7f5f2b215640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f24109930 0x7f5f2419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:37.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.735+0000 7f5f2b215640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5f24113520 con 0x7f5f24108fe0 2024-07-27T03:18:37.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.735+0000 7f5f2b215640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241133a0 con 0x7f5f241036a0 2024-07-27T03:18:37.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.735+0000 7f5f2b215640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241136a0 con 0x7f5f24109930 2024-07-27T03:18:37.771 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.735+0000 7f5f23fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f24108fe0 0x7f5f2419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:37.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.735+0000 7f5f23fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f24108fe0 0x7f5f2419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56108/0 (socket says 172.21.15.67:56108) 2024-07-27T03:18:37.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.735+0000 7f5f23fff640 1 -- 172.21.15.67:0/1055553299 learned_addr learned my addr 172.21.15.67:0/1055553299 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:37.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.735+0000 7f5f28fb1640 1 --2- 172.21.15.67:0/1055553299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f241036a0 0x7f5f24199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:37.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.735+0000 7f5f297b2640 1 --2- 172.21.15.67:0/1055553299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f24109930 0x7f5f2419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:37.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.735+0000 7f5f28fb1640 1 -- 172.21.15.67:0/1055553299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f24109930 msgr2=0x7f5f2419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:37.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.735+0000 7f5f28fb1640 1 --2- 172.21.15.67:0/1055553299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f24109930 0x7f5f2419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.735+0000 7f5f28fb1640 1 -- 172.21.15.67:0/1055553299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f24108fe0 msgr2=0x7f5f2419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:37.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.735+0000 7f5f28fb1640 1 --2- 172.21.15.67:0/1055553299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f24108fe0 0x7f5f2419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.736+0000 7f5f28fb1640 1 -- 172.21.15.67:0/1055553299 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2419fec0 con 0x7f5f241036a0 2024-07-27T03:18:37.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.736+0000 7f5f297b2640 1 --2- 172.21.15.67:0/1055553299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f24109930 0x7f5f2419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:18:37.772 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.736+0000 7f5f23fff640 1 --2- 172.21.15.67:0/1055553299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f24108fe0 0x7f5f2419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:18:37.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.736+0000 7f5f28fb1640 1 --2- 172.21.15.67:0/1055553299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f241036a0 0x7f5f24199f50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800e9f0 tx=0x7f5f1800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:37.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.736+0000 7f5f21ffb640 1 -- 172.21.15.67:0/1055553299 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1800ccc0 con 0x7f5f241036a0 2024-07-27T03:18:37.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.736+0000 7f5f2b215640 1 -- 172.21.15.67:0/1055553299 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f241a01b0 con 0x7f5f241036a0 2024-07-27T03:18:37.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.736+0000 7f5f21ffb640 1 -- 172.21.15.67:0/1055553299 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f18005020 con 0x7f5f241036a0 2024-07-27T03:18:37.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.736+0000 7f5f21ffb640 1 -- 172.21.15.67:0/1055553299 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f18004a00 con 0x7f5f241036a0 2024-07-27T03:18:37.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.736+0000 7f5f2b215640 1 -- 172.21.15.67:0/1055553299 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f241a6910 con 0x7f5f241036a0 2024-07-27T03:18:37.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.739+0000 7f5f21ffb640 1 -- 172.21.15.67:0/1055553299 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f5f18004ba0 con 0x7f5f241036a0 2024-07-27T03:18:37.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.740+0000 7f5f21ffb640 1 --2- 172.21.15.67:0/1055553299 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5f140b3c50 0x7f5f140b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:37.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.740+0000 7f5f23fff640 1 --2- 172.21.15.67:0/1055553299 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5f140b3c50 0x7f5f140b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:37.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.741+0000 7f5f21ffb640 1 -- 172.21.15.67:0/1055553299 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f5f180e3fd0 con 0x7f5f241036a0 2024-07-27T03:18:37.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.741+0000 7f5f3179a740 1 Processor -- start 2024-07-27T03:18:37.773 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.741+0000 7f5f3179a740 1 -- start start 2024-07-27T03:18:37.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.741+0000 7f5f23fff640 1 --2- 172.21.15.67:0/1055553299 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5f140b3c50 0x7f5f140b6160 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5f24107050 tx=0x7f5f10032ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:37.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.741+0000 7f5f3179a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8ca1070 0x557cd9045e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:37.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.741+0000 7f5f3179a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cd8c781b0 0x557cd9046370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:37.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.741+0000 7f5f3179a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8c80390 0x557cd90468b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:37.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.741+0000 7f5f3179a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557cd8f2e400 con 0x557cd8c80390 2024-07-27T03:18:37.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.741+0000 7f5f3179a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557cd8fbcb20 con 0x557cd8ca1070 2024-07-27T03:18:37.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.741+0000 7f5f3179a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557cd8fac020 con 0x557cd8c781b0 2024-07-27T03:18:37.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.741+0000 7f5f28fb1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8ca1070 0x557cd9045e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:37.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.741+0000 7f5f297b2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8c80390 0x557cd90468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:37.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.741+0000 7f5f28fb1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8ca1070 0x557cd9045e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.67:42986/0 (socket says 172.21.15.67:42986) 2024-07-27T03:18:37.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.741+0000 7f5f28fb1640 1 -- 172.21.15.67:0/1988372908 learned_addr learned my addr 172.21.15.67:0/1988372908 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:37.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.742+0000 7f5f23fff640 1 --2- 172.21.15.67:0/1988372908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cd8c781b0 0x557cd9046370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:37.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.742+0000 7f5f297b2640 1 -- 172.21.15.67:0/1988372908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cd8c781b0 msgr2=0x557cd9046370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:37.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.742+0000 7f5f297b2640 1 --2- 172.21.15.67:0/1988372908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cd8c781b0 0x557cd9046370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.742+0000 7f5f297b2640 1 -- 172.21.15.67:0/1988372908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8ca1070 msgr2=0x557cd9045e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:37.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.742+0000 7f5f297b2640 1 --2- 172.21.15.67:0/1988372908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8ca1070 0x557cd9045e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.742+0000 7f5f297b2640 1 -- 172.21.15.67:0/1988372908 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cd8f41770 con 0x557cd8c80390 2024-07-27T03:18:37.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.742+0000 7f5f28fb1640 1 --2- 172.21.15.67:0/1988372908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8ca1070 0x557cd9045e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:18:37.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.742+0000 7f5f23fff640 1 --2- 172.21.15.67:0/1988372908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cd8c781b0 0x557cd9046370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:18:37.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.742+0000 7f5f297b2640 1 --2- 172.21.15.67:0/1988372908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8c80390 0x557cd90468b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009f90 tx=0x7f5f1c010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96378efdda68051 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:37.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.742+0000 7f5f0effd640 1 -- 172.21.15.67:0/1988372908 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c004d20 con 0x557cd8c80390 2024-07-27T03:18:37.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.742+0000 7f5f0effd640 1 -- 172.21.15.67:0/1988372908 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f1c0119b0 con 0x557cd8c80390 2024-07-27T03:18:37.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.742+0000 7f5f0effd640 1 -- 172.21.15.67:0/1988372908 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c0054f0 con 0x557cd8c80390 2024-07-27T03:18:37.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.744+0000 7f5f3179a740 1 -- 172.21.15.67:0/1988372908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8c80390 msgr2=0x557cd90468b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:37.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.744+0000 7f5f3179a740 1 --2- 172.21.15.67:0/1988372908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8c80390 0x557cd90468b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009f90 tx=0x7f5f1c010860 comp rx=0 tx=0).stop 2024-07-27T03:18:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.744+0000 7f5f3179a740 1 -- 172.21.15.67:0/1988372908 shutdown_connections 2024-07-27T03:18:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.744+0000 7f5f3179a740 1 --2- 172.21.15.67:0/1988372908 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8c80390 0x557cd90468b0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.744+0000 7f5f3179a740 1 --2- 172.21.15.67:0/1988372908 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cd8c781b0 0x557cd9046370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.744+0000 7f5f3179a740 1 --2- 172.21.15.67:0/1988372908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8ca1070 0x557cd9045e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.744+0000 7f5f3179a740 1 -- 172.21.15.67:0/1988372908 >> 172.21.15.67:0/1988372908 conn(0x557cd8c95460 msgr2=0x557cd9043d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.744+0000 7f5f3179a740 1 -- 172.21.15.67:0/1988372908 shutdown_connections 2024-07-27T03:18:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.744+0000 7f5f3179a740 1 -- 172.21.15.67:0/1988372908 wait complete. 2024-07-27T03:18:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.744+0000 7f5f3179a740 1 -- start start 2024-07-27T03:18:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.745+0000 7f5f3179a740 1 Processor -- start 2024-07-27T03:18:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.745+0000 7f5f3179a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8c781b0 0x557cd90519c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.745+0000 7f5f3179a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cd8c80390 0x557cd9051f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.745+0000 7f5f3179a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8ca1070 0x557cd905d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.745+0000 7f5f3179a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x557cd8f4e900 con 0x557cd8ca1070 2024-07-27T03:18:37.776 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.745+0000 7f5f3179a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x557cd8fb0060 con 0x557cd8c781b0 2024-07-27T03:18:37.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.745+0000 7f5f3179a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x557cd8e0f700 con 0x557cd8c80390 2024-07-27T03:18:37.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.745+0000 7f5f3179a740 10 client.0 _ll_register_callbacks cb 0x557cd8f56d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:37.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.746+0000 7f5f28fb1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8c781b0 0x557cd90519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:37.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.746+0000 7f5f297b2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8ca1070 0x557cd905d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:37.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.746+0000 7f5f28fb1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8c781b0 0x557cd90519c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.67:42998/0 (socket says 172.21.15.67:42998) 2024-07-27T03:18:37.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.746+0000 7f5f28fb1640 1 -- 172.21.15.67:0/557450747 learned_addr learned my addr 172.21.15.67:0/557450747 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:37.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.746+0000 7f5f297b2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8ca1070 0x557cd905d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56126/0 (socket says 172.21.15.67:56126) 2024-07-27T03:18:37.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.746+0000 7f5f23fff640 1 --2- 172.21.15.67:0/557450747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cd8c80390 0x557cd9051f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:37.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.746+0000 7f5f297b2640 1 -- 172.21.15.67:0/557450747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cd8c80390 msgr2=0x557cd9051f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:37.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.746+0000 7f5f297b2640 1 --2- 172.21.15.67:0/557450747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cd8c80390 0x557cd9051f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.746+0000 7f5f297b2640 1 -- 172.21.15.67:0/557450747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8c781b0 msgr2=0x557cd90519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:37.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.746+0000 7f5f297b2640 1 --2- 172.21.15.67:0/557450747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8c781b0 0x557cd90519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.746+0000 7f5f297b2640 1 -- 172.21.15.67:0/557450747 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cd8f52bf0 con 0x557cd8ca1070 2024-07-27T03:18:37.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.746+0000 7f5f28fb1640 1 --2- 172.21.15.67:0/557450747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8c781b0 0x557cd90519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:18:37.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.746+0000 7f5f297b2640 1 --2- 172.21.15.67:0/557450747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8ca1070 0x557cd905d7b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c000f80 tx=0x7f5f1c011730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5851b3e156e67e6 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:37.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.746+0000 7f5f03fff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:37.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.746+0000 7f5f03fff640 1 -- 172.21.15.67:0/557450747 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c02b060 con 0x557cd8ca1070 2024-07-27T03:18:37.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.747+0000 7f5f03fff640 1 -- 172.21.15.67:0/557450747 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f5f1c004c50 con 0x557cd8ca1070 2024-07-27T03:18:37.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.747+0000 7f5f3179a740 1 -- 172.21.15.67:0/557450747 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f1c0119b0 con 0x557cd8ca1070 2024-07-27T03:18:37.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.747+0000 7f5f03fff640 1 -- 172.21.15.67:0/557450747 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5f1c032050 con 0x557cd8ca1070 2024-07-27T03:18:37.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.747+0000 7f5f3179a740 10 client.15885 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:37.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.747+0000 7f5f3179a740 1 -- 172.21.15.67:0/557450747 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557cd8d0c650 con 0x557cd8ca1070 2024-07-27T03:18:37.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.747+0000 7f5f03fff640 1 -- 172.21.15.67:0/557450747 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f1c0119b0 con 0x557cd8ca1070 2024-07-27T03:18:37.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.747+0000 7f5f03fff640 1 -- 172.21.15.67:0/557450747 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5f1c01b550 con 0x557cd8ca1070 2024-07-27T03:18:37.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.747+0000 7f5f3179a740 10 client.15885 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:37.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.747+0000 7f5f3179a740 10 client.15885 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-07-27T03:18:37.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.747+0000 7f5f3179a740 20 client.15885 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:37.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.747+0000 7f5f3179a740 1 --2- 172.21.15.67:0/557450747 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x557cd8c5f9f0 0x557cd90636e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:37.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.747+0000 7f5f3179a740 4 client.15885 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:18:37.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.747+0000 7f5f3179a740 1 -- 172.21.15.67:0/557450747 --> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cd8fbea70 con 0x557cd8c5f9f0 2024-07-27T03:18:37.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.748+0000 7f5f28fb1640 1 --2- 172.21.15.67:0/557450747 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x557cd8c5f9f0 0x557cd90636e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:37.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.748+0000 7f5f03fff640 10 client.15885 ms_handle_connect on v2:172.21.15.149:6836/2932951845 2024-07-27T03:18:37.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.748+0000 7f5f28fb1640 1 --2- 172.21.15.67:0/557450747 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x557cd8c5f9f0 0x557cd90636e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c667dd32803a241a server_cookie=d8705ca80fe82f10 in_seq=0 out_seq=0 2024-07-27T03:18:37.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.749+0000 7f5f03fff640 1 -- 172.21.15.67:0/557450747 <== mds.1 v2:172.21.15.149:6836/2932951845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557cd8fbea70 con 0x557cd8c5f9f0 2024-07-27T03:18:37.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.749+0000 7f5f03fff640 10 client.15885 handle_command_reply: tid=0 2024-07-27T03:18:37.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.749+0000 7f5f3179a740 1 client.15885 shutdown 2024-07-27T03:18:37.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.749+0000 7f5f3179a740 1 -- 172.21.15.67:0/557450747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8ca1070 msgr2=0x557cd905d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:37.779 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.749+0000 7f5f3179a740 1 --2- 172.21.15.67:0/557450747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8ca1070 0x557cd905d7b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c000f80 tx=0x7f5f1c011730 comp rx=0 tx=0).stop 2024-07-27T03:18:37.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.749+0000 7f5f3179a740 1 -- 172.21.15.67:0/557450747 shutdown_connections 2024-07-27T03:18:37.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.749+0000 7f5f3179a740 1 --2- 172.21.15.67:0/557450747 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x557cd8c5f9f0 0x557cd90636e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.749+0000 7f5f3179a740 1 --2- 172.21.15.67:0/557450747 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8ca1070 0x557cd905d7b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.750+0000 7f5f03fff640 0 client.15885 ms_handle_reset on v2:172.21.15.149:6836/2932951845 2024-07-27T03:18:37.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.750+0000 7f5f3179a740 1 --2- 172.21.15.67:0/557450747 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cd8c80390 0x557cd9051f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.750+0000 7f5f3179a740 1 --2- 172.21.15.67:0/557450747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8c781b0 0x557cd90519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.750+0000 7f5f3179a740 1 -- 172.21.15.67:0/557450747 >> 172.21.15.67:0/557450747 conn(0x557cd8c95460 msgr2=0x557cd9048fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:37.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.750+0000 7f5f3179a740 1 -- 172.21.15.67:0/557450747 shutdown_connections 2024-07-27T03:18:37.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.750+0000 7f5f3179a740 1 -- 172.21.15.67:0/557450747 wait complete. 2024-07-27T03:18:37.780 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.750+0000 7f5f3179a740 20 client.15885 trim_cache size 0 max 16384 2024-07-27T03:18:37.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.775+0000 7f5f3179a740 1 Processor -- start 2024-07-27T03:18:37.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.775+0000 7f5f3179a740 1 -- start start 2024-07-27T03:18:37.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.775+0000 7f5f3179a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8c5f9f0 0x557cd9042cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:37.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.775+0000 7f5f3179a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8c781b0 0x557cd9047570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:37.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.775+0000 7f5f3179a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cd8c80390 0x557cd905d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:37.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.775+0000 7f5f3179a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241b06d0 con 0x557cd8c5f9f0 2024-07-27T03:18:37.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.775+0000 7f5f3179a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241b2210 con 0x557cd8c781b0 2024-07-27T03:18:37.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.775+0000 7f5f3179a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241b0da0 con 0x557cd8c80390 2024-07-27T03:18:37.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.775+0000 7f5f28fb1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8c5f9f0 0x557cd9042cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:37.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.775+0000 7f5f28fb1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8c5f9f0 0x557cd9042cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56128/0 (socket says 172.21.15.67:56128) 2024-07-27T03:18:37.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.775+0000 7f5f28fb1640 1 -- 172.21.15.67:0/3733923742 learned_addr learned my addr 172.21.15.67:0/3733923742 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:37.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.775+0000 7f5f23fff640 1 --2- 172.21.15.67:0/3733923742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8c781b0 0x557cd9047570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:37.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.775+0000 7f5f297b2640 1 --2- 172.21.15.67:0/3733923742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cd8c80390 0x557cd905d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:37.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.776+0000 7f5f28fb1640 1 -- 172.21.15.67:0/3733923742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cd8c80390 msgr2=0x557cd905d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:37.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.776+0000 7f5f28fb1640 1 --2- 172.21.15.67:0/3733923742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cd8c80390 0x557cd905d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.776+0000 7f5f28fb1640 1 -- 172.21.15.67:0/3733923742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8c781b0 msgr2=0x557cd9047570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:37.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.776+0000 7f5f28fb1640 1 --2- 172.21.15.67:0/3733923742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8c781b0 0x557cd9047570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.776+0000 7f5f28fb1640 1 -- 172.21.15.67:0/3733923742 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cd9052a80 con 0x557cd8c5f9f0 2024-07-27T03:18:37.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.776+0000 7f5f28fb1640 1 --2- 172.21.15.67:0/3733923742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8c5f9f0 0x557cd9042cc0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5f180aaaa0 tx=0x7f5f180096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02abe7b8bd7a837 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:37.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.776+0000 7f5f0e7fc640 1 -- 172.21.15.67:0/3733923742 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1801c040 con 0x557cd8c5f9f0 2024-07-27T03:18:37.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.776+0000 7f5f0e7fc640 1 -- 172.21.15.67:0/3733923742 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f180af110 con 0x557cd8c5f9f0 2024-07-27T03:18:37.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.776+0000 7f5f0e7fc640 1 -- 172.21.15.67:0/3733923742 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f180aafd0 con 0x557cd8c5f9f0 2024-07-27T03:18:37.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.778+0000 7f5f3179a740 1 -- 172.21.15.67:0/3733923742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8c5f9f0 msgr2=0x557cd9042cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:37.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.778+0000 7f5f3179a740 1 --2- 172.21.15.67:0/3733923742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8c5f9f0 0x557cd9042cc0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5f180aaaa0 tx=0x7f5f180096e0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.778+0000 7f5f3179a740 1 -- 172.21.15.67:0/3733923742 shutdown_connections 2024-07-27T03:18:37.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.778+0000 7f5f3179a740 1 --2- 172.21.15.67:0/3733923742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cd8c80390 0x557cd905d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.778+0000 7f5f3179a740 1 --2- 172.21.15.67:0/3733923742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8c781b0 0x557cd9047570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.778+0000 7f5f3179a740 1 --2- 172.21.15.67:0/3733923742 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8c5f9f0 0x557cd9042cc0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.778+0000 7f5f3179a740 1 -- 172.21.15.67:0/3733923742 >> 172.21.15.67:0/3733923742 conn(0x557cd8c95460 msgr2=0x557cd9046e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:37.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.778+0000 7f5f3179a740 1 -- 172.21.15.67:0/3733923742 shutdown_connections 2024-07-27T03:18:37.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.778+0000 7f5f3179a740 1 -- 172.21.15.67:0/3733923742 wait complete. 2024-07-27T03:18:37.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.778+0000 7f5f3179a740 1 -- start start 2024-07-27T03:18:37.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.778+0000 7f5f3179a740 1 Processor -- start 2024-07-27T03:18:37.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.779+0000 7f5f3179a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cd8c5f9f0 0x557cd9064a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:37.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.779+0000 7f5f3179a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8c781b0 0x557cd9065000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:37.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.779+0000 7f5f3179a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8c80390 0x557cd90cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:37.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.779+0000 7f5f3179a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241b1b40 con 0x557cd8c80390 2024-07-27T03:18:37.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.779+0000 7f5f3179a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241b1470 con 0x557cd8c781b0 2024-07-27T03:18:37.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.779+0000 7f5f3179a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f241ae6b0 con 0x557cd8c5f9f0 2024-07-27T03:18:37.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.779+0000 7f5f3179a740 10 client.0 _ll_register_callbacks cb 0x7f5f18004f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:37.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.779+0000 7f5f297b2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8c80390 0x557cd90cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:37.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.779+0000 7f5f297b2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8c80390 0x557cd90cac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56144/0 (socket says 172.21.15.67:56144) 2024-07-27T03:18:37.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.779+0000 7f5f297b2640 1 -- 172.21.15.67:0/3847247641 learned_addr learned my addr 172.21.15.67:0/3847247641 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:37.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.779+0000 7f5f28fb1640 1 --2- 172.21.15.67:0/3847247641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cd8c5f9f0 0x557cd9064a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:37.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.779+0000 7f5f23fff640 1 --2- 172.21.15.67:0/3847247641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8c781b0 0x557cd9065000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:37.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.779+0000 7f5f297b2640 1 -- 172.21.15.67:0/3847247641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cd8c5f9f0 msgr2=0x557cd9064a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:37.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.779+0000 7f5f297b2640 1 --2- 172.21.15.67:0/3847247641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cd8c5f9f0 0x557cd9064a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.779+0000 7f5f297b2640 1 -- 172.21.15.67:0/3847247641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8c781b0 msgr2=0x557cd9065000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:37.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.779+0000 7f5f297b2640 1 --2- 172.21.15.67:0/3847247641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8c781b0 0x557cd9065000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.779+0000 7f5f297b2640 1 -- 172.21.15.67:0/3847247641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cd90cb170 con 0x557cd8c80390 2024-07-27T03:18:37.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.780+0000 7f5f297b2640 1 --2- 172.21.15.67:0/3847247641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8c80390 0x557cd90cac30 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0092a0 tx=0x7f5f1c009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ad900f62e75bfe server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:37.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.780+0000 7f5f03fff640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:37.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.780+0000 7f5f03fff640 1 -- 172.21.15.67:0/3847247641 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c03b060 con 0x557cd8c80390 2024-07-27T03:18:37.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.780+0000 7f5f03fff640 1 -- 172.21.15.67:0/3847247641 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f5f1c0117a0 con 0x557cd8c80390 2024-07-27T03:18:37.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.780+0000 7f5f3179a740 1 -- 172.21.15.67:0/3847247641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f180af110 con 0x557cd8c80390 2024-07-27T03:18:37.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.780+0000 7f5f03fff640 1 -- 172.21.15.67:0/3847247641 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5f1c042050 con 0x557cd8c80390 2024-07-27T03:18:37.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.780+0000 7f5f3179a740 10 client.15891 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:37.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.780+0000 7f5f3179a740 1 -- 172.21.15.67:0/3847247641 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557cd90cc300 con 0x557cd8c80390 2024-07-27T03:18:37.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.780+0000 7f5f03fff640 1 -- 172.21.15.67:0/3847247641 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f180af110 con 0x557cd8c80390 2024-07-27T03:18:37.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.781+0000 7f5f03fff640 1 -- 172.21.15.67:0/3847247641 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5f1c026880 con 0x557cd8c80390 2024-07-27T03:18:37.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.781+0000 7f5f3179a740 10 client.15891 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:37.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.781+0000 7f5f3179a740 10 client.15891 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-07-27T03:18:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.781+0000 7f5f3179a740 20 client.15891 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.781+0000 7f5f3179a740 1 --2- 172.21.15.67:0/3847247641 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x557cd8ca1070 0x557cd90cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.781+0000 7f5f3179a740 4 client.15891 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-27T03:18:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.781+0000 7f5f3179a740 1 -- 172.21.15.67:0/3847247641 --> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557cd8fba510 con 0x557cd8ca1070 2024-07-27T03:18:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.781+0000 7f5f28fb1640 1 --2- 172.21.15.67:0/3847247641 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x557cd8ca1070 0x557cd90cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.781+0000 7f5f03fff640 10 client.15891 ms_handle_connect on v2:172.21.15.149:6836/2932951845 2024-07-27T03:18:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.781+0000 7f5f28fb1640 1 --2- 172.21.15.67:0/3847247641 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x557cd8ca1070 0x557cd90cea30 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a35c0ddd53c192e server_cookie=b8b8a50ce1d8276d in_seq=0 out_seq=0 2024-07-27T03:18:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.782+0000 7f5f03fff640 1 -- 172.21.15.67:0/3847247641 <== mds.1 v2:172.21.15.149:6836/2932951845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557cd8fba510 con 0x557cd8ca1070 2024-07-27T03:18:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.782+0000 7f5f03fff640 10 client.15891 handle_command_reply: tid=0 2024-07-27T03:18:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.782+0000 7f5f3179a740 1 client.15891 shutdown 2024-07-27T03:18:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.782+0000 7f5f3179a740 1 -- 172.21.15.67:0/3847247641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8c80390 msgr2=0x557cd90cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:37.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.782+0000 7f5f3179a740 1 --2- 172.21.15.67:0/3847247641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8c80390 0x557cd90cac30 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0092a0 tx=0x7f5f1c009e10 comp rx=0 tx=0).stop 2024-07-27T03:18:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.782+0000 7f5f3179a740 1 -- 172.21.15.67:0/3847247641 shutdown_connections 2024-07-27T03:18:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.782+0000 7f5f3179a740 1 --2- 172.21.15.67:0/3847247641 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x557cd8ca1070 0x557cd90cea30 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.782+0000 7f5f3179a740 1 --2- 172.21.15.67:0/3847247641 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x557cd8c80390 0x557cd90cac30 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.782+0000 7f5f3179a740 1 --2- 172.21.15.67:0/3847247641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cd8c781b0 0x557cd9065000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.782+0000 7f5f03fff640 0 client.15891 ms_handle_reset on v2:172.21.15.149:6836/2932951845 2024-07-27T03:18:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.782+0000 7f5f3179a740 1 --2- 172.21.15.67:0/3847247641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x557cd8c5f9f0 0x557cd9064a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.782+0000 7f5f3179a740 1 -- 172.21.15.67:0/3847247641 >> 172.21.15.67:0/3847247641 conn(0x557cd8c95460 msgr2=0x557cd9042a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.782+0000 7f5f3179a740 1 -- 172.21.15.67:0/3847247641 shutdown_connections 2024-07-27T03:18:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.782+0000 7f5f3179a740 1 -- 172.21.15.67:0/3847247641 wait complete. 2024-07-27T03:18:37.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.782+0000 7f5f3179a740 20 client.15891 trim_cache size 0 max 16384 2024-07-27T03:18:37.793 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-27T03:18:37.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.783+0000 7f5f2b215640 1 -- 172.21.15.67:0/1055553299 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5f140b3c50 msgr2=0x7f5f140b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:37.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.783+0000 7f5f2b215640 1 --2- 172.21.15.67:0/1055553299 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5f140b3c50 0x7f5f140b6160 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5f24107050 tx=0x7f5f10032ed0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.783+0000 7f5f2b215640 1 -- 172.21.15.67:0/1055553299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f241036a0 msgr2=0x7f5f24199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:37.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.783+0000 7f5f2b215640 1 --2- 172.21.15.67:0/1055553299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f241036a0 0x7f5f24199f50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800e9f0 tx=0x7f5f1800eeb0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.783+0000 7f5f2b215640 1 -- 172.21.15.67:0/1055553299 shutdown_connections 2024-07-27T03:18:37.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.783+0000 7f5f2b215640 1 --2- 172.21.15.67:0/1055553299 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f5f140b3c50 0x7f5f140b6160 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.783+0000 7f5f2b215640 1 --2- 172.21.15.67:0/1055553299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f24109930 0x7f5f2419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.783+0000 7f5f2b215640 1 --2- 172.21.15.67:0/1055553299 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5f24108fe0 0x7f5f2419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.783+0000 7f5f2b215640 1 --2- 172.21.15.67:0/1055553299 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f241036a0 0x7f5f24199f50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:37.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.783+0000 7f5f2b215640 1 -- 172.21.15.67:0/1055553299 >> 172.21.15.67:0/1055553299 conn(0x7f5f2407b340 msgr2=0x7f5f24105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:37.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.783+0000 7f5f2b215640 1 -- 172.21.15.67:0/1055553299 shutdown_connections 2024-07-27T03:18:37.794 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:37.783+0000 7f5f2b215640 1 -- 172.21.15.67:0/1055553299 wait complete. 2024-07-27T03:18:38.585 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-27T03:18:38.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:38 smithi175 ceph-mon[37125]: pgmap v492: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 31 MiB/s wr, 20 op/s 2024-07-27T03:18:38.879 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:18:38.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:38 smithi067 ceph-mon[32814]: pgmap v492: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 31 MiB/s wr, 20 op/s 2024-07-27T03:18:39.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:38 smithi149 ceph-mon[37173]: pgmap v492: 129 pgs: 129 active+clean; 258 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 31 MiB/s wr, 20 op/s 2024-07-27T03:18:40.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.632+0000 7f053e066640 1 -- 172.21.15.67:0/2487684722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0538108760 msgr2=0x7f053810ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:40.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.632+0000 7f053e066640 1 --2- 172.21.15.67:0/2487684722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0538108760 0x7f053810ef90 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0530009ca0 tx=0x7f053002a8c0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.632+0000 7f053e066640 1 -- 172.21.15.67:0/2487684722 shutdown_connections 2024-07-27T03:18:40.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.632+0000 7f053e066640 1 --2- 172.21.15.67:0/2487684722 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0538108760 0x7f053810ef90 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.632+0000 7f053e066640 1 --2- 172.21.15.67:0/2487684722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0538107e10 0x7f0538108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.635 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.632+0000 7f053e066640 1 --2- 172.21.15.67:0/2487684722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05381024d0 0x7f05381028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.632+0000 7f053e066640 1 -- 172.21.15.67:0/2487684722 >> 172.21.15.67:0/2487684722 conn(0x7f05380791a0 msgr2=0x7f05380795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:40.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.633+0000 7f053e066640 1 -- 172.21.15.67:0/2487684722 shutdown_connections 2024-07-27T03:18:40.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.633+0000 7f053e066640 1 -- 172.21.15.67:0/2487684722 wait complete. 2024-07-27T03:18:40.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.633+0000 7f053e066640 1 Processor -- start 2024-07-27T03:18:40.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.633+0000 7f053e066640 1 -- start start 2024-07-27T03:18:40.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.634+0000 7f053e066640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05381024d0 0x7f05380ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:40.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.634+0000 7f053e066640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0538107e10 0x7f05380ff620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:40.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.634+0000 7f053e066640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0538108760 0x7f05380ffb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:40.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.634+0000 7f053e066640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f05381123c0 con 0x7f05381024d0 2024-07-27T03:18:40.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.634+0000 7f053e066640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0538112240 con 0x7f0538107e10 2024-07-27T03:18:40.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.634+0000 7f053e066640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0538112540 con 0x7f0538108760 2024-07-27T03:18:40.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.634+0000 7f05377fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05381024d0 0x7f05380ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:40.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.634+0000 7f05377fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05381024d0 0x7f05380ff0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56158/0 (socket says 172.21.15.67:56158) 2024-07-27T03:18:40.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.634+0000 7f05377fe640 1 -- 172.21.15.67:0/631571667 learned_addr learned my addr 172.21.15.67:0/631571667 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:40.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.634+0000 7f0537fff640 1 --2- 172.21.15.67:0/631571667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0538108760 0x7f05380ffb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:40.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.634+0000 7f0536ffd640 1 --2- 172.21.15.67:0/631571667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0538107e10 0x7f05380ff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:40.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.634+0000 7f05377fe640 1 -- 172.21.15.67:0/631571667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0538108760 msgr2=0x7f05380ffb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:40.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.634+0000 7f05377fe640 1 --2- 172.21.15.67:0/631571667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0538108760 0x7f05380ffb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.634+0000 7f05377fe640 1 -- 172.21.15.67:0/631571667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0538107e10 msgr2=0x7f05380ff620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:40.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.634+0000 7f05377fe640 1 --2- 172.21.15.67:0/631571667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0538107e10 0x7f05380ff620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.634+0000 7f05377fe640 1 -- 172.21.15.67:0/631571667 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05381a81c0 con 0x7f05381024d0 2024-07-27T03:18:40.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.635+0000 7f05377fe640 1 --2- 172.21.15.67:0/631571667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05381024d0 0x7f05380ff0e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f052800e9f0 tx=0x7f052800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:40.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.635+0000 7f0534ff9640 1 -- 172.21.15.67:0/631571667 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052800cd80 con 0x7f05381024d0 2024-07-27T03:18:40.639 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:40 smithi067 ceph-mon[32814]: pgmap v493: 129 pgs: 129 active+clean; 258 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 32 MiB/s wr, 20 op/s 2024-07-27T03:18:40.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.635+0000 7f053e066640 1 -- 172.21.15.67:0/631571667 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05381a84b0 con 0x7f05381024d0 2024-07-27T03:18:40.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.635+0000 7f053e066640 1 -- 172.21.15.67:0/631571667 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05381a8a40 con 0x7f05381024d0 2024-07-27T03:18:40.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.635+0000 7f0534ff9640 1 -- 172.21.15.67:0/631571667 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05280108e0 con 0x7f05381024d0 2024-07-27T03:18:40.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.635+0000 7f0534ff9640 1 -- 172.21.15.67:0/631571667 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528018980 con 0x7f05381024d0 2024-07-27T03:18:40.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.638+0000 7f05445eb740 1 Processor -- start 2024-07-27T03:18:40.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.638+0000 7f05445eb740 1 -- start start 2024-07-27T03:18:40.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.639+0000 7f05445eb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae9f9b070 0x55eaea33fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:40.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.639+0000 7f05445eb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f721b0 0x55eaea340370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:40.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.639+0000 7f05445eb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f7a390 0x55eaea3408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:40.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.639+0000 7f05445eb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55eaea228400 con 0x55eae9f7a390 2024-07-27T03:18:40.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.639+0000 7f05445eb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55eaea2b6b20 con 0x55eae9f721b0 2024-07-27T03:18:40.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.639+0000 7f05445eb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55eaea2a6020 con 0x55eae9f9b070 2024-07-27T03:18:40.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.639+0000 7f0537fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae9f9b070 0x55eaea33fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:40.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.639+0000 7f0537fff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae9f9b070 0x55eaea33fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.67:36956/0 (socket says 172.21.15.67:36956) 2024-07-27T03:18:40.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.639+0000 7f0537fff640 1 -- 172.21.15.67:0/1453208130 learned_addr learned my addr 172.21.15.67:0/1453208130 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:40.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.639+0000 7f0534ff9640 1 -- 172.21.15.67:0/631571667 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f0528018b20 con 0x7f05381024d0 2024-07-27T03:18:40.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.640+0000 7f0536ffd640 1 --2- 172.21.15.67:0/1453208130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f7a390 0x55eaea3408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:40.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.640+0000 7f0537fff640 1 -- 172.21.15.67:0/1453208130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f721b0 msgr2=0x55eaea340370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:40.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.640+0000 7f05377fe640 1 --2- 172.21.15.67:0/1453208130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f721b0 0x55eaea340370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:40.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.640+0000 7f0537fff640 1 --2- 172.21.15.67:0/1453208130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f721b0 0x55eaea340370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.640+0000 7f0537fff640 1 -- 172.21.15.67:0/1453208130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f7a390 msgr2=0x55eaea3408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:40.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.640+0000 7f0537fff640 1 --2- 172.21.15.67:0/1453208130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f7a390 0x55eaea3408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.640+0000 7f0537fff640 1 -- 172.21.15.67:0/1453208130 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaea23b770 con 0x55eae9f9b070 2024-07-27T03:18:40.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.640+0000 7f05377fe640 1 --2- 172.21.15.67:0/1453208130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f721b0 0x55eaea340370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:18:40.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.640+0000 7f0536ffd640 1 --2- 172.21.15.67:0/1453208130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f7a390 0x55eaea3408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:18:40.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.640+0000 7f0537fff640 1 --2- 172.21.15.67:0/1453208130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae9f9b070 0x55eaea33fe30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f053002f4c0 tx=0x7f0530009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b69ff1b2a2f2e39 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.641+0000 7f052dffb640 1 -- 172.21.15.67:0/1453208130 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530031cf0 con 0x55eae9f9b070 2024-07-27T03:18:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.641+0000 7f052dffb640 1 -- 172.21.15.67:0/1453208130 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0530036030 con 0x55eae9f9b070 2024-07-27T03:18:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.641+0000 7f052dffb640 1 -- 172.21.15.67:0/1453208130 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05300042a0 con 0x55eae9f9b070 2024-07-27T03:18:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.641+0000 7f0534ff9640 1 --2- 172.21.15.67:0/631571667 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f05240b3c50 0x7f05240b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.641+0000 7f05377fe640 1 --2- 172.21.15.67:0/631571667 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f05240b3c50 0x7f05240b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.641+0000 7f0534ff9640 1 -- 172.21.15.67:0/631571667 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f05280e4270 con 0x7f05381024d0 2024-07-27T03:18:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.642+0000 7f05445eb740 1 -- 172.21.15.67:0/1453208130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae9f9b070 msgr2=0x55eaea33fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.642+0000 7f05445eb740 1 --2- 172.21.15.67:0/1453208130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae9f9b070 0x55eaea33fe30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f053002f4c0 tx=0x7f0530009800 comp rx=0 tx=0).stop 2024-07-27T03:18:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.642+0000 7f05445eb740 1 -- 172.21.15.67:0/1453208130 shutdown_connections 2024-07-27T03:18:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.642+0000 7f05445eb740 1 --2- 172.21.15.67:0/1453208130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f7a390 0x55eaea3408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.642+0000 7f05445eb740 1 --2- 172.21.15.67:0/1453208130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f721b0 0x55eaea340370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.648 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.642+0000 7f05445eb740 1 --2- 172.21.15.67:0/1453208130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae9f9b070 0x55eaea33fe30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.642+0000 7f05445eb740 1 -- 172.21.15.67:0/1453208130 >> 172.21.15.67:0/1453208130 conn(0x55eae9f8f460 msgr2=0x55eaea33dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.642+0000 7f05377fe640 1 --2- 172.21.15.67:0/631571667 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f05240b3c50 0x7f05240b6160 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55eaea33eab0 tx=0x7f05280a6e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.642+0000 7f05445eb740 1 -- 172.21.15.67:0/1453208130 shutdown_connections 2024-07-27T03:18:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.642+0000 7f05445eb740 1 -- 172.21.15.67:0/1453208130 wait complete. 2024-07-27T03:18:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.642+0000 7f05445eb740 1 -- start start 2024-07-27T03:18:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.643+0000 7f05445eb740 1 Processor -- start 2024-07-27T03:18:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.643+0000 7f05445eb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae9f721b0 0x55eaea34ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.643+0000 7f05445eb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f7a390 0x55eaea34bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.643+0000 7f05445eb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f9b070 0x55eaea3575d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.643+0000 7f05445eb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55eaea248900 con 0x55eae9f9b070 2024-07-27T03:18:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.643+0000 7f05445eb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55eaea2aa060 con 0x55eae9f7a390 2024-07-27T03:18:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.643+0000 7f05445eb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55eaea109700 con 0x55eae9f721b0 2024-07-27T03:18:40.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.643+0000 7f05445eb740 10 client.0 _ll_register_callbacks cb 0x55eaea250d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:40.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.643+0000 7f0537fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f9b070 0x55eaea3575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:40.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.643+0000 7f0537fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f9b070 0x55eaea3575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56180/0 (socket says 172.21.15.67:56180) 2024-07-27T03:18:40.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.643+0000 7f0537fff640 1 -- 172.21.15.67:0/3809380630 learned_addr learned my addr 172.21.15.67:0/3809380630 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:40.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.643+0000 7f0537fff640 1 --2- 172.21.15.67:0/3809380630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae9f721b0 0x55eaea34ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:40.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.643+0000 7f0536ffd640 1 --2- 172.21.15.67:0/3809380630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f7a390 0x55eaea34bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:40.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.644+0000 7f0537fff640 1 -- 172.21.15.67:0/3809380630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae9f721b0 msgr2=0x55eaea34ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:40.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.644+0000 7f0537fff640 1 --2- 172.21.15.67:0/3809380630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae9f721b0 0x55eaea34ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.644+0000 7f0537fff640 1 -- 172.21.15.67:0/3809380630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f7a390 msgr2=0x55eaea34bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:40.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.644+0000 7f0537fff640 1 --2- 172.21.15.67:0/3809380630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f7a390 0x55eaea34bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.644+0000 7f0537fff640 1 -- 172.21.15.67:0/3809380630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaea24cbf0 con 0x55eae9f9b070 2024-07-27T03:18:40.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.644+0000 7f0537fff640 1 --2- 172.21.15.67:0/3809380630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f9b070 0x55eaea3575d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0530005c90 tx=0x7f0530003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c7351051beda5d server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:40.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.644+0000 7f0516ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:40.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.644+0000 7f0516ffd640 1 -- 172.21.15.67:0/3809380630 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530059060 con 0x55eae9f9b070 2024-07-27T03:18:40.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.644+0000 7f0516ffd640 1 -- 172.21.15.67:0/3809380630 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f0530031450 con 0x55eae9f9b070 2024-07-27T03:18:40.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.644+0000 7f05445eb740 1 -- 172.21.15.67:0/3809380630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0530036030 con 0x55eae9f9b070 2024-07-27T03:18:40.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.644+0000 7f0516ffd640 1 -- 172.21.15.67:0/3809380630 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0530031690 con 0x55eae9f9b070 2024-07-27T03:18:40.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.645+0000 7f05445eb740 10 client.15903 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:40.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.645+0000 7f05445eb740 1 -- 172.21.15.67:0/3809380630 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eaea006650 con 0x55eae9f9b070 2024-07-27T03:18:40.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.645+0000 7f0516ffd640 1 -- 172.21.15.67:0/3809380630 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0530036030 con 0x55eae9f9b070 2024-07-27T03:18:40.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.645+0000 7f0516ffd640 1 -- 172.21.15.67:0/3809380630 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0530043070 con 0x55eae9f9b070 2024-07-27T03:18:40.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.645+0000 7f05445eb740 10 client.15903 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:40.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.645+0000 7f05445eb740 10 client.15903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:18:40.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.645+0000 7f05445eb740 20 client.15903 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:40.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.645+0000 7f05445eb740 1 --2- 172.21.15.67:0/3809380630 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55eae9f599f0 0x55eaea35d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:40.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.645+0000 7f05445eb740 4 client.15903 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:18:40.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.645+0000 7f05445eb740 1 -- 172.21.15.67:0/3809380630 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eaea2b8a70 con 0x55eae9f599f0 2024-07-27T03:18:40.651 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.645+0000 7f0536ffd640 1 --2- 172.21.15.67:0/3809380630 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55eae9f599f0 0x55eaea35d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:40.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.646+0000 7f0536ffd640 1 --2- 172.21.15.67:0/3809380630 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55eae9f599f0 0x55eaea35d440 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69644e02caeb8892 server_cookie=7c54fc216d97cdf6 in_seq=0 out_seq=0 2024-07-27T03:18:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.646+0000 7f0516ffd640 10 client.15903 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.647+0000 7f0516ffd640 1 -- 172.21.15.67:0/3809380630 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eaea2b8a70 con 0x55eae9f599f0 2024-07-27T03:18:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.647+0000 7f0516ffd640 10 client.15903 handle_command_reply: tid=0 2024-07-27T03:18:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.647+0000 7f05445eb740 1 client.15903 shutdown 2024-07-27T03:18:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.647+0000 7f05445eb740 1 -- 172.21.15.67:0/3809380630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f9b070 msgr2=0x55eaea3575d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.647+0000 7f05445eb740 1 --2- 172.21.15.67:0/3809380630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f9b070 0x55eaea3575d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0530005c90 tx=0x7f0530003d70 comp rx=0 tx=0).stop 2024-07-27T03:18:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.647+0000 7f05445eb740 1 -- 172.21.15.67:0/3809380630 shutdown_connections 2024-07-27T03:18:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.647+0000 7f05445eb740 1 --2- 172.21.15.67:0/3809380630 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55eae9f599f0 0x55eaea35d440 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.647+0000 7f05445eb740 1 --2- 172.21.15.67:0/3809380630 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f9b070 0x55eaea3575d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.647+0000 7f05445eb740 1 --2- 172.21.15.67:0/3809380630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f7a390 0x55eaea34bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.647+0000 7f0516ffd640 0 client.15903 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.647+0000 7f05445eb740 1 --2- 172.21.15.67:0/3809380630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae9f721b0 0x55eaea34ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.647+0000 7f05445eb740 1 -- 172.21.15.67:0/3809380630 >> 172.21.15.67:0/3809380630 conn(0x55eae9f8f460 msgr2=0x55eaea342fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:40.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.647+0000 7f05445eb740 1 -- 172.21.15.67:0/3809380630 shutdown_connections 2024-07-27T03:18:40.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.647+0000 7f05445eb740 1 -- 172.21.15.67:0/3809380630 wait complete. 2024-07-27T03:18:40.654 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.648+0000 7f05445eb740 20 client.15903 trim_cache size 0 max 16384 2024-07-27T03:18:40.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.661+0000 7f05445eb740 1 Processor -- start 2024-07-27T03:18:40.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.661+0000 7f05445eb740 1 -- start start 2024-07-27T03:18:40.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.661+0000 7f05445eb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f599f0 0x55eaea33ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.661+0000 7f05445eb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f721b0 0x55eaea357540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.661+0000 7f05445eb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae9f7a390 0x55eaea341570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.661+0000 7f05445eb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f05381b28d0 con 0x55eae9f721b0 2024-07-27T03:18:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.661+0000 7f05445eb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f05381b4410 con 0x55eae9f599f0 2024-07-27T03:18:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.661+0000 7f05445eb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05381b2fa0 con 0x55eae9f7a390 2024-07-27T03:18:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.661+0000 7f0536ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f721b0 0x55eaea357540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.661+0000 7f0536ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f721b0 0x55eaea357540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56190/0 (socket says 172.21.15.67:56190) 2024-07-27T03:18:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.661+0000 7f0536ffd640 1 -- 172.21.15.67:0/784818971 learned_addr learned my addr 172.21.15.67:0/784818971 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.662+0000 7f0537fff640 1 --2- 172.21.15.67:0/784818971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae9f7a390 0x55eaea341570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.662+0000 7f0537fff640 1 --2- 172.21.15.67:0/784818971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f599f0 0x55eaea33ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.662+0000 7f0536ffd640 1 -- 172.21.15.67:0/784818971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae9f7a390 msgr2=0x55eaea341570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.662+0000 7f0536ffd640 1 --2- 172.21.15.67:0/784818971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae9f7a390 0x55eaea341570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.662+0000 7f0536ffd640 1 -- 172.21.15.67:0/784818971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f599f0 msgr2=0x55eaea33ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.662+0000 7f0536ffd640 1 --2- 172.21.15.67:0/784818971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f599f0 0x55eaea33ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.662+0000 7f0536ffd640 1 -- 172.21.15.67:0/784818971 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaea34cae0 con 0x55eae9f721b0 2024-07-27T03:18:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.662+0000 7f0537fff640 1 --2- 172.21.15.67:0/784818971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f599f0 0x55eaea33ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:18:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.662+0000 7f0536ffd640 1 --2- 172.21.15.67:0/784818971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f721b0 0x55eaea357540 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0520005d00 tx=0x7f05200138f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3519d817930d370f server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.662+0000 7f052d7fa640 1 -- 172.21.15.67:0/784818971 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0520004020 con 0x55eae9f721b0 2024-07-27T03:18:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.662+0000 7f052d7fa640 1 -- 172.21.15.67:0/784818971 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05200041c0 con 0x55eae9f721b0 2024-07-27T03:18:40.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.662+0000 7f052d7fa640 1 -- 172.21.15.67:0/784818971 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0520010750 con 0x55eae9f721b0 2024-07-27T03:18:40.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.663+0000 7f05445eb740 1 -- 172.21.15.67:0/784818971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f721b0 msgr2=0x55eaea357540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:40.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.663+0000 7f05445eb740 1 --2- 172.21.15.67:0/784818971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f721b0 0x55eaea357540 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0520005d00 tx=0x7f05200138f0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.669 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.663+0000 7f05445eb740 1 -- 172.21.15.67:0/784818971 shutdown_connections 2024-07-27T03:18:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.663+0000 7f05445eb740 1 --2- 172.21.15.67:0/784818971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae9f7a390 0x55eaea341570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.663+0000 7f05445eb740 1 --2- 172.21.15.67:0/784818971 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f721b0 0x55eaea357540 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.663+0000 7f05445eb740 1 --2- 172.21.15.67:0/784818971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f599f0 0x55eaea33ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.663+0000 7f05445eb740 1 -- 172.21.15.67:0/784818971 >> 172.21.15.67:0/784818971 conn(0x55eae9f8f460 msgr2=0x55eaea340e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.663+0000 7f05445eb740 1 -- 172.21.15.67:0/784818971 shutdown_connections 2024-07-27T03:18:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.663+0000 7f05445eb740 1 -- 172.21.15.67:0/784818971 wait complete. 2024-07-27T03:18:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.664+0000 7f05445eb740 1 -- start start 2024-07-27T03:18:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.664+0000 7f05445eb740 1 Processor -- start 2024-07-27T03:18:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.664+0000 7f05445eb740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f599f0 0x55eaea35c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.664+0000 7f05445eb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f721b0 0x55eaea35ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.664+0000 7f05445eb740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae9f7a390 0x55eaea35cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:40.670 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.664+0000 7f05445eb740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f05381b3d40 con 0x55eae9f599f0 2024-07-27T03:18:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.664+0000 7f05445eb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f05381b3670 con 0x55eae9f721b0 2024-07-27T03:18:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.664+0000 7f05445eb740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05381b08b0 con 0x55eae9f7a390 2024-07-27T03:18:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.664+0000 7f05445eb740 10 client.0 _ll_register_callbacks cb 0x7f053002f410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.664+0000 7f0537fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f599f0 0x55eaea35c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.665+0000 7f0537fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f599f0 0x55eaea35c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56194/0 (socket says 172.21.15.67:56194) 2024-07-27T03:18:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.665+0000 7f0537fff640 1 -- 172.21.15.67:0/242986942 learned_addr learned my addr 172.21.15.67:0/242986942 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.665+0000 7f0536ffd640 1 --2- 172.21.15.67:0/242986942 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f721b0 0x55eaea35ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.665+0000 7f0537fff640 1 --2- 172.21.15.67:0/242986942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae9f7a390 0x55eaea35cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.665+0000 7f0537fff640 1 -- 172.21.15.67:0/242986942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae9f7a390 msgr2=0x55eaea35cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.665+0000 7f0537fff640 1 --2- 172.21.15.67:0/242986942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae9f7a390 0x55eaea35cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.665+0000 7f0537fff640 1 -- 172.21.15.67:0/242986942 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f721b0 msgr2=0x55eaea35ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.665+0000 7f0537fff640 1 --2- 172.21.15.67:0/242986942 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f721b0 0x55eaea35ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.665+0000 7f0537fff640 1 -- 172.21.15.67:0/242986942 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaea3cd2c0 con 0x55eae9f599f0 2024-07-27T03:18:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.665+0000 7f0536ffd640 1 --2- 172.21.15.67:0/242986942 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f721b0 0x55eaea35ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:18:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.665+0000 7f0537fff640 1 --2- 172.21.15.67:0/242986942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f599f0 0x55eaea35c4c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f053002faf0 tx=0x7f0530003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b961374c558e9f5 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.665+0000 7f0516ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.665+0000 7f0516ffd640 1 -- 172.21.15.67:0/242986942 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530059040 con 0x55eae9f599f0 2024-07-27T03:18:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.665+0000 7f0516ffd640 1 -- 172.21.15.67:0/242986942 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f0530003aa0 con 0x55eae9f599f0 2024-07-27T03:18:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.665+0000 7f05445eb740 1 -- 172.21.15.67:0/242986942 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05200041c0 con 0x55eae9f599f0 2024-07-27T03:18:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.666+0000 7f0516ffd640 1 -- 172.21.15.67:0/242986942 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f053006d050 con 0x55eae9f599f0 2024-07-27T03:18:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.666+0000 7f05445eb740 10 client.15909 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.666+0000 7f05445eb740 1 -- 172.21.15.67:0/242986942 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55eaea3ce570 con 0x55eae9f599f0 2024-07-27T03:18:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.666+0000 7f0516ffd640 1 -- 172.21.15.67:0/242986942 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0530036040 con 0x55eae9f599f0 2024-07-27T03:18:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.666+0000 7f0516ffd640 1 -- 172.21.15.67:0/242986942 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f0530043040 con 0x55eae9f599f0 2024-07-27T03:18:40.672 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.666+0000 7f05445eb740 10 client.15909 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:40.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.666+0000 7f05445eb740 10 client.15909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:18:40.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.666+0000 7f05445eb740 20 client.15909 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:40.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.666+0000 7f05445eb740 1 --2- 172.21.15.67:0/242986942 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55eae9f9b070 0x55eaea3d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:40.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.666+0000 7f05445eb740 4 client.15909 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-27T03:18:40.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.666+0000 7f05445eb740 1 -- 172.21.15.67:0/242986942 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f05200041c0 con 0x55eae9f9b070 2024-07-27T03:18:40.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.667+0000 7f0536ffd640 1 --2- 172.21.15.67:0/242986942 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55eae9f9b070 0x55eaea3d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:40.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.667+0000 7f0516ffd640 10 client.15909 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:40.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.667+0000 7f0536ffd640 1 --2- 172.21.15.67:0/242986942 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55eae9f9b070 0x55eaea3d0d00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8692f5c41a722a7 server_cookie=992011002bfa9442 in_seq=0 out_seq=0 2024-07-27T03:18:40.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.667+0000 7f0516ffd640 1 -- 172.21.15.67:0/242986942 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f05200041c0 con 0x55eae9f9b070 2024-07-27T03:18:40.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.667+0000 7f0516ffd640 10 client.15909 handle_command_reply: tid=0 2024-07-27T03:18:40.673 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.667+0000 7f05445eb740 1 client.15909 shutdown 2024-07-27T03:18:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.668+0000 7f05445eb740 1 -- 172.21.15.67:0/242986942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f599f0 msgr2=0x55eaea35c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:40.674 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.668+0000 7f05445eb740 1 --2- 172.21.15.67:0/242986942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f599f0 0x55eaea35c4c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f053002faf0 tx=0x7f0530003cd0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.668+0000 7f05445eb740 1 -- 172.21.15.67:0/242986942 shutdown_connections 2024-07-27T03:18:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.668+0000 7f05445eb740 1 --2- 172.21.15.67:0/242986942 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55eae9f9b070 0x55eaea3d0d00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.668+0000 7f05445eb740 1 --2- 172.21.15.67:0/242986942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55eae9f7a390 0x55eaea35cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.668+0000 7f05445eb740 1 --2- 172.21.15.67:0/242986942 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55eae9f721b0 0x55eaea35ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.668+0000 7f05445eb740 1 --2- 172.21.15.67:0/242986942 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55eae9f599f0 0x55eaea35c4c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.668+0000 7f05445eb740 1 -- 172.21.15.67:0/242986942 >> 172.21.15.67:0/242986942 conn(0x55eae9f8f460 msgr2=0x55eaea33ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.668+0000 7f0516ffd640 0 client.15909 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.668+0000 7f05445eb740 1 -- 172.21.15.67:0/242986942 shutdown_connections 2024-07-27T03:18:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.668+0000 7f05445eb740 1 -- 172.21.15.67:0/242986942 wait complete. 2024-07-27T03:18:40.675 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.668+0000 7f05445eb740 20 client.15909 trim_cache size 0 max 16384 2024-07-27T03:18:40.676 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-27T03:18:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.669+0000 7f053e066640 1 -- 172.21.15.67:0/631571667 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f05240b3c50 msgr2=0x7f05240b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.669+0000 7f053e066640 1 --2- 172.21.15.67:0/631571667 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f05240b3c50 0x7f05240b6160 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55eaea33eab0 tx=0x7f05280a6e10 comp rx=0 tx=0).stop 2024-07-27T03:18:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.669+0000 7f053e066640 1 -- 172.21.15.67:0/631571667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05381024d0 msgr2=0x7f05380ff0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:40.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.669+0000 7f053e066640 1 --2- 172.21.15.67:0/631571667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05381024d0 0x7f05380ff0e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f052800e9f0 tx=0x7f052800eeb0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.669+0000 7f053e066640 1 -- 172.21.15.67:0/631571667 shutdown_connections 2024-07-27T03:18:40.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.669+0000 7f053e066640 1 --2- 172.21.15.67:0/631571667 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f05240b3c50 0x7f05240b6160 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.669+0000 7f053e066640 1 --2- 172.21.15.67:0/631571667 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0538108760 0x7f05380ffb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.669+0000 7f053e066640 1 --2- 172.21.15.67:0/631571667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0538107e10 0x7f05380ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.669+0000 7f053e066640 1 --2- 172.21.15.67:0/631571667 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f05381024d0 0x7f05380ff0e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:40.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.669+0000 7f053e066640 1 -- 172.21.15.67:0/631571667 >> 172.21.15.67:0/631571667 conn(0x7f05380791a0 msgr2=0x7f053806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:40.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.669+0000 7f053e066640 1 -- 172.21.15.67:0/631571667 shutdown_connections 2024-07-27T03:18:40.677 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:40.669+0000 7f053e066640 1 -- 172.21.15.67:0/631571667 wait complete. 2024-07-27T03:18:40.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:40 smithi175 ceph-mon[37125]: pgmap v493: 129 pgs: 129 active+clean; 258 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 32 MiB/s wr, 20 op/s 2024-07-27T03:18:41.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:40 smithi149 ceph-mon[37173]: pgmap v493: 129 pgs: 129 active+clean; 258 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 32 MiB/s wr, 20 op/s 2024-07-27T03:18:42.451 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-07-27T03:18:42.452 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-07-27T03:18:42.745 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:18:42.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:42 smithi067 ceph-mon[32814]: pgmap v494: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-07-27T03:18:42.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:42 smithi175 ceph-mon[37125]: pgmap v494: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-07-27T03:18:42.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:42 smithi149 ceph-mon[37173]: pgmap v494: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 22 op/s 2024-07-27T03:18:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.542+0000 7f85bdd0f640 1 -- 172.21.15.67:0/1537325531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b8104d10 msgr2=0x7f85b810d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.542+0000 7f85bdd0f640 1 --2- 172.21.15.67:0/1537325531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b8104d10 0x7f85b810d3a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009900 tx=0x7f85a002a410 comp rx=0 tx=0).stop 2024-07-27T03:18:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.543+0000 7f85bdd0f640 1 -- 172.21.15.67:0/1537325531 shutdown_connections 2024-07-27T03:18:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.543+0000 7f85bdd0f640 1 --2- 172.21.15.67:0/1537325531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85b810d8e0 0x7f85b810fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.543+0000 7f85bdd0f640 1 --2- 172.21.15.67:0/1537325531 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b8104d10 0x7f85b810d3a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.543+0000 7f85bdd0f640 1 --2- 172.21.15.67:0/1537325531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85b8104330 0x7f85b8104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.543+0000 7f85bdd0f640 1 -- 172.21.15.67:0/1537325531 >> 172.21.15.67:0/1537325531 conn(0x7f85b8079220 msgr2=0x7f85b8079630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:44.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.543+0000 7f85bdd0f640 1 -- 172.21.15.67:0/1537325531 shutdown_connections 2024-07-27T03:18:44.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.543+0000 7f85bdd0f640 1 -- 172.21.15.67:0/1537325531 wait complete. 2024-07-27T03:18:44.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.543+0000 7f85bdd0f640 1 Processor -- start 2024-07-27T03:18:44.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.544+0000 7f85bdd0f640 1 -- start start 2024-07-27T03:18:44.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.544+0000 7f85bdd0f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b8104330 0x7f85b8074940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:44.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.544+0000 7f85bdd0f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85b8104d10 0x7f85b806dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:44.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.544+0000 7f85bdd0f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85b810d8e0 0x7f85b806e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:44.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.544+0000 7f85bdd0f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81134f0 con 0x7f85b8104330 2024-07-27T03:18:44.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.544+0000 7f85bdd0f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f85b8113370 con 0x7f85b8104d10 2024-07-27T03:18:44.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.544+0000 7f85bdd0f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85b8113670 con 0x7f85b810d8e0 2024-07-27T03:18:44.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.544+0000 7f85b77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b8104330 0x7f85b8074940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:44.554 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.544+0000 7f85b77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b8104330 0x7f85b8074940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56214/0 (socket says 172.21.15.67:56214) 2024-07-27T03:18:44.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.544+0000 7f85b77fe640 1 -- 172.21.15.67:0/4086853756 learned_addr learned my addr 172.21.15.67:0/4086853756 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:44.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.544+0000 7f85b6ffd640 1 --2- 172.21.15.67:0/4086853756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85b8104d10 0x7f85b806dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:44.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.544+0000 7f85b7fff640 1 --2- 172.21.15.67:0/4086853756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85b810d8e0 0x7f85b806e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:44.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.545+0000 7f85b77fe640 1 -- 172.21.15.67:0/4086853756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85b810d8e0 msgr2=0x7f85b806e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:44.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.545+0000 7f85b77fe640 1 --2- 172.21.15.67:0/4086853756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85b810d8e0 0x7f85b806e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.545+0000 7f85b77fe640 1 -- 172.21.15.67:0/4086853756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85b8104d10 msgr2=0x7f85b806dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:44.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.545+0000 7f85b77fe640 1 --2- 172.21.15.67:0/4086853756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85b8104d10 0x7f85b806dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.545+0000 7f85b77fe640 1 -- 172.21.15.67:0/4086853756 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b806ea20 con 0x7f85b8104330 2024-07-27T03:18:44.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.545+0000 7f85b6ffd640 1 --2- 172.21.15.67:0/4086853756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85b8104d10 0x7f85b806dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:18:44.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.545+0000 7f85b77fe640 1 --2- 172.21.15.67:0/4086853756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b8104330 0x7f85b8074940 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f85a800c890 tx=0x7f85a800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:44.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.545+0000 7f85b4ff9640 1 -- 172.21.15.67:0/4086853756 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8004220 con 0x7f85b8104330 2024-07-27T03:18:44.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.545+0000 7f85b4ff9640 1 -- 172.21.15.67:0/4086853756 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a80043c0 con 0x7f85b8104330 2024-07-27T03:18:44.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.545+0000 7f85bdd0f640 1 -- 172.21.15.67:0/4086853756 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b80736d0 con 0x7f85b8104330 2024-07-27T03:18:44.555 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.545+0000 7f85b4ff9640 1 -- 172.21.15.67:0/4086853756 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8004a00 con 0x7f85b8104330 2024-07-27T03:18:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.545+0000 7f85bdd0f640 1 -- 172.21.15.67:0/4086853756 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b8073be0 con 0x7f85b8104330 2024-07-27T03:18:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.548+0000 7f85c4294740 1 Processor -- start 2024-07-27T03:18:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.548+0000 7f85b4ff9640 1 -- 172.21.15.67:0/4086853756 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f85a8004ba0 con 0x7f85b8104330 2024-07-27T03:18:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.548+0000 7f85c4294740 1 -- start start 2024-07-27T03:18:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.548+0000 7f85c4294740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582007df350 0x558200b83e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.548+0000 7f85c4294740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582007b6490 0x558200b84350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.549+0000 7f85c4294740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5582007be670 0x558200b84890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.549+0000 7f85c4294740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x558200afaa60 con 0x5582007be670 2024-07-27T03:18:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.549+0000 7f85c4294740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558200ae9f60 con 0x5582007df350 2024-07-27T03:18:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.549+0000 7f85c4294740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558200aedfa0 con 0x5582007b6490 2024-07-27T03:18:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.549+0000 7f85b6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5582007be670 0x558200b84890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.549+0000 7f85b77fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582007b6490 0x558200b84350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:44.556 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.549+0000 7f85b6ffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5582007be670 0x558200b84890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56230/0 (socket says 172.21.15.67:56230) 2024-07-27T03:18:44.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.549+0000 7f85b6ffd640 1 -- 172.21.15.67:0/1261991759 learned_addr learned my addr 172.21.15.67:0/1261991759 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:44.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.549+0000 7f85b7fff640 1 --2- 172.21.15.67:0/1261991759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582007df350 0x558200b83e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:44.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.549+0000 7f85b4ff9640 1 --2- 172.21.15.67:0/4086853756 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f85a40b3bd0 0x7f85a40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:44.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.549+0000 7f85b6ffd640 1 -- 172.21.15.67:0/1261991759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582007b6490 msgr2=0x558200b84350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:44.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.549+0000 7f85b6ffd640 1 --2- 172.21.15.67:0/1261991759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582007b6490 0x558200b84350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.549+0000 7f85b6ffd640 1 -- 172.21.15.67:0/1261991759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582007df350 msgr2=0x558200b83e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:44.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.549+0000 7f85b6ffd640 1 --2- 172.21.15.67:0/1261991759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582007df350 0x558200b83e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.549+0000 7f85b6ffd640 1 -- 172.21.15.67:0/1261991759 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55820084a930 con 0x5582007be670 2024-07-27T03:18:44.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.549+0000 7f85b7fff640 1 --2- 172.21.15.67:0/1261991759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582007df350 0x558200b83e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:18:44.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.549+0000 7f85b7fff640 1 --2- 172.21.15.67:0/4086853756 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f85a40b3bd0 0x7f85a40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:44.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.549+0000 7f85b4ff9640 1 -- 172.21.15.67:0/4086853756 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f85a80e3e00 con 0x7f85b8104330 2024-07-27T03:18:44.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.550+0000 7f85b6ffd640 1 --2- 172.21.15.67:0/1261991759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5582007be670 0x558200b84890 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f85a00096c0 tx=0x7f85a0031e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38cfd2dac4f9c1e1 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:44.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.550+0000 7f85b7fff640 1 --2- 172.21.15.67:0/4086853756 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f85a40b3bd0 0x7f85a40b60e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f85b00114a0 tx=0x7f85b0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:44.557 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.550+0000 7f85adffb640 1 -- 172.21.15.67:0/1261991759 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a003f070 con 0x5582007be670 2024-07-27T03:18:44.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.550+0000 7f85adffb640 1 -- 172.21.15.67:0/1261991759 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a0031760 con 0x5582007be670 2024-07-27T03:18:44.558 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.550+0000 7f85adffb640 1 -- 172.21.15.67:0/1261991759 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0035070 con 0x5582007be670 2024-07-27T03:18:44.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.552+0000 7f85c4294740 1 -- 172.21.15.67:0/1261991759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5582007be670 msgr2=0x558200b84890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:44.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.552+0000 7f85c4294740 1 --2- 172.21.15.67:0/1261991759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5582007be670 0x558200b84890 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f85a00096c0 tx=0x7f85a0031e80 comp rx=0 tx=0).stop 2024-07-27T03:18:44.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.552+0000 7f85c4294740 1 -- 172.21.15.67:0/1261991759 shutdown_connections 2024-07-27T03:18:44.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.552+0000 7f85c4294740 1 --2- 172.21.15.67:0/1261991759 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5582007be670 0x558200b84890 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.552+0000 7f85c4294740 1 --2- 172.21.15.67:0/1261991759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582007b6490 0x558200b84350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.552+0000 7f85c4294740 1 --2- 172.21.15.67:0/1261991759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582007df350 0x558200b83e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.552+0000 7f85c4294740 1 -- 172.21.15.67:0/1261991759 >> 172.21.15.67:0/1261991759 conn(0x5582007d3740 msgr2=0x558200b81d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:44.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.552+0000 7f85c4294740 1 -- 172.21.15.67:0/1261991759 shutdown_connections 2024-07-27T03:18:44.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.552+0000 7f85c4294740 1 -- 172.21.15.67:0/1261991759 wait complete. 2024-07-27T03:18:44.562 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.553+0000 7f85c4294740 1 -- start start 2024-07-27T03:18:44.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.553+0000 7f85c4294740 1 Processor -- start 2024-07-27T03:18:44.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.554+0000 7f85c4294740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582007b6490 0x558200b8f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:44.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.554+0000 7f85c4294740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582007be670 0x558200b8feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:44.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.554+0000 7f85c4294740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5582007df350 0x558200b9b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:44.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.554+0000 7f85c4294740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55820094d890 con 0x5582007df350 2024-07-27T03:18:44.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.554+0000 7f85c4294740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558200a8c580 con 0x5582007be670 2024-07-27T03:18:44.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.554+0000 7f85c4294740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558200b92590 con 0x5582007b6490 2024-07-27T03:18:44.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.554+0000 7f85c4294740 10 client.0 _ll_register_callbacks cb 0x558200ae3f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:44.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.554+0000 7f85b77fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582007b6490 0x558200b8f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:44.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.554+0000 7f85b77fe640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582007b6490 0x558200b8f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.67:37016/0 (socket says 172.21.15.67:37016) 2024-07-27T03:18:44.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.554+0000 7f85b77fe640 1 -- 172.21.15.67:0/4043176176 learned_addr learned my addr 172.21.15.67:0/4043176176 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:44.563 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.554+0000 7f85b6ffd640 1 --2- 172.21.15.67:0/4043176176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582007be670 0x558200b8feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.554+0000 7f85b7fff640 1 --2- 172.21.15.67:0/4043176176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5582007df350 0x558200b9b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.554+0000 7f85b77fe640 1 -- 172.21.15.67:0/4043176176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582007be670 msgr2=0x558200b8feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.554+0000 7f85b77fe640 1 --2- 172.21.15.67:0/4043176176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582007be670 0x558200b8feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.554+0000 7f85b77fe640 1 -- 172.21.15.67:0/4043176176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5582007df350 msgr2=0x558200b9b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.554+0000 7f85b77fe640 1 --2- 172.21.15.67:0/4043176176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5582007df350 0x558200b9b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.554+0000 7f85b77fe640 1 -- 172.21.15.67:0/4043176176 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558200ba0230 con 0x5582007b6490 2024-07-27T03:18:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.554+0000 7f85b7fff640 1 --2- 172.21.15.67:0/4043176176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5582007df350 0x558200b9b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:18:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.555+0000 7f85b77fe640 1 --2- 172.21.15.67:0/4043176176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582007b6490 0x558200b8f970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f85a80058c0 tx=0x7f85a800a5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a4344a2a392b3b7 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.555+0000 7f8596ffd640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-27T03:18:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.555+0000 7f8596ffd640 1 -- 172.21.15.67:0/4043176176 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a801d070 con 0x5582007b6490 2024-07-27T03:18:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.555+0000 7f8596ffd640 1 -- 172.21.15.67:0/4043176176 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f85a8004e90 con 0x5582007b6490 2024-07-27T03:18:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.555+0000 7f85c4294740 1 -- 172.21.15.67:0/4043176176 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a0031760 con 0x5582007b6490 2024-07-27T03:18:44.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.555+0000 7f8596ffd640 1 -- 172.21.15.67:0/4043176176 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f85a80a6130 con 0x5582007b6490 2024-07-27T03:18:44.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.555+0000 7f85c4294740 10 client.25516 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:44.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.555+0000 7f85c4294740 1 -- 172.21.15.67:0/4043176176 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558200ba2b60 con 0x5582007b6490 2024-07-27T03:18:44.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.556+0000 7f8596ffd640 1 -- 172.21.15.67:0/4043176176 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a0031760 con 0x5582007b6490 2024-07-27T03:18:44.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.556+0000 7f8596ffd640 1 -- 172.21.15.67:0/4043176176 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f85a80b35c0 con 0x5582007b6490 2024-07-27T03:18:44.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.556+0000 7f85c4294740 10 client.25516 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:44.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.556+0000 7f85c4294740 10 client.25516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:18:44.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.556+0000 7f85c4294740 20 client.25516 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:44.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.556+0000 7f85c4294740 1 --2- 172.21.15.67:0/4043176176 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55820079dcd0 0x558200ba64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:44.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.556+0000 7f85c4294740 4 client.25516 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:18:44.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.556+0000 7f85c4294740 1 -- 172.21.15.67:0/4043176176 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558200afc9b0 con 0x55820079dcd0 2024-07-27T03:18:44.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.557+0000 7f85b6ffd640 1 --2- 172.21.15.67:0/4043176176 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55820079dcd0 0x558200ba64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:44.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.557+0000 7f8596ffd640 10 client.25516 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:44.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.557+0000 7f85b6ffd640 1 --2- 172.21.15.67:0/4043176176 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55820079dcd0 0x558200ba64e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=698494d5d87dfab1 server_cookie=51cb02de0727c8ab in_seq=0 out_seq=0 2024-07-27T03:18:44.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.558+0000 7f8596ffd640 1 -- 172.21.15.67:0/4043176176 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558200afc9b0 con 0x55820079dcd0 2024-07-27T03:18:44.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.558+0000 7f8596ffd640 10 client.25516 handle_command_reply: tid=0 2024-07-27T03:18:44.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.558+0000 7f85c4294740 1 client.25516 shutdown 2024-07-27T03:18:44.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.558+0000 7f85c4294740 1 -- 172.21.15.67:0/4043176176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582007b6490 msgr2=0x558200b8f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:44.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.558+0000 7f85c4294740 1 --2- 172.21.15.67:0/4043176176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582007b6490 0x558200b8f970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f85a80058c0 tx=0x7f85a800a5e0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.558+0000 7f85c4294740 1 -- 172.21.15.67:0/4043176176 shutdown_connections 2024-07-27T03:18:44.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.558+0000 7f85c4294740 1 --2- 172.21.15.67:0/4043176176 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55820079dcd0 0x558200ba64e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.559+0000 7f85c4294740 1 --2- 172.21.15.67:0/4043176176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5582007df350 0x558200b9b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.559+0000 7f8596ffd640 0 client.25516 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:44.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.559+0000 7f85c4294740 1 --2- 172.21.15.67:0/4043176176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582007be670 0x558200b8feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.559+0000 7f85c4294740 1 --2- 172.21.15.67:0/4043176176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582007b6490 0x558200b8f970 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.559+0000 7f85c4294740 1 -- 172.21.15.67:0/4043176176 >> 172.21.15.67:0/4043176176 conn(0x5582007d3740 msgr2=0x558200b86fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:44.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.559+0000 7f85c4294740 1 -- 172.21.15.67:0/4043176176 shutdown_connections 2024-07-27T03:18:44.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.559+0000 7f85c4294740 1 -- 172.21.15.67:0/4043176176 wait complete. 2024-07-27T03:18:44.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.559+0000 7f85c4294740 20 client.25516 trim_cache size 0 max 16384 2024-07-27T03:18:44.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.594+0000 7f85c4294740 1 Processor -- start 2024-07-27T03:18:44.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.594+0000 7f85c4294740 1 -- start start 2024-07-27T03:18:44.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.594+0000 7f85c4294740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820079dcd0 0x558200b81ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:44.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.594+0000 7f85c4294740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582007b6490 0x558200b89f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:44.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.594+0000 7f85c4294740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5582007be670 0x558200b9b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:44.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.594+0000 7f85c4294740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81b6bc0 con 0x5582007be670 2024-07-27T03:18:44.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.594+0000 7f85c4294740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81b8700 con 0x55820079dcd0 2024-07-27T03:18:44.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.594+0000 7f85c4294740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81b7290 con 0x5582007b6490 2024-07-27T03:18:44.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.594+0000 7f85b7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5582007be670 0x558200b9b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:44.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.594+0000 7f85b7fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5582007be670 0x558200b9b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56252/0 (socket says 172.21.15.67:56252) 2024-07-27T03:18:44.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.594+0000 7f85b7fff640 1 -- 172.21.15.67:0/194344066 learned_addr learned my addr 172.21.15.67:0/194344066 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:44.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.594+0000 7f85b6ffd640 1 --2- 172.21.15.67:0/194344066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582007b6490 0x558200b89f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:44.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.594+0000 7f85b77fe640 1 --2- 172.21.15.67:0/194344066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820079dcd0 0x558200b81ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:44.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.595+0000 7f85b7fff640 1 -- 172.21.15.67:0/194344066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582007b6490 msgr2=0x558200b89f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:44.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.595+0000 7f85b7fff640 1 --2- 172.21.15.67:0/194344066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582007b6490 0x558200b89f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.595+0000 7f85b7fff640 1 -- 172.21.15.67:0/194344066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820079dcd0 msgr2=0x558200b81ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:44.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.595+0000 7f85b7fff640 1 --2- 172.21.15.67:0/194344066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820079dcd0 0x558200b81ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.595+0000 7f85b7fff640 1 -- 172.21.15.67:0/194344066 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558200b84c20 con 0x5582007be670 2024-07-27T03:18:44.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.595+0000 7f85b7fff640 1 --2- 172.21.15.67:0/194344066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5582007be670 0x558200b9b6a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f85b00128b0 tx=0x7f85b0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a703d92f3f428d17 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:44.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.595+0000 7f85ad7fa640 1 -- 172.21.15.67:0/194344066 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b001fb60 con 0x5582007be670 2024-07-27T03:18:44.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.595+0000 7f85ad7fa640 1 -- 172.21.15.67:0/194344066 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85b0016070 con 0x5582007be670 2024-07-27T03:18:44.599 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.595+0000 7f85ad7fa640 1 -- 172.21.15.67:0/194344066 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b001d690 con 0x5582007be670 2024-07-27T03:18:44.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.596+0000 7f85c4294740 1 -- 172.21.15.67:0/194344066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5582007be670 msgr2=0x558200b9b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:44.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.596+0000 7f85c4294740 1 --2- 172.21.15.67:0/194344066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5582007be670 0x558200b9b6a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f85b00128b0 tx=0x7f85b0014000 comp rx=0 tx=0).stop 2024-07-27T03:18:44.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.596+0000 7f85c4294740 1 -- 172.21.15.67:0/194344066 shutdown_connections 2024-07-27T03:18:44.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.596+0000 7f85c4294740 1 --2- 172.21.15.67:0/194344066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5582007be670 0x558200b9b6a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.596+0000 7f85c4294740 1 --2- 172.21.15.67:0/194344066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582007b6490 0x558200b89f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.596+0000 7f85c4294740 1 --2- 172.21.15.67:0/194344066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55820079dcd0 0x558200b81ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.596+0000 7f85c4294740 1 -- 172.21.15.67:0/194344066 >> 172.21.15.67:0/194344066 conn(0x5582007d3740 msgr2=0x558200b90c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:44.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.596+0000 7f85c4294740 1 -- 172.21.15.67:0/194344066 shutdown_connections 2024-07-27T03:18:44.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.596+0000 7f85c4294740 1 -- 172.21.15.67:0/194344066 wait complete. 2024-07-27T03:18:44.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.596+0000 7f85c4294740 1 -- start start 2024-07-27T03:18:44.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.597+0000 7f85c4294740 1 Processor -- start 2024-07-27T03:18:44.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.597+0000 7f85c4294740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55820079dcd0 0x558200b97e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:44.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.597+0000 7f85c4294740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582007b6490 0x558200b932e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:44.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.597+0000 7f85c4294740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582007be670 0x558200b93820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:44.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.597+0000 7f85c4294740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81b8030 con 0x55820079dcd0 2024-07-27T03:18:44.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.597+0000 7f85c4294740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81b7960 con 0x5582007be670 2024-07-27T03:18:44.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.597+0000 7f85c4294740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81b4ba0 con 0x5582007b6490 2024-07-27T03:18:44.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.597+0000 7f85c4294740 10 client.0 _ll_register_callbacks cb 0x7f85a8005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:44.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.597+0000 7f85b77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55820079dcd0 0x558200b97e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:44.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.597+0000 7f85b77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55820079dcd0 0x558200b97e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56258/0 (socket says 172.21.15.67:56258) 2024-07-27T03:18:44.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.597+0000 7f85b77fe640 1 -- 172.21.15.67:0/2256081000 learned_addr learned my addr 172.21.15.67:0/2256081000 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:44.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.597+0000 7f85b6ffd640 1 --2- 172.21.15.67:0/2256081000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582007b6490 0x558200b932e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:44.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.597+0000 7f85b7fff640 1 --2- 172.21.15.67:0/2256081000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582007be670 0x558200b93820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:44.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.598+0000 7f85b77fe640 1 -- 172.21.15.67:0/2256081000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582007b6490 msgr2=0x558200b932e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:44.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.598+0000 7f85b77fe640 1 --2- 172.21.15.67:0/2256081000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582007b6490 0x558200b932e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.598+0000 7f85b77fe640 1 -- 172.21.15.67:0/2256081000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582007be670 msgr2=0x558200b93820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:44.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.598+0000 7f85b77fe640 1 --2- 172.21.15.67:0/2256081000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582007be670 0x558200b93820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.598+0000 7f85b77fe640 1 -- 172.21.15.67:0/2256081000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558200b93f10 con 0x55820079dcd0 2024-07-27T03:18:44.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.598+0000 7f85b77fe640 1 --2- 172.21.15.67:0/2256081000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55820079dcd0 0x558200b97e30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f85a80a6a20 tx=0x7f85a80050c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca13c79e88fa0f0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:44.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.598+0000 7f8596ffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:44.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.598+0000 7f8596ffd640 1 -- 172.21.15.67:0/2256081000 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a806d030 con 0x55820079dcd0 2024-07-27T03:18:44.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.598+0000 7f8596ffd640 1 -- 172.21.15.67:0/2256081000 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f85a80a9c00 con 0x55820079dcd0 2024-07-27T03:18:44.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.598+0000 7f85c4294740 1 -- 172.21.15.67:0/2256081000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85b0016070 con 0x55820079dcd0 2024-07-27T03:18:44.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.598+0000 7f8596ffd640 1 -- 172.21.15.67:0/2256081000 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f85a801d040 con 0x55820079dcd0 2024-07-27T03:18:44.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.598+0000 7f85c4294740 10 client.15927 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:44.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.598+0000 7f85c4294740 1 -- 172.21.15.67:0/2256081000 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558200c0ebb0 con 0x55820079dcd0 2024-07-27T03:18:44.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.599+0000 7f8596ffd640 1 -- 172.21.15.67:0/2256081000 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85b0016070 con 0x55820079dcd0 2024-07-27T03:18:44.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.599+0000 7f8596ffd640 1 -- 172.21.15.67:0/2256081000 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f85a80a9670 con 0x55820079dcd0 2024-07-27T03:18:44.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.599+0000 7f85c4294740 10 client.15927 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:44.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.599+0000 7f85c4294740 10 client.15927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:18:44.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.599+0000 7f85c4294740 20 client.15927 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:44.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.599+0000 7f85c4294740 1 --2- 172.21.15.67:0/2256081000 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5582007df350 0x558200c113a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:44.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.599+0000 7f85c4294740 4 client.15927 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-07-27T03:18:44.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.599+0000 7f85c4294740 1 -- 172.21.15.67:0/2256081000 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558200af8450 con 0x5582007df350 2024-07-27T03:18:44.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.599+0000 7f85b6ffd640 1 --2- 172.21.15.67:0/2256081000 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5582007df350 0x558200c113a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:44.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.600+0000 7f8596ffd640 10 client.15927 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:44.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.600+0000 7f85b6ffd640 1 --2- 172.21.15.67:0/2256081000 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5582007df350 0x558200c113a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55bc292f917894c5 server_cookie=c6f190b39ee30af1 in_seq=0 out_seq=0 2024-07-27T03:18:44.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.601+0000 7f8596ffd640 1 -- 172.21.15.67:0/2256081000 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558200af8450 con 0x5582007df350 2024-07-27T03:18:44.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.601+0000 7f8596ffd640 10 client.15927 handle_command_reply: tid=0 2024-07-27T03:18:44.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.601+0000 7f85c4294740 1 client.15927 shutdown 2024-07-27T03:18:44.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.601+0000 7f85c4294740 1 -- 172.21.15.67:0/2256081000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55820079dcd0 msgr2=0x558200b97e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:44.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.601+0000 7f85c4294740 1 --2- 172.21.15.67:0/2256081000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55820079dcd0 0x558200b97e30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f85a80a6a20 tx=0x7f85a80050c0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.602+0000 7f85c4294740 1 -- 172.21.15.67:0/2256081000 shutdown_connections 2024-07-27T03:18:44.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.602+0000 7f85c4294740 1 --2- 172.21.15.67:0/2256081000 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x5582007df350 0x558200c113a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.602+0000 7f85c4294740 1 --2- 172.21.15.67:0/2256081000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5582007be670 0x558200b93820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.602+0000 7f85c4294740 1 --2- 172.21.15.67:0/2256081000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5582007b6490 0x558200b932e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.602+0000 7f85c4294740 1 --2- 172.21.15.67:0/2256081000 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55820079dcd0 0x558200b97e30 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.602+0000 7f8596ffd640 0 client.15927 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:44.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.602+0000 7f85c4294740 1 -- 172.21.15.67:0/2256081000 >> 172.21.15.67:0/2256081000 conn(0x5582007d3740 msgr2=0x558200b84e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:44.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.602+0000 7f85c4294740 1 -- 172.21.15.67:0/2256081000 shutdown_connections 2024-07-27T03:18:44.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.602+0000 7f85c4294740 1 -- 172.21.15.67:0/2256081000 wait complete. 2024-07-27T03:18:44.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.602+0000 7f85c4294740 20 client.15927 trim_cache size 0 max 16384 2024-07-27T03:18:44.608 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:18:44.608 INFO:teuthology.orchestra.run.smithi067.stdout: "return_code": 0, 2024-07-27T03:18:44.608 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_tag": "39c7ce3d-9f37-4d79-bcf6-913b7d4c5e54", 2024-07-27T03:18:44.608 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": "asynchronous" 2024-07-27T03:18:44.608 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:18:44.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.603+0000 7f85bdd0f640 1 -- 172.21.15.67:0/4086853756 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f85a40b3bd0 msgr2=0x7f85a40b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:44.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.603+0000 7f85bdd0f640 1 --2- 172.21.15.67:0/4086853756 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f85a40b3bd0 0x7f85a40b60e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f85b00114a0 tx=0x7f85b0004a40 comp rx=0 tx=0).stop 2024-07-27T03:18:44.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.603+0000 7f85bdd0f640 1 -- 172.21.15.67:0/4086853756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b8104330 msgr2=0x7f85b8074940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:44.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.603+0000 7f85bdd0f640 1 --2- 172.21.15.67:0/4086853756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b8104330 0x7f85b8074940 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f85a800c890 tx=0x7f85a800cd50 comp rx=0 tx=0).stop 2024-07-27T03:18:44.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.603+0000 7f85bdd0f640 1 -- 172.21.15.67:0/4086853756 shutdown_connections 2024-07-27T03:18:44.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.603+0000 7f85bdd0f640 1 --2- 172.21.15.67:0/4086853756 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f85a40b3bd0 0x7f85a40b60e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.603+0000 7f85bdd0f640 1 --2- 172.21.15.67:0/4086853756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85b810d8e0 0x7f85b806e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.603+0000 7f85bdd0f640 1 --2- 172.21.15.67:0/4086853756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85b8104d10 0x7f85b806dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.603+0000 7f85bdd0f640 1 --2- 172.21.15.67:0/4086853756 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f85b8104330 0x7f85b8074940 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:44.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.603+0000 7f85bdd0f640 1 -- 172.21.15.67:0/4086853756 >> 172.21.15.67:0/4086853756 conn(0x7f85b8079220 msgr2=0x7f85b8100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:44.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.603+0000 7f85bdd0f640 1 -- 172.21.15.67:0/4086853756 shutdown_connections 2024-07-27T03:18:44.610 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:44.603+0000 7f85bdd0f640 1 -- 172.21.15.67:0/4086853756 wait complete. 2024-07-27T03:18:44.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:44 smithi175 ceph-mon[37125]: pgmap v495: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-07-27T03:18:44.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:44 smithi067 ceph-mon[32814]: pgmap v495: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-07-27T03:18:45.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:44 smithi149 ceph-mon[37173]: pgmap v495: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-07-27T03:18:45.350 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-07-27T03:18:45.649 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:18:45.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:45 smithi175 ceph-mon[37125]: scrub queued for path: / 2024-07-27T03:18:45.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:45 smithi175 ceph-mon[37125]: scrub summary: idle+waiting paths [/] 2024-07-27T03:18:45.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:45 smithi175 ceph-mon[37125]: scrub summary: active paths [/] 2024-07-27T03:18:45.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:45 smithi175 ceph-mon[37125]: scrub summary: idle+waiting paths [/] 2024-07-27T03:18:45.875 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:45 smithi175 ceph-mon[37125]: scrub summary: idle 2024-07-27T03:18:45.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:45 smithi067 ceph-mon[32814]: scrub queued for path: / 2024-07-27T03:18:45.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:45 smithi067 ceph-mon[32814]: scrub summary: idle+waiting paths [/] 2024-07-27T03:18:45.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:45 smithi067 ceph-mon[32814]: scrub summary: active paths [/] 2024-07-27T03:18:45.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:45 smithi067 ceph-mon[32814]: scrub summary: idle+waiting paths [/] 2024-07-27T03:18:45.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:45 smithi067 ceph-mon[32814]: scrub summary: idle 2024-07-27T03:18:46.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:45 smithi149 ceph-mon[37173]: scrub queued for path: / 2024-07-27T03:18:46.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:45 smithi149 ceph-mon[37173]: scrub summary: idle+waiting paths [/] 2024-07-27T03:18:46.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:45 smithi149 ceph-mon[37173]: scrub summary: active paths [/] 2024-07-27T03:18:46.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:45 smithi149 ceph-mon[37173]: scrub summary: idle+waiting paths [/] 2024-07-27T03:18:46.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:45 smithi149 ceph-mon[37173]: scrub summary: idle 2024-07-27T03:18:46.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:46 smithi175 ceph-mon[37125]: pgmap v496: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 18 op/s 2024-07-27T03:18:46.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:46 smithi067 ceph-mon[32814]: pgmap v496: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 18 op/s 2024-07-27T03:18:47.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:46 smithi149 ceph-mon[37173]: pgmap v496: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 18 op/s 2024-07-27T03:18:47.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.561+0000 7f1db6361640 1 -- 172.21.15.67:0/3490878155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1db0102550 msgr2=0x7f1db0102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:47.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.561+0000 7f1db6361640 1 --2- 172.21.15.67:0/3490878155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1db0102550 0x7f1db0102960 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1da0009900 tx=0x7f1da002a410 comp rx=0 tx=0).stop 2024-07-27T03:18:47.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.561+0000 7f1db6361640 1 -- 172.21.15.67:0/3490878155 shutdown_connections 2024-07-27T03:18:47.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.561+0000 7f1db6361640 1 --2- 172.21.15.67:0/3490878155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1db0108790 0x7f1db010efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.561+0000 7f1db6361640 1 --2- 172.21.15.67:0/3490878155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1db0107e40 0x7f1db0108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.561+0000 7f1db6361640 1 --2- 172.21.15.67:0/3490878155 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1db0102550 0x7f1db0102960 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.564 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.561+0000 7f1db6361640 1 -- 172.21.15.67:0/3490878155 >> 172.21.15.67:0/3490878155 conn(0x7f1db0079220 msgr2=0x7f1db0079630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:47.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.561+0000 7f1db6361640 1 -- 172.21.15.67:0/3490878155 shutdown_connections 2024-07-27T03:18:47.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.561+0000 7f1db6361640 1 -- 172.21.15.67:0/3490878155 wait complete. 2024-07-27T03:18:47.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.563+0000 7f1db6361640 1 Processor -- start 2024-07-27T03:18:47.565 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.563+0000 7f1db6361640 1 -- start start 2024-07-27T03:18:47.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.563+0000 7f1db6361640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1db0102550 0x7f1db0199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:47.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.563+0000 7f1db6361640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1db0107e40 0x7f1db019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:47.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.563+0000 7f1db6361640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1db0108790 0x7f1db019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:47.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.563+0000 7f1db6361640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1db0113570 con 0x7f1db0108790 2024-07-27T03:18:47.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.563+0000 7f1db6361640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01133f0 con 0x7f1db0107e40 2024-07-27T03:18:47.566 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.563+0000 7f1db6361640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01136f0 con 0x7f1db0102550 2024-07-27T03:18:47.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.564+0000 7f1db48fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1db0108790 0x7f1db019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:47.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.564+0000 7f1db48fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1db0108790 0x7f1db019f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35980/0 (socket says 172.21.15.67:35980) 2024-07-27T03:18:47.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.564+0000 7f1db48fe640 1 -- 172.21.15.67:0/3834465576 learned_addr learned my addr 172.21.15.67:0/3834465576 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:47.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.564+0000 7f1daffff640 1 --2- 172.21.15.67:0/3834465576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1db0102550 0x7f1db0199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:47.567 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.564+0000 7f1daf7fe640 1 --2- 172.21.15.67:0/3834465576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1db0107e40 0x7f1db019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:47.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.564+0000 7f1db48fe640 1 -- 172.21.15.67:0/3834465576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1db0102550 msgr2=0x7f1db0199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:47.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.564+0000 7f1db48fe640 1 --2- 172.21.15.67:0/3834465576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1db0102550 0x7f1db0199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.564+0000 7f1db48fe640 1 -- 172.21.15.67:0/3834465576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1db0107e40 msgr2=0x7f1db019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:47.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.564+0000 7f1db48fe640 1 --2- 172.21.15.67:0/3834465576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1db0107e40 0x7f1db019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.564+0000 7f1db48fe640 1 -- 172.21.15.67:0/3834465576 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db019fef0 con 0x7f1db0108790 2024-07-27T03:18:47.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.564+0000 7f1db48fe640 1 --2- 172.21.15.67:0/3834465576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1db0108790 0x7f1db019f7c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1da800eb10 tx=0x7f1da800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:47.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.564+0000 7f1dad7fa640 1 -- 172.21.15.67:0/3834465576 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8004b20 con 0x7f1db0108790 2024-07-27T03:18:47.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.564+0000 7f1db6361640 1 -- 172.21.15.67:0/3834465576 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db01a01e0 con 0x7f1db0108790 2024-07-27T03:18:47.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.564+0000 7f1dad7fa640 1 -- 172.21.15.67:0/3834465576 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1da8004cc0 con 0x7f1db0108790 2024-07-27T03:18:47.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.564+0000 7f1dad7fa640 1 -- 172.21.15.67:0/3834465576 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8010670 con 0x7f1db0108790 2024-07-27T03:18:47.568 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.565+0000 7f1db6361640 1 -- 172.21.15.67:0/3834465576 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db01a6940 con 0x7f1db0108790 2024-07-27T03:18:47.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.567+0000 7f1dbc8e6740 1 Processor -- start 2024-07-27T03:18:47.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.568+0000 7f1dad7fa640 1 -- 172.21.15.67:0/3834465576 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f1da8010830 con 0x7f1db0108790 2024-07-27T03:18:47.570 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.568+0000 7f1dbc8e6740 1 -- start start 2024-07-27T03:18:47.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.568+0000 7f1dbc8e6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1f5070 0x56523e599cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:47.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.568+0000 7f1dbc8e6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56523e1cc1b0 0x56523e59a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:47.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.568+0000 7f1dbc8e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56523e1d4390 0x56523e59a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:47.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.568+0000 7f1dbc8e6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56523e4823b0 con 0x56523e1f5070 2024-07-27T03:18:47.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.568+0000 7f1dbc8e6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56523e510ad0 con 0x56523e1cc1b0 2024-07-27T03:18:47.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.568+0000 7f1dbc8e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56523e4fffd0 con 0x56523e1d4390 2024-07-27T03:18:47.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.568+0000 7f1daf7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1f5070 0x56523e599cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:47.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.568+0000 7f1daf7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1f5070 0x56523e599cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35988/0 (socket says 172.21.15.67:35988) 2024-07-27T03:18:47.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.568+0000 7f1daf7fe640 1 -- 172.21.15.67:0/1290821564 learned_addr learned my addr 172.21.15.67:0/1290821564 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:47.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.568+0000 7f1daffff640 1 --2- 172.21.15.67:0/1290821564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56523e1cc1b0 0x56523e59a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:47.572 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.569+0000 7f1dad7fa640 1 --2- 172.21.15.67:0/3834465576 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f1d9c0b3c50 0x7f1d9c0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:47.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.569+0000 7f1daf7fe640 1 -- 172.21.15.67:0/1290821564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56523e1d4390 msgr2=0x56523e59a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:47.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.569+0000 7f1daf7fe640 1 --2- 172.21.15.67:0/1290821564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56523e1d4390 0x56523e59a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.569+0000 7f1daf7fe640 1 -- 172.21.15.67:0/1290821564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56523e1cc1b0 msgr2=0x56523e59a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:47.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.569+0000 7f1daf7fe640 1 --2- 172.21.15.67:0/1290821564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56523e1cc1b0 0x56523e59a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.569+0000 7f1daf7fe640 1 -- 172.21.15.67:0/1290821564 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56523e495720 con 0x56523e1f5070 2024-07-27T03:18:47.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.569+0000 7f1daffff640 1 --2- 172.21.15.67:0/1290821564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56523e1cc1b0 0x56523e59a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:18:47.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.569+0000 7f1db48fe640 1 --2- 172.21.15.67:0/3834465576 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f1d9c0b3c50 0x7f1d9c0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:47.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.569+0000 7f1daf7fe640 1 --2- 172.21.15.67:0/1290821564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1f5070 0x56523e599cf0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009cd0 tx=0x7f1d98013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f392d2148a9d3d server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:47.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.569+0000 7f1dad7fa640 1 -- 172.21.15.67:0/3834465576 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f1da80e4390 con 0x7f1db0108790 2024-07-27T03:18:47.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.569+0000 7f1da67fc640 1 -- 172.21.15.67:0/1290821564 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98013180 con 0x56523e1f5070 2024-07-27T03:18:47.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.569+0000 7f1da67fc640 1 -- 172.21.15.67:0/1290821564 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d98017030 con 0x56523e1f5070 2024-07-27T03:18:47.573 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.569+0000 7f1da67fc640 1 -- 172.21.15.67:0/1290821564 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98010430 con 0x56523e1f5070 2024-07-27T03:18:47.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.570+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/1290821564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1f5070 msgr2=0x56523e599cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:47.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.570+0000 7f1db48fe640 1 --2- 172.21.15.67:0/3834465576 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f1d9c0b3c50 0x7f1d9c0b6160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1da80aaf60 tx=0x7f1da800efd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:47.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.570+0000 7f1dbc8e6740 1 --2- 172.21.15.67:0/1290821564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1f5070 0x56523e599cf0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009cd0 tx=0x7f1d98013830 comp rx=0 tx=0).stop 2024-07-27T03:18:47.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.570+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/1290821564 shutdown_connections 2024-07-27T03:18:47.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.570+0000 7f1dbc8e6740 1 --2- 172.21.15.67:0/1290821564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56523e1d4390 0x56523e59a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.570+0000 7f1dbc8e6740 1 --2- 172.21.15.67:0/1290821564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56523e1cc1b0 0x56523e59a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.570+0000 7f1dbc8e6740 1 --2- 172.21.15.67:0/1290821564 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1f5070 0x56523e599cf0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.570+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/1290821564 >> 172.21.15.67:0/1290821564 conn(0x56523e1e9460 msgr2=0x56523e597be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:47.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.570+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/1290821564 shutdown_connections 2024-07-27T03:18:47.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.570+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/1290821564 wait complete. 2024-07-27T03:18:47.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.571+0000 7f1dbc8e6740 1 -- start start 2024-07-27T03:18:47.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.571+0000 7f1dbc8e6740 1 Processor -- start 2024-07-27T03:18:47.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.572+0000 7f1dbc8e6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1cc1b0 0x56523e5a9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:47.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.572+0000 7f1dbc8e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56523e1d4390 0x56523e5aa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:47.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.572+0000 7f1dbc8e6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56523e1f5070 0x56523e5b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:47.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.572+0000 7f1dbc8e6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x56523e4a28b0 con 0x56523e1cc1b0 2024-07-27T03:18:47.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.572+0000 7f1dbc8e6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x56523e504010 con 0x56523e1f5070 2024-07-27T03:18:47.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.572+0000 7f1dbc8e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56523e363700 con 0x56523e1d4390 2024-07-27T03:18:47.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.572+0000 7f1daf7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1cc1b0 0x56523e5a9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:47.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.572+0000 7f1dbc8e6740 10 client.0 _ll_register_callbacks cb 0x56523e4aad40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:47.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.572+0000 7f1daf7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1cc1b0 0x56523e5a9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36002/0 (socket says 172.21.15.67:36002) 2024-07-27T03:18:47.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.572+0000 7f1daf7fe640 1 -- 172.21.15.67:0/443376956 learned_addr learned my addr 172.21.15.67:0/443376956 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:47.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.572+0000 7f1daffff640 1 --2- 172.21.15.67:0/443376956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56523e1d4390 0x56523e5aa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:47.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.572+0000 7f1daf7fe640 1 --2- 172.21.15.67:0/443376956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56523e1f5070 0x56523e5b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:47.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.572+0000 7f1daf7fe640 1 -- 172.21.15.67:0/443376956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56523e1d4390 msgr2=0x56523e5aa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:47.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.572+0000 7f1daf7fe640 1 --2- 172.21.15.67:0/443376956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56523e1d4390 0x56523e5aa270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.572+0000 7f1daf7fe640 1 -- 172.21.15.67:0/443376956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56523e1f5070 msgr2=0x56523e5b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:47.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.572+0000 7f1daf7fe640 1 --2- 172.21.15.67:0/443376956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56523e1f5070 0x56523e5b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.572+0000 7f1daf7fe640 1 -- 172.21.15.67:0/443376956 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56523e4a6ba0 con 0x56523e1cc1b0 2024-07-27T03:18:47.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.573+0000 7f1daf7fe640 1 --2- 172.21.15.67:0/443376956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1cc1b0 0x56523e5a9d10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1d98003b40 tx=0x7f1d98005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7d20a76fe2ecce server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:47.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.573+0000 7f1d8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:47.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.573+0000 7f1d8f7fe640 1 -- 172.21.15.67:0/443376956 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98025040 con 0x56523e1cc1b0 2024-07-27T03:18:47.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.573+0000 7f1d8f7fe640 1 -- 172.21.15.67:0/443376956 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f1d98013180 con 0x56523e1cc1b0 2024-07-27T03:18:47.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.573+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/443376956 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d98017030 con 0x56523e1cc1b0 2024-07-27T03:18:47.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.573+0000 7f1d8f7fe640 1 -- 172.21.15.67:0/443376956 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d98032050 con 0x56523e1cc1b0 2024-07-27T03:18:47.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.573+0000 7f1dbc8e6740 10 client.15939 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:47.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.573+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/443376956 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56523e260650 con 0x56523e1cc1b0 2024-07-27T03:18:47.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.573+0000 7f1d8f7fe640 1 -- 172.21.15.67:0/443376956 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d98017030 con 0x56523e1cc1b0 2024-07-27T03:18:47.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.574+0000 7f1d8f7fe640 1 -- 172.21.15.67:0/443376956 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1d98010bc0 con 0x56523e1cc1b0 2024-07-27T03:18:47.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.574+0000 7f1dbc8e6740 10 client.15939 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:47.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.574+0000 7f1dbc8e6740 10 client.15939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:18:47.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.574+0000 7f1dbc8e6740 20 client.15939 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:47.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.574+0000 7f1dbc8e6740 1 --2- 172.21.15.67:0/443376956 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x56523e1b39f0 0x56523e5bb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:47.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.574+0000 7f1dbc8e6740 4 client.15939 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:18:47.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.574+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/443376956 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56523e512a20 con 0x56523e1b39f0 2024-07-27T03:18:47.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.574+0000 7f1daffff640 1 --2- 172.21.15.67:0/443376956 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x56523e1b39f0 0x56523e5bb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:47.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.575+0000 7f1daffff640 1 --2- 172.21.15.67:0/443376956 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x56523e1b39f0 0x56523e5bb920 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7feeeca33e29dd3c server_cookie=aa35c8edb96058ef in_seq=0 out_seq=0 2024-07-27T03:18:47.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.575+0000 7f1d8f7fe640 10 client.15939 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:47.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.575+0000 7f1d8f7fe640 1 -- 172.21.15.67:0/443376956 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56523e512a20 con 0x56523e1b39f0 2024-07-27T03:18:47.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.575+0000 7f1d8f7fe640 10 client.15939 handle_command_reply: tid=0 2024-07-27T03:18:47.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.575+0000 7f1dbc8e6740 1 client.15939 shutdown 2024-07-27T03:18:47.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.576+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/443376956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1cc1b0 msgr2=0x56523e5a9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:47.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.576+0000 7f1dbc8e6740 1 --2- 172.21.15.67:0/443376956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1cc1b0 0x56523e5a9d10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1d98003b40 tx=0x7f1d98005b90 comp rx=0 tx=0).stop 2024-07-27T03:18:47.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.576+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/443376956 shutdown_connections 2024-07-27T03:18:47.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.576+0000 7f1dbc8e6740 1 --2- 172.21.15.67:0/443376956 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x56523e1b39f0 0x56523e5bb920 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.576+0000 7f1dbc8e6740 1 --2- 172.21.15.67:0/443376956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56523e1f5070 0x56523e5b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.576+0000 7f1dbc8e6740 1 --2- 172.21.15.67:0/443376956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56523e1d4390 0x56523e5aa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.576+0000 7f1dbc8e6740 1 --2- 172.21.15.67:0/443376956 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1cc1b0 0x56523e5a9d10 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.576+0000 7f1d8f7fe640 0 client.15939 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:47.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.576+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/443376956 >> 172.21.15.67:0/443376956 conn(0x56523e1e9460 msgr2=0x56523e59b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:47.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.576+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/443376956 shutdown_connections 2024-07-27T03:18:47.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.576+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/443376956 wait complete. 2024-07-27T03:18:47.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.576+0000 7f1dbc8e6740 20 client.15939 trim_cache size 0 max 16384 2024-07-27T03:18:47.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.594+0000 7f1dbc8e6740 1 Processor -- start 2024-07-27T03:18:47.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.594+0000 7f1dbc8e6740 1 -- start start 2024-07-27T03:18:47.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.594+0000 7f1dbc8e6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56523e1cc1b0 0x56523e596b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:47.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.594+0000 7f1dbc8e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56523e1d4390 0x56523e5b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:47.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.594+0000 7f1dbc8e6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1f5070 0x56523e5c0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:47.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.594+0000 7f1dbc8e6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01ae4a0 con 0x56523e1f5070 2024-07-27T03:18:47.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.594+0000 7f1dbc8e6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01affe0 con 0x56523e1cc1b0 2024-07-27T03:18:47.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.594+0000 7f1dbc8e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01aeb70 con 0x56523e1d4390 2024-07-27T03:18:47.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.594+0000 7f1daf7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1f5070 0x56523e5c0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:47.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.594+0000 7f1daf7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1f5070 0x56523e5c0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36010/0 (socket says 172.21.15.67:36010) 2024-07-27T03:18:47.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.594+0000 7f1daf7fe640 1 -- 172.21.15.67:0/1683054841 learned_addr learned my addr 172.21.15.67:0/1683054841 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:47.597 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.594+0000 7f1daf7fe640 1 --2- 172.21.15.67:0/1683054841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56523e1cc1b0 0x56523e596b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:47.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.594+0000 7f1daffff640 1 --2- 172.21.15.67:0/1683054841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56523e1d4390 0x56523e5b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:47.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.594+0000 7f1daf7fe640 1 -- 172.21.15.67:0/1683054841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56523e1d4390 msgr2=0x56523e5b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:47.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.594+0000 7f1daf7fe640 1 --2- 172.21.15.67:0/1683054841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56523e1d4390 0x56523e5b57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.594+0000 7f1daf7fe640 1 -- 172.21.15.67:0/1683054841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56523e1cc1b0 msgr2=0x56523e596b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:47.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.594+0000 7f1daf7fe640 1 --2- 172.21.15.67:0/1683054841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56523e1cc1b0 0x56523e596b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.594+0000 7f1daf7fe640 1 -- 172.21.15.67:0/1683054841 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56523e59cab0 con 0x56523e1f5070 2024-07-27T03:18:47.598 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.595+0000 7f1daf7fe640 1 --2- 172.21.15.67:0/1683054841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1f5070 0x56523e5c0f80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1d98003ad0 tx=0x7f1d9802c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7660372bf40d3863 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:47.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.595+0000 7f1da5ffb640 1 -- 172.21.15.67:0/1683054841 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9802c0b0 con 0x56523e1f5070 2024-07-27T03:18:47.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.595+0000 7f1da5ffb640 1 -- 172.21.15.67:0/1683054841 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d98017030 con 0x56523e1f5070 2024-07-27T03:18:47.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.595+0000 7f1da5ffb640 1 -- 172.21.15.67:0/1683054841 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9802ca40 con 0x56523e1f5070 2024-07-27T03:18:47.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.596+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/1683054841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1f5070 msgr2=0x56523e5c0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:47.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.596+0000 7f1dbc8e6740 1 --2- 172.21.15.67:0/1683054841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1f5070 0x56523e5c0f80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1d98003ad0 tx=0x7f1d9802c040 comp rx=0 tx=0).stop 2024-07-27T03:18:47.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.596+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/1683054841 shutdown_connections 2024-07-27T03:18:47.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.596+0000 7f1dbc8e6740 1 --2- 172.21.15.67:0/1683054841 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1f5070 0x56523e5c0f80 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.596+0000 7f1dbc8e6740 1 --2- 172.21.15.67:0/1683054841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56523e1d4390 0x56523e5b57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.596+0000 7f1dbc8e6740 1 --2- 172.21.15.67:0/1683054841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56523e1cc1b0 0x56523e596b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.596+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/1683054841 >> 172.21.15.67:0/1683054841 conn(0x56523e1e9460 msgr2=0x56523e597a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:47.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.596+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/1683054841 shutdown_connections 2024-07-27T03:18:47.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.596+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/1683054841 wait complete. 2024-07-27T03:18:47.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.596+0000 7f1dbc8e6740 1 -- start start 2024-07-27T03:18:47.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.597+0000 7f1dbc8e6740 1 Processor -- start 2024-07-27T03:18:47.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.597+0000 7f1dbc8e6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56523e1cc1b0 0x56523e5ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:47.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.597+0000 7f1dbc8e6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56523e1d4390 0x56523e5adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:47.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.597+0000 7f1dbc8e6740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1f5070 0x56523e5ae320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:47.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.597+0000 7f1dbc8e6740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01af910 con 0x56523e1f5070 2024-07-27T03:18:47.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.597+0000 7f1dbc8e6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01af240 con 0x56523e1d4390 2024-07-27T03:18:47.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.597+0000 7f1dbc8e6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1db01ac480 con 0x56523e1cc1b0 2024-07-27T03:18:47.602 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.597+0000 7f1dbc8e6740 10 client.0 _ll_register_callbacks cb 0x7f1d98009bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:47.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.597+0000 7f1daf7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1f5070 0x56523e5ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:47.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.597+0000 7f1daf7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1f5070 0x56523e5ae320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36020/0 (socket says 172.21.15.67:36020) 2024-07-27T03:18:47.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.597+0000 7f1daf7fe640 1 -- 172.21.15.67:0/1777160595 learned_addr learned my addr 172.21.15.67:0/1777160595 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:47.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.597+0000 7f1daf7fe640 1 --2- 172.21.15.67:0/1777160595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56523e1cc1b0 0x56523e5ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:47.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.597+0000 7f1daffff640 1 --2- 172.21.15.67:0/1777160595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56523e1d4390 0x56523e5adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:47.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.598+0000 7f1daf7fe640 1 -- 172.21.15.67:0/1777160595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56523e1cc1b0 msgr2=0x56523e5ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:47.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.598+0000 7f1daf7fe640 1 --2- 172.21.15.67:0/1777160595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56523e1cc1b0 0x56523e5ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.598+0000 7f1daf7fe640 1 -- 172.21.15.67:0/1777160595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56523e1d4390 msgr2=0x56523e5adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:47.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.598+0000 7f1daf7fe640 1 --2- 172.21.15.67:0/1777160595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56523e1d4390 0x56523e5adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.598+0000 7f1daf7fe640 1 -- 172.21.15.67:0/1777160595 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56523e622fa0 con 0x56523e1f5070 2024-07-27T03:18:47.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.598+0000 7f1daffff640 1 --2- 172.21.15.67:0/1777160595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56523e1d4390 0x56523e5adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:18:47.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.598+0000 7f1daf7fe640 1 --2- 172.21.15.67:0/1777160595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1f5070 0x56523e5ae320 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x56523e5ac170 tx=0x7f1d9802c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5dd0617ac04d2ad server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:47.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.598+0000 7f1d8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:47.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.598+0000 7f1d8f7fe640 1 -- 172.21.15.67:0/1777160595 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98005420 con 0x56523e1f5070 2024-07-27T03:18:47.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.598+0000 7f1d8f7fe640 1 -- 172.21.15.67:0/1777160595 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f1d980035f0 con 0x56523e1f5070 2024-07-27T03:18:47.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.598+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/1777160595 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d98017030 con 0x56523e1f5070 2024-07-27T03:18:47.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.598+0000 7f1d8f7fe640 1 -- 172.21.15.67:0/1777160595 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1d980037f0 con 0x56523e1f5070 2024-07-27T03:18:47.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.598+0000 7f1dbc8e6740 10 client.15945 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:47.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.598+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/1777160595 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56523e624230 con 0x56523e1f5070 2024-07-27T03:18:47.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.598+0000 7f1d8f7fe640 1 -- 172.21.15.67:0/1777160595 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d98017030 con 0x56523e1f5070 2024-07-27T03:18:47.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.599+0000 7f1d8f7fe640 1 -- 172.21.15.67:0/1777160595 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f1d980106d0 con 0x56523e1f5070 2024-07-27T03:18:47.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.599+0000 7f1dbc8e6740 10 client.15945 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:47.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.599+0000 7f1dbc8e6740 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:18:47.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.599+0000 7f1dbc8e6740 20 client.15945 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:47.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.599+0000 7f1dbc8e6740 1 --2- 172.21.15.67:0/1777160595 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x56523e188da0 0x56523e626880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:47.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.599+0000 7f1dbc8e6740 4 client.15945 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-07-27T03:18:47.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.599+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/1777160595 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56523e50e4c0 con 0x56523e188da0 2024-07-27T03:18:47.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.599+0000 7f1daffff640 1 --2- 172.21.15.67:0/1777160595 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x56523e188da0 0x56523e626880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:47.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.600+0000 7f1daffff640 1 --2- 172.21.15.67:0/1777160595 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x56523e188da0 0x56523e626880 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43dae68921400acd server_cookie=5fd8ac2694bb2130 in_seq=0 out_seq=0 2024-07-27T03:18:47.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.600+0000 7f1d8f7fe640 10 client.15945 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:47.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.600+0000 7f1d8f7fe640 1 -- 172.21.15.67:0/1777160595 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56523e50e4c0 con 0x56523e188da0 2024-07-27T03:18:47.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.600+0000 7f1d8f7fe640 10 client.15945 handle_command_reply: tid=0 2024-07-27T03:18:47.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.600+0000 7f1dbc8e6740 1 client.15945 shutdown 2024-07-27T03:18:47.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.600+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/1777160595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1f5070 msgr2=0x56523e5ae320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:47.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.600+0000 7f1dbc8e6740 1 --2- 172.21.15.67:0/1777160595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1f5070 0x56523e5ae320 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x56523e5ac170 tx=0x7f1d9802c880 comp rx=0 tx=0).stop 2024-07-27T03:18:47.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.600+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/1777160595 shutdown_connections 2024-07-27T03:18:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.600+0000 7f1dbc8e6740 1 --2- 172.21.15.67:0/1777160595 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x56523e188da0 0x56523e626880 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.600+0000 7f1d8f7fe640 0 client.15945 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:18:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.600+0000 7f1dbc8e6740 1 --2- 172.21.15.67:0/1777160595 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x56523e1f5070 0x56523e5ae320 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.600+0000 7f1dbc8e6740 1 --2- 172.21.15.67:0/1777160595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56523e1d4390 0x56523e5adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.600+0000 7f1dbc8e6740 1 --2- 172.21.15.67:0/1777160595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56523e1cc1b0 0x56523e5ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.600+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/1777160595 >> 172.21.15.67:0/1777160595 conn(0x56523e1e9460 msgr2=0x56523e597a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.600+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/1777160595 shutdown_connections 2024-07-27T03:18:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.600+0000 7f1dbc8e6740 1 -- 172.21.15.67:0/1777160595 wait complete. 2024-07-27T03:18:47.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.601+0000 7f1dbc8e6740 20 client.15945 trim_cache size 0 max 16384 2024-07-27T03:18:47.607 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:18:47.607 INFO:teuthology.orchestra.run.smithi067.stdout: "status": "no active scrubs running", 2024-07-27T03:18:47.608 INFO:teuthology.orchestra.run.smithi067.stdout: "scrubs": {} 2024-07-27T03:18:47.608 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:18:47.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.601+0000 7f1db6361640 1 -- 172.21.15.67:0/3834465576 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f1d9c0b3c50 msgr2=0x7f1d9c0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:47.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.601+0000 7f1db6361640 1 --2- 172.21.15.67:0/3834465576 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f1d9c0b3c50 0x7f1d9c0b6160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1da80aaf60 tx=0x7f1da800efd0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.601+0000 7f1db6361640 1 -- 172.21.15.67:0/3834465576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1db0108790 msgr2=0x7f1db019f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:47.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.601+0000 7f1db6361640 1 --2- 172.21.15.67:0/3834465576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1db0108790 0x7f1db019f7c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1da800eb10 tx=0x7f1da800c2d0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.608 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.601+0000 7f1db6361640 1 -- 172.21.15.67:0/3834465576 shutdown_connections 2024-07-27T03:18:47.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.601+0000 7f1db6361640 1 --2- 172.21.15.67:0/3834465576 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f1d9c0b3c50 0x7f1d9c0b6160 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.601+0000 7f1db6361640 1 --2- 172.21.15.67:0/3834465576 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1db0108790 0x7f1db019f7c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.601+0000 7f1db6361640 1 --2- 172.21.15.67:0/3834465576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1db0107e40 0x7f1db019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.601+0000 7f1db6361640 1 --2- 172.21.15.67:0/3834465576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1db0102550 0x7f1db0199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:47.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.601+0000 7f1db6361640 1 -- 172.21.15.67:0/3834465576 >> 172.21.15.67:0/3834465576 conn(0x7f1db0079220 msgr2=0x7f1db0105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:47.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.601+0000 7f1db6361640 1 -- 172.21.15.67:0/3834465576 shutdown_connections 2024-07-27T03:18:47.609 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:47.601+0000 7f1db6361640 1 -- 172.21.15.67:0/3834465576 wait complete. 2024-07-27T03:18:48.488 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-07-27T03:18:48.489 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:18:48.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:48 smithi067 ceph-mon[32814]: pgmap v497: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 14 op/s 2024-07-27T03:18:48.777 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:18:48.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:48 smithi175 ceph-mon[37125]: pgmap v497: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 14 op/s 2024-07-27T03:18:49.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:48 smithi149 ceph-mon[37173]: pgmap v497: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 MiB/s wr, 14 op/s 2024-07-27T03:18:50.541 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.535+0000 7f5755c44640 1 -- 172.21.15.67:0/3710761285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5750107e10 msgr2=0x7f5750108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:50.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.535+0000 7f5755c44640 1 --2- 172.21.15.67:0/3710761285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5750107e10 0x7f5750108220 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5744009900 tx=0x7f574402a410 comp rx=0 tx=0).stop 2024-07-27T03:18:50.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.535+0000 7f5755c44640 1 -- 172.21.15.67:0/3710761285 shutdown_connections 2024-07-27T03:18:50.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.535+0000 7f5755c44640 1 --2- 172.21.15.67:0/3710761285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5750108760 0x7f575010ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:50.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.535+0000 7f5755c44640 1 --2- 172.21.15.67:0/3710761285 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5750107e10 0x7f5750108220 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:50.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.535+0000 7f5755c44640 1 --2- 172.21.15.67:0/3710761285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57501024d0 0x7f57501028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:50.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.535+0000 7f5755c44640 1 -- 172.21.15.67:0/3710761285 >> 172.21.15.67:0/3710761285 conn(0x7f57500791a0 msgr2=0x7f57500795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:50.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.535+0000 7f5755c44640 1 -- 172.21.15.67:0/3710761285 shutdown_connections 2024-07-27T03:18:50.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.535+0000 7f5755c44640 1 -- 172.21.15.67:0/3710761285 wait complete. 2024-07-27T03:18:50.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.536+0000 7f5755c44640 1 Processor -- start 2024-07-27T03:18:50.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.536+0000 7f5755c44640 1 -- start start 2024-07-27T03:18:50.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.536+0000 7f5755c44640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57501024d0 0x7f5750199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:50.542 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.536+0000 7f5755c44640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5750107e10 0x7f575019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.536+0000 7f5755c44640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5750108760 0x7f575019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.536+0000 7f5755c44640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f57501134b0 con 0x7f57501024d0 2024-07-27T03:18:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.536+0000 7f5755c44640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5750113330 con 0x7f5750108760 2024-07-27T03:18:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.536+0000 7f5755c44640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5750113630 con 0x7f5750107e10 2024-07-27T03:18:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.537+0000 7f574f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57501024d0 0x7f5750199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.537+0000 7f574f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57501024d0 0x7f5750199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36038/0 (socket says 172.21.15.67:36038) 2024-07-27T03:18:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.537+0000 7f574f7fe640 1 -- 172.21.15.67:0/684744991 learned_addr learned my addr 172.21.15.67:0/684744991 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.537+0000 7f574effd640 1 --2- 172.21.15.67:0/684744991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5750107e10 0x7f575019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.537+0000 7f574ffff640 1 --2- 172.21.15.67:0/684744991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5750108760 0x7f575019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.537+0000 7f574f7fe640 1 -- 172.21.15.67:0/684744991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5750107e10 msgr2=0x7f575019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.537+0000 7f574f7fe640 1 --2- 172.21.15.67:0/684744991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5750107e10 0x7f575019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:50.543 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.537+0000 7f574f7fe640 1 -- 172.21.15.67:0/684744991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5750108760 msgr2=0x7f575019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.537+0000 7f574f7fe640 1 --2- 172.21.15.67:0/684744991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5750108760 0x7f575019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.537+0000 7f574f7fe640 1 -- 172.21.15.67:0/684744991 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575019fe50 con 0x7f57501024d0 2024-07-27T03:18:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.537+0000 7f574f7fe640 1 --2- 172.21.15.67:0/684744991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57501024d0 0x7f5750199ee0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f574000d770 tx=0x7f574000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.537+0000 7f574cff9640 1 -- 172.21.15.67:0/684744991 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740017070 con 0x7f57501024d0 2024-07-27T03:18:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.537+0000 7f574cff9640 1 -- 172.21.15.67:0/684744991 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57400054d0 con 0x7f57501024d0 2024-07-27T03:18:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.537+0000 7f5755c44640 1 -- 172.21.15.67:0/684744991 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57501a0140 con 0x7f57501024d0 2024-07-27T03:18:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.537+0000 7f574cff9640 1 -- 172.21.15.67:0/684744991 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740004c30 con 0x7f57501024d0 2024-07-27T03:18:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.538+0000 7f5755c44640 1 -- 172.21.15.67:0/684744991 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57501a68a0 con 0x7f57501024d0 2024-07-27T03:18:50.544 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.539+0000 7f5755c44640 1 -- 172.21.15.67:0/684744991 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57501028e0 con 0x7f57501024d0 2024-07-27T03:18:50.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.542+0000 7f574cff9640 1 -- 172.21.15.67:0/684744991 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f5740004dd0 con 0x7f57501024d0 2024-07-27T03:18:50.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.542+0000 7f574cff9640 1 --2- 172.21.15.67:0/684744991 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f57380b3c50 0x7f57380b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:50.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.542+0000 7f574cff9640 1 -- 172.21.15.67:0/684744991 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f57400e3d10 con 0x7f57501024d0 2024-07-27T03:18:50.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.543+0000 7f574effd640 1 --2- 172.21.15.67:0/684744991 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f57380b3c50 0x7f57380b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:50.546 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.543+0000 7f574effd640 1 --2- 172.21.15.67:0/684744991 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f57380b3c50 0x7f57380b6160 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5744000c00 tx=0x7f5744031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:50.547 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.545+0000 7f574cff9640 1 -- 172.21.15.67:0/684744991 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f57400aa9e0 con 0x7f57501024d0 2024-07-27T03:18:50.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:50 smithi067 ceph-mon[32814]: pgmap v498: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.3 MiB/s wr, 12 op/s 2024-07-27T03:18:50.774 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.772+0000 7f5755c44640 1 -- 172.21.15.67:0/684744991 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f575019b010 con 0x7f57501024d0 2024-07-27T03:18:50.775 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.773+0000 7f574cff9640 1 -- 172.21.15.67:0/684744991 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11983 (secure 0 0 0) 0x7f575019b010 con 0x7f57501024d0 2024-07-27T03:18:50.775 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:18:50.775 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"btime":"2024-07-27T03:16:40:350577+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14679,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/252686369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":252686369},{"type":"v1","addr":"172.21.15.67:6835","nonce":252686369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/3358832762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3358832762},{"type":"v1","addr":"172.21.15.67:6837","nonce":3358832762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-27T03:12:34.320208+0000","modified":"2024-07-27T03:16:39.741149+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24620,"mds_1":24587,"mds_2":24583,"mds_3":24581,"mds_4":24563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14736":{"gid":14736,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/2162708317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2162708317},{"type":"v1","addr":"172.21.15.67:6839","nonce":2162708317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14748":{"gid":14748,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/795704244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":795704244},{"type":"v1","addr":"172.21.15.67:6841","nonce":795704244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/3868325215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3868325215},{"type":"v1","addr":"172.21.15.149:6833","nonce":3868325215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2462749027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2462749027},{"type":"v1","addr":"172.21.15.175:6835","nonce":2462749027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.149:6835/3730964806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3730964806},{"type":"v1","addr":"172.21.15.149:6835","nonce":3730964806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.175:6837/1304803502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1304803502},{"type":"v1","addr":"172.21.15.175:6837","nonce":1304803502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.175:6839/1670568853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1670568853},{"type":"v1","addr":"172.21.15.175:6839","nonce":1670568853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24587":{"gid":24587,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.149:6837/2932951845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2932951845},{"type":"v1","addr":"172.21.15.149:6837","nonce":2932951845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.149:6839/281746498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":281746498},{"type":"v1","addr":"172.21.15.149:6839","nonce":281746498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24620,"qdb_cluster":[24587,24583,24620,24581,24563]},"id":1}]} 2024-07-27T03:18:50.775 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 30 2024-07-27T03:18:50.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.775+0000 7f5755c44640 1 -- 172.21.15.67:0/684744991 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f57380b3c50 msgr2=0x7f57380b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:50.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.775+0000 7f5755c44640 1 --2- 172.21.15.67:0/684744991 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f57380b3c50 0x7f57380b6160 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5744000c00 tx=0x7f5744031400 comp rx=0 tx=0).stop 2024-07-27T03:18:50.777 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.775+0000 7f5755c44640 1 -- 172.21.15.67:0/684744991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57501024d0 msgr2=0x7f5750199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:50.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.775+0000 7f5755c44640 1 --2- 172.21.15.67:0/684744991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57501024d0 0x7f5750199ee0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f574000d770 tx=0x7f574000dc30 comp rx=0 tx=0).stop 2024-07-27T03:18:50.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.775+0000 7f5755c44640 1 -- 172.21.15.67:0/684744991 shutdown_connections 2024-07-27T03:18:50.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.775+0000 7f5755c44640 1 --2- 172.21.15.67:0/684744991 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f57380b3c50 0x7f57380b6160 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:50.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.775+0000 7f5755c44640 1 --2- 172.21.15.67:0/684744991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5750108760 0x7f575019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:50.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.775+0000 7f5755c44640 1 --2- 172.21.15.67:0/684744991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5750107e10 0x7f575019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:50.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.775+0000 7f5755c44640 1 --2- 172.21.15.67:0/684744991 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57501024d0 0x7f5750199ee0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:50.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.775+0000 7f5755c44640 1 -- 172.21.15.67:0/684744991 >> 172.21.15.67:0/684744991 conn(0x7f57500791a0 msgr2=0x7f5750103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:50.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.775+0000 7f5755c44640 1 -- 172.21.15.67:0/684744991 shutdown_connections 2024-07-27T03:18:50.778 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:50.775+0000 7f5755c44640 1 -- 172.21.15.67:0/684744991 wait complete. 2024-07-27T03:18:50.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:50 smithi175 ceph-mon[37125]: pgmap v498: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.3 MiB/s wr, 12 op/s 2024-07-27T03:18:51.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:50 smithi149 ceph-mon[37173]: pgmap v498: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.3 MiB/s wr, 12 op/s 2024-07-27T03:18:51.598 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-07-27T03:18:51.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:51 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/684744991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:18:51.879 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:18:51.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:51 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/684744991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:18:52.002 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:51 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/684744991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:18:52.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:52 smithi067 ceph-mon[32814]: pgmap v499: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 863 KiB/s wr, 15 op/s 2024-07-27T03:18:52.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:52 smithi149 ceph-mon[37173]: pgmap v499: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 863 KiB/s wr, 15 op/s 2024-07-27T03:18:53.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:52 smithi175 ceph-mon[37125]: pgmap v499: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 863 KiB/s wr, 15 op/s 2024-07-27T03:18:53.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.573+0000 7fe2944af640 1 -- 172.21.15.67:0/3208649264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe28c1024d0 msgr2=0x7fe28c1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:53.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.573+0000 7fe2944af640 1 --2- 172.21.15.67:0/3208649264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe28c1024d0 0x7fe28c1028e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe288009900 tx=0x7fe28802a410 comp rx=0 tx=0).stop 2024-07-27T03:18:53.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.573+0000 7fe2944af640 1 -- 172.21.15.67:0/3208649264 shutdown_connections 2024-07-27T03:18:53.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.573+0000 7fe2944af640 1 --2- 172.21.15.67:0/3208649264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe28c108760 0x7fe28c10ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.573+0000 7fe2944af640 1 --2- 172.21.15.67:0/3208649264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe28c107e10 0x7fe28c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.573+0000 7fe2944af640 1 --2- 172.21.15.67:0/3208649264 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe28c1024d0 0x7fe28c1028e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.573+0000 7fe2944af640 1 -- 172.21.15.67:0/3208649264 >> 172.21.15.67:0/3208649264 conn(0x7fe28c0791a0 msgr2=0x7fe28c0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:53.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.573+0000 7fe2944af640 1 -- 172.21.15.67:0/3208649264 shutdown_connections 2024-07-27T03:18:53.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.573+0000 7fe2944af640 1 -- 172.21.15.67:0/3208649264 wait complete. 2024-07-27T03:18:53.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.574+0000 7fe2944af640 1 Processor -- start 2024-07-27T03:18:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.574+0000 7fe2944af640 1 -- start start 2024-07-27T03:18:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.575+0000 7fe2944af640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe28c1024d0 0x7fe28c06b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.575+0000 7fe2944af640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe28c107e10 0x7fe28c06fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.575+0000 7fe2944af640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe28c108760 0x7fe28c0703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.575+0000 7fe2944af640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c1123c0 con 0x7fe28c108760 2024-07-27T03:18:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.575+0000 7fe2944af640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c112240 con 0x7fe28c107e10 2024-07-27T03:18:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.575+0000 7fe2944af640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c112540 con 0x7fe28c1024d0 2024-07-27T03:18:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.575+0000 7fe292a4c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe28c108760 0x7fe28c0703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:53.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.575+0000 7fe292a4c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe28c108760 0x7fe28c0703d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36062/0 (socket says 172.21.15.67:36062) 2024-07-27T03:18:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.575+0000 7fe292a4c640 1 -- 172.21.15.67:0/1463712476 learned_addr learned my addr 172.21.15.67:0/1463712476 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.575+0000 7fe291a4a640 1 --2- 172.21.15.67:0/1463712476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe28c107e10 0x7fe28c06fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.575+0000 7fe29224b640 1 --2- 172.21.15.67:0/1463712476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe28c1024d0 0x7fe28c06b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.575+0000 7fe292a4c640 1 -- 172.21.15.67:0/1463712476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe28c1024d0 msgr2=0x7fe28c06b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.575+0000 7fe292a4c640 1 --2- 172.21.15.67:0/1463712476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe28c1024d0 0x7fe28c06b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.575+0000 7fe292a4c640 1 -- 172.21.15.67:0/1463712476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe28c107e10 msgr2=0x7fe28c06fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.575+0000 7fe292a4c640 1 --2- 172.21.15.67:0/1463712476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe28c107e10 0x7fe28c06fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.575+0000 7fe292a4c640 1 -- 172.21.15.67:0/1463712476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28c1ae8d0 con 0x7fe28c108760 2024-07-27T03:18:53.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.576+0000 7fe292a4c640 1 --2- 172.21.15.67:0/1463712476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe28c108760 0x7fe28c0703d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe28400eaa0 tx=0x7fe28400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:53.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.576+0000 7fe2837fe640 1 -- 172.21.15.67:0/1463712476 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28400cdb0 con 0x7fe28c108760 2024-07-27T03:18:53.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.576+0000 7fe2944af640 1 -- 172.21.15.67:0/1463712476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe28c1aeb00 con 0x7fe28c108760 2024-07-27T03:18:53.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.576+0000 7fe2837fe640 1 -- 172.21.15.67:0/1463712476 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe284005020 con 0x7fe28c108760 2024-07-27T03:18:53.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.576+0000 7fe2837fe640 1 -- 172.21.15.67:0/1463712476 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2840116d0 con 0x7fe28c108760 2024-07-27T03:18:53.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.576+0000 7fe2944af640 1 -- 172.21.15.67:0/1463712476 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe28c1aefc0 con 0x7fe28c108760 2024-07-27T03:18:53.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.579+0000 7fe2837fe640 1 -- 172.21.15.67:0/1463712476 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7fe2840051d0 con 0x7fe28c108760 2024-07-27T03:18:53.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.579+0000 7fe29aa34740 1 Processor -- start 2024-07-27T03:18:53.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.579+0000 7fe29aa34740 1 -- start start 2024-07-27T03:18:53.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.580+0000 7fe29aa34740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d22ffe070 0x563d233a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:53.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.580+0000 7fe29aa34740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d22fd51b0 0x563d233a3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:53.585 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.580+0000 7fe29aa34740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22fdd390 0x563d233a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:53.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.580+0000 7fe29aa34740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563d2328b400 con 0x563d22fdd390 2024-07-27T03:18:53.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.580+0000 7fe29aa34740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563d23319b20 con 0x563d22ffe070 2024-07-27T03:18:53.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.580+0000 7fe29aa34740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563d23309020 con 0x563d22fd51b0 2024-07-27T03:18:53.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.580+0000 7fe291a4a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22fdd390 0x563d233a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:53.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.580+0000 7fe291a4a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22fdd390 0x563d233a37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36068/0 (socket says 172.21.15.67:36068) 2024-07-27T03:18:53.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.580+0000 7fe291a4a640 1 -- 172.21.15.67:0/830397570 learned_addr learned my addr 172.21.15.67:0/830397570 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:53.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.580+0000 7fe29224b640 1 --2- 172.21.15.67:0/830397570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d22fd51b0 0x563d233a3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:53.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.580+0000 7fe291a4a640 1 --2- 172.21.15.67:0/830397570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d22ffe070 0x563d233a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:53.586 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.580+0000 7fe2837fe640 1 --2- 172.21.15.67:0/1463712476 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe2780b3bd0 0x7fe2780b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:53.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.581+0000 7fe291a4a640 1 -- 172.21.15.67:0/830397570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d22fd51b0 msgr2=0x563d233a3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:53.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.581+0000 7fe291a4a640 1 --2- 172.21.15.67:0/830397570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d22fd51b0 0x563d233a3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.581+0000 7fe291a4a640 1 -- 172.21.15.67:0/830397570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d22ffe070 msgr2=0x563d233a2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:53.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.581+0000 7fe291a4a640 1 --2- 172.21.15.67:0/830397570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d22ffe070 0x563d233a2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.581+0000 7fe291a4a640 1 -- 172.21.15.67:0/830397570 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d2329e770 con 0x563d22fdd390 2024-07-27T03:18:53.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.581+0000 7fe292a4c640 1 --2- 172.21.15.67:0/1463712476 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe2780b3bd0 0x7fe2780b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:53.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.581+0000 7fe291a4a640 1 --2- 172.21.15.67:0/830397570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22fdd390 0x563d233a37c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe28c06a510 tx=0x7fe27c00b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7cf417c307e952 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:53.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.581+0000 7fe2837fe640 1 -- 172.21.15.67:0/1463712476 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fe2840e4080 con 0x7fe28c108760 2024-07-27T03:18:53.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.581+0000 7fe280d9a640 1 -- 172.21.15.67:0/830397570 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c0106b0 con 0x563d22fdd390 2024-07-27T03:18:53.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.581+0000 7fe292a4c640 1 --2- 172.21.15.67:0/1463712476 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe2780b3bd0 0x7fe2780b60e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe2840e4430 tx=0x7fe2840a6bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:53.587 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.581+0000 7fe280d9a640 1 -- 172.21.15.67:0/830397570 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe27c010d00 con 0x563d22fdd390 2024-07-27T03:18:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.581+0000 7fe280d9a640 1 -- 172.21.15.67:0/830397570 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c01dc30 con 0x563d22fdd390 2024-07-27T03:18:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.582+0000 7fe29aa34740 1 -- 172.21.15.67:0/830397570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22fdd390 msgr2=0x563d233a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.582+0000 7fe29aa34740 1 --2- 172.21.15.67:0/830397570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22fdd390 0x563d233a37c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe28c06a510 tx=0x7fe27c00b650 comp rx=0 tx=0).stop 2024-07-27T03:18:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.582+0000 7fe29aa34740 1 -- 172.21.15.67:0/830397570 shutdown_connections 2024-07-27T03:18:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.582+0000 7fe29aa34740 1 --2- 172.21.15.67:0/830397570 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22fdd390 0x563d233a37c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.582+0000 7fe29aa34740 1 --2- 172.21.15.67:0/830397570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d22fd51b0 0x563d233a3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.582+0000 7fe29aa34740 1 --2- 172.21.15.67:0/830397570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d22ffe070 0x563d233a2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.582+0000 7fe29aa34740 1 -- 172.21.15.67:0/830397570 >> 172.21.15.67:0/830397570 conn(0x563d22ff2460 msgr2=0x563d233a0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.582+0000 7fe29aa34740 1 -- 172.21.15.67:0/830397570 shutdown_connections 2024-07-27T03:18:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.582+0000 7fe29aa34740 1 -- 172.21.15.67:0/830397570 wait complete. 2024-07-27T03:18:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.583+0000 7fe29aa34740 1 -- start start 2024-07-27T03:18:53.589 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.583+0000 7fe29aa34740 1 Processor -- start 2024-07-27T03:18:53.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.583+0000 7fe29aa34740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22fd51b0 0x563d233ae950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:53.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.583+0000 7fe29aa34740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d22fdd390 0x563d233aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:53.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.583+0000 7fe29aa34740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d22ffe070 0x563d233ba690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:53.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.583+0000 7fe29aa34740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x563d232ab900 con 0x563d22fd51b0 2024-07-27T03:18:53.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.583+0000 7fe29aa34740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x563d2330d060 con 0x563d22fdd390 2024-07-27T03:18:53.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.583+0000 7fe29aa34740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563d2316c700 con 0x563d22ffe070 2024-07-27T03:18:53.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.583+0000 7fe29aa34740 10 client.0 _ll_register_callbacks cb 0x563d232b3d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:53.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.583+0000 7fe291a4a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22fd51b0 0x563d233ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:53.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.583+0000 7fe291a4a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22fd51b0 0x563d233ae950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36074/0 (socket says 172.21.15.67:36074) 2024-07-27T03:18:53.592 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.583+0000 7fe291a4a640 1 -- 172.21.15.67:0/1972704280 learned_addr learned my addr 172.21.15.67:0/1972704280 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:53.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.584+0000 7fe29224b640 1 --2- 172.21.15.67:0/1972704280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d22fdd390 0x563d233aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:53.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.584+0000 7fe291a4a640 1 --2- 172.21.15.67:0/1972704280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d22ffe070 0x563d233ba690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:53.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.584+0000 7fe291a4a640 1 -- 172.21.15.67:0/1972704280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d22ffe070 msgr2=0x563d233ba690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:53.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.584+0000 7fe291a4a640 1 --2- 172.21.15.67:0/1972704280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d22ffe070 0x563d233ba690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.584+0000 7fe291a4a640 1 -- 172.21.15.67:0/1972704280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d22fdd390 msgr2=0x563d233aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:53.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.584+0000 7fe291a4a640 1 --2- 172.21.15.67:0/1972704280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d22fdd390 0x563d233aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.584+0000 7fe291a4a640 1 -- 172.21.15.67:0/1972704280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d232afbf0 con 0x563d22fd51b0 2024-07-27T03:18:53.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.584+0000 7fe291a4a640 1 --2- 172.21.15.67:0/1972704280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22fd51b0 0x563d233ae950 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe27c0057a0 tx=0x7fe27c0056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749ccb30714e4d61 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:53.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.584+0000 7fe275ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:53.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.584+0000 7fe275ffb640 1 -- 172.21.15.67:0/1972704280 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c005850 con 0x563d22fd51b0 2024-07-27T03:18:53.593 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.584+0000 7fe275ffb640 1 -- 172.21.15.67:0/1972704280 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fe27c0104b0 con 0x563d22fd51b0 2024-07-27T03:18:53.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.584+0000 7fe29aa34740 1 -- 172.21.15.67:0/1972704280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe27c010d00 con 0x563d22fd51b0 2024-07-27T03:18:53.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.585+0000 7fe275ffb640 1 -- 172.21.15.67:0/1972704280 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe27c03b050 con 0x563d22fd51b0 2024-07-27T03:18:53.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.585+0000 7fe29aa34740 10 client.15963 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:53.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.585+0000 7fe29aa34740 1 -- 172.21.15.67:0/1972704280 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563d23069650 con 0x563d22fd51b0 2024-07-27T03:18:53.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.585+0000 7fe275ffb640 1 -- 172.21.15.67:0/1972704280 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe27c010d00 con 0x563d22fd51b0 2024-07-27T03:18:53.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.585+0000 7fe275ffb640 1 -- 172.21.15.67:0/1972704280 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe27c01c070 con 0x563d22fd51b0 2024-07-27T03:18:53.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.585+0000 7fe29aa34740 10 client.15963 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:53.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.585+0000 7fe29aa34740 10 client.15963 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-07-27T03:18:53.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.585+0000 7fe29aa34740 20 client.15963 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:53.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.585+0000 7fe29aa34740 1 --2- 172.21.15.67:0/1972704280 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x563d22fbc9f0 0x563d233c0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:53.594 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.585+0000 7fe29aa34740 4 client.15963 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:18:53.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.585+0000 7fe29aa34740 1 -- 172.21.15.67:0/1972704280 --> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d2331ba70 con 0x563d22fbc9f0 2024-07-27T03:18:53.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.586+0000 7fe29224b640 1 --2- 172.21.15.67:0/1972704280 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x563d22fbc9f0 0x563d233c0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:53.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.586+0000 7fe275ffb640 10 client.15963 ms_handle_connect on v2:172.21.15.149:6834/3730964806 2024-07-27T03:18:53.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.586+0000 7fe29224b640 1 --2- 172.21.15.67:0/1972704280 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x563d22fbc9f0 0x563d233c0590 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eec9fbe1624afac2 server_cookie=ab053057973f7c68 in_seq=0 out_seq=0 2024-07-27T03:18:53.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.587+0000 7fe275ffb640 1 -- 172.21.15.67:0/1972704280 <== mds.4 v2:172.21.15.149:6834/3730964806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d2331ba70 con 0x563d22fbc9f0 2024-07-27T03:18:53.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.587+0000 7fe275ffb640 10 client.15963 handle_command_reply: tid=0 2024-07-27T03:18:53.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.587+0000 7fe29aa34740 1 client.15963 shutdown 2024-07-27T03:18:53.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.587+0000 7fe29aa34740 1 -- 172.21.15.67:0/1972704280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22fd51b0 msgr2=0x563d233ae950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:53.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.587+0000 7fe29aa34740 1 --2- 172.21.15.67:0/1972704280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22fd51b0 0x563d233ae950 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe27c0057a0 tx=0x7fe27c0056f0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.595 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.587+0000 7fe29aa34740 1 -- 172.21.15.67:0/1972704280 shutdown_connections 2024-07-27T03:18:53.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.587+0000 7fe29aa34740 1 --2- 172.21.15.67:0/1972704280 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x563d22fbc9f0 0x563d233c0590 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.587+0000 7fe29aa34740 1 --2- 172.21.15.67:0/1972704280 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d22ffe070 0x563d233ba690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.587+0000 7fe29aa34740 1 --2- 172.21.15.67:0/1972704280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d22fdd390 0x563d233aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.587+0000 7fe29aa34740 1 --2- 172.21.15.67:0/1972704280 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22fd51b0 0x563d233ae950 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.587+0000 7fe275ffb640 0 client.15963 ms_handle_reset on v2:172.21.15.149:6834/3730964806 2024-07-27T03:18:53.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.587+0000 7fe29aa34740 1 -- 172.21.15.67:0/1972704280 >> 172.21.15.67:0/1972704280 conn(0x563d22ff2460 msgr2=0x563d233a5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:53.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.587+0000 7fe29aa34740 1 -- 172.21.15.67:0/1972704280 shutdown_connections 2024-07-27T03:18:53.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.587+0000 7fe29aa34740 1 -- 172.21.15.67:0/1972704280 wait complete. 2024-07-27T03:18:53.596 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.588+0000 7fe29aa34740 20 client.15963 trim_cache size 0 max 16384 2024-07-27T03:18:53.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.601+0000 7fe29aa34740 1 Processor -- start 2024-07-27T03:18:53.603 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.601+0000 7fe29aa34740 1 -- start start 2024-07-27T03:18:53.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.601+0000 7fe29aa34740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d22fd51b0 0x563d233ba480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:53.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.601+0000 7fe29aa34740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d22fdd390 0x563d233b0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:53.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.601+0000 7fe29aa34740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22ffe070 0x563d233c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:53.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.601+0000 7fe29aa34740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c1b6c00 con 0x563d22ffe070 2024-07-27T03:18:53.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.601+0000 7fe29aa34740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c1b8740 con 0x563d22fd51b0 2024-07-27T03:18:53.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.601+0000 7fe29aa34740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c1b72d0 con 0x563d22fdd390 2024-07-27T03:18:53.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.601+0000 7fe291a4a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22ffe070 0x563d233c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:53.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.602+0000 7fe291a4a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d22fd51b0 0x563d233ba480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:53.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.602+0000 7fe291a4a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d22fd51b0 0x563d233ba480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.67:48766/0 (socket says 172.21.15.67:48766) 2024-07-27T03:18:53.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.602+0000 7fe291a4a640 1 -- 172.21.15.67:0/1756867286 learned_addr learned my addr 172.21.15.67:0/1756867286 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:53.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.602+0000 7fe29224b640 1 --2- 172.21.15.67:0/1756867286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d22fdd390 0x563d233b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:53.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.602+0000 7fe291a4a640 1 -- 172.21.15.67:0/1756867286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d22fdd390 msgr2=0x563d233b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:53.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.602+0000 7fe291a4a640 1 --2- 172.21.15.67:0/1756867286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d22fdd390 0x563d233b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.602+0000 7fe291a4a640 1 -- 172.21.15.67:0/1756867286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d22fd51b0 msgr2=0x563d233ba480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:53.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.602+0000 7fe291a4a640 1 --2- 172.21.15.67:0/1756867286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d22fd51b0 0x563d233ba480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.602+0000 7fe291a4a640 1 -- 172.21.15.67:0/1756867286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d233afa10 con 0x563d22ffe070 2024-07-27T03:18:53.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.602+0000 7fe291a4a640 1 --2- 172.21.15.67:0/1756867286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22ffe070 0x563d233c5bf0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe27c01e4e0 tx=0x7fe27c01d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd98a059e84bf05 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:53.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.602+0000 7fe277fff640 1 -- 172.21.15.67:0/1756867286 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c00b410 con 0x563d22ffe070 2024-07-27T03:18:53.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.603+0000 7fe277fff640 1 -- 172.21.15.67:0/1756867286 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe27c02f040 con 0x563d22ffe070 2024-07-27T03:18:53.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.603+0000 7fe277fff640 1 -- 172.21.15.67:0/1756867286 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c004280 con 0x563d22ffe070 2024-07-27T03:18:53.611 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.603+0000 7fe29aa34740 1 -- 172.21.15.67:0/1756867286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22ffe070 msgr2=0x563d233c5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:53.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.603+0000 7fe29aa34740 1 --2- 172.21.15.67:0/1756867286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22ffe070 0x563d233c5bf0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe27c01e4e0 tx=0x7fe27c01d940 comp rx=0 tx=0).stop 2024-07-27T03:18:53.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.604+0000 7fe29aa34740 1 -- 172.21.15.67:0/1756867286 shutdown_connections 2024-07-27T03:18:53.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.604+0000 7fe29aa34740 1 --2- 172.21.15.67:0/1756867286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22ffe070 0x563d233c5bf0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.604+0000 7fe29aa34740 1 --2- 172.21.15.67:0/1756867286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d22fdd390 0x563d233b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.604+0000 7fe29aa34740 1 --2- 172.21.15.67:0/1756867286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d22fd51b0 0x563d233ba480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.604+0000 7fe29aa34740 1 -- 172.21.15.67:0/1756867286 >> 172.21.15.67:0/1756867286 conn(0x563d22ff2460 msgr2=0x563d233afc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:53.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.604+0000 7fe29aa34740 1 -- 172.21.15.67:0/1756867286 shutdown_connections 2024-07-27T03:18:53.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.604+0000 7fe29aa34740 1 -- 172.21.15.67:0/1756867286 wait complete. 2024-07-27T03:18:53.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.604+0000 7fe29aa34740 1 -- start start 2024-07-27T03:18:53.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.604+0000 7fe29aa34740 1 Processor -- start 2024-07-27T03:18:53.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.605+0000 7fe29aa34740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d22fd51b0 0x563d233b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:53.612 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.605+0000 7fe29aa34740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d22fdd390 0x563d233b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:53.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.605+0000 7fe29aa34740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22ffe070 0x563d233b2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:53.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.605+0000 7fe29aa34740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c1b8070 con 0x563d22ffe070 2024-07-27T03:18:53.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.605+0000 7fe29aa34740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c1b79a0 con 0x563d22fdd390 2024-07-27T03:18:53.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.605+0000 7fe29aa34740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c1b4be0 con 0x563d22fd51b0 2024-07-27T03:18:53.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.605+0000 7fe29aa34740 10 client.0 _ll_register_callbacks cb 0x7fe288009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:53.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.605+0000 7fe291a4a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22ffe070 0x563d233b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:53.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.605+0000 7fe291a4a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22ffe070 0x563d233b2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36088/0 (socket says 172.21.15.67:36088) 2024-07-27T03:18:53.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.605+0000 7fe291a4a640 1 -- 172.21.15.67:0/768506573 learned_addr learned my addr 172.21.15.67:0/768506573 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:53.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.605+0000 7fe291a4a640 1 --2- 172.21.15.67:0/768506573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d22fd51b0 0x563d233b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:53.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.605+0000 7fe29224b640 1 --2- 172.21.15.67:0/768506573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d22fdd390 0x563d233b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:53.613 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.605+0000 7fe291a4a640 1 -- 172.21.15.67:0/768506573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d22fd51b0 msgr2=0x563d233b6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:53.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.605+0000 7fe291a4a640 1 --2- 172.21.15.67:0/768506573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d22fd51b0 0x563d233b6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.605+0000 7fe291a4a640 1 -- 172.21.15.67:0/768506573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d22fdd390 msgr2=0x563d233b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:53.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.605+0000 7fe291a4a640 1 --2- 172.21.15.67:0/768506573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d22fdd390 0x563d233b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.605+0000 7fe291a4a640 1 -- 172.21.15.67:0/768506573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d233b2960 con 0x563d22ffe070 2024-07-27T03:18:53.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.605+0000 7fe29224b640 1 --2- 172.21.15.67:0/768506573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d22fdd390 0x563d233b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:18:53.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.606+0000 7fe291a4a640 1 --2- 172.21.15.67:0/768506573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22ffe070 0x563d233b2420 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x563d233a59c0 tx=0x7fe27c012200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf79a52838b09ade server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:53.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.606+0000 7fe275ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:53.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.606+0000 7fe275ffb640 1 -- 172.21.15.67:0/768506573 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c0122a0 con 0x563d22ffe070 2024-07-27T03:18:53.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.606+0000 7fe275ffb640 1 -- 172.21.15.67:0/768506573 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fe27c025570 con 0x563d22ffe070 2024-07-27T03:18:53.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.606+0000 7fe29aa34740 1 -- 172.21.15.67:0/768506573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe27c02f040 con 0x563d22ffe070 2024-07-27T03:18:53.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.606+0000 7fe275ffb640 1 -- 172.21.15.67:0/768506573 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe27c012440 con 0x563d22ffe070 2024-07-27T03:18:53.614 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.606+0000 7fe29aa34740 10 client.15969 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:53.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.606+0000 7fe29aa34740 1 -- 172.21.15.67:0/768506573 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563d23428860 con 0x563d22ffe070 2024-07-27T03:18:53.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.606+0000 7fe275ffb640 1 -- 172.21.15.67:0/768506573 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe27c02f040 con 0x563d22ffe070 2024-07-27T03:18:53.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.606+0000 7fe275ffb640 1 -- 172.21.15.67:0/768506573 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fe27c01c040 con 0x563d22ffe070 2024-07-27T03:18:53.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.607+0000 7fe29aa34740 10 client.15969 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:53.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.607+0000 7fe29aa34740 10 client.15969 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-07-27T03:18:53.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.607+0000 7fe29aa34740 20 client.15969 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:53.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.607+0000 7fe29aa34740 1 --2- 172.21.15.67:0/768506573 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x563d22f91da0 0x563d2342b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:53.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.607+0000 7fe29aa34740 4 client.15969 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-27T03:18:53.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.607+0000 7fe29aa34740 1 -- 172.21.15.67:0/768506573 --> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d23317510 con 0x563d22f91da0 2024-07-27T03:18:53.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.607+0000 7fe29224b640 1 --2- 172.21.15.67:0/768506573 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x563d22f91da0 0x563d2342b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:53.615 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.607+0000 7fe275ffb640 10 client.15969 ms_handle_connect on v2:172.21.15.149:6834/3730964806 2024-07-27T03:18:53.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.607+0000 7fe29224b640 1 --2- 172.21.15.67:0/768506573 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x563d22f91da0 0x563d2342b040 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=98010b6e792940e7 server_cookie=d685442a346ae052 in_seq=0 out_seq=0 2024-07-27T03:18:53.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.608+0000 7fe275ffb640 1 -- 172.21.15.67:0/768506573 <== mds.4 v2:172.21.15.149:6834/3730964806 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d23317510 con 0x563d22f91da0 2024-07-27T03:18:53.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.608+0000 7fe275ffb640 10 client.15969 handle_command_reply: tid=0 2024-07-27T03:18:53.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.608+0000 7fe29aa34740 1 client.15969 shutdown 2024-07-27T03:18:53.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.608+0000 7fe29aa34740 1 -- 172.21.15.67:0/768506573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22ffe070 msgr2=0x563d233b2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:53.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.608+0000 7fe29aa34740 1 --2- 172.21.15.67:0/768506573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22ffe070 0x563d233b2420 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x563d233a59c0 tx=0x7fe27c012200 comp rx=0 tx=0).stop 2024-07-27T03:18:53.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.608+0000 7fe29aa34740 1 -- 172.21.15.67:0/768506573 shutdown_connections 2024-07-27T03:18:53.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.608+0000 7fe29aa34740 1 --2- 172.21.15.67:0/768506573 >> [v2:172.21.15.149:6834/3730964806,v1:172.21.15.149:6835/3730964806] conn(0x563d22f91da0 0x563d2342b040 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.608+0000 7fe29aa34740 1 --2- 172.21.15.67:0/768506573 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x563d22ffe070 0x563d233b2420 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.608+0000 7fe29aa34740 1 --2- 172.21.15.67:0/768506573 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563d22fdd390 0x563d233b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.608+0000 7fe29aa34740 1 --2- 172.21.15.67:0/768506573 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563d22fd51b0 0x563d233b6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.608+0000 7fe275ffb640 0 client.15969 ms_handle_reset on v2:172.21.15.149:6834/3730964806 2024-07-27T03:18:53.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.608+0000 7fe29aa34740 1 -- 172.21.15.67:0/768506573 >> 172.21.15.67:0/768506573 conn(0x563d22ff2460 msgr2=0x563d233a3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:53.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.608+0000 7fe29aa34740 1 -- 172.21.15.67:0/768506573 shutdown_connections 2024-07-27T03:18:53.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.608+0000 7fe29aa34740 1 -- 172.21.15.67:0/768506573 wait complete. 2024-07-27T03:18:53.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.608+0000 7fe29aa34740 20 client.15969 trim_cache size 0 max 16384 2024-07-27T03:18:53.617 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-27T03:18:53.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.609+0000 7fe2944af640 1 -- 172.21.15.67:0/1463712476 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe2780b3bd0 msgr2=0x7fe2780b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:53.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.609+0000 7fe2944af640 1 --2- 172.21.15.67:0/1463712476 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe2780b3bd0 0x7fe2780b60e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe2840e4430 tx=0x7fe2840a6bd0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.609+0000 7fe2944af640 1 -- 172.21.15.67:0/1463712476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe28c108760 msgr2=0x7fe28c0703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:53.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.609+0000 7fe2944af640 1 --2- 172.21.15.67:0/1463712476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe28c108760 0x7fe28c0703d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe28400eaa0 tx=0x7fe28400ef60 comp rx=0 tx=0).stop 2024-07-27T03:18:53.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.609+0000 7fe2944af640 1 -- 172.21.15.67:0/1463712476 shutdown_connections 2024-07-27T03:18:53.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.609+0000 7fe2944af640 1 --2- 172.21.15.67:0/1463712476 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe2780b3bd0 0x7fe2780b60e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.609+0000 7fe2944af640 1 --2- 172.21.15.67:0/1463712476 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe28c108760 0x7fe28c0703d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.609+0000 7fe2944af640 1 --2- 172.21.15.67:0/1463712476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe28c107e10 0x7fe28c06fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.609+0000 7fe2944af640 1 --2- 172.21.15.67:0/1463712476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe28c1024d0 0x7fe28c06b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:53.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.609+0000 7fe2944af640 1 -- 172.21.15.67:0/1463712476 >> 172.21.15.67:0/1463712476 conn(0x7fe28c0791a0 msgr2=0x7fe28c100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:53.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.609+0000 7fe2944af640 1 -- 172.21.15.67:0/1463712476 shutdown_connections 2024-07-27T03:18:53.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:53.609+0000 7fe2944af640 1 -- 172.21.15.67:0/1463712476 wait complete. 2024-07-27T03:18:54.406 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-07-27T03:18:54.689 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:18:54.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:54 smithi067 ceph-mon[32814]: pgmap v500: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 859 KiB/s wr, 11 op/s 2024-07-27T03:18:55.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:54 smithi149 ceph-mon[37173]: pgmap v500: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 859 KiB/s wr, 11 op/s 2024-07-27T03:18:55.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:54 smithi175 ceph-mon[37125]: pgmap v500: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 859 KiB/s wr, 11 op/s 2024-07-27T03:18:56.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.349+0000 7efdd4efa640 1 -- 172.21.15.67:0/1695815581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdd0107e30 msgr2=0x7efdd0108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:56.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.349+0000 7efdd4efa640 1 --2- 172.21.15.67:0/1695815581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdd0107e30 0x7efdd0108240 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009900 tx=0x7efdb802a410 comp rx=0 tx=0).stop 2024-07-27T03:18:56.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.349+0000 7efdd4efa640 1 -- 172.21.15.67:0/1695815581 shutdown_connections 2024-07-27T03:18:56.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.349+0000 7efdd4efa640 1 --2- 172.21.15.67:0/1695815581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efdd0108780 0x7efdd010efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.349+0000 7efdd4efa640 1 --2- 172.21.15.67:0/1695815581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdd0107e30 0x7efdd0108240 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.352 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.349+0000 7efdd4efa640 1 --2- 172.21.15.67:0/1695815581 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd01024f0 0x7efdd0102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.350+0000 7efdd4efa640 1 -- 172.21.15.67:0/1695815581 >> 172.21.15.67:0/1695815581 conn(0x7efdd00791b0 msgr2=0x7efdd00795c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:56.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.350+0000 7efdd4efa640 1 -- 172.21.15.67:0/1695815581 shutdown_connections 2024-07-27T03:18:56.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.350+0000 7efdd4efa640 1 -- 172.21.15.67:0/1695815581 wait complete. 2024-07-27T03:18:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.350+0000 7efdd4efa640 1 Processor -- start 2024-07-27T03:18:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.351+0000 7efdd4efa640 1 -- start start 2024-07-27T03:18:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.351+0000 7efdd4efa640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdd01024f0 0x7efdd0104b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.351+0000 7efdd4efa640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd0107e30 0x7efdd01050b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.351+0000 7efdd4efa640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efdd0108780 0x7efdd01055f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.351+0000 7efdd4efa640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efdd01123d0 con 0x7efdd01024f0 2024-07-27T03:18:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.351+0000 7efdd4efa640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efdd0112250 con 0x7efdd0107e30 2024-07-27T03:18:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.351+0000 7efdd4efa640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efdd0112550 con 0x7efdd0108780 2024-07-27T03:18:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.351+0000 7efdce59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdd01024f0 0x7efdd0104b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.351+0000 7efdce59c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdd01024f0 0x7efdd0104b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40362/0 (socket says 172.21.15.67:40362) 2024-07-27T03:18:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.351+0000 7efdce59c640 1 -- 172.21.15.67:0/1999802355 learned_addr learned my addr 172.21.15.67:0/1999802355 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:56.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.351+0000 7efdcdd9b640 1 --2- 172.21.15.67:0/1999802355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd0107e30 0x7efdd01050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:56.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.351+0000 7efdced9d640 1 --2- 172.21.15.67:0/1999802355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efdd0108780 0x7efdd01055f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:56.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.351+0000 7efdce59c640 1 -- 172.21.15.67:0/1999802355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efdd0108780 msgr2=0x7efdd01055f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:56.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.351+0000 7efdce59c640 1 --2- 172.21.15.67:0/1999802355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efdd0108780 0x7efdd01055f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.351+0000 7efdce59c640 1 -- 172.21.15.67:0/1999802355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd0107e30 msgr2=0x7efdd01050b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:56.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.351+0000 7efdce59c640 1 --2- 172.21.15.67:0/1999802355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd0107e30 0x7efdd01050b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.352+0000 7efdce59c640 1 -- 172.21.15.67:0/1999802355 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd01a81c0 con 0x7efdd01024f0 2024-07-27T03:18:56.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.352+0000 7efdcdd9b640 1 --2- 172.21.15.67:0/1999802355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd0107e30 0x7efdd01050b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:18:56.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.352+0000 7efdce59c640 1 --2- 172.21.15.67:0/1999802355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdd01024f0 0x7efdd0104b70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7efdc000eab0 tx=0x7efdc000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:56.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.352+0000 7efdc77fe640 1 -- 172.21.15.67:0/1999802355 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc000ce40 con 0x7efdd01024f0 2024-07-27T03:18:56.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.352+0000 7efdc77fe640 1 -- 172.21.15.67:0/1999802355 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdc0011920 con 0x7efdd01024f0 2024-07-27T03:18:56.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.352+0000 7efdd4efa640 1 -- 172.21.15.67:0/1999802355 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd01a84b0 con 0x7efdd01024f0 2024-07-27T03:18:56.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.352+0000 7efdc77fe640 1 -- 172.21.15.67:0/1999802355 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc0010a30 con 0x7efdd01024f0 2024-07-27T03:18:56.356 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.352+0000 7efdd4efa640 1 -- 172.21.15.67:0/1999802355 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd01a89c0 con 0x7efdd01024f0 2024-07-27T03:18:56.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.355+0000 7efddb47f740 1 Processor -- start 2024-07-27T03:18:56.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.355+0000 7efddb47f740 1 -- start start 2024-07-27T03:18:56.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.355+0000 7efddb47f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4cc4070 0x5617a5068d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:56.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.355+0000 7efddb47f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a4c9b1b0 0x5617a5069280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:56.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.355+0000 7efddb47f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a4ca3390 0x5617a50697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:56.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.355+0000 7efddb47f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5617a4f51400 con 0x5617a4cc4070 2024-07-27T03:18:56.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.355+0000 7efddb47f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5617a4fdfb20 con 0x5617a4ca3390 2024-07-27T03:18:56.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.355+0000 7efddb47f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5617a4fcf020 con 0x5617a4c9b1b0 2024-07-27T03:18:56.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.355+0000 7efdc77fe640 1 -- 172.21.15.67:0/1999802355 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7efdc0010bd0 con 0x7efdd01024f0 2024-07-27T03:18:56.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.356+0000 7efdced9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4cc4070 0x5617a5068d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:56.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.356+0000 7efdced9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4cc4070 0x5617a5068d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40374/0 (socket says 172.21.15.67:40374) 2024-07-27T03:18:56.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.356+0000 7efdced9d640 1 -- 172.21.15.67:0/3111850500 learned_addr learned my addr 172.21.15.67:0/3111850500 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:56.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.356+0000 7efdcdd9b640 1 --2- 172.21.15.67:0/3111850500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a4ca3390 0x5617a50697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:56.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.356+0000 7efdce59c640 1 --2- 172.21.15.67:0/3111850500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a4c9b1b0 0x5617a5069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:56.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.356+0000 7efdced9d640 1 -- 172.21.15.67:0/3111850500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a4c9b1b0 msgr2=0x5617a5069280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:56.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.356+0000 7efdced9d640 1 --2- 172.21.15.67:0/3111850500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a4c9b1b0 0x5617a5069280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.356+0000 7efdced9d640 1 -- 172.21.15.67:0/3111850500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a4ca3390 msgr2=0x5617a50697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:56.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.356+0000 7efdced9d640 1 --2- 172.21.15.67:0/3111850500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a4ca3390 0x5617a50697c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.356+0000 7efdced9d640 1 -- 172.21.15.67:0/3111850500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617a4f64770 con 0x5617a4cc4070 2024-07-27T03:18:56.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.356+0000 7efdce59c640 1 --2- 172.21.15.67:0/3111850500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a4c9b1b0 0x5617a5069280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:18:56.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.356+0000 7efdcdd9b640 1 --2- 172.21.15.67:0/3111850500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a4ca3390 0x5617a50697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:18:56.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.356+0000 7efdced9d640 1 --2- 172.21.15.67:0/3111850500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4cc4070 0x5617a5068d40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7efdc800cb40 tx=0x7efdc800e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26909a06a49e843c server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:56.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.357+0000 7efdc77fe640 1 --2- 172.21.15.67:0/1999802355 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7efdbc0b3bd0 0x7efdbc0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:56.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.357+0000 7efdc4ff9640 1 -- 172.21.15.67:0/3111850500 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8005020 con 0x5617a4cc4070 2024-07-27T03:18:56.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.357+0000 7efdce59c640 1 --2- 172.21.15.67:0/1999802355 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7efdbc0b3bd0 0x7efdbc0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:56.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.357+0000 7efdc4ff9640 1 -- 172.21.15.67:0/3111850500 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdc80047a0 con 0x5617a4cc4070 2024-07-27T03:18:56.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.357+0000 7efdc4ff9640 1 -- 172.21.15.67:0/3111850500 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc80106b0 con 0x5617a4cc4070 2024-07-27T03:18:56.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.357+0000 7efdc77fe640 1 -- 172.21.15.67:0/1999802355 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7efdc00e4710 con 0x7efdd01024f0 2024-07-27T03:18:56.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.358+0000 7efddb47f740 1 -- 172.21.15.67:0/3111850500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4cc4070 msgr2=0x5617a5068d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:56.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.358+0000 7efdce59c640 1 --2- 172.21.15.67:0/1999802355 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7efdbc0b3bd0 0x7efdbc0b60e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efdc0000f80 tx=0x7efdc00094a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:56.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.358+0000 7efddb47f740 1 --2- 172.21.15.67:0/3111850500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4cc4070 0x5617a5068d40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7efdc800cb40 tx=0x7efdc800e350 comp rx=0 tx=0).stop 2024-07-27T03:18:56.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.358+0000 7efddb47f740 1 -- 172.21.15.67:0/3111850500 shutdown_connections 2024-07-27T03:18:56.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.358+0000 7efddb47f740 1 --2- 172.21.15.67:0/3111850500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a4ca3390 0x5617a50697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.358+0000 7efddb47f740 1 --2- 172.21.15.67:0/3111850500 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a4c9b1b0 0x5617a5069280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.358+0000 7efddb47f740 1 --2- 172.21.15.67:0/3111850500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4cc4070 0x5617a5068d40 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.358+0000 7efddb47f740 1 -- 172.21.15.67:0/3111850500 >> 172.21.15.67:0/3111850500 conn(0x5617a4cb8460 msgr2=0x5617a5066c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:56.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.358+0000 7efddb47f740 1 -- 172.21.15.67:0/3111850500 shutdown_connections 2024-07-27T03:18:56.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.358+0000 7efddb47f740 1 -- 172.21.15.67:0/3111850500 wait complete. 2024-07-27T03:18:56.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.359+0000 7efddb47f740 1 -- start start 2024-07-27T03:18:56.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.359+0000 7efddb47f740 1 Processor -- start 2024-07-27T03:18:56.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.360+0000 7efddb47f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4c9b1b0 0x5617a5074950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:56.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.360+0000 7efddb47f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a4ca3390 0x5617a5074eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:56.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.360+0000 7efddb47f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a4cc4070 0x5617a50806c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:56.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.360+0000 7efddb47f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x5617a4f71900 con 0x5617a4c9b1b0 2024-07-27T03:18:56.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.360+0000 7efddb47f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5617a4fd3060 con 0x5617a4ca3390 2024-07-27T03:18:56.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.360+0000 7efddb47f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5617a4e32700 con 0x5617a4cc4070 2024-07-27T03:18:56.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.360+0000 7efddb47f740 10 client.0 _ll_register_callbacks cb 0x5617a4f79d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:56.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.360+0000 7efdced9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4c9b1b0 0x5617a5074950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:56.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.360+0000 7efdced9d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4c9b1b0 0x5617a5074950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40390/0 (socket says 172.21.15.67:40390) 2024-07-27T03:18:56.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.360+0000 7efdced9d640 1 -- 172.21.15.67:0/2880195227 learned_addr learned my addr 172.21.15.67:0/2880195227 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:56.365 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.360+0000 7efdcdd9b640 1 --2- 172.21.15.67:0/2880195227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a4ca3390 0x5617a5074eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:56.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.360+0000 7efdced9d640 1 --2- 172.21.15.67:0/2880195227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a4cc4070 0x5617a50806c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:56.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.360+0000 7efdced9d640 1 -- 172.21.15.67:0/2880195227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a4cc4070 msgr2=0x5617a50806c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:56.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.360+0000 7efdced9d640 1 --2- 172.21.15.67:0/2880195227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a4cc4070 0x5617a50806c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.360+0000 7efdced9d640 1 -- 172.21.15.67:0/2880195227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a4ca3390 msgr2=0x5617a5074eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:56.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.360+0000 7efdced9d640 1 --2- 172.21.15.67:0/2880195227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a4ca3390 0x5617a5074eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.360+0000 7efdced9d640 1 -- 172.21.15.67:0/2880195227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617a4f75bf0 con 0x5617a4c9b1b0 2024-07-27T03:18:56.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.360+0000 7efdced9d640 1 --2- 172.21.15.67:0/2880195227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4c9b1b0 0x5617a5074950 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7efdc80058c0 tx=0x7efdc8005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac16b3df5ca797d server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:56.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.360+0000 7efdb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:56.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.361+0000 7efdb5ffb640 1 -- 172.21.15.67:0/2880195227 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8004020 con 0x5617a4c9b1b0 2024-07-27T03:18:56.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.361+0000 7efdb5ffb640 1 -- 172.21.15.67:0/2880195227 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7efdc80041c0 con 0x5617a4c9b1b0 2024-07-27T03:18:56.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.361+0000 7efddb47f740 1 -- 172.21.15.67:0/2880195227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdc80047a0 con 0x5617a4c9b1b0 2024-07-27T03:18:56.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.361+0000 7efdb5ffb640 1 -- 172.21.15.67:0/2880195227 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efdc80043c0 con 0x5617a4c9b1b0 2024-07-27T03:18:56.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.361+0000 7efddb47f740 10 client.15981 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:56.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.361+0000 7efddb47f740 1 -- 172.21.15.67:0/2880195227 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5617a4d2f650 con 0x5617a4c9b1b0 2024-07-27T03:18:56.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.362+0000 7efdb5ffb640 1 -- 172.21.15.67:0/2880195227 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdc80047a0 con 0x5617a4c9b1b0 2024-07-27T03:18:56.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.362+0000 7efdb5ffb640 1 -- 172.21.15.67:0/2880195227 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7efdc8028030 con 0x5617a4c9b1b0 2024-07-27T03:18:56.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.362+0000 7efddb47f740 10 client.15981 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:56.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.362+0000 7efddb47f740 10 client.15981 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-27T03:18:56.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.362+0000 7efddb47f740 20 client.15981 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:56.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.362+0000 7efddb47f740 1 --2- 172.21.15.67:0/2880195227 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x5617a4c829f0 0x5617a5086670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:56.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.362+0000 7efddb47f740 4 client.15981 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:18:56.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.362+0000 7efddb47f740 1 -- 172.21.15.67:0/2880195227 --> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617a4fe1a70 con 0x5617a4c829f0 2024-07-27T03:18:56.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.363+0000 7efdcdd9b640 1 --2- 172.21.15.67:0/2880195227 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x5617a4c829f0 0x5617a5086670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:56.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.363+0000 7efdb5ffb640 10 client.15981 ms_handle_connect on v2:172.21.15.175:6836/1304803502 2024-07-27T03:18:56.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.364+0000 7efdcdd9b640 1 --2- 172.21.15.67:0/2880195227 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x5617a4c829f0 0x5617a5086670 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f1292471c38ea876 server_cookie=34d037565081fbfa in_seq=0 out_seq=0 2024-07-27T03:18:56.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.365+0000 7efdb5ffb640 1 -- 172.21.15.67:0/2880195227 <== mds.3 v2:172.21.15.175:6836/1304803502 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5617a4fe1a70 con 0x5617a4c829f0 2024-07-27T03:18:56.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.365+0000 7efdb5ffb640 10 client.15981 handle_command_reply: tid=0 2024-07-27T03:18:56.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.365+0000 7efddb47f740 1 client.15981 shutdown 2024-07-27T03:18:56.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.366+0000 7efddb47f740 1 -- 172.21.15.67:0/2880195227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4c9b1b0 msgr2=0x5617a5074950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:56.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.366+0000 7efddb47f740 1 --2- 172.21.15.67:0/2880195227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4c9b1b0 0x5617a5074950 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7efdc80058c0 tx=0x7efdc8005ca0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.366+0000 7efddb47f740 1 -- 172.21.15.67:0/2880195227 shutdown_connections 2024-07-27T03:18:56.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.366+0000 7efddb47f740 1 --2- 172.21.15.67:0/2880195227 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x5617a4c829f0 0x5617a5086670 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.366+0000 7efddb47f740 1 --2- 172.21.15.67:0/2880195227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a4cc4070 0x5617a50806c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.366+0000 7efddb47f740 1 --2- 172.21.15.67:0/2880195227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a4ca3390 0x5617a5074eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.366+0000 7efdb5ffb640 0 client.15981 ms_handle_reset on v2:172.21.15.175:6836/1304803502 2024-07-27T03:18:56.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.366+0000 7efddb47f740 1 --2- 172.21.15.67:0/2880195227 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4c9b1b0 0x5617a5074950 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.366+0000 7efddb47f740 1 -- 172.21.15.67:0/2880195227 >> 172.21.15.67:0/2880195227 conn(0x5617a4cb8460 msgr2=0x5617a506bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:56.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.366+0000 7efddb47f740 1 -- 172.21.15.67:0/2880195227 shutdown_connections 2024-07-27T03:18:56.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.366+0000 7efddb47f740 1 -- 172.21.15.67:0/2880195227 wait complete. 2024-07-27T03:18:56.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.366+0000 7efddb47f740 20 client.15981 trim_cache size 0 max 16384 2024-07-27T03:18:56.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.379+0000 7efddb47f740 1 Processor -- start 2024-07-27T03:18:56.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.379+0000 7efddb47f740 1 -- start start 2024-07-27T03:18:56.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.380+0000 7efddb47f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a4c9b1b0 0x5617a5066c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:56.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.380+0000 7efddb47f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4ca3390 0x5617a5080630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:56.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.380+0000 7efddb47f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a4cc4070 0x5617a5080b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:56.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.380+0000 7efddb47f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efdd01b2860 con 0x5617a4ca3390 2024-07-27T03:18:56.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.380+0000 7efddb47f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efdd01b43a0 con 0x5617a4cc4070 2024-07-27T03:18:56.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.380+0000 7efddb47f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efdd01b2f30 con 0x5617a4c9b1b0 2024-07-27T03:18:56.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.380+0000 7efdcdd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4ca3390 0x5617a5080630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:56.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.380+0000 7efdcdd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4ca3390 0x5617a5080630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40402/0 (socket says 172.21.15.67:40402) 2024-07-27T03:18:56.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.380+0000 7efdcdd9b640 1 -- 172.21.15.67:0/2513142819 learned_addr learned my addr 172.21.15.67:0/2513142819 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:56.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.380+0000 7efdced9d640 1 --2- 172.21.15.67:0/2513142819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a4c9b1b0 0x5617a5066c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:56.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.380+0000 7efdced9d640 1 --2- 172.21.15.67:0/2513142819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a4cc4070 0x5617a5080b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:56.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.381+0000 7efdcdd9b640 1 -- 172.21.15.67:0/2513142819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a4c9b1b0 msgr2=0x5617a5066c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:56.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.381+0000 7efdcdd9b640 1 --2- 172.21.15.67:0/2513142819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a4c9b1b0 0x5617a5066c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.381+0000 7efdcdd9b640 1 -- 172.21.15.67:0/2513142819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a4cc4070 msgr2=0x5617a5080b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:56.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.381+0000 7efdcdd9b640 1 --2- 172.21.15.67:0/2513142819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a4cc4070 0x5617a5080b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.381+0000 7efdcdd9b640 1 -- 172.21.15.67:0/2513142819 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617a50810b0 con 0x5617a4ca3390 2024-07-27T03:18:56.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.381+0000 7efdcdd9b640 1 --2- 172.21.15.67:0/2513142819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4ca3390 0x5617a5080630 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7efdb8031440 tx=0x7efdb8005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52cd2a746c093eb5 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:56.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.381+0000 7efdb7fff640 1 -- 172.21.15.67:0/2513142819 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb804a070 con 0x5617a4ca3390 2024-07-27T03:18:56.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.381+0000 7efdb7fff640 1 -- 172.21.15.67:0/2513142819 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdb80054f0 con 0x5617a4ca3390 2024-07-27T03:18:56.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.381+0000 7efdb7fff640 1 -- 172.21.15.67:0/2513142819 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb8035040 con 0x5617a4ca3390 2024-07-27T03:18:56.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.382+0000 7efddb47f740 1 -- 172.21.15.67:0/2513142819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4ca3390 msgr2=0x5617a5080630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:56.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.382+0000 7efddb47f740 1 --2- 172.21.15.67:0/2513142819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4ca3390 0x5617a5080630 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7efdb8031440 tx=0x7efdb8005a20 comp rx=0 tx=0).stop 2024-07-27T03:18:56.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.382+0000 7efddb47f740 1 -- 172.21.15.67:0/2513142819 shutdown_connections 2024-07-27T03:18:56.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.382+0000 7efddb47f740 1 --2- 172.21.15.67:0/2513142819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a4cc4070 0x5617a5080b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.382+0000 7efddb47f740 1 --2- 172.21.15.67:0/2513142819 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4ca3390 0x5617a5080630 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.382+0000 7efddb47f740 1 --2- 172.21.15.67:0/2513142819 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a4c9b1b0 0x5617a5066c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.382+0000 7efddb47f740 1 -- 172.21.15.67:0/2513142819 >> 172.21.15.67:0/2513142819 conn(0x5617a4cb8460 msgr2=0x5617a5066550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:56.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.382+0000 7efddb47f740 1 -- 172.21.15.67:0/2513142819 shutdown_connections 2024-07-27T03:18:56.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.382+0000 7efddb47f740 1 -- 172.21.15.67:0/2513142819 wait complete. 2024-07-27T03:18:56.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.383+0000 7efddb47f740 1 -- start start 2024-07-27T03:18:56.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.383+0000 7efddb47f740 1 Processor -- start 2024-07-27T03:18:56.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.383+0000 7efddb47f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a4c9b1b0 0x5617a5089af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:56.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.383+0000 7efddb47f740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4ca3390 0x5617a5086fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:56.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.383+0000 7efddb47f740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a4cc4070 0x5617a5087500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:56.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.383+0000 7efddb47f740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efdd01b3cd0 con 0x5617a4ca3390 2024-07-27T03:18:56.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.383+0000 7efddb47f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efdd01b3600 con 0x5617a4c9b1b0 2024-07-27T03:18:56.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.383+0000 7efddb47f740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efdd01b0840 con 0x5617a4cc4070 2024-07-27T03:18:56.387 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.383+0000 7efddb47f740 10 client.0 _ll_register_callbacks cb 0x7efdc8014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:56.390 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.384+0000 7efdcdd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4ca3390 0x5617a5086fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:56.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.384+0000 7efdced9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a4c9b1b0 0x5617a5089af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:56.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.384+0000 7efdced9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a4c9b1b0 0x5617a5089af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.67:41598/0 (socket says 172.21.15.67:41598) 2024-07-27T03:18:56.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.384+0000 7efdced9d640 1 -- 172.21.15.67:0/1793850540 learned_addr learned my addr 172.21.15.67:0/1793850540 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:56.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.384+0000 7efdcdd9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4ca3390 0x5617a5086fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40408/0 (socket says 172.21.15.67:40408) 2024-07-27T03:18:56.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.384+0000 7efdced9d640 1 --2- 172.21.15.67:0/1793850540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a4cc4070 0x5617a5087500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:56.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.384+0000 7efdcdd9b640 1 -- 172.21.15.67:0/1793850540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a4cc4070 msgr2=0x5617a5087500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:56.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.384+0000 7efdcdd9b640 1 --2- 172.21.15.67:0/1793850540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a4cc4070 0x5617a5087500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.384+0000 7efdcdd9b640 1 -- 172.21.15.67:0/1793850540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a4c9b1b0 msgr2=0x5617a5089af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:56.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.384+0000 7efdcdd9b640 1 --2- 172.21.15.67:0/1793850540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a4c9b1b0 0x5617a5089af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.384+0000 7efdcdd9b640 1 -- 172.21.15.67:0/1793850540 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617a5087a40 con 0x5617a4ca3390 2024-07-27T03:18:56.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.384+0000 7efdcdd9b640 1 --2- 172.21.15.67:0/1793850540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4ca3390 0x5617a5086fc0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009230 tx=0x7efdb8032e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2144ce1ed35dd5bd server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:56.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.384+0000 7efdb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:56.391 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.385+0000 7efdb5ffb640 1 -- 172.21.15.67:0/1793850540 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb8035070 con 0x5617a4ca3390 2024-07-27T03:18:56.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.385+0000 7efdb5ffb640 1 -- 172.21.15.67:0/1793850540 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7efdb8004220 con 0x5617a4ca3390 2024-07-27T03:18:56.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.385+0000 7efddb47f740 1 -- 172.21.15.67:0/1793850540 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdb80054f0 con 0x5617a4ca3390 2024-07-27T03:18:56.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.385+0000 7efdb5ffb640 1 -- 172.21.15.67:0/1793850540 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efdb804a040 con 0x5617a4ca3390 2024-07-27T03:18:56.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.385+0000 7efddb47f740 10 client.15987 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:56.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.385+0000 7efddb47f740 1 -- 172.21.15.67:0/1793850540 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5617a50f7080 con 0x5617a4ca3390 2024-07-27T03:18:56.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.385+0000 7efdb5ffb640 1 -- 172.21.15.67:0/1793850540 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdb80054f0 con 0x5617a4ca3390 2024-07-27T03:18:56.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.386+0000 7efdb5ffb640 1 -- 172.21.15.67:0/1793850540 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7efdb804d070 con 0x5617a4ca3390 2024-07-27T03:18:56.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.387+0000 7efddb47f740 10 client.15987 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:56.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.387+0000 7efddb47f740 10 client.15987 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-07-27T03:18:56.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.387+0000 7efddb47f740 20 client.15987 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:56.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.387+0000 7efddb47f740 1 --2- 172.21.15.67:0/1793850540 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x5617a4c57da0 0x5617a50f9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:56.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.387+0000 7efddb47f740 4 client.15987 mds_command: new command op to 24581 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-27T03:18:56.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.387+0000 7efddb47f740 1 -- 172.21.15.67:0/1793850540 --> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5617a4fdd510 con 0x5617a4c57da0 2024-07-27T03:18:56.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.387+0000 7efdced9d640 1 --2- 172.21.15.67:0/1793850540 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x5617a4c57da0 0x5617a50f9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:56.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.387+0000 7efdb5ffb640 10 client.15987 ms_handle_connect on v2:172.21.15.175:6836/1304803502 2024-07-27T03:18:56.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.387+0000 7efdced9d640 1 --2- 172.21.15.67:0/1793850540 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x5617a4c57da0 0x5617a50f9710 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5741b8beda8a8450 server_cookie=f12225a6ad26df1b in_seq=0 out_seq=0 2024-07-27T03:18:56.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.388+0000 7efdb5ffb640 1 -- 172.21.15.67:0/1793850540 <== mds.3 v2:172.21.15.175:6836/1304803502 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5617a4fdd510 con 0x5617a4c57da0 2024-07-27T03:18:56.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.388+0000 7efdb5ffb640 10 client.15987 handle_command_reply: tid=0 2024-07-27T03:18:56.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.388+0000 7efddb47f740 1 client.15987 shutdown 2024-07-27T03:18:56.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.388+0000 7efddb47f740 1 -- 172.21.15.67:0/1793850540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4ca3390 msgr2=0x5617a5086fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:56.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.388+0000 7efddb47f740 1 --2- 172.21.15.67:0/1793850540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4ca3390 0x5617a5086fc0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009230 tx=0x7efdb8032e30 comp rx=0 tx=0).stop 2024-07-27T03:18:56.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.388+0000 7efddb47f740 1 -- 172.21.15.67:0/1793850540 shutdown_connections 2024-07-27T03:18:56.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.388+0000 7efddb47f740 1 --2- 172.21.15.67:0/1793850540 >> [v2:172.21.15.175:6836/1304803502,v1:172.21.15.175:6837/1304803502] conn(0x5617a4c57da0 0x5617a50f9710 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.388+0000 7efddb47f740 1 --2- 172.21.15.67:0/1793850540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5617a4cc4070 0x5617a5087500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.388+0000 7efddb47f740 1 --2- 172.21.15.67:0/1793850540 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5617a4ca3390 0x5617a5086fc0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.388+0000 7efddb47f740 1 --2- 172.21.15.67:0/1793850540 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5617a4c9b1b0 0x5617a5089af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.388+0000 7efddb47f740 1 -- 172.21.15.67:0/1793850540 >> 172.21.15.67:0/1793850540 conn(0x5617a4cb8460 msgr2=0x5617a50651d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:56.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.388+0000 7efdb5ffb640 0 client.15987 ms_handle_reset on v2:172.21.15.175:6836/1304803502 2024-07-27T03:18:56.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.388+0000 7efddb47f740 1 -- 172.21.15.67:0/1793850540 shutdown_connections 2024-07-27T03:18:56.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.388+0000 7efddb47f740 1 -- 172.21.15.67:0/1793850540 wait complete. 2024-07-27T03:18:56.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.388+0000 7efddb47f740 20 client.15987 trim_cache size 0 max 16384 2024-07-27T03:18:56.394 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-27T03:18:56.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.389+0000 7efdd4efa640 1 -- 172.21.15.67:0/1999802355 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7efdbc0b3bd0 msgr2=0x7efdbc0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:56.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.389+0000 7efdd4efa640 1 --2- 172.21.15.67:0/1999802355 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7efdbc0b3bd0 0x7efdbc0b60e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efdc0000f80 tx=0x7efdc00094a0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.389+0000 7efdd4efa640 1 -- 172.21.15.67:0/1999802355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdd01024f0 msgr2=0x7efdd0104b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:56.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.389+0000 7efdd4efa640 1 --2- 172.21.15.67:0/1999802355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdd01024f0 0x7efdd0104b70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7efdc000eab0 tx=0x7efdc000ef70 comp rx=0 tx=0).stop 2024-07-27T03:18:56.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.389+0000 7efdd4efa640 1 -- 172.21.15.67:0/1999802355 shutdown_connections 2024-07-27T03:18:56.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.389+0000 7efdd4efa640 1 --2- 172.21.15.67:0/1999802355 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7efdbc0b3bd0 0x7efdbc0b60e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.389+0000 7efdd4efa640 1 --2- 172.21.15.67:0/1999802355 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efdd0108780 0x7efdd01055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.389+0000 7efdd4efa640 1 --2- 172.21.15.67:0/1999802355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efdd0107e30 0x7efdd01050b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.389+0000 7efdd4efa640 1 --2- 172.21.15.67:0/1999802355 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efdd01024f0 0x7efdd0104b70 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:56.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.389+0000 7efdd4efa640 1 -- 172.21.15.67:0/1999802355 >> 172.21.15.67:0/1999802355 conn(0x7efdd00791b0 msgr2=0x7efdd01006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:56.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.389+0000 7efdd4efa640 1 -- 172.21.15.67:0/1999802355 shutdown_connections 2024-07-27T03:18:56.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:56.390+0000 7efdd4efa640 1 -- 172.21.15.67:0/1999802355 wait complete. 2024-07-27T03:18:56.715 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:56 smithi067 ceph-mon[32814]: pgmap v501: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-07-27T03:18:57.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:56 smithi149 ceph-mon[37173]: pgmap v501: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-07-27T03:18:57.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:56 smithi175 ceph-mon[37125]: pgmap v501: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.3 MiB/s wr, 15 op/s 2024-07-27T03:18:57.141 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-07-27T03:18:57.367 INFO:tasks.workunit.client.0.smithi067.stdout:setting up random write file 2024-07-27T03:18:57.367 INFO:tasks.workunit.client.0.smithi067.stdout:done setting up random write file 2024-07-27T03:18:57.367 INFO:tasks.workunit.client.0.smithi067.stdout:starting fsync run 2024-07-27T03:18:57.367 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0003s fsync time: 0.0096s 2024-07-27T03:18:57.367 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0003s fsync time: 0.0084s 2024-07-27T03:18:57.367 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0063s 2024-07-27T03:18:57.368 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0063s 2024-07-27T03:18:57.368 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0067s 2024-07-27T03:18:57.368 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0071s 2024-07-27T03:18:57.368 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0003s fsync time: 0.0071s 2024-07-27T03:18:57.368 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0060s 2024-07-27T03:18:57.368 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0063s 2024-07-27T03:18:57.368 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0058s 2024-07-27T03:18:57.368 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0061s 2024-07-27T03:18:57.368 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0058s 2024-07-27T03:18:57.368 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0003s fsync time: 0.0066s 2024-07-27T03:18:57.368 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0060s 2024-07-27T03:18:57.368 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0061s 2024-07-27T03:18:57.368 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0062s 2024-07-27T03:18:57.368 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0056s 2024-07-27T03:18:57.369 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0003s fsync time: 0.0066s 2024-07-27T03:18:57.369 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0003s fsync time: 0.0063s 2024-07-27T03:18:57.369 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0065s 2024-07-27T03:18:57.369 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0056s 2024-07-27T03:18:57.369 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0003s fsync time: 0.0068s 2024-07-27T03:18:57.369 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0058s 2024-07-27T03:18:57.369 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0057s 2024-07-27T03:18:57.369 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0067s 2024-07-27T03:18:57.369 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0058s 2024-07-27T03:18:57.369 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0065s 2024-07-27T03:18:57.369 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0065s 2024-07-27T03:18:57.369 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0064s 2024-07-27T03:18:57.369 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0003s fsync time: 0.0065s 2024-07-27T03:18:57.369 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0003s fsync time: 0.0061s 2024-07-27T03:18:57.370 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0057s 2024-07-27T03:18:57.370 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0072s 2024-07-27T03:18:57.370 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0003s fsync time: 0.0058s 2024-07-27T03:18:57.370 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0061s 2024-07-27T03:18:57.370 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0071s 2024-07-27T03:18:57.370 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0003s fsync time: 0.0067s 2024-07-27T03:18:57.370 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0057s 2024-07-27T03:18:57.370 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0061s 2024-07-27T03:18:57.370 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0064s 2024-07-27T03:18:57.370 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0070s 2024-07-27T03:18:57.370 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0058s 2024-07-27T03:18:57.370 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0064s 2024-07-27T03:18:57.370 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0067s 2024-07-27T03:18:57.371 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0058s 2024-07-27T03:18:57.371 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0061s 2024-07-27T03:18:57.371 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0058s 2024-07-27T03:18:57.371 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0003s fsync time: 0.0081s 2024-07-27T03:18:57.371 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0064s 2024-07-27T03:18:57.371 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0060s 2024-07-27T03:18:57.371 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0064s 2024-07-27T03:18:57.371 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0058s 2024-07-27T03:18:57.371 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0060s 2024-07-27T03:18:57.371 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0071s 2024-07-27T03:18:57.371 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0065s 2024-07-27T03:18:57.371 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0071s 2024-07-27T03:18:57.371 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0065s 2024-07-27T03:18:57.371 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0003s fsync time: 0.0071s 2024-07-27T03:18:57.372 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0002s fsync time: 0.0067s 2024-07-27T03:18:57.372 INFO:tasks.workunit.client.0.smithi067.stdout:write time: 0.0003s fsync time: 0.0071s 2024-07-27T03:18:57.372 INFO:tasks.workunit.client.0.smithi067.stdout:run done 61 fsyncs total, killing random writer 2024-07-27T03:18:57.372 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-07-27T03:18:57.372 INFO:tasks.workunit.client.0.smithi067.stdout:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-07-27T03:18:57.372 INFO:tasks.workunit.client.0.smithi067.stderr:+ whereis lsof 2024-07-27T03:18:57.401 INFO:tasks.workunit.client.0.smithi067.stdout:lsof: /usr/bin/lsof /usr/share/man/man1/lsof.1.gz 2024-07-27T03:18:57.402 INFO:tasks.workunit.client.0.smithi067.stderr:+ lsof 2024-07-27T03:18:57.422 INFO:tasks.workunit.client.0.smithi067.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/12bcfc4d12ec0a04ec4c60fcff70d14f47c466d670d89b15a9f74e828ec54b23/merged 2024-07-27T03:18:57.422 INFO:tasks.workunit.client.0.smithi067.stderr: Output information may be incomplete. 2024-07-27T03:18:57.422 INFO:tasks.workunit.client.0.smithi067.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/674bfc8892a054f27e7aa4ede34f89129086cfe49790ac9804f6ef6506431cf9/merged 2024-07-27T03:18:57.422 INFO:tasks.workunit.client.0.smithi067.stderr: Output information may be incomplete. 2024-07-27T03:18:57.422 INFO:tasks.workunit.client.0.smithi067.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/70789b57cd5c4d6eaf30237a2972d191e48a922ceedd8c943b786379b1d13b6c/merged 2024-07-27T03:18:57.422 INFO:tasks.workunit.client.0.smithi067.stderr: Output information may be incomplete. 2024-07-27T03:18:57.422 INFO:tasks.workunit.client.0.smithi067.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/6f3c9de00f6ae917a7c76ef47dc9ab8dc6f76e61a196bccedef3ca7f4706f9d2/merged 2024-07-27T03:18:57.423 INFO:tasks.workunit.client.0.smithi067.stderr: Output information may be incomplete. 2024-07-27T03:18:57.423 INFO:tasks.workunit.client.0.smithi067.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/f40c4da81a2247311fc4f6bfcdf14ab414446f413225781ac486830336b3034b/merged 2024-07-27T03:18:57.423 INFO:tasks.workunit.client.0.smithi067.stderr: Output information may be incomplete. 2024-07-27T03:18:57.423 INFO:tasks.workunit.client.0.smithi067.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/b35eb33d96823b9b6d2b36aa3e83e1d0094e4bd16f0c99046b2cafe2d9fe2aff/merged 2024-07-27T03:18:57.423 INFO:tasks.workunit.client.0.smithi067.stderr: Output information may be incomplete. 2024-07-27T03:18:57.423 INFO:tasks.workunit.client.0.smithi067.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/66d157c7abe04527f02430d1f3b695d5509aab24f1d8a45f2266325fd19dac06/merged 2024-07-27T03:18:57.423 INFO:tasks.workunit.client.0.smithi067.stderr: Output information may be incomplete. 2024-07-27T03:18:57.423 INFO:tasks.workunit.client.0.smithi067.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/50118fe2307939a64213443e190c2f88261c9f7463467574d2e7f0d72822ade5/merged 2024-07-27T03:18:57.423 INFO:tasks.workunit.client.0.smithi067.stderr: Output information may be incomplete. 2024-07-27T03:18:57.423 INFO:tasks.workunit.client.0.smithi067.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/8bd5cabbcf787f24ed6004034e53d3d940512dc8fe0cbeeba288c16c88c87211/merged 2024-07-27T03:18:57.423 INFO:tasks.workunit.client.0.smithi067.stderr: Output information may be incomplete. 2024-07-27T03:18:57.423 INFO:tasks.workunit.client.0.smithi067.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/cb40d4687fce95ded9315bfa3a4ddf2e61b2ffc7a83a8d87e72c9faa8031cf8a/merged 2024-07-27T03:18:57.423 INFO:tasks.workunit.client.0.smithi067.stderr: Output information may be incomplete. 2024-07-27T03:18:57.424 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:18:57.488 INFO:tasks.workunit.client.0.smithi067.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2024-07-27T03:18:57.489 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 1 root cwd unknown /proc/1/cwd (readlink: Permission denied) 2024-07-27T03:18:57.489 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 1 root rtd unknown /proc/1/root (readlink: Permission denied) 2024-07-27T03:18:57.489 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 1 root txt unknown /proc/1/exe (readlink: Permission denied) 2024-07-27T03:18:57.489 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 1 root NOFD /proc/1/fd (opendir: Permission denied) 2024-07-27T03:18:57.489 INFO:tasks.workunit.client.0.smithi067.stdout:kthreadd 2 root cwd unknown /proc/2/cwd (readlink: Permission denied) 2024-07-27T03:18:57.489 INFO:tasks.workunit.client.0.smithi067.stdout:kthreadd 2 root rtd unknown /proc/2/root (readlink: Permission denied) 2024-07-27T03:18:57.489 INFO:tasks.workunit.client.0.smithi067.stdout:kthreadd 2 root txt unknown /proc/2/exe (readlink: Permission denied) 2024-07-27T03:18:57.489 INFO:tasks.workunit.client.0.smithi067.stdout:kthreadd 2 root NOFD /proc/2/fd (opendir: Permission denied) 2024-07-27T03:18:57.489 INFO:tasks.workunit.client.0.smithi067.stdout:pool_work 3 root cwd unknown /proc/3/cwd (readlink: Permission denied) 2024-07-27T03:18:57.489 INFO:tasks.workunit.client.0.smithi067.stdout:pool_work 3 root rtd unknown /proc/3/root (readlink: Permission denied) 2024-07-27T03:18:57.489 INFO:tasks.workunit.client.0.smithi067.stdout:pool_work 3 root txt unknown /proc/3/exe (readlink: Permission denied) 2024-07-27T03:18:57.489 INFO:tasks.workunit.client.0.smithi067.stdout:pool_work 3 root NOFD /proc/3/fd (opendir: Permission denied) 2024-07-27T03:18:57.489 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 4 root cwd unknown /proc/4/cwd (readlink: Permission denied) 2024-07-27T03:18:57.490 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 4 root rtd unknown /proc/4/root (readlink: Permission denied) 2024-07-27T03:18:57.490 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 4 root txt unknown /proc/4/exe (readlink: Permission denied) 2024-07-27T03:18:57.490 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 4 root NOFD /proc/4/fd (opendir: Permission denied) 2024-07-27T03:18:57.490 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 5 root cwd unknown /proc/5/cwd (readlink: Permission denied) 2024-07-27T03:18:57.490 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 5 root rtd unknown /proc/5/root (readlink: Permission denied) 2024-07-27T03:18:57.490 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 5 root txt unknown /proc/5/exe (readlink: Permission denied) 2024-07-27T03:18:57.490 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 5 root NOFD /proc/5/fd (opendir: Permission denied) 2024-07-27T03:18:57.490 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 6 root cwd unknown /proc/6/cwd (readlink: Permission denied) 2024-07-27T03:18:57.490 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 6 root rtd unknown /proc/6/root (readlink: Permission denied) 2024-07-27T03:18:57.490 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 6 root txt unknown /proc/6/exe (readlink: Permission denied) 2024-07-27T03:18:57.491 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 6 root NOFD /proc/6/fd (opendir: Permission denied) 2024-07-27T03:18:57.491 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 7 root cwd unknown /proc/7/cwd (readlink: Permission denied) 2024-07-27T03:18:57.491 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 7 root rtd unknown /proc/7/root (readlink: Permission denied) 2024-07-27T03:18:57.491 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 7 root txt unknown /proc/7/exe (readlink: Permission denied) 2024-07-27T03:18:57.491 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 7 root NOFD /proc/7/fd (opendir: Permission denied) 2024-07-27T03:18:57.491 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 9 root cwd unknown /proc/9/cwd (readlink: Permission denied) 2024-07-27T03:18:57.491 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 9 root rtd unknown /proc/9/root (readlink: Permission denied) 2024-07-27T03:18:57.491 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 9 root txt unknown /proc/9/exe (readlink: Permission denied) 2024-07-27T03:18:57.491 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 9 root NOFD /proc/9/fd (opendir: Permission denied) 2024-07-27T03:18:57.491 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 11 root cwd unknown /proc/11/cwd (readlink: Permission denied) 2024-07-27T03:18:57.491 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 11 root rtd unknown /proc/11/root (readlink: Permission denied) 2024-07-27T03:18:57.491 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 11 root txt unknown /proc/11/exe (readlink: Permission denied) 2024-07-27T03:18:57.491 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 11 root NOFD /proc/11/fd (opendir: Permission denied) 2024-07-27T03:18:57.491 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 12 root cwd unknown /proc/12/cwd (readlink: Permission denied) 2024-07-27T03:18:57.492 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 12 root rtd unknown /proc/12/root (readlink: Permission denied) 2024-07-27T03:18:57.492 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 12 root txt unknown /proc/12/exe (readlink: Permission denied) 2024-07-27T03:18:57.492 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 12 root NOFD /proc/12/fd (opendir: Permission denied) 2024-07-27T03:18:57.492 INFO:tasks.workunit.client.0.smithi067.stdout:rcu_tasks 13 root cwd unknown /proc/13/cwd (readlink: Permission denied) 2024-07-27T03:18:57.492 INFO:tasks.workunit.client.0.smithi067.stdout:rcu_tasks 13 root rtd unknown /proc/13/root (readlink: Permission denied) 2024-07-27T03:18:57.492 INFO:tasks.workunit.client.0.smithi067.stdout:rcu_tasks 13 root txt unknown /proc/13/exe (readlink: Permission denied) 2024-07-27T03:18:57.492 INFO:tasks.workunit.client.0.smithi067.stdout:rcu_tasks 13 root NOFD /proc/13/fd (opendir: Permission denied) 2024-07-27T03:18:57.492 INFO:tasks.workunit.client.0.smithi067.stdout:rcu_tasks 14 root cwd unknown /proc/14/cwd (readlink: Permission denied) 2024-07-27T03:18:57.492 INFO:tasks.workunit.client.0.smithi067.stdout:rcu_tasks 14 root rtd unknown /proc/14/root (readlink: Permission denied) 2024-07-27T03:18:57.492 INFO:tasks.workunit.client.0.smithi067.stdout:rcu_tasks 14 root txt unknown /proc/14/exe (readlink: Permission denied) 2024-07-27T03:18:57.492 INFO:tasks.workunit.client.0.smithi067.stdout:rcu_tasks 14 root NOFD /proc/14/fd (opendir: Permission denied) 2024-07-27T03:18:57.492 INFO:tasks.workunit.client.0.smithi067.stdout:rcu_tasks 15 root cwd unknown /proc/15/cwd (readlink: Permission denied) 2024-07-27T03:18:57.492 INFO:tasks.workunit.client.0.smithi067.stdout:rcu_tasks 15 root rtd unknown /proc/15/root (readlink: Permission denied) 2024-07-27T03:18:57.492 INFO:tasks.workunit.client.0.smithi067.stdout:rcu_tasks 15 root txt unknown /proc/15/exe (readlink: Permission denied) 2024-07-27T03:18:57.493 INFO:tasks.workunit.client.0.smithi067.stdout:rcu_tasks 15 root NOFD /proc/15/fd (opendir: Permission denied) 2024-07-27T03:18:57.493 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 16 root cwd unknown /proc/16/cwd (readlink: Permission denied) 2024-07-27T03:18:57.493 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 16 root rtd unknown /proc/16/root (readlink: Permission denied) 2024-07-27T03:18:57.493 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 16 root txt unknown /proc/16/exe (readlink: Permission denied) 2024-07-27T03:18:57.493 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 16 root NOFD /proc/16/fd (opendir: Permission denied) 2024-07-27T03:18:57.493 INFO:tasks.workunit.client.0.smithi067.stdout:pr/ttyS1 17 root cwd unknown /proc/17/cwd (readlink: Permission denied) 2024-07-27T03:18:57.493 INFO:tasks.workunit.client.0.smithi067.stdout:pr/ttyS1 17 root rtd unknown /proc/17/root (readlink: Permission denied) 2024-07-27T03:18:57.493 INFO:tasks.workunit.client.0.smithi067.stdout:pr/ttyS1 17 root txt unknown /proc/17/exe (readlink: Permission denied) 2024-07-27T03:18:57.493 INFO:tasks.workunit.client.0.smithi067.stdout:pr/ttyS1 17 root NOFD /proc/17/fd (opendir: Permission denied) 2024-07-27T03:18:57.493 INFO:tasks.workunit.client.0.smithi067.stdout:rcu_preem 18 root cwd unknown /proc/18/cwd (readlink: Permission denied) 2024-07-27T03:18:57.493 INFO:tasks.workunit.client.0.smithi067.stdout:rcu_preem 18 root rtd unknown /proc/18/root (readlink: Permission denied) 2024-07-27T03:18:57.493 INFO:tasks.workunit.client.0.smithi067.stdout:rcu_preem 18 root txt unknown /proc/18/exe (readlink: Permission denied) 2024-07-27T03:18:57.494 INFO:tasks.workunit.client.0.smithi067.stdout:rcu_preem 18 root NOFD /proc/18/fd (opendir: Permission denied) 2024-07-27T03:18:57.494 INFO:tasks.workunit.client.0.smithi067.stdout:migration 19 root cwd unknown /proc/19/cwd (readlink: Permission denied) 2024-07-27T03:18:57.494 INFO:tasks.workunit.client.0.smithi067.stdout:migration 19 root rtd unknown /proc/19/root (readlink: Permission denied) 2024-07-27T03:18:57.494 INFO:tasks.workunit.client.0.smithi067.stdout:migration 19 root txt unknown /proc/19/exe (readlink: Permission denied) 2024-07-27T03:18:57.494 INFO:tasks.workunit.client.0.smithi067.stdout:migration 19 root NOFD /proc/19/fd (opendir: Permission denied) 2024-07-27T03:18:57.494 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 20 root cwd unknown /proc/20/cwd (readlink: Permission denied) 2024-07-27T03:18:57.494 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 20 root rtd unknown /proc/20/root (readlink: Permission denied) 2024-07-27T03:18:57.494 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 20 root txt unknown /proc/20/exe (readlink: Permission denied) 2024-07-27T03:18:57.494 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 20 root NOFD /proc/20/fd (opendir: Permission denied) 2024-07-27T03:18:57.494 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 21 root cwd unknown /proc/21/cwd (readlink: Permission denied) 2024-07-27T03:18:57.494 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 21 root rtd unknown /proc/21/root (readlink: Permission denied) 2024-07-27T03:18:57.494 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 21 root txt unknown /proc/21/exe (readlink: Permission denied) 2024-07-27T03:18:57.494 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 21 root NOFD /proc/21/fd (opendir: Permission denied) 2024-07-27T03:18:57.495 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/0 22 root cwd unknown /proc/22/cwd (readlink: Permission denied) 2024-07-27T03:18:57.495 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/0 22 root rtd unknown /proc/22/root (readlink: Permission denied) 2024-07-27T03:18:57.495 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/0 22 root txt unknown /proc/22/exe (readlink: Permission denied) 2024-07-27T03:18:57.495 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/0 22 root NOFD /proc/22/fd (opendir: Permission denied) 2024-07-27T03:18:57.495 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/1 23 root cwd unknown /proc/23/cwd (readlink: Permission denied) 2024-07-27T03:18:57.495 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/1 23 root rtd unknown /proc/23/root (readlink: Permission denied) 2024-07-27T03:18:57.495 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/1 23 root txt unknown /proc/23/exe (readlink: Permission denied) 2024-07-27T03:18:57.495 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/1 23 root NOFD /proc/23/fd (opendir: Permission denied) 2024-07-27T03:18:57.495 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 24 root cwd unknown /proc/24/cwd (readlink: Permission denied) 2024-07-27T03:18:57.495 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 24 root rtd unknown /proc/24/root (readlink: Permission denied) 2024-07-27T03:18:57.495 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 24 root txt unknown /proc/24/exe (readlink: Permission denied) 2024-07-27T03:18:57.495 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 24 root NOFD /proc/24/fd (opendir: Permission denied) 2024-07-27T03:18:57.495 INFO:tasks.workunit.client.0.smithi067.stdout:migration 25 root cwd unknown /proc/25/cwd (readlink: Permission denied) 2024-07-27T03:18:57.496 INFO:tasks.workunit.client.0.smithi067.stdout:migration 25 root rtd unknown /proc/25/root (readlink: Permission denied) 2024-07-27T03:18:57.496 INFO:tasks.workunit.client.0.smithi067.stdout:migration 25 root txt unknown /proc/25/exe (readlink: Permission denied) 2024-07-27T03:18:57.496 INFO:tasks.workunit.client.0.smithi067.stdout:migration 25 root NOFD /proc/25/fd (opendir: Permission denied) 2024-07-27T03:18:57.496 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 26 root cwd unknown /proc/26/cwd (readlink: Permission denied) 2024-07-27T03:18:57.496 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 26 root rtd unknown /proc/26/root (readlink: Permission denied) 2024-07-27T03:18:57.496 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 26 root txt unknown /proc/26/exe (readlink: Permission denied) 2024-07-27T03:18:57.496 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 26 root NOFD /proc/26/fd (opendir: Permission denied) 2024-07-27T03:18:57.496 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 28 root cwd unknown /proc/28/cwd (readlink: Permission denied) 2024-07-27T03:18:57.496 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 28 root rtd unknown /proc/28/root (readlink: Permission denied) 2024-07-27T03:18:57.496 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 28 root txt unknown /proc/28/exe (readlink: Permission denied) 2024-07-27T03:18:57.496 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 28 root NOFD /proc/28/fd (opendir: Permission denied) 2024-07-27T03:18:57.496 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/2 29 root cwd unknown /proc/29/cwd (readlink: Permission denied) 2024-07-27T03:18:57.496 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/2 29 root rtd unknown /proc/29/root (readlink: Permission denied) 2024-07-27T03:18:57.496 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/2 29 root txt unknown /proc/29/exe (readlink: Permission denied) 2024-07-27T03:18:57.497 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/2 29 root NOFD /proc/29/fd (opendir: Permission denied) 2024-07-27T03:18:57.497 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 30 root cwd unknown /proc/30/cwd (readlink: Permission denied) 2024-07-27T03:18:57.497 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 30 root rtd unknown /proc/30/root (readlink: Permission denied) 2024-07-27T03:18:57.497 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 30 root txt unknown /proc/30/exe (readlink: Permission denied) 2024-07-27T03:18:57.497 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 30 root NOFD /proc/30/fd (opendir: Permission denied) 2024-07-27T03:18:57.497 INFO:tasks.workunit.client.0.smithi067.stdout:migration 31 root cwd unknown /proc/31/cwd (readlink: Permission denied) 2024-07-27T03:18:57.497 INFO:tasks.workunit.client.0.smithi067.stdout:migration 31 root rtd unknown /proc/31/root (readlink: Permission denied) 2024-07-27T03:18:57.497 INFO:tasks.workunit.client.0.smithi067.stdout:migration 31 root txt unknown /proc/31/exe (readlink: Permission denied) 2024-07-27T03:18:57.497 INFO:tasks.workunit.client.0.smithi067.stdout:migration 31 root NOFD /proc/31/fd (opendir: Permission denied) 2024-07-27T03:18:57.498 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 32 root cwd unknown /proc/32/cwd (readlink: Permission denied) 2024-07-27T03:18:57.498 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 32 root rtd unknown /proc/32/root (readlink: Permission denied) 2024-07-27T03:18:57.498 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 32 root txt unknown /proc/32/exe (readlink: Permission denied) 2024-07-27T03:18:57.498 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 32 root NOFD /proc/32/fd (opendir: Permission denied) 2024-07-27T03:18:57.498 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/2 34 root cwd unknown /proc/34/cwd (readlink: Permission denied) 2024-07-27T03:18:57.498 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/2 34 root rtd unknown /proc/34/root (readlink: Permission denied) 2024-07-27T03:18:57.498 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/2 34 root txt unknown /proc/34/exe (readlink: Permission denied) 2024-07-27T03:18:57.498 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/2 34 root NOFD /proc/34/fd (opendir: Permission denied) 2024-07-27T03:18:57.498 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/3 35 root cwd unknown /proc/35/cwd (readlink: Permission denied) 2024-07-27T03:18:57.498 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/3 35 root rtd unknown /proc/35/root (readlink: Permission denied) 2024-07-27T03:18:57.498 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/3 35 root txt unknown /proc/35/exe (readlink: Permission denied) 2024-07-27T03:18:57.498 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/3 35 root NOFD /proc/35/fd (opendir: Permission denied) 2024-07-27T03:18:57.498 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 36 root cwd unknown /proc/36/cwd (readlink: Permission denied) 2024-07-27T03:18:57.499 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 36 root rtd unknown /proc/36/root (readlink: Permission denied) 2024-07-27T03:18:57.499 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 36 root txt unknown /proc/36/exe (readlink: Permission denied) 2024-07-27T03:18:57.499 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 36 root NOFD /proc/36/fd (opendir: Permission denied) 2024-07-27T03:18:57.499 INFO:tasks.workunit.client.0.smithi067.stdout:migration 37 root cwd unknown /proc/37/cwd (readlink: Permission denied) 2024-07-27T03:18:57.499 INFO:tasks.workunit.client.0.smithi067.stdout:migration 37 root rtd unknown /proc/37/root (readlink: Permission denied) 2024-07-27T03:18:57.499 INFO:tasks.workunit.client.0.smithi067.stdout:migration 37 root txt unknown /proc/37/exe (readlink: Permission denied) 2024-07-27T03:18:57.499 INFO:tasks.workunit.client.0.smithi067.stdout:migration 37 root NOFD /proc/37/fd (opendir: Permission denied) 2024-07-27T03:18:57.499 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 38 root cwd unknown /proc/38/cwd (readlink: Permission denied) 2024-07-27T03:18:57.499 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 38 root rtd unknown /proc/38/root (readlink: Permission denied) 2024-07-27T03:18:57.499 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 38 root txt unknown /proc/38/exe (readlink: Permission denied) 2024-07-27T03:18:57.499 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 38 root NOFD /proc/38/fd (opendir: Permission denied) 2024-07-27T03:18:57.499 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 40 root cwd unknown /proc/40/cwd (readlink: Permission denied) 2024-07-27T03:18:57.499 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 40 root rtd unknown /proc/40/root (readlink: Permission denied) 2024-07-27T03:18:57.500 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 40 root txt unknown /proc/40/exe (readlink: Permission denied) 2024-07-27T03:18:57.500 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 40 root NOFD /proc/40/fd (opendir: Permission denied) 2024-07-27T03:18:57.500 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/4 41 root cwd unknown /proc/41/cwd (readlink: Permission denied) 2024-07-27T03:18:57.500 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/4 41 root rtd unknown /proc/41/root (readlink: Permission denied) 2024-07-27T03:18:57.500 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/4 41 root txt unknown /proc/41/exe (readlink: Permission denied) 2024-07-27T03:18:57.500 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/4 41 root NOFD /proc/41/fd (opendir: Permission denied) 2024-07-27T03:18:57.500 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 42 root cwd unknown /proc/42/cwd (readlink: Permission denied) 2024-07-27T03:18:57.500 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 42 root rtd unknown /proc/42/root (readlink: Permission denied) 2024-07-27T03:18:57.500 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 42 root txt unknown /proc/42/exe (readlink: Permission denied) 2024-07-27T03:18:57.500 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 42 root NOFD /proc/42/fd (opendir: Permission denied) 2024-07-27T03:18:57.500 INFO:tasks.workunit.client.0.smithi067.stdout:migration 43 root cwd unknown /proc/43/cwd (readlink: Permission denied) 2024-07-27T03:18:57.500 INFO:tasks.workunit.client.0.smithi067.stdout:migration 43 root rtd unknown /proc/43/root (readlink: Permission denied) 2024-07-27T03:18:57.500 INFO:tasks.workunit.client.0.smithi067.stdout:migration 43 root txt unknown /proc/43/exe (readlink: Permission denied) 2024-07-27T03:18:57.501 INFO:tasks.workunit.client.0.smithi067.stdout:migration 43 root NOFD /proc/43/fd (opendir: Permission denied) 2024-07-27T03:18:57.501 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 44 root cwd unknown /proc/44/cwd (readlink: Permission denied) 2024-07-27T03:18:57.501 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 44 root rtd unknown /proc/44/root (readlink: Permission denied) 2024-07-27T03:18:57.501 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 44 root txt unknown /proc/44/exe (readlink: Permission denied) 2024-07-27T03:18:57.501 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 44 root NOFD /proc/44/fd (opendir: Permission denied) 2024-07-27T03:18:57.501 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/4 46 root cwd unknown /proc/46/cwd (readlink: Permission denied) 2024-07-27T03:18:57.501 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/4 46 root rtd unknown /proc/46/root (readlink: Permission denied) 2024-07-27T03:18:57.501 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/4 46 root txt unknown /proc/46/exe (readlink: Permission denied) 2024-07-27T03:18:57.501 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/4 46 root NOFD /proc/46/fd (opendir: Permission denied) 2024-07-27T03:18:57.501 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/5 47 root cwd unknown /proc/47/cwd (readlink: Permission denied) 2024-07-27T03:18:57.501 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/5 47 root rtd unknown /proc/47/root (readlink: Permission denied) 2024-07-27T03:18:57.501 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/5 47 root txt unknown /proc/47/exe (readlink: Permission denied) 2024-07-27T03:18:57.501 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/5 47 root NOFD /proc/47/fd (opendir: Permission denied) 2024-07-27T03:18:57.501 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 48 root cwd unknown /proc/48/cwd (readlink: Permission denied) 2024-07-27T03:18:57.502 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 48 root rtd unknown /proc/48/root (readlink: Permission denied) 2024-07-27T03:18:57.502 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 48 root txt unknown /proc/48/exe (readlink: Permission denied) 2024-07-27T03:18:57.502 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 48 root NOFD /proc/48/fd (opendir: Permission denied) 2024-07-27T03:18:57.502 INFO:tasks.workunit.client.0.smithi067.stdout:migration 49 root cwd unknown /proc/49/cwd (readlink: Permission denied) 2024-07-27T03:18:57.502 INFO:tasks.workunit.client.0.smithi067.stdout:migration 49 root rtd unknown /proc/49/root (readlink: Permission denied) 2024-07-27T03:18:57.502 INFO:tasks.workunit.client.0.smithi067.stdout:migration 49 root txt unknown /proc/49/exe (readlink: Permission denied) 2024-07-27T03:18:57.502 INFO:tasks.workunit.client.0.smithi067.stdout:migration 49 root NOFD /proc/49/fd (opendir: Permission denied) 2024-07-27T03:18:57.502 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 50 root cwd unknown /proc/50/cwd (readlink: Permission denied) 2024-07-27T03:18:57.502 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 50 root rtd unknown /proc/50/root (readlink: Permission denied) 2024-07-27T03:18:57.502 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 50 root txt unknown /proc/50/exe (readlink: Permission denied) 2024-07-27T03:18:57.502 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 50 root NOFD /proc/50/fd (opendir: Permission denied) 2024-07-27T03:18:57.502 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 52 root cwd unknown /proc/52/cwd (readlink: Permission denied) 2024-07-27T03:18:57.502 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 52 root rtd unknown /proc/52/root (readlink: Permission denied) 2024-07-27T03:18:57.503 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 52 root txt unknown /proc/52/exe (readlink: Permission denied) 2024-07-27T03:18:57.503 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 52 root NOFD /proc/52/fd (opendir: Permission denied) 2024-07-27T03:18:57.503 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/6 53 root cwd unknown /proc/53/cwd (readlink: Permission denied) 2024-07-27T03:18:57.503 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/6 53 root rtd unknown /proc/53/root (readlink: Permission denied) 2024-07-27T03:18:57.503 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/6 53 root txt unknown /proc/53/exe (readlink: Permission denied) 2024-07-27T03:18:57.503 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/6 53 root NOFD /proc/53/fd (opendir: Permission denied) 2024-07-27T03:18:57.503 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 54 root cwd unknown /proc/54/cwd (readlink: Permission denied) 2024-07-27T03:18:57.503 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 54 root rtd unknown /proc/54/root (readlink: Permission denied) 2024-07-27T03:18:57.503 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 54 root txt unknown /proc/54/exe (readlink: Permission denied) 2024-07-27T03:18:57.503 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 54 root NOFD /proc/54/fd (opendir: Permission denied) 2024-07-27T03:18:57.503 INFO:tasks.workunit.client.0.smithi067.stdout:migration 55 root cwd unknown /proc/55/cwd (readlink: Permission denied) 2024-07-27T03:18:57.503 INFO:tasks.workunit.client.0.smithi067.stdout:migration 55 root rtd unknown /proc/55/root (readlink: Permission denied) 2024-07-27T03:18:57.503 INFO:tasks.workunit.client.0.smithi067.stdout:migration 55 root txt unknown /proc/55/exe (readlink: Permission denied) 2024-07-27T03:18:57.503 INFO:tasks.workunit.client.0.smithi067.stdout:migration 55 root NOFD /proc/55/fd (opendir: Permission denied) 2024-07-27T03:18:57.504 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 56 root cwd unknown /proc/56/cwd (readlink: Permission denied) 2024-07-27T03:18:57.504 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 56 root rtd unknown /proc/56/root (readlink: Permission denied) 2024-07-27T03:18:57.504 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 56 root txt unknown /proc/56/exe (readlink: Permission denied) 2024-07-27T03:18:57.504 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 56 root NOFD /proc/56/fd (opendir: Permission denied) 2024-07-27T03:18:57.504 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 57 root cwd unknown /proc/57/cwd (readlink: Permission denied) 2024-07-27T03:18:57.504 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 57 root rtd unknown /proc/57/root (readlink: Permission denied) 2024-07-27T03:18:57.504 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 57 root txt unknown /proc/57/exe (readlink: Permission denied) 2024-07-27T03:18:57.504 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 57 root NOFD /proc/57/fd (opendir: Permission denied) 2024-07-27T03:18:57.504 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 58 root cwd unknown /proc/58/cwd (readlink: Permission denied) 2024-07-27T03:18:57.504 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 58 root rtd unknown /proc/58/root (readlink: Permission denied) 2024-07-27T03:18:57.504 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 58 root txt unknown /proc/58/exe (readlink: Permission denied) 2024-07-27T03:18:57.504 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 58 root NOFD /proc/58/fd (opendir: Permission denied) 2024-07-27T03:18:57.504 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/7 59 root cwd unknown /proc/59/cwd (readlink: Permission denied) 2024-07-27T03:18:57.504 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/7 59 root rtd unknown /proc/59/root (readlink: Permission denied) 2024-07-27T03:18:57.505 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/7 59 root txt unknown /proc/59/exe (readlink: Permission denied) 2024-07-27T03:18:57.505 INFO:tasks.workunit.client.0.smithi067.stdout:cpuhp/7 59 root NOFD /proc/59/fd (opendir: Permission denied) 2024-07-27T03:18:57.505 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 60 root cwd unknown /proc/60/cwd (readlink: Permission denied) 2024-07-27T03:18:57.505 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 60 root rtd unknown /proc/60/root (readlink: Permission denied) 2024-07-27T03:18:57.505 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 60 root txt unknown /proc/60/exe (readlink: Permission denied) 2024-07-27T03:18:57.505 INFO:tasks.workunit.client.0.smithi067.stdout:idle_inje 60 root NOFD /proc/60/fd (opendir: Permission denied) 2024-07-27T03:18:57.505 INFO:tasks.workunit.client.0.smithi067.stdout:migration 61 root cwd unknown /proc/61/cwd (readlink: Permission denied) 2024-07-27T03:18:57.505 INFO:tasks.workunit.client.0.smithi067.stdout:migration 61 root rtd unknown /proc/61/root (readlink: Permission denied) 2024-07-27T03:18:57.505 INFO:tasks.workunit.client.0.smithi067.stdout:migration 61 root txt unknown /proc/61/exe (readlink: Permission denied) 2024-07-27T03:18:57.505 INFO:tasks.workunit.client.0.smithi067.stdout:migration 61 root NOFD /proc/61/fd (opendir: Permission denied) 2024-07-27T03:18:57.505 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 62 root cwd unknown /proc/62/cwd (readlink: Permission denied) 2024-07-27T03:18:57.505 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 62 root rtd unknown /proc/62/root (readlink: Permission denied) 2024-07-27T03:18:57.505 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 62 root txt unknown /proc/62/exe (readlink: Permission denied) 2024-07-27T03:18:57.506 INFO:tasks.workunit.client.0.smithi067.stdout:ksoftirqd 62 root NOFD /proc/62/fd (opendir: Permission denied) 2024-07-27T03:18:57.506 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 64 root cwd unknown /proc/64/cwd (readlink: Permission denied) 2024-07-27T03:18:57.506 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 64 root rtd unknown /proc/64/root (readlink: Permission denied) 2024-07-27T03:18:57.506 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 64 root txt unknown /proc/64/exe (readlink: Permission denied) 2024-07-27T03:18:57.506 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 64 root NOFD /proc/64/fd (opendir: Permission denied) 2024-07-27T03:18:57.506 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 69 root cwd unknown /proc/69/cwd (readlink: Permission denied) 2024-07-27T03:18:57.506 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 69 root rtd unknown /proc/69/root (readlink: Permission denied) 2024-07-27T03:18:57.506 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 69 root txt unknown /proc/69/exe (readlink: Permission denied) 2024-07-27T03:18:57.506 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 69 root NOFD /proc/69/fd (opendir: Permission denied) 2024-07-27T03:18:57.506 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 71 root cwd unknown /proc/71/cwd (readlink: Permission denied) 2024-07-27T03:18:57.506 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 71 root rtd unknown /proc/71/root (readlink: Permission denied) 2024-07-27T03:18:57.506 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 71 root txt unknown /proc/71/exe (readlink: Permission denied) 2024-07-27T03:18:57.506 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 71 root NOFD /proc/71/fd (opendir: Permission denied) 2024-07-27T03:18:57.507 INFO:tasks.workunit.client.0.smithi067.stdout:kdevtmpfs 72 root cwd unknown /proc/72/cwd (readlink: Permission denied) 2024-07-27T03:18:57.507 INFO:tasks.workunit.client.0.smithi067.stdout:kdevtmpfs 72 root rtd unknown /proc/72/root (readlink: Permission denied) 2024-07-27T03:18:57.507 INFO:tasks.workunit.client.0.smithi067.stdout:kdevtmpfs 72 root txt unknown /proc/72/exe (readlink: Permission denied) 2024-07-27T03:18:57.507 INFO:tasks.workunit.client.0.smithi067.stdout:kdevtmpfs 72 root NOFD /proc/72/fd (opendir: Permission denied) 2024-07-27T03:18:57.507 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 73 root cwd unknown /proc/73/cwd (readlink: Permission denied) 2024-07-27T03:18:57.507 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 73 root rtd unknown /proc/73/root (readlink: Permission denied) 2024-07-27T03:18:57.507 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 73 root txt unknown /proc/73/exe (readlink: Permission denied) 2024-07-27T03:18:57.507 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 73 root NOFD /proc/73/fd (opendir: Permission denied) 2024-07-27T03:18:57.507 INFO:tasks.workunit.client.0.smithi067.stdout:kauditd 74 root cwd unknown /proc/74/cwd (readlink: Permission denied) 2024-07-27T03:18:57.507 INFO:tasks.workunit.client.0.smithi067.stdout:kauditd 74 root rtd unknown /proc/74/root (readlink: Permission denied) 2024-07-27T03:18:57.507 INFO:tasks.workunit.client.0.smithi067.stdout:kauditd 74 root txt unknown /proc/74/exe (readlink: Permission denied) 2024-07-27T03:18:57.507 INFO:tasks.workunit.client.0.smithi067.stdout:kauditd 74 root NOFD /proc/74/fd (opendir: Permission denied) 2024-07-27T03:18:57.507 INFO:tasks.workunit.client.0.smithi067.stdout:khungtask 76 root cwd unknown /proc/76/cwd (readlink: Permission denied) 2024-07-27T03:18:57.507 INFO:tasks.workunit.client.0.smithi067.stdout:khungtask 76 root rtd unknown /proc/76/root (readlink: Permission denied) 2024-07-27T03:18:57.508 INFO:tasks.workunit.client.0.smithi067.stdout:khungtask 76 root txt unknown /proc/76/exe (readlink: Permission denied) 2024-07-27T03:18:57.508 INFO:tasks.workunit.client.0.smithi067.stdout:khungtask 76 root NOFD /proc/76/fd (opendir: Permission denied) 2024-07-27T03:18:57.508 INFO:tasks.workunit.client.0.smithi067.stdout:oom_reape 77 root cwd unknown /proc/77/cwd (readlink: Permission denied) 2024-07-27T03:18:57.508 INFO:tasks.workunit.client.0.smithi067.stdout:oom_reape 77 root rtd unknown /proc/77/root (readlink: Permission denied) 2024-07-27T03:18:57.508 INFO:tasks.workunit.client.0.smithi067.stdout:oom_reape 77 root txt unknown /proc/77/exe (readlink: Permission denied) 2024-07-27T03:18:57.508 INFO:tasks.workunit.client.0.smithi067.stdout:oom_reape 77 root NOFD /proc/77/fd (opendir: Permission denied) 2024-07-27T03:18:57.508 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 78 root cwd unknown /proc/78/cwd (readlink: Permission denied) 2024-07-27T03:18:57.508 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 78 root rtd unknown /proc/78/root (readlink: Permission denied) 2024-07-27T03:18:57.508 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 78 root txt unknown /proc/78/exe (readlink: Permission denied) 2024-07-27T03:18:57.508 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 78 root NOFD /proc/78/fd (opendir: Permission denied) 2024-07-27T03:18:57.508 INFO:tasks.workunit.client.0.smithi067.stdout:kcompactd 79 root cwd unknown /proc/79/cwd (readlink: Permission denied) 2024-07-27T03:18:57.508 INFO:tasks.workunit.client.0.smithi067.stdout:kcompactd 79 root rtd unknown /proc/79/root (readlink: Permission denied) 2024-07-27T03:18:57.508 INFO:tasks.workunit.client.0.smithi067.stdout:kcompactd 79 root txt unknown /proc/79/exe (readlink: Permission denied) 2024-07-27T03:18:57.508 INFO:tasks.workunit.client.0.smithi067.stdout:kcompactd 79 root NOFD /proc/79/fd (opendir: Permission denied) 2024-07-27T03:18:57.509 INFO:tasks.workunit.client.0.smithi067.stdout:ksmd 80 root cwd unknown /proc/80/cwd (readlink: Permission denied) 2024-07-27T03:18:57.509 INFO:tasks.workunit.client.0.smithi067.stdout:ksmd 80 root rtd unknown /proc/80/root (readlink: Permission denied) 2024-07-27T03:18:57.509 INFO:tasks.workunit.client.0.smithi067.stdout:ksmd 80 root txt unknown /proc/80/exe (readlink: Permission denied) 2024-07-27T03:18:57.509 INFO:tasks.workunit.client.0.smithi067.stdout:ksmd 80 root NOFD /proc/80/fd (opendir: Permission denied) 2024-07-27T03:18:57.509 INFO:tasks.workunit.client.0.smithi067.stdout:khugepage 82 root cwd unknown /proc/82/cwd (readlink: Permission denied) 2024-07-27T03:18:57.509 INFO:tasks.workunit.client.0.smithi067.stdout:khugepage 82 root rtd unknown /proc/82/root (readlink: Permission denied) 2024-07-27T03:18:57.509 INFO:tasks.workunit.client.0.smithi067.stdout:khugepage 82 root txt unknown /proc/82/exe (readlink: Permission denied) 2024-07-27T03:18:57.509 INFO:tasks.workunit.client.0.smithi067.stdout:khugepage 82 root NOFD /proc/82/fd (opendir: Permission denied) 2024-07-27T03:18:57.509 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 83 root cwd unknown /proc/83/cwd (readlink: Permission denied) 2024-07-27T03:18:57.509 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 83 root rtd unknown /proc/83/root (readlink: Permission denied) 2024-07-27T03:18:57.509 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 83 root txt unknown /proc/83/exe (readlink: Permission denied) 2024-07-27T03:18:57.509 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 83 root NOFD /proc/83/fd (opendir: Permission denied) 2024-07-27T03:18:57.509 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 84 root cwd unknown /proc/84/cwd (readlink: Permission denied) 2024-07-27T03:18:57.509 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 84 root rtd unknown /proc/84/root (readlink: Permission denied) 2024-07-27T03:18:57.510 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 84 root txt unknown /proc/84/exe (readlink: Permission denied) 2024-07-27T03:18:57.510 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 84 root NOFD /proc/84/fd (opendir: Permission denied) 2024-07-27T03:18:57.510 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 85 root cwd unknown /proc/85/cwd (readlink: Permission denied) 2024-07-27T03:18:57.510 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 85 root rtd unknown /proc/85/root (readlink: Permission denied) 2024-07-27T03:18:57.510 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 85 root txt unknown /proc/85/exe (readlink: Permission denied) 2024-07-27T03:18:57.510 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 85 root NOFD /proc/85/fd (opendir: Permission denied) 2024-07-27T03:18:57.510 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 86 root cwd unknown /proc/86/cwd (readlink: Permission denied) 2024-07-27T03:18:57.510 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 86 root rtd unknown /proc/86/root (readlink: Permission denied) 2024-07-27T03:18:57.510 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 86 root txt unknown /proc/86/exe (readlink: Permission denied) 2024-07-27T03:18:57.510 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 86 root NOFD /proc/86/fd (opendir: Permission denied) 2024-07-27T03:18:57.510 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 87 root cwd unknown /proc/87/cwd (readlink: Permission denied) 2024-07-27T03:18:57.510 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 87 root rtd unknown /proc/87/root (readlink: Permission denied) 2024-07-27T03:18:57.510 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 87 root txt unknown /proc/87/exe (readlink: Permission denied) 2024-07-27T03:18:57.511 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 87 root NOFD /proc/87/fd (opendir: Permission denied) 2024-07-27T03:18:57.511 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 88 root cwd unknown /proc/88/cwd (readlink: Permission denied) 2024-07-27T03:18:57.511 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 88 root rtd unknown /proc/88/root (readlink: Permission denied) 2024-07-27T03:18:57.511 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 88 root txt unknown /proc/88/exe (readlink: Permission denied) 2024-07-27T03:18:57.511 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 88 root NOFD /proc/88/fd (opendir: Permission denied) 2024-07-27T03:18:57.511 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 89 root cwd unknown /proc/89/cwd (readlink: Permission denied) 2024-07-27T03:18:57.511 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 89 root rtd unknown /proc/89/root (readlink: Permission denied) 2024-07-27T03:18:57.511 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 89 root txt unknown /proc/89/exe (readlink: Permission denied) 2024-07-27T03:18:57.511 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 89 root NOFD /proc/89/fd (opendir: Permission denied) 2024-07-27T03:18:57.511 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 90 root cwd unknown /proc/90/cwd (readlink: Permission denied) 2024-07-27T03:18:57.511 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 90 root rtd unknown /proc/90/root (readlink: Permission denied) 2024-07-27T03:18:57.511 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 90 root txt unknown /proc/90/exe (readlink: Permission denied) 2024-07-27T03:18:57.511 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 90 root NOFD /proc/90/fd (opendir: Permission denied) 2024-07-27T03:18:57.512 INFO:tasks.workunit.client.0.smithi067.stdout:watchdogd 91 root cwd unknown /proc/91/cwd (readlink: Permission denied) 2024-07-27T03:18:57.512 INFO:tasks.workunit.client.0.smithi067.stdout:watchdogd 91 root rtd unknown /proc/91/root (readlink: Permission denied) 2024-07-27T03:18:57.512 INFO:tasks.workunit.client.0.smithi067.stdout:watchdogd 91 root txt unknown /proc/91/exe (readlink: Permission denied) 2024-07-27T03:18:57.512 INFO:tasks.workunit.client.0.smithi067.stdout:watchdogd 91 root NOFD /proc/91/fd (opendir: Permission denied) 2024-07-27T03:18:57.512 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 92 root cwd unknown /proc/92/cwd (readlink: Permission denied) 2024-07-27T03:18:57.512 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 92 root rtd unknown /proc/92/root (readlink: Permission denied) 2024-07-27T03:18:57.512 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 92 root txt unknown /proc/92/exe (readlink: Permission denied) 2024-07-27T03:18:57.512 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 92 root NOFD /proc/92/fd (opendir: Permission denied) 2024-07-27T03:18:57.512 INFO:tasks.workunit.client.0.smithi067.stdout:kswapd0 94 root cwd unknown /proc/94/cwd (readlink: Permission denied) 2024-07-27T03:18:57.512 INFO:tasks.workunit.client.0.smithi067.stdout:kswapd0 94 root rtd unknown /proc/94/root (readlink: Permission denied) 2024-07-27T03:18:57.512 INFO:tasks.workunit.client.0.smithi067.stdout:kswapd0 94 root txt unknown /proc/94/exe (readlink: Permission denied) 2024-07-27T03:18:57.512 INFO:tasks.workunit.client.0.smithi067.stdout:kswapd0 94 root NOFD /proc/94/fd (opendir: Permission denied) 2024-07-27T03:18:57.512 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 101 root cwd unknown /proc/101/cwd (readlink: Permission denied) 2024-07-27T03:18:57.512 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 101 root rtd unknown /proc/101/root (readlink: Permission denied) 2024-07-27T03:18:57.513 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 101 root txt unknown /proc/101/exe (readlink: Permission denied) 2024-07-27T03:18:57.513 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 101 root NOFD /proc/101/fd (opendir: Permission denied) 2024-07-27T03:18:57.513 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 110 root cwd unknown /proc/110/cwd (readlink: Permission denied) 2024-07-27T03:18:57.513 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 110 root rtd unknown /proc/110/root (readlink: Permission denied) 2024-07-27T03:18:57.513 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 110 root txt unknown /proc/110/exe (readlink: Permission denied) 2024-07-27T03:18:57.513 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 110 root NOFD /proc/110/fd (opendir: Permission denied) 2024-07-27T03:18:57.513 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 111 root cwd unknown /proc/111/cwd (readlink: Permission denied) 2024-07-27T03:18:57.513 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 111 root rtd unknown /proc/111/root (readlink: Permission denied) 2024-07-27T03:18:57.513 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 111 root txt unknown /proc/111/exe (readlink: Permission denied) 2024-07-27T03:18:57.513 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 111 root NOFD /proc/111/fd (opendir: Permission denied) 2024-07-27T03:18:57.513 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 112 root cwd unknown /proc/112/cwd (readlink: Permission denied) 2024-07-27T03:18:57.513 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 112 root rtd unknown /proc/112/root (readlink: Permission denied) 2024-07-27T03:18:57.513 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 112 root txt unknown /proc/112/exe (readlink: Permission denied) 2024-07-27T03:18:57.513 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 112 root NOFD /proc/112/fd (opendir: Permission denied) 2024-07-27T03:18:57.513 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 115 root cwd unknown /proc/115/cwd (readlink: Permission denied) 2024-07-27T03:18:57.514 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 115 root rtd unknown /proc/115/root (readlink: Permission denied) 2024-07-27T03:18:57.514 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 115 root txt unknown /proc/115/exe (readlink: Permission denied) 2024-07-27T03:18:57.514 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 115 root NOFD /proc/115/fd (opendir: Permission denied) 2024-07-27T03:18:57.514 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/4 116 root cwd unknown /proc/116/cwd (readlink: Permission denied) 2024-07-27T03:18:57.514 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/4 116 root rtd unknown /proc/116/root (readlink: Permission denied) 2024-07-27T03:18:57.514 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/4 116 root txt unknown /proc/116/exe (readlink: Permission denied) 2024-07-27T03:18:57.514 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/4 116 root NOFD /proc/116/fd (opendir: Permission denied) 2024-07-27T03:18:57.514 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 117 root cwd unknown /proc/117/cwd (readlink: Permission denied) 2024-07-27T03:18:57.514 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 117 root rtd unknown /proc/117/root (readlink: Permission denied) 2024-07-27T03:18:57.514 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 117 root txt unknown /proc/117/exe (readlink: Permission denied) 2024-07-27T03:18:57.514 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 117 root NOFD /proc/117/fd (opendir: Permission denied) 2024-07-27T03:18:57.514 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 127 root cwd unknown /proc/127/cwd (readlink: Permission denied) 2024-07-27T03:18:57.514 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 127 root rtd unknown /proc/127/root (readlink: Permission denied) 2024-07-27T03:18:57.515 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 127 root txt unknown /proc/127/exe (readlink: Permission denied) 2024-07-27T03:18:57.515 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 127 root NOFD /proc/127/fd (opendir: Permission denied) 2024-07-27T03:18:57.515 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 238 root cwd unknown /proc/238/cwd (readlink: Permission denied) 2024-07-27T03:18:57.515 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 238 root rtd unknown /proc/238/root (readlink: Permission denied) 2024-07-27T03:18:57.515 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 238 root txt unknown /proc/238/exe (readlink: Permission denied) 2024-07-27T03:18:57.515 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 238 root NOFD /proc/238/fd (opendir: Permission denied) 2024-07-27T03:18:57.515 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/2 239 root cwd unknown /proc/239/cwd (readlink: Permission denied) 2024-07-27T03:18:57.515 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/2 239 root rtd unknown /proc/239/root (readlink: Permission denied) 2024-07-27T03:18:57.515 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/2 239 root txt unknown /proc/239/exe (readlink: Permission denied) 2024-07-27T03:18:57.515 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/2 239 root NOFD /proc/239/fd (opendir: Permission denied) 2024-07-27T03:18:57.515 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 241 root cwd unknown /proc/241/cwd (readlink: Permission denied) 2024-07-27T03:18:57.515 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 241 root rtd unknown /proc/241/root (readlink: Permission denied) 2024-07-27T03:18:57.515 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 241 root txt unknown /proc/241/exe (readlink: Permission denied) 2024-07-27T03:18:57.516 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 241 root NOFD /proc/241/fd (opendir: Permission denied) 2024-07-27T03:18:57.516 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 280 root cwd unknown /proc/280/cwd (readlink: Permission denied) 2024-07-27T03:18:57.516 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 280 root rtd unknown /proc/280/root (readlink: Permission denied) 2024-07-27T03:18:57.516 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 280 root txt unknown /proc/280/exe (readlink: Permission denied) 2024-07-27T03:18:57.516 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 280 root NOFD /proc/280/fd (opendir: Permission denied) 2024-07-27T03:18:57.516 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 284 root cwd unknown /proc/284/cwd (readlink: Permission denied) 2024-07-27T03:18:57.516 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 284 root rtd unknown /proc/284/root (readlink: Permission denied) 2024-07-27T03:18:57.516 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 284 root txt unknown /proc/284/exe (readlink: Permission denied) 2024-07-27T03:18:57.516 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 284 root NOFD /proc/284/fd (opendir: Permission denied) 2024-07-27T03:18:57.516 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 294 root cwd unknown /proc/294/cwd (readlink: Permission denied) 2024-07-27T03:18:57.516 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 294 root rtd unknown /proc/294/root (readlink: Permission denied) 2024-07-27T03:18:57.516 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 294 root txt unknown /proc/294/exe (readlink: Permission denied) 2024-07-27T03:18:57.516 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 294 root NOFD /proc/294/fd (opendir: Permission denied) 2024-07-27T03:18:57.516 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 296 root cwd unknown /proc/296/cwd (readlink: Permission denied) 2024-07-27T03:18:57.517 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 296 root rtd unknown /proc/296/root (readlink: Permission denied) 2024-07-27T03:18:57.517 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 296 root txt unknown /proc/296/exe (readlink: Permission denied) 2024-07-27T03:18:57.517 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 296 root NOFD /proc/296/fd (opendir: Permission denied) 2024-07-27T03:18:57.517 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 428 root cwd unknown /proc/428/cwd (readlink: Permission denied) 2024-07-27T03:18:57.517 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 428 root rtd unknown /proc/428/root (readlink: Permission denied) 2024-07-27T03:18:57.517 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 428 root txt unknown /proc/428/exe (readlink: Permission denied) 2024-07-27T03:18:57.517 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 428 root NOFD /proc/428/fd (opendir: Permission denied) 2024-07-27T03:18:57.517 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 429 root cwd unknown /proc/429/cwd (readlink: Permission denied) 2024-07-27T03:18:57.517 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 429 root rtd unknown /proc/429/root (readlink: Permission denied) 2024-07-27T03:18:57.517 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 429 root txt unknown /proc/429/exe (readlink: Permission denied) 2024-07-27T03:18:57.517 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 429 root NOFD /proc/429/fd (opendir: Permission denied) 2024-07-27T03:18:57.517 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 430 root cwd unknown /proc/430/cwd (readlink: Permission denied) 2024-07-27T03:18:57.517 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 430 root rtd unknown /proc/430/root (readlink: Permission denied) 2024-07-27T03:18:57.517 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 430 root txt unknown /proc/430/exe (readlink: Permission denied) 2024-07-27T03:18:57.517 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 430 root NOFD /proc/430/fd (opendir: Permission denied) 2024-07-27T03:18:57.518 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 431 root cwd unknown /proc/431/cwd (readlink: Permission denied) 2024-07-27T03:18:57.518 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 431 root rtd unknown /proc/431/root (readlink: Permission denied) 2024-07-27T03:18:57.518 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 431 root txt unknown /proc/431/exe (readlink: Permission denied) 2024-07-27T03:18:57.518 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 431 root NOFD /proc/431/fd (opendir: Permission denied) 2024-07-27T03:18:57.518 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 433 root cwd unknown /proc/433/cwd (readlink: Permission denied) 2024-07-27T03:18:57.518 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 433 root rtd unknown /proc/433/root (readlink: Permission denied) 2024-07-27T03:18:57.518 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 433 root txt unknown /proc/433/exe (readlink: Permission denied) 2024-07-27T03:18:57.518 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 433 root NOFD /proc/433/fd (opendir: Permission denied) 2024-07-27T03:18:57.518 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 438 root cwd unknown /proc/438/cwd (readlink: Permission denied) 2024-07-27T03:18:57.518 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 438 root rtd unknown /proc/438/root (readlink: Permission denied) 2024-07-27T03:18:57.518 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 438 root txt unknown /proc/438/exe (readlink: Permission denied) 2024-07-27T03:18:57.518 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 438 root NOFD /proc/438/fd (opendir: Permission denied) 2024-07-27T03:18:57.519 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_0 447 root cwd unknown /proc/447/cwd (readlink: Permission denied) 2024-07-27T03:18:57.519 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_0 447 root rtd unknown /proc/447/root (readlink: Permission denied) 2024-07-27T03:18:57.519 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_0 447 root txt unknown /proc/447/exe (readlink: Permission denied) 2024-07-27T03:18:57.519 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_0 447 root NOFD /proc/447/fd (opendir: Permission denied) 2024-07-27T03:18:57.519 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 448 root cwd unknown /proc/448/cwd (readlink: Permission denied) 2024-07-27T03:18:57.519 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 448 root rtd unknown /proc/448/root (readlink: Permission denied) 2024-07-27T03:18:57.519 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 448 root txt unknown /proc/448/exe (readlink: Permission denied) 2024-07-27T03:18:57.519 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 448 root NOFD /proc/448/fd (opendir: Permission denied) 2024-07-27T03:18:57.519 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_1 449 root cwd unknown /proc/449/cwd (readlink: Permission denied) 2024-07-27T03:18:57.519 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_1 449 root rtd unknown /proc/449/root (readlink: Permission denied) 2024-07-27T03:18:57.519 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_1 449 root txt unknown /proc/449/exe (readlink: Permission denied) 2024-07-27T03:18:57.519 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_1 449 root NOFD /proc/449/fd (opendir: Permission denied) 2024-07-27T03:18:57.519 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 450 root cwd unknown /proc/450/cwd (readlink: Permission denied) 2024-07-27T03:18:57.519 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 450 root rtd unknown /proc/450/root (readlink: Permission denied) 2024-07-27T03:18:57.520 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 450 root txt unknown /proc/450/exe (readlink: Permission denied) 2024-07-27T03:18:57.520 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 450 root NOFD /proc/450/fd (opendir: Permission denied) 2024-07-27T03:18:57.520 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_2 451 root cwd unknown /proc/451/cwd (readlink: Permission denied) 2024-07-27T03:18:57.520 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_2 451 root rtd unknown /proc/451/root (readlink: Permission denied) 2024-07-27T03:18:57.520 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_2 451 root txt unknown /proc/451/exe (readlink: Permission denied) 2024-07-27T03:18:57.520 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_2 451 root NOFD /proc/451/fd (opendir: Permission denied) 2024-07-27T03:18:57.520 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 452 root cwd unknown /proc/452/cwd (readlink: Permission denied) 2024-07-27T03:18:57.520 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 452 root rtd unknown /proc/452/root (readlink: Permission denied) 2024-07-27T03:18:57.520 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 452 root txt unknown /proc/452/exe (readlink: Permission denied) 2024-07-27T03:18:57.520 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 452 root NOFD /proc/452/fd (opendir: Permission denied) 2024-07-27T03:18:57.520 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_3 453 root cwd unknown /proc/453/cwd (readlink: Permission denied) 2024-07-27T03:18:57.520 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_3 453 root rtd unknown /proc/453/root (readlink: Permission denied) 2024-07-27T03:18:57.520 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_3 453 root txt unknown /proc/453/exe (readlink: Permission denied) 2024-07-27T03:18:57.520 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_3 453 root NOFD /proc/453/fd (opendir: Permission denied) 2024-07-27T03:18:57.521 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 454 root cwd unknown /proc/454/cwd (readlink: Permission denied) 2024-07-27T03:18:57.521 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 454 root rtd unknown /proc/454/root (readlink: Permission denied) 2024-07-27T03:18:57.521 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 454 root txt unknown /proc/454/exe (readlink: Permission denied) 2024-07-27T03:18:57.521 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 454 root NOFD /proc/454/fd (opendir: Permission denied) 2024-07-27T03:18:57.521 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_4 456 root cwd unknown /proc/456/cwd (readlink: Permission denied) 2024-07-27T03:18:57.521 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_4 456 root rtd unknown /proc/456/root (readlink: Permission denied) 2024-07-27T03:18:57.521 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_4 456 root txt unknown /proc/456/exe (readlink: Permission denied) 2024-07-27T03:18:57.521 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_4 456 root NOFD /proc/456/fd (opendir: Permission denied) 2024-07-27T03:18:57.521 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 457 root cwd unknown /proc/457/cwd (readlink: Permission denied) 2024-07-27T03:18:57.521 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 457 root rtd unknown /proc/457/root (readlink: Permission denied) 2024-07-27T03:18:57.521 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 457 root txt unknown /proc/457/exe (readlink: Permission denied) 2024-07-27T03:18:57.521 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 457 root NOFD /proc/457/fd (opendir: Permission denied) 2024-07-27T03:18:57.521 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_5 458 root cwd unknown /proc/458/cwd (readlink: Permission denied) 2024-07-27T03:18:57.521 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_5 458 root rtd unknown /proc/458/root (readlink: Permission denied) 2024-07-27T03:18:57.522 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_5 458 root txt unknown /proc/458/exe (readlink: Permission denied) 2024-07-27T03:18:57.522 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_5 458 root NOFD /proc/458/fd (opendir: Permission denied) 2024-07-27T03:18:57.522 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 459 root cwd unknown /proc/459/cwd (readlink: Permission denied) 2024-07-27T03:18:57.522 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 459 root rtd unknown /proc/459/root (readlink: Permission denied) 2024-07-27T03:18:57.522 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 459 root txt unknown /proc/459/exe (readlink: Permission denied) 2024-07-27T03:18:57.522 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 459 root NOFD /proc/459/fd (opendir: Permission denied) 2024-07-27T03:18:57.522 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_6 460 root cwd unknown /proc/460/cwd (readlink: Permission denied) 2024-07-27T03:18:57.522 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_6 460 root rtd unknown /proc/460/root (readlink: Permission denied) 2024-07-27T03:18:57.522 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_6 460 root txt unknown /proc/460/exe (readlink: Permission denied) 2024-07-27T03:18:57.522 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_6 460 root NOFD /proc/460/fd (opendir: Permission denied) 2024-07-27T03:18:57.522 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 461 root cwd unknown /proc/461/cwd (readlink: Permission denied) 2024-07-27T03:18:57.522 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 461 root rtd unknown /proc/461/root (readlink: Permission denied) 2024-07-27T03:18:57.523 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 461 root txt unknown /proc/461/exe (readlink: Permission denied) 2024-07-27T03:18:57.523 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 461 root NOFD /proc/461/fd (opendir: Permission denied) 2024-07-27T03:18:57.523 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_7 462 root cwd unknown /proc/462/cwd (readlink: Permission denied) 2024-07-27T03:18:57.523 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_7 462 root rtd unknown /proc/462/root (readlink: Permission denied) 2024-07-27T03:18:57.523 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_7 462 root txt unknown /proc/462/exe (readlink: Permission denied) 2024-07-27T03:18:57.523 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_7 462 root NOFD /proc/462/fd (opendir: Permission denied) 2024-07-27T03:18:57.523 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 463 root cwd unknown /proc/463/cwd (readlink: Permission denied) 2024-07-27T03:18:57.523 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 463 root rtd unknown /proc/463/root (readlink: Permission denied) 2024-07-27T03:18:57.523 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 463 root txt unknown /proc/463/exe (readlink: Permission denied) 2024-07-27T03:18:57.523 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 463 root NOFD /proc/463/fd (opendir: Permission denied) 2024-07-27T03:18:57.523 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_8 464 root cwd unknown /proc/464/cwd (readlink: Permission denied) 2024-07-27T03:18:57.523 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_8 464 root rtd unknown /proc/464/root (readlink: Permission denied) 2024-07-27T03:18:57.523 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_8 464 root txt unknown /proc/464/exe (readlink: Permission denied) 2024-07-27T03:18:57.523 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_8 464 root NOFD /proc/464/fd (opendir: Permission denied) 2024-07-27T03:18:57.524 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 465 root cwd unknown /proc/465/cwd (readlink: Permission denied) 2024-07-27T03:18:57.524 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 465 root rtd unknown /proc/465/root (readlink: Permission denied) 2024-07-27T03:18:57.524 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 465 root txt unknown /proc/465/exe (readlink: Permission denied) 2024-07-27T03:18:57.524 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 465 root NOFD /proc/465/fd (opendir: Permission denied) 2024-07-27T03:18:57.524 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_9 466 root cwd unknown /proc/466/cwd (readlink: Permission denied) 2024-07-27T03:18:57.524 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_9 466 root rtd unknown /proc/466/root (readlink: Permission denied) 2024-07-27T03:18:57.524 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_9 466 root txt unknown /proc/466/exe (readlink: Permission denied) 2024-07-27T03:18:57.524 INFO:tasks.workunit.client.0.smithi067.stdout:scsi_eh_9 466 root NOFD /proc/466/fd (opendir: Permission denied) 2024-07-27T03:18:57.524 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 467 root cwd unknown /proc/467/cwd (readlink: Permission denied) 2024-07-27T03:18:57.524 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 467 root rtd unknown /proc/467/root (readlink: Permission denied) 2024-07-27T03:18:57.524 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 467 root txt unknown /proc/467/exe (readlink: Permission denied) 2024-07-27T03:18:57.524 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 467 root NOFD /proc/467/fd (opendir: Permission denied) 2024-07-27T03:18:57.524 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 468 root cwd unknown /proc/468/cwd (readlink: Permission denied) 2024-07-27T03:18:57.524 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 468 root rtd unknown /proc/468/root (readlink: Permission denied) 2024-07-27T03:18:57.525 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 468 root txt unknown /proc/468/exe (readlink: Permission denied) 2024-07-27T03:18:57.525 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 468 root NOFD /proc/468/fd (opendir: Permission denied) 2024-07-27T03:18:57.525 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 469 root cwd unknown /proc/469/cwd (readlink: Permission denied) 2024-07-27T03:18:57.525 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 469 root rtd unknown /proc/469/root (readlink: Permission denied) 2024-07-27T03:18:57.525 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 469 root txt unknown /proc/469/exe (readlink: Permission denied) 2024-07-27T03:18:57.525 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 469 root NOFD /proc/469/fd (opendir: Permission denied) 2024-07-27T03:18:57.525 INFO:tasks.workunit.client.0.smithi067.stdout:jbd2/sda1 496 root cwd unknown /proc/496/cwd (readlink: Permission denied) 2024-07-27T03:18:57.525 INFO:tasks.workunit.client.0.smithi067.stdout:jbd2/sda1 496 root rtd unknown /proc/496/root (readlink: Permission denied) 2024-07-27T03:18:57.525 INFO:tasks.workunit.client.0.smithi067.stdout:jbd2/sda1 496 root txt unknown /proc/496/exe (readlink: Permission denied) 2024-07-27T03:18:57.525 INFO:tasks.workunit.client.0.smithi067.stdout:jbd2/sda1 496 root NOFD /proc/496/fd (opendir: Permission denied) 2024-07-27T03:18:57.525 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 497 root cwd unknown /proc/497/cwd (readlink: Permission denied) 2024-07-27T03:18:57.525 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 497 root rtd unknown /proc/497/root (readlink: Permission denied) 2024-07-27T03:18:57.525 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 497 root txt unknown /proc/497/exe (readlink: Permission denied) 2024-07-27T03:18:57.525 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 497 root NOFD /proc/497/fd (opendir: Permission denied) 2024-07-27T03:18:57.526 INFO:tasks.workunit.client.0.smithi067.stdout:systemd-j 573 root cwd unknown /proc/573/cwd (readlink: Permission denied) 2024-07-27T03:18:57.526 INFO:tasks.workunit.client.0.smithi067.stdout:systemd-j 573 root rtd unknown /proc/573/root (readlink: Permission denied) 2024-07-27T03:18:57.526 INFO:tasks.workunit.client.0.smithi067.stdout:systemd-j 573 root txt unknown /proc/573/exe (readlink: Permission denied) 2024-07-27T03:18:57.526 INFO:tasks.workunit.client.0.smithi067.stdout:systemd-j 573 root NOFD /proc/573/fd (opendir: Permission denied) 2024-07-27T03:18:57.526 INFO:tasks.workunit.client.0.smithi067.stdout:ext4lazyi 585 root cwd unknown /proc/585/cwd (readlink: Permission denied) 2024-07-27T03:18:57.526 INFO:tasks.workunit.client.0.smithi067.stdout:ext4lazyi 585 root rtd unknown /proc/585/root (readlink: Permission denied) 2024-07-27T03:18:57.526 INFO:tasks.workunit.client.0.smithi067.stdout:ext4lazyi 585 root txt unknown /proc/585/exe (readlink: Permission denied) 2024-07-27T03:18:57.526 INFO:tasks.workunit.client.0.smithi067.stdout:ext4lazyi 585 root NOFD /proc/585/fd (opendir: Permission denied) 2024-07-27T03:18:57.526 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/4 586 root cwd unknown /proc/586/cwd (readlink: Permission denied) 2024-07-27T03:18:57.526 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/4 586 root rtd unknown /proc/586/root (readlink: Permission denied) 2024-07-27T03:18:57.526 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/4 586 root txt unknown /proc/586/exe (readlink: Permission denied) 2024-07-27T03:18:57.526 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/4 586 root NOFD /proc/586/fd (opendir: Permission denied) 2024-07-27T03:18:57.527 INFO:tasks.workunit.client.0.smithi067.stdout:systemd-u 594 root cwd unknown /proc/594/cwd (readlink: Permission denied) 2024-07-27T03:18:57.527 INFO:tasks.workunit.client.0.smithi067.stdout:systemd-u 594 root rtd unknown /proc/594/root (readlink: Permission denied) 2024-07-27T03:18:57.527 INFO:tasks.workunit.client.0.smithi067.stdout:systemd-u 594 root txt unknown /proc/594/exe (readlink: Permission denied) 2024-07-27T03:18:57.527 INFO:tasks.workunit.client.0.smithi067.stdout:systemd-u 594 root NOFD /proc/594/fd (opendir: Permission denied) 2024-07-27T03:18:57.527 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 644 root cwd unknown /proc/644/cwd (readlink: Permission denied) 2024-07-27T03:18:57.527 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 644 root rtd unknown /proc/644/root (readlink: Permission denied) 2024-07-27T03:18:57.527 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 644 root txt unknown /proc/644/exe (readlink: Permission denied) 2024-07-27T03:18:57.527 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 644 root NOFD /proc/644/fd (opendir: Permission denied) 2024-07-27T03:18:57.527 INFO:tasks.workunit.client.0.smithi067.stdout:rpcbind 658 rpc cwd unknown /proc/658/cwd (readlink: Permission denied) 2024-07-27T03:18:57.527 INFO:tasks.workunit.client.0.smithi067.stdout:rpcbind 658 rpc rtd unknown /proc/658/root (readlink: Permission denied) 2024-07-27T03:18:57.527 INFO:tasks.workunit.client.0.smithi067.stdout:rpcbind 658 rpc txt unknown /proc/658/exe (readlink: Permission denied) 2024-07-27T03:18:57.527 INFO:tasks.workunit.client.0.smithi067.stdout:rpcbind 658 rpc NOFD /proc/658/fd (opendir: Permission denied) 2024-07-27T03:18:57.527 INFO:tasks.workunit.client.0.smithi067.stdout:kipmi0 660 root cwd unknown /proc/660/cwd (readlink: Permission denied) 2024-07-27T03:18:57.527 INFO:tasks.workunit.client.0.smithi067.stdout:kipmi0 660 root rtd unknown /proc/660/root (readlink: Permission denied) 2024-07-27T03:18:57.528 INFO:tasks.workunit.client.0.smithi067.stdout:kipmi0 660 root txt unknown /proc/660/exe (readlink: Permission denied) 2024-07-27T03:18:57.528 INFO:tasks.workunit.client.0.smithi067.stdout:kipmi0 660 root NOFD /proc/660/fd (opendir: Permission denied) 2024-07-27T03:18:57.528 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 664 root cwd unknown /proc/664/cwd (readlink: Permission denied) 2024-07-27T03:18:57.528 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 664 root rtd unknown /proc/664/root (readlink: Permission denied) 2024-07-27T03:18:57.528 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 664 root txt unknown /proc/664/exe (readlink: Permission denied) 2024-07-27T03:18:57.528 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 664 root NOFD /proc/664/fd (opendir: Permission denied) 2024-07-27T03:18:57.528 INFO:tasks.workunit.client.0.smithi067.stdout:auditd 669 root cwd unknown /proc/669/cwd (readlink: Permission denied) 2024-07-27T03:18:57.528 INFO:tasks.workunit.client.0.smithi067.stdout:auditd 669 root rtd unknown /proc/669/root (readlink: Permission denied) 2024-07-27T03:18:57.528 INFO:tasks.workunit.client.0.smithi067.stdout:auditd 669 root txt unknown /proc/669/exe (readlink: Permission denied) 2024-07-27T03:18:57.528 INFO:tasks.workunit.client.0.smithi067.stdout:auditd 669 root NOFD /proc/669/fd (opendir: Permission denied) 2024-07-27T03:18:57.528 INFO:tasks.workunit.client.0.smithi067.stdout:auditd 669 670 auditd root cwd unknown /proc/669/task/670/cwd (readlink: Permission denied) 2024-07-27T03:18:57.528 INFO:tasks.workunit.client.0.smithi067.stdout:auditd 669 670 auditd root rtd unknown /proc/669/task/670/root (readlink: Permission denied) 2024-07-27T03:18:57.528 INFO:tasks.workunit.client.0.smithi067.stdout:auditd 669 670 auditd root txt unknown /proc/669/task/670/exe (readlink: Permission denied) 2024-07-27T03:18:57.529 INFO:tasks.workunit.client.0.smithi067.stdout:auditd 669 670 auditd root NOFD /proc/669/task/670/fd (opendir: Permission denied) 2024-07-27T03:18:57.529 INFO:tasks.workunit.client.0.smithi067.stdout:auditd 669 672 auditd root cwd unknown /proc/669/task/672/cwd (readlink: Permission denied) 2024-07-27T03:18:57.529 INFO:tasks.workunit.client.0.smithi067.stdout:auditd 669 672 auditd root rtd unknown /proc/669/task/672/root (readlink: Permission denied) 2024-07-27T03:18:57.529 INFO:tasks.workunit.client.0.smithi067.stdout:auditd 669 672 auditd root txt unknown /proc/669/task/672/exe (readlink: Permission denied) 2024-07-27T03:18:57.529 INFO:tasks.workunit.client.0.smithi067.stdout:auditd 669 672 auditd root NOFD /proc/669/task/672/fd (opendir: Permission denied) 2024-07-27T03:18:57.529 INFO:tasks.workunit.client.0.smithi067.stdout:sedispatc 671 root cwd unknown /proc/671/cwd (readlink: Permission denied) 2024-07-27T03:18:57.529 INFO:tasks.workunit.client.0.smithi067.stdout:sedispatc 671 root rtd unknown /proc/671/root (readlink: Permission denied) 2024-07-27T03:18:57.529 INFO:tasks.workunit.client.0.smithi067.stdout:sedispatc 671 root txt unknown /proc/671/exe (readlink: Permission denied) 2024-07-27T03:18:57.529 INFO:tasks.workunit.client.0.smithi067.stdout:sedispatc 671 root NOFD /proc/671/fd (opendir: Permission denied) 2024-07-27T03:18:57.529 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 688 root cwd unknown /proc/688/cwd (readlink: Permission denied) 2024-07-27T03:18:57.529 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 688 root rtd unknown /proc/688/root (readlink: Permission denied) 2024-07-27T03:18:57.529 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 688 root txt unknown /proc/688/exe (readlink: Permission denied) 2024-07-27T03:18:57.529 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 688 root NOFD /proc/688/fd (opendir: Permission denied) 2024-07-27T03:18:57.529 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 689 root cwd unknown /proc/689/cwd (readlink: Permission denied) 2024-07-27T03:18:57.529 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 689 root rtd unknown /proc/689/root (readlink: Permission denied) 2024-07-27T03:18:57.530 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 689 root txt unknown /proc/689/exe (readlink: Permission denied) 2024-07-27T03:18:57.530 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 689 root NOFD /proc/689/fd (opendir: Permission denied) 2024-07-27T03:18:57.530 INFO:tasks.workunit.client.0.smithi067.stdout:dbus-brok 717 dbus cwd unknown /proc/717/cwd (readlink: Permission denied) 2024-07-27T03:18:57.530 INFO:tasks.workunit.client.0.smithi067.stdout:dbus-brok 717 dbus rtd unknown /proc/717/root (readlink: Permission denied) 2024-07-27T03:18:57.530 INFO:tasks.workunit.client.0.smithi067.stdout:dbus-brok 717 dbus txt unknown /proc/717/exe (readlink: Permission denied) 2024-07-27T03:18:57.530 INFO:tasks.workunit.client.0.smithi067.stdout:dbus-brok 717 dbus NOFD /proc/717/fd (opendir: Permission denied) 2024-07-27T03:18:57.530 INFO:tasks.workunit.client.0.smithi067.stdout:dbus-brok 718 dbus cwd unknown /proc/718/cwd (readlink: Permission denied) 2024-07-27T03:18:57.530 INFO:tasks.workunit.client.0.smithi067.stdout:dbus-brok 718 dbus rtd unknown /proc/718/root (readlink: Permission denied) 2024-07-27T03:18:57.530 INFO:tasks.workunit.client.0.smithi067.stdout:dbus-brok 718 dbus txt unknown /proc/718/exe (readlink: Permission denied) 2024-07-27T03:18:57.530 INFO:tasks.workunit.client.0.smithi067.stdout:dbus-brok 718 dbus NOFD /proc/718/fd (opendir: Permission denied) 2024-07-27T03:18:57.530 INFO:tasks.workunit.client.0.smithi067.stdout:NetworkMa 719 root cwd unknown /proc/719/cwd (readlink: Permission denied) 2024-07-27T03:18:57.530 INFO:tasks.workunit.client.0.smithi067.stdout:NetworkMa 719 root rtd unknown /proc/719/root (readlink: Permission denied) 2024-07-27T03:18:57.530 INFO:tasks.workunit.client.0.smithi067.stdout:NetworkMa 719 root txt unknown /proc/719/exe (readlink: Permission denied) 2024-07-27T03:18:57.531 INFO:tasks.workunit.client.0.smithi067.stdout:NetworkMa 719 root NOFD /proc/719/fd (opendir: Permission denied) 2024-07-27T03:18:57.531 INFO:tasks.workunit.client.0.smithi067.stdout:NetworkMa 719 735 gmain root cwd unknown /proc/719/task/735/cwd (readlink: Permission denied) 2024-07-27T03:18:57.531 INFO:tasks.workunit.client.0.smithi067.stdout:NetworkMa 719 735 gmain root rtd unknown /proc/719/task/735/root (readlink: Permission denied) 2024-07-27T03:18:57.531 INFO:tasks.workunit.client.0.smithi067.stdout:NetworkMa 719 735 gmain root txt unknown /proc/719/task/735/exe (readlink: Permission denied) 2024-07-27T03:18:57.531 INFO:tasks.workunit.client.0.smithi067.stdout:NetworkMa 719 735 gmain root NOFD /proc/719/task/735/fd (opendir: Permission denied) 2024-07-27T03:18:57.531 INFO:tasks.workunit.client.0.smithi067.stdout:NetworkMa 719 736 gdbus root cwd unknown /proc/719/task/736/cwd (readlink: Permission denied) 2024-07-27T03:18:57.531 INFO:tasks.workunit.client.0.smithi067.stdout:NetworkMa 719 736 gdbus root rtd unknown /proc/719/task/736/root (readlink: Permission denied) 2024-07-27T03:18:57.531 INFO:tasks.workunit.client.0.smithi067.stdout:NetworkMa 719 736 gdbus root txt unknown /proc/719/task/736/exe (readlink: Permission denied) 2024-07-27T03:18:57.531 INFO:tasks.workunit.client.0.smithi067.stdout:NetworkMa 719 736 gdbus root NOFD /proc/719/task/736/fd (opendir: Permission denied) 2024-07-27T03:18:57.531 INFO:tasks.workunit.client.0.smithi067.stdout:irqbalanc 722 root cwd unknown /proc/722/cwd (readlink: Permission denied) 2024-07-27T03:18:57.531 INFO:tasks.workunit.client.0.smithi067.stdout:irqbalanc 722 root rtd unknown /proc/722/root (readlink: Permission denied) 2024-07-27T03:18:57.531 INFO:tasks.workunit.client.0.smithi067.stdout:irqbalanc 722 root txt unknown /proc/722/exe (readlink: Permission denied) 2024-07-27T03:18:57.531 INFO:tasks.workunit.client.0.smithi067.stdout:irqbalanc 722 root NOFD /proc/722/fd (opendir: Permission denied) 2024-07-27T03:18:57.532 INFO:tasks.workunit.client.0.smithi067.stdout:irqbalanc 722 728 gmain root cwd unknown /proc/722/task/728/cwd (readlink: Permission denied) 2024-07-27T03:18:57.532 INFO:tasks.workunit.client.0.smithi067.stdout:irqbalanc 722 728 gmain root rtd unknown /proc/722/task/728/root (readlink: Permission denied) 2024-07-27T03:18:57.532 INFO:tasks.workunit.client.0.smithi067.stdout:irqbalanc 722 728 gmain root txt unknown /proc/722/task/728/exe (readlink: Permission denied) 2024-07-27T03:18:57.532 INFO:tasks.workunit.client.0.smithi067.stdout:irqbalanc 722 728 gmain root NOFD /proc/722/task/728/fd (opendir: Permission denied) 2024-07-27T03:18:57.532 INFO:tasks.workunit.client.0.smithi067.stdout:lsmd 723 libstoragemgmt cwd unknown /proc/723/cwd (readlink: Permission denied) 2024-07-27T03:18:57.532 INFO:tasks.workunit.client.0.smithi067.stdout:lsmd 723 libstoragemgmt rtd unknown /proc/723/root (readlink: Permission denied) 2024-07-27T03:18:57.532 INFO:tasks.workunit.client.0.smithi067.stdout:lsmd 723 libstoragemgmt txt unknown /proc/723/exe (readlink: Permission denied) 2024-07-27T03:18:57.532 INFO:tasks.workunit.client.0.smithi067.stdout:lsmd 723 libstoragemgmt NOFD /proc/723/fd (opendir: Permission denied) 2024-07-27T03:18:57.532 INFO:tasks.workunit.client.0.smithi067.stdout:mcelog 724 root cwd unknown /proc/724/cwd (readlink: Permission denied) 2024-07-27T03:18:57.532 INFO:tasks.workunit.client.0.smithi067.stdout:mcelog 724 root rtd unknown /proc/724/root (readlink: Permission denied) 2024-07-27T03:18:57.532 INFO:tasks.workunit.client.0.smithi067.stdout:mcelog 724 root txt unknown /proc/724/exe (readlink: Permission denied) 2024-07-27T03:18:57.532 INFO:tasks.workunit.client.0.smithi067.stdout:mcelog 724 root NOFD /proc/724/fd (opendir: Permission denied) 2024-07-27T03:18:57.532 INFO:tasks.workunit.client.0.smithi067.stdout:smartd 725 root cwd unknown /proc/725/cwd (readlink: Permission denied) 2024-07-27T03:18:57.532 INFO:tasks.workunit.client.0.smithi067.stdout:smartd 725 root rtd unknown /proc/725/root (readlink: Permission denied) 2024-07-27T03:18:57.533 INFO:tasks.workunit.client.0.smithi067.stdout:smartd 725 root txt unknown /proc/725/exe (readlink: Permission denied) 2024-07-27T03:18:57.533 INFO:tasks.workunit.client.0.smithi067.stdout:smartd 725 root NOFD /proc/725/fd (opendir: Permission denied) 2024-07-27T03:18:57.533 INFO:tasks.workunit.client.0.smithi067.stdout:systemd-l 727 root cwd unknown /proc/727/cwd (readlink: Permission denied) 2024-07-27T03:18:57.533 INFO:tasks.workunit.client.0.smithi067.stdout:systemd-l 727 root rtd unknown /proc/727/root (readlink: Permission denied) 2024-07-27T03:18:57.533 INFO:tasks.workunit.client.0.smithi067.stdout:systemd-l 727 root txt unknown /proc/727/exe (readlink: Permission denied) 2024-07-27T03:18:57.533 INFO:tasks.workunit.client.0.smithi067.stdout:systemd-l 727 root NOFD /proc/727/fd (opendir: Permission denied) 2024-07-27T03:18:57.533 INFO:tasks.workunit.client.0.smithi067.stdout:sshd 767 root cwd unknown /proc/767/cwd (readlink: Permission denied) 2024-07-27T03:18:57.533 INFO:tasks.workunit.client.0.smithi067.stdout:sshd 767 root rtd unknown /proc/767/root (readlink: Permission denied) 2024-07-27T03:18:57.533 INFO:tasks.workunit.client.0.smithi067.stdout:sshd 767 root txt unknown /proc/767/exe (readlink: Permission denied) 2024-07-27T03:18:57.533 INFO:tasks.workunit.client.0.smithi067.stdout:sshd 767 root NOFD /proc/767/fd (opendir: Permission denied) 2024-07-27T03:18:57.533 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 root cwd unknown /proc/771/cwd (readlink: Permission denied) 2024-07-27T03:18:57.533 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 root rtd unknown /proc/771/root (readlink: Permission denied) 2024-07-27T03:18:57.533 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 root txt unknown /proc/771/exe (readlink: Permission denied) 2024-07-27T03:18:57.533 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 root NOFD /proc/771/fd (opendir: Permission denied) 2024-07-27T03:18:57.534 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 772 gssproxy root cwd unknown /proc/771/task/772/cwd (readlink: Permission denied) 2024-07-27T03:18:57.534 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 772 gssproxy root rtd unknown /proc/771/task/772/root (readlink: Permission denied) 2024-07-27T03:18:57.534 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 772 gssproxy root txt unknown /proc/771/task/772/exe (readlink: Permission denied) 2024-07-27T03:18:57.534 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 772 gssproxy root NOFD /proc/771/task/772/fd (opendir: Permission denied) 2024-07-27T03:18:57.534 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 773 gssproxy root cwd unknown /proc/771/task/773/cwd (readlink: Permission denied) 2024-07-27T03:18:57.534 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 773 gssproxy root rtd unknown /proc/771/task/773/root (readlink: Permission denied) 2024-07-27T03:18:57.534 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 773 gssproxy root txt unknown /proc/771/task/773/exe (readlink: Permission denied) 2024-07-27T03:18:57.534 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 773 gssproxy root NOFD /proc/771/task/773/fd (opendir: Permission denied) 2024-07-27T03:18:57.534 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 774 gssproxy root cwd unknown /proc/771/task/774/cwd (readlink: Permission denied) 2024-07-27T03:18:57.534 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 774 gssproxy root rtd unknown /proc/771/task/774/root (readlink: Permission denied) 2024-07-27T03:18:57.534 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 774 gssproxy root txt unknown /proc/771/task/774/exe (readlink: Permission denied) 2024-07-27T03:18:57.534 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 774 gssproxy root NOFD /proc/771/task/774/fd (opendir: Permission denied) 2024-07-27T03:18:57.535 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 775 gssproxy root cwd unknown /proc/771/task/775/cwd (readlink: Permission denied) 2024-07-27T03:18:57.535 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 775 gssproxy root rtd unknown /proc/771/task/775/root (readlink: Permission denied) 2024-07-27T03:18:57.535 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 775 gssproxy root txt unknown /proc/771/task/775/exe (readlink: Permission denied) 2024-07-27T03:18:57.535 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 775 gssproxy root NOFD /proc/771/task/775/fd (opendir: Permission denied) 2024-07-27T03:18:57.535 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 776 gssproxy root cwd unknown /proc/771/task/776/cwd (readlink: Permission denied) 2024-07-27T03:18:57.535 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 776 gssproxy root rtd unknown /proc/771/task/776/root (readlink: Permission denied) 2024-07-27T03:18:57.535 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 776 gssproxy root txt unknown /proc/771/task/776/exe (readlink: Permission denied) 2024-07-27T03:18:57.535 INFO:tasks.workunit.client.0.smithi067.stdout:gssproxy 771 776 gssproxy root NOFD /proc/771/task/776/fd (opendir: Permission denied) 2024-07-27T03:18:57.535 INFO:tasks.workunit.client.0.smithi067.stdout:atd 778 root cwd unknown /proc/778/cwd (readlink: Permission denied) 2024-07-27T03:18:57.535 INFO:tasks.workunit.client.0.smithi067.stdout:atd 778 root rtd unknown /proc/778/root (readlink: Permission denied) 2024-07-27T03:18:57.535 INFO:tasks.workunit.client.0.smithi067.stdout:atd 778 root txt unknown /proc/778/exe (readlink: Permission denied) 2024-07-27T03:18:57.535 INFO:tasks.workunit.client.0.smithi067.stdout:atd 778 root NOFD /proc/778/fd (opendir: Permission denied) 2024-07-27T03:18:57.535 INFO:tasks.workunit.client.0.smithi067.stdout:crond 779 root cwd unknown /proc/779/cwd (readlink: Permission denied) 2024-07-27T03:18:57.536 INFO:tasks.workunit.client.0.smithi067.stdout:crond 779 root rtd unknown /proc/779/root (readlink: Permission denied) 2024-07-27T03:18:57.536 INFO:tasks.workunit.client.0.smithi067.stdout:crond 779 root txt unknown /proc/779/exe (readlink: Permission denied) 2024-07-27T03:18:57.536 INFO:tasks.workunit.client.0.smithi067.stdout:crond 779 root NOFD /proc/779/fd (opendir: Permission denied) 2024-07-27T03:18:57.536 INFO:tasks.workunit.client.0.smithi067.stdout:nrpe 801 nrpe cwd unknown /proc/801/cwd (readlink: Permission denied) 2024-07-27T03:18:57.536 INFO:tasks.workunit.client.0.smithi067.stdout:nrpe 801 nrpe rtd unknown /proc/801/root (readlink: Permission denied) 2024-07-27T03:18:57.536 INFO:tasks.workunit.client.0.smithi067.stdout:nrpe 801 nrpe txt unknown /proc/801/exe (readlink: Permission denied) 2024-07-27T03:18:57.536 INFO:tasks.workunit.client.0.smithi067.stdout:nrpe 801 nrpe NOFD /proc/801/fd (opendir: Permission denied) 2024-07-27T03:18:57.536 INFO:tasks.workunit.client.0.smithi067.stdout:agetty 1175 root cwd unknown /proc/1175/cwd (readlink: Permission denied) 2024-07-27T03:18:57.536 INFO:tasks.workunit.client.0.smithi067.stdout:agetty 1175 root rtd unknown /proc/1175/root (readlink: Permission denied) 2024-07-27T03:18:57.536 INFO:tasks.workunit.client.0.smithi067.stdout:agetty 1175 root txt unknown /proc/1175/exe (readlink: Permission denied) 2024-07-27T03:18:57.536 INFO:tasks.workunit.client.0.smithi067.stdout:agetty 1175 root NOFD /proc/1175/fd (opendir: Permission denied) 2024-07-27T03:18:57.536 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 1176 root cwd unknown /proc/1176/cwd (readlink: Permission denied) 2024-07-27T03:18:57.536 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 1176 root rtd unknown /proc/1176/root (readlink: Permission denied) 2024-07-27T03:18:57.536 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 1176 root txt unknown /proc/1176/exe (readlink: Permission denied) 2024-07-27T03:18:57.537 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 1176 root NOFD /proc/1176/fd (opendir: Permission denied) 2024-07-27T03:18:57.537 INFO:tasks.workunit.client.0.smithi067.stdout:agetty 1177 root cwd unknown /proc/1177/cwd (readlink: Permission denied) 2024-07-27T03:18:57.537 INFO:tasks.workunit.client.0.smithi067.stdout:agetty 1177 root rtd unknown /proc/1177/root (readlink: Permission denied) 2024-07-27T03:18:57.537 INFO:tasks.workunit.client.0.smithi067.stdout:agetty 1177 root txt unknown /proc/1177/exe (readlink: Permission denied) 2024-07-27T03:18:57.537 INFO:tasks.workunit.client.0.smithi067.stdout:agetty 1177 root NOFD /proc/1177/fd (opendir: Permission denied) 2024-07-27T03:18:57.537 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 1179 root cwd unknown /proc/1179/cwd (readlink: Permission denied) 2024-07-27T03:18:57.537 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 1179 root rtd unknown /proc/1179/root (readlink: Permission denied) 2024-07-27T03:18:57.537 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 1179 root txt unknown /proc/1179/exe (readlink: Permission denied) 2024-07-27T03:18:57.537 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 1179 root NOFD /proc/1179/fd (opendir: Permission denied) 2024-07-27T03:18:57.537 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu cwd DIR 8,1 4096 2 / 2024-07-27T03:18:57.537 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu rtd DIR 8,1 4096 2 / 2024-07-27T03:18:57.537 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu txt REG 8,1 98216 18583 /usr/lib/systemd/systemd 2024-07-27T03:18:57.537 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu DEL REG 8,1 25843 /etc/selinux/targeted/contexts/files/file_contexts.bin 2024-07-27T03:18:57.537 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-07-27T03:18:57.538 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-07-27T03:18:57.538 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-07-27T03:18:57.538 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-07-27T03:18:57.538 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu DEL REG 8,1 7490 /usr/lib64/libcrypto.so.3.0.7 2024-07-27T03:18:57.538 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-07-27T03:18:57.538 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-07-27T03:18:57.538 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu DEL REG 8,1 52266 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2024-07-27T03:18:57.538 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-07-27T03:18:57.538 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-07-27T03:18:57.538 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 28552 7287 /usr/lib64/libattr.so.1.1.2501 2024-07-27T03:18:57.538 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-07-27T03:18:57.538 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-07-27T03:18:57.539 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-07-27T03:18:57.539 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-07-27T03:18:57.539 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-07-27T03:18:57.539 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-07-27T03:18:57.539 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-07-27T03:18:57.539 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-07-27T03:18:57.539 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-07-27T03:18:57.539 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 107424 7663 /usr/lib64/libkmod.so.2.3.6 2024-07-27T03:18:57.539 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 3785856 46432 /usr/lib64/systemd/libsystemd-shared-252.so 2024-07-27T03:18:57.539 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-07-27T03:18:57.539 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-07-27T03:18:57.539 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-07-27T03:18:57.539 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 127680 7455 /usr/lib64/libseccomp.so.2.5.2 2024-07-27T03:18:57.540 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 2157904 46429 /usr/lib64/systemd/libsystemd-core-252.so 2024-07-27T03:18:57.540 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 40496 7288 /usr/lib64/libacl.so.1.1.2301 2024-07-27T03:18:57.540 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-07-27T03:18:57.540 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-07-27T03:18:57.540 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu mem REG 0,19 0 19 /sys/fs/selinux/status 2024-07-27T03:18:57.540 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2024-07-27T03:18:57.540 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu 1u unix 0x0000000000000000 0t0 654 type=STREAM (CONNECTED) 2024-07-27T03:18:57.540 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu 2u unix 0x0000000000000000 0t0 654 type=STREAM (CONNECTED) 2024-07-27T03:18:57.540 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu 3u unix 0x0000000000000000 0t0 18946 type=DGRAM (CONNECTED) 2024-07-27T03:18:57.540 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu 4u a_inode 0,14 0 3105 [eventpoll:5,6,8,9,11,12,14,15,16,17,19,20,21] 2024-07-27T03:18:57.540 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu 5u a_inode 0,14 0 3105 [signalfd] 2024-07-27T03:18:57.540 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu 6r a_inode 0,14 0 3105 inotify 2024-07-27T03:18:57.540 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu 7r DIR 0,25 0 3417 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service 2024-07-27T03:18:57.541 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu 8u a_inode 0,14 0 3105 [timerfd] 2024-07-27T03:18:57.541 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu 9u a_inode 0,14 0 3105 [eventpoll:10,13] 2024-07-27T03:18:57.541 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu 10r REG 0,20 0 18935 /proc/2010/mountinfo 2024-07-27T03:18:57.541 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu 11r a_inode 0,14 0 3105 inotify 2024-07-27T03:18:57.541 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu 12u unix 0x0000000000000000 0t0 18966 /run/user/1000/bus type=STREAM (LISTEN) 2024-07-27T03:18:57.541 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu 13r a_inode 0,14 0 3105 inotify 2024-07-27T03:18:57.541 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu 14r REG 0,20 0 4026532068 /proc/swaps 2024-07-27T03:18:57.541 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu 15u netlink 0t0 18951 KOBJECT_UEVENT 2024-07-27T03:18:57.541 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu 16u unix 0x0000000000000000 0t0 18956 /run/user/1000/systemd/notify type=DGRAM (UNCONNECTED) 2024-07-27T03:18:57.541 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu 17u unix 0x0000000000000000 0t0 18957 type=DGRAM (CONNECTED) 2024-07-27T03:18:57.541 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu 18u unix 0x0000000000000000 0t0 18958 type=DGRAM (CONNECTED) 2024-07-27T03:18:57.541 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu 19u unix 0x0000000000000000 0t0 18959 /run/user/1000/systemd/private type=STREAM (LISTEN) 2024-07-27T03:18:57.541 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu 20u unix 0x0000000000000000 0t0 18961 type=STREAM (CONNECTED) 2024-07-27T03:18:57.541 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 2010 ubuntu 21u a_inode 0,14 0 3105 [timerfd] 2024-07-27T03:18:57.542 INFO:tasks.workunit.client.0.smithi067.stdout:(sd-pam) 2032 ubuntu cwd unknown /proc/2032/cwd (readlink: Permission denied) 2024-07-27T03:18:57.542 INFO:tasks.workunit.client.0.smithi067.stdout:(sd-pam) 2032 ubuntu rtd unknown /proc/2032/root (readlink: Permission denied) 2024-07-27T03:18:57.542 INFO:tasks.workunit.client.0.smithi067.stdout:(sd-pam) 2032 ubuntu txt unknown /proc/2032/exe (readlink: Permission denied) 2024-07-27T03:18:57.542 INFO:tasks.workunit.client.0.smithi067.stdout:(sd-pam) 2032 ubuntu NOFD /proc/2032/fd (opendir: Permission denied) 2024-07-27T03:18:57.542 INFO:tasks.workunit.client.0.smithi067.stdout:rsyslogd 3891 root cwd unknown /proc/3891/cwd (readlink: Permission denied) 2024-07-27T03:18:57.542 INFO:tasks.workunit.client.0.smithi067.stdout:rsyslogd 3891 root rtd unknown /proc/3891/root (readlink: Permission denied) 2024-07-27T03:18:57.542 INFO:tasks.workunit.client.0.smithi067.stdout:rsyslogd 3891 root txt unknown /proc/3891/exe (readlink: Permission denied) 2024-07-27T03:18:57.542 INFO:tasks.workunit.client.0.smithi067.stdout:rsyslogd 3891 root NOFD /proc/3891/fd (opendir: Permission denied) 2024-07-27T03:18:57.542 INFO:tasks.workunit.client.0.smithi067.stdout:rsyslogd 3891 3894 in:imjour root cwd unknown /proc/3891/task/3894/cwd (readlink: Permission denied) 2024-07-27T03:18:57.542 INFO:tasks.workunit.client.0.smithi067.stdout:rsyslogd 3891 3894 in:imjour root rtd unknown /proc/3891/task/3894/root (readlink: Permission denied) 2024-07-27T03:18:57.542 INFO:tasks.workunit.client.0.smithi067.stdout:rsyslogd 3891 3894 in:imjour root txt unknown /proc/3891/task/3894/exe (readlink: Permission denied) 2024-07-27T03:18:57.542 INFO:tasks.workunit.client.0.smithi067.stdout:rsyslogd 3891 3894 in:imjour root NOFD /proc/3891/task/3894/fd (opendir: Permission denied) 2024-07-27T03:18:57.542 INFO:tasks.workunit.client.0.smithi067.stdout:rsyslogd 3891 3895 rs:main root cwd unknown /proc/3891/task/3895/cwd (readlink: Permission denied) 2024-07-27T03:18:57.545 INFO:tasks.workunit.client.0.smithi067.stdout:rsyslogd 3891 3895 rs:main root rtd unknown /proc/3891/task/3895/root (readlink: Permission denied) 2024-07-27T03:18:57.545 INFO:tasks.workunit.client.0.smithi067.stdout:rsyslogd 3891 3895 rs:main root txt unknown /proc/3891/task/3895/exe (readlink: Permission denied) 2024-07-27T03:18:57.545 INFO:tasks.workunit.client.0.smithi067.stdout:rsyslogd 3891 3895 rs:main root NOFD /proc/3891/task/3895/fd (opendir: Permission denied) 2024-07-27T03:18:57.545 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 17209 root cwd unknown /proc/17209/cwd (readlink: Permission denied) 2024-07-27T03:18:57.545 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 17209 root rtd unknown /proc/17209/root (readlink: Permission denied) 2024-07-27T03:18:57.545 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 17209 root txt unknown /proc/17209/exe (readlink: Permission denied) 2024-07-27T03:18:57.545 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 17209 root NOFD /proc/17209/fd (opendir: Permission denied) 2024-07-27T03:18:57.545 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 polkitd cwd unknown /proc/17221/cwd (readlink: Permission denied) 2024-07-27T03:18:57.546 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 polkitd rtd unknown /proc/17221/root (readlink: Permission denied) 2024-07-27T03:18:57.546 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 polkitd txt unknown /proc/17221/exe (readlink: Permission denied) 2024-07-27T03:18:57.546 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 polkitd NOFD /proc/17221/fd (opendir: Permission denied) 2024-07-27T03:18:57.546 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17224 gmain polkitd cwd unknown /proc/17221/task/17224/cwd (readlink: Permission denied) 2024-07-27T03:18:57.546 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17224 gmain polkitd rtd unknown /proc/17221/task/17224/root (readlink: Permission denied) 2024-07-27T03:18:57.546 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17224 gmain polkitd txt unknown /proc/17221/task/17224/exe (readlink: Permission denied) 2024-07-27T03:18:57.546 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17224 gmain polkitd NOFD /proc/17221/task/17224/fd (opendir: Permission denied) 2024-07-27T03:18:57.546 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17225 gdbus polkitd cwd unknown /proc/17221/task/17225/cwd (readlink: Permission denied) 2024-07-27T03:18:57.546 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17225 gdbus polkitd rtd unknown /proc/17221/task/17225/root (readlink: Permission denied) 2024-07-27T03:18:57.547 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17225 gdbus polkitd txt unknown /proc/17221/task/17225/exe (readlink: Permission denied) 2024-07-27T03:18:57.547 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17225 gdbus polkitd NOFD /proc/17221/task/17225/fd (opendir: Permission denied) 2024-07-27T03:18:57.547 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17226 JS\x20Hel polkitd cwd unknown /proc/17221/task/17226/cwd (readlink: Permission denied) 2024-07-27T03:18:57.547 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17226 JS\x20Hel polkitd rtd unknown /proc/17221/task/17226/root (readlink: Permission denied) 2024-07-27T03:18:57.547 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17226 JS\x20Hel polkitd txt unknown /proc/17221/task/17226/exe (readlink: Permission denied) 2024-07-27T03:18:57.547 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17226 JS\x20Hel polkitd NOFD /proc/17221/task/17226/fd (opendir: Permission denied) 2024-07-27T03:18:57.547 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17227 JS\x20Hel polkitd cwd unknown /proc/17221/task/17227/cwd (readlink: Permission denied) 2024-07-27T03:18:57.547 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17227 JS\x20Hel polkitd rtd unknown /proc/17221/task/17227/root (readlink: Permission denied) 2024-07-27T03:18:57.547 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17227 JS\x20Hel polkitd txt unknown /proc/17221/task/17227/exe (readlink: Permission denied) 2024-07-27T03:18:57.547 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17227 JS\x20Hel polkitd NOFD /proc/17221/task/17227/fd (opendir: Permission denied) 2024-07-27T03:18:57.547 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17228 JS\x20Hel polkitd cwd unknown /proc/17221/task/17228/cwd (readlink: Permission denied) 2024-07-27T03:18:57.547 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17228 JS\x20Hel polkitd rtd unknown /proc/17221/task/17228/root (readlink: Permission denied) 2024-07-27T03:18:57.548 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17228 JS\x20Hel polkitd txt unknown /proc/17221/task/17228/exe (readlink: Permission denied) 2024-07-27T03:18:57.548 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17228 JS\x20Hel polkitd NOFD /proc/17221/task/17228/fd (opendir: Permission denied) 2024-07-27T03:18:57.548 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17229 JS\x20Hel polkitd cwd unknown /proc/17221/task/17229/cwd (readlink: Permission denied) 2024-07-27T03:18:57.548 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17229 JS\x20Hel polkitd rtd unknown /proc/17221/task/17229/root (readlink: Permission denied) 2024-07-27T03:18:57.548 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17229 JS\x20Hel polkitd txt unknown /proc/17221/task/17229/exe (readlink: Permission denied) 2024-07-27T03:18:57.548 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17229 JS\x20Hel polkitd NOFD /proc/17221/task/17229/fd (opendir: Permission denied) 2024-07-27T03:18:57.548 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17230 JS\x20Hel polkitd cwd unknown /proc/17221/task/17230/cwd (readlink: Permission denied) 2024-07-27T03:18:57.548 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17230 JS\x20Hel polkitd rtd unknown /proc/17221/task/17230/root (readlink: Permission denied) 2024-07-27T03:18:57.548 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17230 JS\x20Hel polkitd txt unknown /proc/17221/task/17230/exe (readlink: Permission denied) 2024-07-27T03:18:57.548 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17230 JS\x20Hel polkitd NOFD /proc/17221/task/17230/fd (opendir: Permission denied) 2024-07-27T03:18:57.548 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17231 JS\x20Hel polkitd cwd unknown /proc/17221/task/17231/cwd (readlink: Permission denied) 2024-07-27T03:18:57.548 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17231 JS\x20Hel polkitd rtd unknown /proc/17221/task/17231/root (readlink: Permission denied) 2024-07-27T03:18:57.548 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17231 JS\x20Hel polkitd txt unknown /proc/17221/task/17231/exe (readlink: Permission denied) 2024-07-27T03:18:57.548 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17231 JS\x20Hel polkitd NOFD /proc/17221/task/17231/fd (opendir: Permission denied) 2024-07-27T03:18:57.549 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17232 JS\x20Hel polkitd cwd unknown /proc/17221/task/17232/cwd (readlink: Permission denied) 2024-07-27T03:18:57.549 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17232 JS\x20Hel polkitd rtd unknown /proc/17221/task/17232/root (readlink: Permission denied) 2024-07-27T03:18:57.549 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17232 JS\x20Hel polkitd txt unknown /proc/17221/task/17232/exe (readlink: Permission denied) 2024-07-27T03:18:57.549 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17232 JS\x20Hel polkitd NOFD /proc/17221/task/17232/fd (opendir: Permission denied) 2024-07-27T03:18:57.549 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17233 JS\x20Hel polkitd cwd unknown /proc/17221/task/17233/cwd (readlink: Permission denied) 2024-07-27T03:18:57.549 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17233 JS\x20Hel polkitd rtd unknown /proc/17221/task/17233/root (readlink: Permission denied) 2024-07-27T03:18:57.549 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17233 JS\x20Hel polkitd txt unknown /proc/17221/task/17233/exe (readlink: Permission denied) 2024-07-27T03:18:57.549 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17233 JS\x20Hel polkitd NOFD /proc/17221/task/17233/fd (opendir: Permission denied) 2024-07-27T03:18:57.549 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17234 polkitd polkitd cwd unknown /proc/17221/task/17234/cwd (readlink: Permission denied) 2024-07-27T03:18:57.549 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17234 polkitd polkitd rtd unknown /proc/17221/task/17234/root (readlink: Permission denied) 2024-07-27T03:18:57.549 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17234 polkitd polkitd txt unknown /proc/17221/task/17234/exe (readlink: Permission denied) 2024-07-27T03:18:57.549 INFO:tasks.workunit.client.0.smithi067.stdout:polkitd 17221 17234 polkitd polkitd NOFD /proc/17221/task/17234/fd (opendir: Permission denied) 2024-07-27T03:18:57.549 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 19592 root cwd unknown /proc/19592/cwd (readlink: Permission denied) 2024-07-27T03:18:57.550 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 19592 root rtd unknown /proc/19592/root (readlink: Permission denied) 2024-07-27T03:18:57.550 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 19592 root txt unknown /proc/19592/exe (readlink: Permission denied) 2024-07-27T03:18:57.550 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 19592 root NOFD /proc/19592/fd (opendir: Permission denied) 2024-07-27T03:18:57.550 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21465 root cwd unknown /proc/21465/cwd (readlink: Permission denied) 2024-07-27T03:18:57.550 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21465 root rtd unknown /proc/21465/root (readlink: Permission denied) 2024-07-27T03:18:57.550 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21465 root txt unknown /proc/21465/exe (readlink: Permission denied) 2024-07-27T03:18:57.550 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21465 root NOFD /proc/21465/fd (opendir: Permission denied) 2024-07-27T03:18:57.550 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21494 root cwd unknown /proc/21494/cwd (readlink: Permission denied) 2024-07-27T03:18:57.550 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21494 root rtd unknown /proc/21494/root (readlink: Permission denied) 2024-07-27T03:18:57.550 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21494 root txt unknown /proc/21494/exe (readlink: Permission denied) 2024-07-27T03:18:57.550 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21494 root NOFD /proc/21494/fd (opendir: Permission denied) 2024-07-27T03:18:57.550 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21523 root cwd unknown /proc/21523/cwd (readlink: Permission denied) 2024-07-27T03:18:57.550 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21523 root rtd unknown /proc/21523/root (readlink: Permission denied) 2024-07-27T03:18:57.551 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21523 root txt unknown /proc/21523/exe (readlink: Permission denied) 2024-07-27T03:18:57.551 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21523 root NOFD /proc/21523/fd (opendir: Permission denied) 2024-07-27T03:18:57.551 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21552 root cwd unknown /proc/21552/cwd (readlink: Permission denied) 2024-07-27T03:18:57.551 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21552 root rtd unknown /proc/21552/root (readlink: Permission denied) 2024-07-27T03:18:57.551 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21552 root txt unknown /proc/21552/exe (readlink: Permission denied) 2024-07-27T03:18:57.551 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21552 root NOFD /proc/21552/fd (opendir: Permission denied) 2024-07-27T03:18:57.551 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21581 root cwd unknown /proc/21581/cwd (readlink: Permission denied) 2024-07-27T03:18:57.551 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21581 root rtd unknown /proc/21581/root (readlink: Permission denied) 2024-07-27T03:18:57.551 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21581 root txt unknown /proc/21581/exe (readlink: Permission denied) 2024-07-27T03:18:57.551 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21581 root NOFD /proc/21581/fd (opendir: Permission denied) 2024-07-27T03:18:57.551 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21745 root cwd unknown /proc/21745/cwd (readlink: Permission denied) 2024-07-27T03:18:57.551 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21745 root rtd unknown /proc/21745/root (readlink: Permission denied) 2024-07-27T03:18:57.551 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21745 root txt unknown /proc/21745/exe (readlink: Permission denied) 2024-07-27T03:18:57.551 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21745 root NOFD /proc/21745/fd (opendir: Permission denied) 2024-07-27T03:18:57.552 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21746 root cwd unknown /proc/21746/cwd (readlink: Permission denied) 2024-07-27T03:18:57.552 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21746 root rtd unknown /proc/21746/root (readlink: Permission denied) 2024-07-27T03:18:57.552 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21746 root txt unknown /proc/21746/exe (readlink: Permission denied) 2024-07-27T03:18:57.552 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21746 root NOFD /proc/21746/fd (opendir: Permission denied) 2024-07-27T03:18:57.552 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21747 root cwd unknown /proc/21747/cwd (readlink: Permission denied) 2024-07-27T03:18:57.552 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21747 root rtd unknown /proc/21747/root (readlink: Permission denied) 2024-07-27T03:18:57.552 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21747 root txt unknown /proc/21747/exe (readlink: Permission denied) 2024-07-27T03:18:57.552 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21747 root NOFD /proc/21747/fd (opendir: Permission denied) 2024-07-27T03:18:57.552 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21748 root cwd unknown /proc/21748/cwd (readlink: Permission denied) 2024-07-27T03:18:57.552 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21748 root rtd unknown /proc/21748/root (readlink: Permission denied) 2024-07-27T03:18:57.552 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21748 root txt unknown /proc/21748/exe (readlink: Permission denied) 2024-07-27T03:18:57.552 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21748 root NOFD /proc/21748/fd (opendir: Permission denied) 2024-07-27T03:18:57.552 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21749 root cwd unknown /proc/21749/cwd (readlink: Permission denied) 2024-07-27T03:18:57.553 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21749 root rtd unknown /proc/21749/root (readlink: Permission denied) 2024-07-27T03:18:57.553 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21749 root txt unknown /proc/21749/exe (readlink: Permission denied) 2024-07-27T03:18:57.553 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21749 root NOFD /proc/21749/fd (opendir: Permission denied) 2024-07-27T03:18:57.553 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21750 root cwd unknown /proc/21750/cwd (readlink: Permission denied) 2024-07-27T03:18:57.553 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21750 root rtd unknown /proc/21750/root (readlink: Permission denied) 2024-07-27T03:18:57.553 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21750 root txt unknown /proc/21750/exe (readlink: Permission denied) 2024-07-27T03:18:57.553 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21750 root NOFD /proc/21750/fd (opendir: Permission denied) 2024-07-27T03:18:57.553 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21751 root cwd unknown /proc/21751/cwd (readlink: Permission denied) 2024-07-27T03:18:57.553 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21751 root rtd unknown /proc/21751/root (readlink: Permission denied) 2024-07-27T03:18:57.553 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21751 root txt unknown /proc/21751/exe (readlink: Permission denied) 2024-07-27T03:18:57.553 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21751 root NOFD /proc/21751/fd (opendir: Permission denied) 2024-07-27T03:18:57.553 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21752 root cwd unknown /proc/21752/cwd (readlink: Permission denied) 2024-07-27T03:18:57.553 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21752 root rtd unknown /proc/21752/root (readlink: Permission denied) 2024-07-27T03:18:57.554 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21752 root txt unknown /proc/21752/exe (readlink: Permission denied) 2024-07-27T03:18:57.554 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21752 root NOFD /proc/21752/fd (opendir: Permission denied) 2024-07-27T03:18:57.554 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21753 root cwd unknown /proc/21753/cwd (readlink: Permission denied) 2024-07-27T03:18:57.554 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21753 root rtd unknown /proc/21753/root (readlink: Permission denied) 2024-07-27T03:18:57.554 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21753 root txt unknown /proc/21753/exe (readlink: Permission denied) 2024-07-27T03:18:57.554 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 21753 root NOFD /proc/21753/fd (opendir: Permission denied) 2024-07-27T03:18:57.554 INFO:tasks.workunit.client.0.smithi067.stdout:xfsaild/d 21754 root cwd unknown /proc/21754/cwd (readlink: Permission denied) 2024-07-27T03:18:57.554 INFO:tasks.workunit.client.0.smithi067.stdout:xfsaild/d 21754 root rtd unknown /proc/21754/root (readlink: Permission denied) 2024-07-27T03:18:57.554 INFO:tasks.workunit.client.0.smithi067.stdout:xfsaild/d 21754 root txt unknown /proc/21754/exe (readlink: Permission denied) 2024-07-27T03:18:57.554 INFO:tasks.workunit.client.0.smithi067.stdout:xfsaild/d 21754 root NOFD /proc/21754/fd (opendir: Permission denied) 2024-07-27T03:18:57.554 INFO:tasks.workunit.client.0.smithi067.stdout:anacron 22295 root cwd unknown /proc/22295/cwd (readlink: Permission denied) 2024-07-27T03:18:57.554 INFO:tasks.workunit.client.0.smithi067.stdout:anacron 22295 root rtd unknown /proc/22295/root (readlink: Permission denied) 2024-07-27T03:18:57.554 INFO:tasks.workunit.client.0.smithi067.stdout:anacron 22295 root txt unknown /proc/22295/exe (readlink: Permission denied) 2024-07-27T03:18:57.555 INFO:tasks.workunit.client.0.smithi067.stdout:anacron 22295 root NOFD /proc/22295/fd (opendir: Permission denied) 2024-07-27T03:18:57.555 INFO:tasks.workunit.client.0.smithi067.stdout:sshd 24206 root cwd unknown /proc/24206/cwd (readlink: Permission denied) 2024-07-27T03:18:57.555 INFO:tasks.workunit.client.0.smithi067.stdout:sshd 24206 root rtd unknown /proc/24206/root (readlink: Permission denied) 2024-07-27T03:18:57.555 INFO:tasks.workunit.client.0.smithi067.stdout:sshd 24206 root txt unknown /proc/24206/exe (readlink: Permission denied) 2024-07-27T03:18:57.555 INFO:tasks.workunit.client.0.smithi067.stdout:sshd 24206 root NOFD /proc/24206/fd (opendir: Permission denied) 2024-07-27T03:18:57.555 INFO:tasks.workunit.client.0.smithi067.stdout:sshd 24209 ubuntu cwd unknown /proc/24209/cwd (readlink: Permission denied) 2024-07-27T03:18:57.555 INFO:tasks.workunit.client.0.smithi067.stdout:sshd 24209 ubuntu rtd unknown /proc/24209/root (readlink: Permission denied) 2024-07-27T03:18:57.555 INFO:tasks.workunit.client.0.smithi067.stdout:sshd 24209 ubuntu txt unknown /proc/24209/exe (readlink: Permission denied) 2024-07-27T03:18:57.555 INFO:tasks.workunit.client.0.smithi067.stdout:sshd 24209 ubuntu NOFD /proc/24209/fd (opendir: Permission denied) 2024-07-27T03:18:57.555 INFO:tasks.workunit.client.0.smithi067.stdout:chronyd 24269 chrony cwd unknown /proc/24269/cwd (readlink: Permission denied) 2024-07-27T03:18:57.555 INFO:tasks.workunit.client.0.smithi067.stdout:chronyd 24269 chrony rtd unknown /proc/24269/root (readlink: Permission denied) 2024-07-27T03:18:57.555 INFO:tasks.workunit.client.0.smithi067.stdout:chronyd 24269 chrony txt unknown /proc/24269/exe (readlink: Permission denied) 2024-07-27T03:18:57.555 INFO:tasks.workunit.client.0.smithi067.stdout:chronyd 24269 chrony NOFD /proc/24269/fd (opendir: Permission denied) 2024-07-27T03:18:57.555 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-cras 24567 ceph cwd unknown /proc/24567/cwd (readlink: Permission denied) 2024-07-27T03:18:57.556 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-cras 24567 ceph rtd unknown /proc/24567/root (readlink: Permission denied) 2024-07-27T03:18:57.556 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-cras 24567 ceph txt unknown /proc/24567/exe (readlink: Permission denied) 2024-07-27T03:18:57.556 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-cras 24567 ceph NOFD /proc/24567/fd (opendir: Permission denied) 2024-07-27T03:18:57.556 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/2 26414 root cwd unknown /proc/26414/cwd (readlink: Permission denied) 2024-07-27T03:18:57.556 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/2 26414 root rtd unknown /proc/26414/root (readlink: Permission denied) 2024-07-27T03:18:57.556 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/2 26414 root txt unknown /proc/26414/exe (readlink: Permission denied) 2024-07-27T03:18:57.556 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/2 26414 root NOFD /proc/26414/fd (opendir: Permission denied) 2024-07-27T03:18:57.556 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 31820 root cwd unknown /proc/31820/cwd (readlink: Permission denied) 2024-07-27T03:18:57.556 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 31820 root rtd unknown /proc/31820/root (readlink: Permission denied) 2024-07-27T03:18:57.556 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 31820 root txt unknown /proc/31820/exe (readlink: Permission denied) 2024-07-27T03:18:57.556 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 31820 root NOFD /proc/31820/fd (opendir: Permission denied) 2024-07-27T03:18:57.556 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 31837 root cwd unknown /proc/31837/cwd (readlink: Permission denied) 2024-07-27T03:18:57.556 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 31837 root rtd unknown /proc/31837/root (readlink: Permission denied) 2024-07-27T03:18:57.556 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 31837 root txt unknown /proc/31837/exe (readlink: Permission denied) 2024-07-27T03:18:57.557 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 31837 root NOFD /proc/31837/fd (opendir: Permission denied) 2024-07-27T03:18:57.557 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 31838 root cwd unknown /proc/31838/cwd (readlink: Permission denied) 2024-07-27T03:18:57.557 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 31838 root rtd unknown /proc/31838/root (readlink: Permission denied) 2024-07-27T03:18:57.557 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 31838 root txt unknown /proc/31838/exe (readlink: Permission denied) 2024-07-27T03:18:57.557 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 31838 root NOFD /proc/31838/fd (opendir: Permission denied) 2024-07-27T03:18:57.557 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 31855 root cwd unknown /proc/31855/cwd (readlink: Permission denied) 2024-07-27T03:18:57.557 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 31855 root rtd unknown /proc/31855/root (readlink: Permission denied) 2024-07-27T03:18:57.557 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 31855 root txt unknown /proc/31855/exe (readlink: Permission denied) 2024-07-27T03:18:57.557 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 31855 root NOFD /proc/31855/fd (opendir: Permission denied) 2024-07-27T03:18:57.557 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 32761 root cwd unknown /proc/32761/cwd (readlink: Permission denied) 2024-07-27T03:18:57.557 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 32761 root rtd unknown /proc/32761/root (readlink: Permission denied) 2024-07-27T03:18:57.557 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 32761 root txt unknown /proc/32761/exe (readlink: Permission denied) 2024-07-27T03:18:57.557 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 32761 root NOFD /proc/32761/fd (opendir: Permission denied) 2024-07-27T03:18:57.558 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 32809 root cwd unknown /proc/32809/cwd (readlink: Permission denied) 2024-07-27T03:18:57.558 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 32809 root rtd unknown /proc/32809/root (readlink: Permission denied) 2024-07-27T03:18:57.558 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 32809 root txt unknown /proc/32809/exe (readlink: Permission denied) 2024-07-27T03:18:57.558 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 32809 root NOFD /proc/32809/fd (opendir: Permission denied) 2024-07-27T03:18:57.558 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 32811 root cwd unknown /proc/32811/cwd (readlink: Permission denied) 2024-07-27T03:18:57.558 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 32811 root rtd unknown /proc/32811/root (readlink: Permission denied) 2024-07-27T03:18:57.558 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 32811 root txt unknown /proc/32811/exe (readlink: Permission denied) 2024-07-27T03:18:57.558 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 32811 root NOFD /proc/32811/fd (opendir: Permission denied) 2024-07-27T03:18:57.558 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 ceph cwd unknown /proc/32814/cwd (readlink: Permission denied) 2024-07-27T03:18:57.558 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 ceph rtd unknown /proc/32814/root (readlink: Permission denied) 2024-07-27T03:18:57.558 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 ceph txt unknown /proc/32814/exe (readlink: Permission denied) 2024-07-27T03:18:57.558 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 ceph NOFD /proc/32814/fd (opendir: Permission denied) 2024-07-27T03:18:57.558 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32815 log ceph cwd unknown /proc/32814/task/32815/cwd (readlink: Permission denied) 2024-07-27T03:18:57.559 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32815 log ceph rtd unknown /proc/32814/task/32815/root (readlink: Permission denied) 2024-07-27T03:18:57.559 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32815 log ceph txt unknown /proc/32814/task/32815/exe (readlink: Permission denied) 2024-07-27T03:18:57.559 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32815 log ceph NOFD /proc/32814/task/32815/fd (opendir: Permission denied) 2024-07-27T03:18:57.559 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32816 service ceph cwd unknown /proc/32814/task/32816/cwd (readlink: Permission denied) 2024-07-27T03:18:57.559 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32816 service ceph rtd unknown /proc/32814/task/32816/root (readlink: Permission denied) 2024-07-27T03:18:57.559 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32816 service ceph txt unknown /proc/32814/task/32816/exe (readlink: Permission denied) 2024-07-27T03:18:57.559 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32816 service ceph NOFD /proc/32814/task/32816/fd (opendir: Permission denied) 2024-07-27T03:18:57.559 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32817 admin_soc ceph cwd unknown /proc/32814/task/32817/cwd (readlink: Permission denied) 2024-07-27T03:18:57.559 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32817 admin_soc ceph rtd unknown /proc/32814/task/32817/root (readlink: Permission denied) 2024-07-27T03:18:57.559 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32817 admin_soc ceph txt unknown /proc/32814/task/32817/exe (readlink: Permission denied) 2024-07-27T03:18:57.559 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32817 admin_soc ceph NOFD /proc/32814/task/32817/fd (opendir: Permission denied) 2024-07-27T03:18:57.559 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32818 signal_ha ceph cwd unknown /proc/32814/task/32818/cwd (readlink: Permission denied) 2024-07-27T03:18:57.559 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32818 signal_ha ceph rtd unknown /proc/32814/task/32818/root (readlink: Permission denied) 2024-07-27T03:18:57.559 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32818 signal_ha ceph txt unknown /proc/32814/task/32818/exe (readlink: Permission denied) 2024-07-27T03:18:57.560 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32818 signal_ha ceph NOFD /proc/32814/task/32818/fd (opendir: Permission denied) 2024-07-27T03:18:57.560 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32819 rocksdb:l ceph cwd unknown /proc/32814/task/32819/cwd (readlink: Permission denied) 2024-07-27T03:18:57.560 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32819 rocksdb:l ceph rtd unknown /proc/32814/task/32819/root (readlink: Permission denied) 2024-07-27T03:18:57.560 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32819 rocksdb:l ceph txt unknown /proc/32814/task/32819/exe (readlink: Permission denied) 2024-07-27T03:18:57.560 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32819 rocksdb:l ceph NOFD /proc/32814/task/32819/fd (opendir: Permission denied) 2024-07-27T03:18:57.560 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32820 rocksdb:h ceph cwd unknown /proc/32814/task/32820/cwd (readlink: Permission denied) 2024-07-27T03:18:57.560 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32820 rocksdb:h ceph rtd unknown /proc/32814/task/32820/root (readlink: Permission denied) 2024-07-27T03:18:57.560 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32820 rocksdb:h ceph txt unknown /proc/32814/task/32820/exe (readlink: Permission denied) 2024-07-27T03:18:57.560 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32820 rocksdb:h ceph NOFD /proc/32814/task/32820/fd (opendir: Permission denied) 2024-07-27T03:18:57.560 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32836 ceph-mon ceph cwd unknown /proc/32814/task/32836/cwd (readlink: Permission denied) 2024-07-27T03:18:57.560 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32836 ceph-mon ceph rtd unknown /proc/32814/task/32836/root (readlink: Permission denied) 2024-07-27T03:18:57.560 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32836 ceph-mon ceph txt unknown /proc/32814/task/32836/exe (readlink: Permission denied) 2024-07-27T03:18:57.560 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32836 ceph-mon ceph NOFD /proc/32814/task/32836/fd (opendir: Permission denied) 2024-07-27T03:18:57.560 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32837 fn_monsto ceph cwd unknown /proc/32814/task/32837/cwd (readlink: Permission denied) 2024-07-27T03:18:57.561 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32837 fn_monsto ceph rtd unknown /proc/32814/task/32837/root (readlink: Permission denied) 2024-07-27T03:18:57.561 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32837 fn_monsto ceph txt unknown /proc/32814/task/32837/exe (readlink: Permission denied) 2024-07-27T03:18:57.561 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32837 fn_monsto ceph NOFD /proc/32814/task/32837/fd (opendir: Permission denied) 2024-07-27T03:18:57.561 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32838 msgr-work ceph cwd unknown /proc/32814/task/32838/cwd (readlink: Permission denied) 2024-07-27T03:18:57.561 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32838 msgr-work ceph rtd unknown /proc/32814/task/32838/root (readlink: Permission denied) 2024-07-27T03:18:57.561 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32838 msgr-work ceph txt unknown /proc/32814/task/32838/exe (readlink: Permission denied) 2024-07-27T03:18:57.561 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32838 msgr-work ceph NOFD /proc/32814/task/32838/fd (opendir: Permission denied) 2024-07-27T03:18:57.561 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32839 msgr-work ceph cwd unknown /proc/32814/task/32839/cwd (readlink: Permission denied) 2024-07-27T03:18:57.561 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32839 msgr-work ceph rtd unknown /proc/32814/task/32839/root (readlink: Permission denied) 2024-07-27T03:18:57.561 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32839 msgr-work ceph txt unknown /proc/32814/task/32839/exe (readlink: Permission denied) 2024-07-27T03:18:57.561 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32839 msgr-work ceph NOFD /proc/32814/task/32839/fd (opendir: Permission denied) 2024-07-27T03:18:57.561 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32840 msgr-work ceph cwd unknown /proc/32814/task/32840/cwd (readlink: Permission denied) 2024-07-27T03:18:57.561 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32840 msgr-work ceph rtd unknown /proc/32814/task/32840/root (readlink: Permission denied) 2024-07-27T03:18:57.561 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32840 msgr-work ceph txt unknown /proc/32814/task/32840/exe (readlink: Permission denied) 2024-07-27T03:18:57.562 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32840 msgr-work ceph NOFD /proc/32814/task/32840/fd (opendir: Permission denied) 2024-07-27T03:18:57.562 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32841 OpHistory ceph cwd unknown /proc/32814/task/32841/cwd (readlink: Permission denied) 2024-07-27T03:18:57.562 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32841 OpHistory ceph rtd unknown /proc/32814/task/32841/root (readlink: Permission denied) 2024-07-27T03:18:57.562 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32841 OpHistory ceph txt unknown /proc/32814/task/32841/exe (readlink: Permission denied) 2024-07-27T03:18:57.562 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32841 OpHistory ceph NOFD /proc/32814/task/32841/fd (opendir: Permission denied) 2024-07-27T03:18:57.562 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32842 fin ceph cwd unknown /proc/32814/task/32842/cwd (readlink: Permission denied) 2024-07-27T03:18:57.562 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32842 fin ceph rtd unknown /proc/32814/task/32842/root (readlink: Permission denied) 2024-07-27T03:18:57.562 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32842 fin ceph txt unknown /proc/32814/task/32842/exe (readlink: Permission denied) 2024-07-27T03:18:57.562 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32842 fin ceph NOFD /proc/32814/task/32842/fd (opendir: Permission denied) 2024-07-27T03:18:57.562 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32843 safe_time ceph cwd unknown /proc/32814/task/32843/cwd (readlink: Permission denied) 2024-07-27T03:18:57.562 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32843 safe_time ceph rtd unknown /proc/32814/task/32843/root (readlink: Permission denied) 2024-07-27T03:18:57.562 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32843 safe_time ceph txt unknown /proc/32814/task/32843/exe (readlink: Permission denied) 2024-07-27T03:18:57.562 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32843 safe_time ceph NOFD /proc/32814/task/32843/fd (opendir: Permission denied) 2024-07-27T03:18:57.563 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32844 cpu_tp ceph cwd unknown /proc/32814/task/32844/cwd (readlink: Permission denied) 2024-07-27T03:18:57.563 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32844 cpu_tp ceph rtd unknown /proc/32814/task/32844/root (readlink: Permission denied) 2024-07-27T03:18:57.563 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32844 cpu_tp ceph txt unknown /proc/32814/task/32844/exe (readlink: Permission denied) 2024-07-27T03:18:57.563 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32844 cpu_tp ceph NOFD /proc/32814/task/32844/fd (opendir: Permission denied) 2024-07-27T03:18:57.563 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32845 cpu_tp ceph cwd unknown /proc/32814/task/32845/cwd (readlink: Permission denied) 2024-07-27T03:18:57.563 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32845 cpu_tp ceph rtd unknown /proc/32814/task/32845/root (readlink: Permission denied) 2024-07-27T03:18:57.563 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32845 cpu_tp ceph txt unknown /proc/32814/task/32845/exe (readlink: Permission denied) 2024-07-27T03:18:57.563 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32845 cpu_tp ceph NOFD /proc/32814/task/32845/fd (opendir: Permission denied) 2024-07-27T03:18:57.563 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32846 cpu_tp ceph cwd unknown /proc/32814/task/32846/cwd (readlink: Permission denied) 2024-07-27T03:18:57.563 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32846 cpu_tp ceph rtd unknown /proc/32814/task/32846/root (readlink: Permission denied) 2024-07-27T03:18:57.563 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32846 cpu_tp ceph txt unknown /proc/32814/task/32846/exe (readlink: Permission denied) 2024-07-27T03:18:57.563 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32846 cpu_tp ceph NOFD /proc/32814/task/32846/fd (opendir: Permission denied) 2024-07-27T03:18:57.563 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32847 cpu_tp ceph cwd unknown /proc/32814/task/32847/cwd (readlink: Permission denied) 2024-07-27T03:18:57.563 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32847 cpu_tp ceph rtd unknown /proc/32814/task/32847/root (readlink: Permission denied) 2024-07-27T03:18:57.564 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32847 cpu_tp ceph txt unknown /proc/32814/task/32847/exe (readlink: Permission denied) 2024-07-27T03:18:57.564 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32847 cpu_tp ceph NOFD /proc/32814/task/32847/fd (opendir: Permission denied) 2024-07-27T03:18:57.564 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32848 ms_dispat ceph cwd unknown /proc/32814/task/32848/cwd (readlink: Permission denied) 2024-07-27T03:18:57.564 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32848 ms_dispat ceph rtd unknown /proc/32814/task/32848/root (readlink: Permission denied) 2024-07-27T03:18:57.564 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32848 ms_dispat ceph txt unknown /proc/32814/task/32848/exe (readlink: Permission denied) 2024-07-27T03:18:57.564 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32848 ms_dispat ceph NOFD /proc/32814/task/32848/fd (opendir: Permission denied) 2024-07-27T03:18:57.564 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32849 ms_local ceph cwd unknown /proc/32814/task/32849/cwd (readlink: Permission denied) 2024-07-27T03:18:57.564 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32849 ms_local ceph rtd unknown /proc/32814/task/32849/root (readlink: Permission denied) 2024-07-27T03:18:57.564 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32849 ms_local ceph txt unknown /proc/32814/task/32849/exe (readlink: Permission denied) 2024-07-27T03:18:57.564 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32849 ms_local ceph NOFD /proc/32814/task/32849/fd (opendir: Permission denied) 2024-07-27T03:18:57.564 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32850 safe_time ceph cwd unknown /proc/32814/task/32850/cwd (readlink: Permission denied) 2024-07-27T03:18:57.564 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32850 safe_time ceph rtd unknown /proc/32814/task/32850/root (readlink: Permission denied) 2024-07-27T03:18:57.564 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32850 safe_time ceph txt unknown /proc/32814/task/32850/exe (readlink: Permission denied) 2024-07-27T03:18:57.564 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32850 safe_time ceph NOFD /proc/32814/task/32850/fd (opendir: Permission denied) 2024-07-27T03:18:57.565 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32851 ms_dispat ceph cwd unknown /proc/32814/task/32851/cwd (readlink: Permission denied) 2024-07-27T03:18:57.565 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32851 ms_dispat ceph rtd unknown /proc/32814/task/32851/root (readlink: Permission denied) 2024-07-27T03:18:57.565 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32851 ms_dispat ceph txt unknown /proc/32814/task/32851/exe (readlink: Permission denied) 2024-07-27T03:18:57.565 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32851 ms_dispat ceph NOFD /proc/32814/task/32851/fd (opendir: Permission denied) 2024-07-27T03:18:57.565 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32852 ms_local ceph cwd unknown /proc/32814/task/32852/cwd (readlink: Permission denied) 2024-07-27T03:18:57.565 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32852 ms_local ceph rtd unknown /proc/32814/task/32852/root (readlink: Permission denied) 2024-07-27T03:18:57.565 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32852 ms_local ceph txt unknown /proc/32814/task/32852/exe (readlink: Permission denied) 2024-07-27T03:18:57.565 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 32852 ms_local ceph NOFD /proc/32814/task/32852/fd (opendir: Permission denied) 2024-07-27T03:18:57.565 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 75368 rstore_co ceph cwd unknown /proc/32814/task/75368/cwd (readlink: Permission denied) 2024-07-27T03:18:57.565 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 75368 rstore_co ceph rtd unknown /proc/32814/task/75368/root (readlink: Permission denied) 2024-07-27T03:18:57.565 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 75368 rstore_co ceph txt unknown /proc/32814/task/75368/exe (readlink: Permission denied) 2024-07-27T03:18:57.565 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mon 32814 75368 rstore_co ceph NOFD /proc/32814/task/75368/fd (opendir: Permission denied) 2024-07-27T03:18:57.565 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 33058 root cwd unknown /proc/33058/cwd (readlink: Permission denied) 2024-07-27T03:18:57.566 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 33058 root rtd unknown /proc/33058/root (readlink: Permission denied) 2024-07-27T03:18:57.566 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 33058 root txt unknown /proc/33058/exe (readlink: Permission denied) 2024-07-27T03:18:57.566 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 33058 root NOFD /proc/33058/fd (opendir: Permission denied) 2024-07-27T03:18:57.566 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 33060 root cwd unknown /proc/33060/cwd (readlink: Permission denied) 2024-07-27T03:18:57.566 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 33060 root rtd unknown /proc/33060/root (readlink: Permission denied) 2024-07-27T03:18:57.566 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 33060 root txt unknown /proc/33060/exe (readlink: Permission denied) 2024-07-27T03:18:57.566 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 33060 root NOFD /proc/33060/fd (opendir: Permission denied) 2024-07-27T03:18:57.566 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 ceph cwd unknown /proc/33062/cwd (readlink: Permission denied) 2024-07-27T03:18:57.566 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 ceph rtd unknown /proc/33062/root (readlink: Permission denied) 2024-07-27T03:18:57.566 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 ceph txt unknown /proc/33062/exe (readlink: Permission denied) 2024-07-27T03:18:57.566 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 ceph NOFD /proc/33062/fd (opendir: Permission denied) 2024-07-27T03:18:57.566 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35386 log ceph cwd unknown /proc/33062/task/35386/cwd (readlink: Permission denied) 2024-07-27T03:18:57.566 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35386 log ceph rtd unknown /proc/33062/task/35386/root (readlink: Permission denied) 2024-07-27T03:18:57.566 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35386 log ceph txt unknown /proc/33062/task/35386/exe (readlink: Permission denied) 2024-07-27T03:18:57.567 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35386 log ceph NOFD /proc/33062/task/35386/fd (opendir: Permission denied) 2024-07-27T03:18:57.567 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35388 msgr-work ceph cwd unknown /proc/33062/task/35388/cwd (readlink: Permission denied) 2024-07-27T03:18:57.567 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35388 msgr-work ceph rtd unknown /proc/33062/task/35388/root (readlink: Permission denied) 2024-07-27T03:18:57.567 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35388 msgr-work ceph txt unknown /proc/33062/task/35388/exe (readlink: Permission denied) 2024-07-27T03:18:57.567 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35388 msgr-work ceph NOFD /proc/33062/task/35388/fd (opendir: Permission denied) 2024-07-27T03:18:57.567 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35389 msgr-work ceph cwd unknown /proc/33062/task/35389/cwd (readlink: Permission denied) 2024-07-27T03:18:57.567 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35389 msgr-work ceph rtd unknown /proc/33062/task/35389/root (readlink: Permission denied) 2024-07-27T03:18:57.567 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35389 msgr-work ceph txt unknown /proc/33062/task/35389/exe (readlink: Permission denied) 2024-07-27T03:18:57.567 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35389 msgr-work ceph NOFD /proc/33062/task/35389/fd (opendir: Permission denied) 2024-07-27T03:18:57.567 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35390 msgr-work ceph cwd unknown /proc/33062/task/35390/cwd (readlink: Permission denied) 2024-07-27T03:18:57.567 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35390 msgr-work ceph rtd unknown /proc/33062/task/35390/root (readlink: Permission denied) 2024-07-27T03:18:57.567 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35390 msgr-work ceph txt unknown /proc/33062/task/35390/exe (readlink: Permission denied) 2024-07-27T03:18:57.567 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35390 msgr-work ceph NOFD /proc/33062/task/35390/fd (opendir: Permission denied) 2024-07-27T03:18:57.568 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35394 service ceph cwd unknown /proc/33062/task/35394/cwd (readlink: Permission denied) 2024-07-27T03:18:57.568 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35394 service ceph rtd unknown /proc/33062/task/35394/root (readlink: Permission denied) 2024-07-27T03:18:57.568 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35394 service ceph txt unknown /proc/33062/task/35394/exe (readlink: Permission denied) 2024-07-27T03:18:57.568 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35394 service ceph NOFD /proc/33062/task/35394/fd (opendir: Permission denied) 2024-07-27T03:18:57.568 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35395 admin_soc ceph cwd unknown /proc/33062/task/35395/cwd (readlink: Permission denied) 2024-07-27T03:18:57.568 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35395 admin_soc ceph rtd unknown /proc/33062/task/35395/root (readlink: Permission denied) 2024-07-27T03:18:57.568 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35395 admin_soc ceph txt unknown /proc/33062/task/35395/exe (readlink: Permission denied) 2024-07-27T03:18:57.568 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35395 admin_soc ceph NOFD /proc/33062/task/35395/fd (opendir: Permission denied) 2024-07-27T03:18:57.568 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35396 ceph_time ceph cwd unknown /proc/33062/task/35396/cwd (readlink: Permission denied) 2024-07-27T03:18:57.568 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35396 ceph_time ceph rtd unknown /proc/33062/task/35396/root (readlink: Permission denied) 2024-07-27T03:18:57.568 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35396 ceph_time ceph txt unknown /proc/33062/task/35396/exe (readlink: Permission denied) 2024-07-27T03:18:57.568 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35396 ceph_time ceph NOFD /proc/33062/task/35396/fd (opendir: Permission denied) 2024-07-27T03:18:57.568 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35397 fn_anonym ceph cwd unknown /proc/33062/task/35397/cwd (readlink: Permission denied) 2024-07-27T03:18:57.569 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35397 fn_anonym ceph rtd unknown /proc/33062/task/35397/root (readlink: Permission denied) 2024-07-27T03:18:57.569 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35397 fn_anonym ceph txt unknown /proc/33062/task/35397/exe (readlink: Permission denied) 2024-07-27T03:18:57.569 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35397 fn_anonym ceph NOFD /proc/33062/task/35397/fd (opendir: Permission denied) 2024-07-27T03:18:57.569 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35398 signal_ha ceph cwd unknown /proc/33062/task/35398/cwd (readlink: Permission denied) 2024-07-27T03:18:57.569 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35398 signal_ha ceph rtd unknown /proc/33062/task/35398/root (readlink: Permission denied) 2024-07-27T03:18:57.569 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35398 signal_ha ceph txt unknown /proc/33062/task/35398/exe (readlink: Permission denied) 2024-07-27T03:18:57.569 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35398 signal_ha ceph NOFD /proc/33062/task/35398/fd (opendir: Permission denied) 2024-07-27T03:18:57.569 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35399 mgrsb-fin ceph cwd unknown /proc/33062/task/35399/cwd (readlink: Permission denied) 2024-07-27T03:18:57.569 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35399 mgrsb-fin ceph rtd unknown /proc/33062/task/35399/root (readlink: Permission denied) 2024-07-27T03:18:57.569 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35399 mgrsb-fin ceph txt unknown /proc/33062/task/35399/exe (readlink: Permission denied) 2024-07-27T03:18:57.569 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35399 mgrsb-fin ceph NOFD /proc/33062/task/35399/fd (opendir: Permission denied) 2024-07-27T03:18:57.569 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35400 ms_dispat ceph cwd unknown /proc/33062/task/35400/cwd (readlink: Permission denied) 2024-07-27T03:18:57.569 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35400 ms_dispat ceph rtd unknown /proc/33062/task/35400/root (readlink: Permission denied) 2024-07-27T03:18:57.570 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35400 ms_dispat ceph txt unknown /proc/33062/task/35400/exe (readlink: Permission denied) 2024-07-27T03:18:57.570 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35400 ms_dispat ceph NOFD /proc/33062/task/35400/fd (opendir: Permission denied) 2024-07-27T03:18:57.570 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35401 ms_local ceph cwd unknown /proc/33062/task/35401/cwd (readlink: Permission denied) 2024-07-27T03:18:57.570 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35401 ms_local ceph rtd unknown /proc/33062/task/35401/root (readlink: Permission denied) 2024-07-27T03:18:57.570 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35401 ms_local ceph txt unknown /proc/33062/task/35401/exe (readlink: Permission denied) 2024-07-27T03:18:57.570 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35401 ms_local ceph NOFD /proc/33062/task/35401/fd (opendir: Permission denied) 2024-07-27T03:18:57.570 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35402 io_contex ceph cwd unknown /proc/33062/task/35402/cwd (readlink: Permission denied) 2024-07-27T03:18:57.570 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35402 io_contex ceph rtd unknown /proc/33062/task/35402/root (readlink: Permission denied) 2024-07-27T03:18:57.570 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35402 io_contex ceph txt unknown /proc/33062/task/35402/exe (readlink: Permission denied) 2024-07-27T03:18:57.570 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35402 io_contex ceph NOFD /proc/33062/task/35402/fd (opendir: Permission denied) 2024-07-27T03:18:57.570 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35403 io_contex ceph cwd unknown /proc/33062/task/35403/cwd (readlink: Permission denied) 2024-07-27T03:18:57.570 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35403 io_contex ceph rtd unknown /proc/33062/task/35403/root (readlink: Permission denied) 2024-07-27T03:18:57.570 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35403 io_contex ceph txt unknown /proc/33062/task/35403/exe (readlink: Permission denied) 2024-07-27T03:18:57.571 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35403 io_contex ceph NOFD /proc/33062/task/35403/fd (opendir: Permission denied) 2024-07-27T03:18:57.571 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35404 safe_time ceph cwd unknown /proc/33062/task/35404/cwd (readlink: Permission denied) 2024-07-27T03:18:57.571 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35404 safe_time ceph rtd unknown /proc/33062/task/35404/root (readlink: Permission denied) 2024-07-27T03:18:57.571 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35404 safe_time ceph txt unknown /proc/33062/task/35404/exe (readlink: Permission denied) 2024-07-27T03:18:57.571 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35404 safe_time ceph NOFD /proc/33062/task/35404/fd (opendir: Permission denied) 2024-07-27T03:18:57.571 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35405 safe_time ceph cwd unknown /proc/33062/task/35405/cwd (readlink: Permission denied) 2024-07-27T03:18:57.571 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35405 safe_time ceph rtd unknown /proc/33062/task/35405/root (readlink: Permission denied) 2024-07-27T03:18:57.571 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35405 safe_time ceph txt unknown /proc/33062/task/35405/exe (readlink: Permission denied) 2024-07-27T03:18:57.571 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35405 safe_time ceph NOFD /proc/33062/task/35405/fd (opendir: Permission denied) 2024-07-27T03:18:57.571 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35406 safe_time ceph cwd unknown /proc/33062/task/35406/cwd (readlink: Permission denied) 2024-07-27T03:18:57.571 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35406 safe_time ceph rtd unknown /proc/33062/task/35406/root (readlink: Permission denied) 2024-07-27T03:18:57.571 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35406 safe_time ceph txt unknown /proc/33062/task/35406/exe (readlink: Permission denied) 2024-07-27T03:18:57.571 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35406 safe_time ceph NOFD /proc/33062/task/35406/fd (opendir: Permission denied) 2024-07-27T03:18:57.571 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35407 fn_anonym ceph cwd unknown /proc/33062/task/35407/cwd (readlink: Permission denied) 2024-07-27T03:18:57.572 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35407 fn_anonym ceph rtd unknown /proc/33062/task/35407/root (readlink: Permission denied) 2024-07-27T03:18:57.572 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35407 fn_anonym ceph txt unknown /proc/33062/task/35407/exe (readlink: Permission denied) 2024-07-27T03:18:57.572 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35407 fn_anonym ceph NOFD /proc/33062/task/35407/fd (opendir: Permission denied) 2024-07-27T03:18:57.572 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35408 flusher ceph cwd unknown /proc/33062/task/35408/cwd (readlink: Permission denied) 2024-07-27T03:18:57.572 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35408 flusher ceph rtd unknown /proc/33062/task/35408/root (readlink: Permission denied) 2024-07-27T03:18:57.572 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35408 flusher ceph txt unknown /proc/33062/task/35408/exe (readlink: Permission denied) 2024-07-27T03:18:57.572 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35408 flusher ceph NOFD /proc/33062/task/35408/fd (opendir: Permission denied) 2024-07-27T03:18:57.572 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35409 safe_time ceph cwd unknown /proc/33062/task/35409/cwd (readlink: Permission denied) 2024-07-27T03:18:57.572 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35409 safe_time ceph rtd unknown /proc/33062/task/35409/root (readlink: Permission denied) 2024-07-27T03:18:57.572 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35409 safe_time ceph txt unknown /proc/33062/task/35409/exe (readlink: Permission denied) 2024-07-27T03:18:57.572 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35409 safe_time ceph NOFD /proc/33062/task/35409/fd (opendir: Permission denied) 2024-07-27T03:18:57.572 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35543 OpHistory ceph cwd unknown /proc/33062/task/35543/cwd (readlink: Permission denied) 2024-07-27T03:18:57.572 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35543 OpHistory ceph rtd unknown /proc/33062/task/35543/root (readlink: Permission denied) 2024-07-27T03:18:57.573 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35543 OpHistory ceph txt unknown /proc/33062/task/35543/exe (readlink: Permission denied) 2024-07-27T03:18:57.573 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35543 OpHistory ceph NOFD /proc/33062/task/35543/fd (opendir: Permission denied) 2024-07-27T03:18:57.573 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35544 mgr-fin ceph cwd unknown /proc/33062/task/35544/cwd (readlink: Permission denied) 2024-07-27T03:18:57.573 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35544 mgr-fin ceph rtd unknown /proc/33062/task/35544/root (readlink: Permission denied) 2024-07-27T03:18:57.573 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35544 mgr-fin ceph txt unknown /proc/33062/task/35544/exe (readlink: Permission denied) 2024-07-27T03:18:57.573 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35544 mgr-fin ceph NOFD /proc/33062/task/35544/fd (opendir: Permission denied) 2024-07-27T03:18:57.573 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35545 ms_dispat ceph cwd unknown /proc/33062/task/35545/cwd (readlink: Permission denied) 2024-07-27T03:18:57.573 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35545 ms_dispat ceph rtd unknown /proc/33062/task/35545/root (readlink: Permission denied) 2024-07-27T03:18:57.573 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35545 ms_dispat ceph txt unknown /proc/33062/task/35545/exe (readlink: Permission denied) 2024-07-27T03:18:57.573 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35545 ms_dispat ceph NOFD /proc/33062/task/35545/fd (opendir: Permission denied) 2024-07-27T03:18:57.573 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35546 ms_local ceph cwd unknown /proc/33062/task/35546/cwd (readlink: Permission denied) 2024-07-27T03:18:57.573 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35546 ms_local ceph rtd unknown /proc/33062/task/35546/root (readlink: Permission denied) 2024-07-27T03:18:57.573 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35546 ms_local ceph txt unknown /proc/33062/task/35546/exe (readlink: Permission denied) 2024-07-27T03:18:57.574 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35546 ms_local ceph NOFD /proc/33062/task/35546/fd (opendir: Permission denied) 2024-07-27T03:18:57.574 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35547 safe_time ceph cwd unknown /proc/33062/task/35547/cwd (readlink: Permission denied) 2024-07-27T03:18:57.574 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35547 safe_time ceph rtd unknown /proc/33062/task/35547/root (readlink: Permission denied) 2024-07-27T03:18:57.574 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35547 safe_time ceph txt unknown /proc/33062/task/35547/exe (readlink: Permission denied) 2024-07-27T03:18:57.574 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35547 safe_time ceph NOFD /proc/33062/task/35547/fd (opendir: Permission denied) 2024-07-27T03:18:57.574 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35548 cmdfin ceph cwd unknown /proc/33062/task/35548/cwd (readlink: Permission denied) 2024-07-27T03:18:57.574 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35548 cmdfin ceph rtd unknown /proc/33062/task/35548/root (readlink: Permission denied) 2024-07-27T03:18:57.574 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35548 cmdfin ceph txt unknown /proc/33062/task/35548/exe (readlink: Permission denied) 2024-07-27T03:18:57.574 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35548 cmdfin ceph NOFD /proc/33062/task/35548/fd (opendir: Permission denied) 2024-07-27T03:18:57.574 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35552 io_contex ceph cwd unknown /proc/33062/task/35552/cwd (readlink: Permission denied) 2024-07-27T03:18:57.574 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35552 io_contex ceph rtd unknown /proc/33062/task/35552/root (readlink: Permission denied) 2024-07-27T03:18:57.574 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35552 io_contex ceph txt unknown /proc/33062/task/35552/exe (readlink: Permission denied) 2024-07-27T03:18:57.574 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35552 io_contex ceph NOFD /proc/33062/task/35552/fd (opendir: Permission denied) 2024-07-27T03:18:57.575 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35553 io_contex ceph cwd unknown /proc/33062/task/35553/cwd (readlink: Permission denied) 2024-07-27T03:18:57.575 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35553 io_contex ceph rtd unknown /proc/33062/task/35553/root (readlink: Permission denied) 2024-07-27T03:18:57.575 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35553 io_contex ceph txt unknown /proc/33062/task/35553/exe (readlink: Permission denied) 2024-07-27T03:18:57.575 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35553 io_contex ceph NOFD /proc/33062/task/35553/fd (opendir: Permission denied) 2024-07-27T03:18:57.575 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35554 ceph_time ceph cwd unknown /proc/33062/task/35554/cwd (readlink: Permission denied) 2024-07-27T03:18:57.575 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35554 ceph_time ceph rtd unknown /proc/33062/task/35554/root (readlink: Permission denied) 2024-07-27T03:18:57.575 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35554 ceph_time ceph txt unknown /proc/33062/task/35554/exe (readlink: Permission denied) 2024-07-27T03:18:57.575 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35554 ceph_time ceph NOFD /proc/33062/task/35554/fd (opendir: Permission denied) 2024-07-27T03:18:57.575 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35555 ms_dispat ceph cwd unknown /proc/33062/task/35555/cwd (readlink: Permission denied) 2024-07-27T03:18:57.575 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35555 ms_dispat ceph rtd unknown /proc/33062/task/35555/root (readlink: Permission denied) 2024-07-27T03:18:57.575 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35555 ms_dispat ceph txt unknown /proc/33062/task/35555/exe (readlink: Permission denied) 2024-07-27T03:18:57.575 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35555 ms_dispat ceph NOFD /proc/33062/task/35555/fd (opendir: Permission denied) 2024-07-27T03:18:57.575 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35556 ms_local ceph cwd unknown /proc/33062/task/35556/cwd (readlink: Permission denied) 2024-07-27T03:18:57.575 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35556 ms_local ceph rtd unknown /proc/33062/task/35556/root (readlink: Permission denied) 2024-07-27T03:18:57.576 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35556 ms_local ceph txt unknown /proc/33062/task/35556/exe (readlink: Permission denied) 2024-07-27T03:18:57.576 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35556 ms_local ceph NOFD /proc/33062/task/35556/fd (opendir: Permission denied) 2024-07-27T03:18:57.576 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35557 safe_time ceph cwd unknown /proc/33062/task/35557/cwd (readlink: Permission denied) 2024-07-27T03:18:57.576 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35557 safe_time ceph rtd unknown /proc/33062/task/35557/root (readlink: Permission denied) 2024-07-27T03:18:57.576 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35557 safe_time ceph txt unknown /proc/33062/task/35557/exe (readlink: Permission denied) 2024-07-27T03:18:57.576 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35557 safe_time ceph NOFD /proc/33062/task/35557/fd (opendir: Permission denied) 2024-07-27T03:18:57.576 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35558 safe_time ceph cwd unknown /proc/33062/task/35558/cwd (readlink: Permission denied) 2024-07-27T03:18:57.576 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35558 safe_time ceph rtd unknown /proc/33062/task/35558/root (readlink: Permission denied) 2024-07-27T03:18:57.576 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35558 safe_time ceph txt unknown /proc/33062/task/35558/exe (readlink: Permission denied) 2024-07-27T03:18:57.576 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35558 safe_time ceph NOFD /proc/33062/task/35558/fd (opendir: Permission denied) 2024-07-27T03:18:57.576 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35559 balancer ceph cwd unknown /proc/33062/task/35559/cwd (readlink: Permission denied) 2024-07-27T03:18:57.576 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35559 balancer ceph rtd unknown /proc/33062/task/35559/root (readlink: Permission denied) 2024-07-27T03:18:57.576 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35559 balancer ceph txt unknown /proc/33062/task/35559/exe (readlink: Permission denied) 2024-07-27T03:18:57.577 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35559 balancer ceph NOFD /proc/33062/task/35559/fd (opendir: Permission denied) 2024-07-27T03:18:57.577 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35560 m-fin-bal ceph cwd unknown /proc/33062/task/35560/cwd (readlink: Permission denied) 2024-07-27T03:18:57.577 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35560 m-fin-bal ceph rtd unknown /proc/33062/task/35560/root (readlink: Permission denied) 2024-07-27T03:18:57.577 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35560 m-fin-bal ceph txt unknown /proc/33062/task/35560/exe (readlink: Permission denied) 2024-07-27T03:18:57.577 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35560 m-fin-bal ceph NOFD /proc/33062/task/35560/fd (opendir: Permission denied) 2024-07-27T03:18:57.577 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35561 mgr-fin ceph cwd unknown /proc/33062/task/35561/cwd (readlink: Permission denied) 2024-07-27T03:18:57.577 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35561 mgr-fin ceph rtd unknown /proc/33062/task/35561/root (readlink: Permission denied) 2024-07-27T03:18:57.577 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35561 mgr-fin ceph txt unknown /proc/33062/task/35561/exe (readlink: Permission denied) 2024-07-27T03:18:57.577 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35561 mgr-fin ceph NOFD /proc/33062/task/35561/fd (opendir: Permission denied) 2024-07-27T03:18:57.577 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35562 mgr-fin ceph cwd unknown /proc/33062/task/35562/cwd (readlink: Permission denied) 2024-07-27T03:18:57.577 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35562 mgr-fin ceph rtd unknown /proc/33062/task/35562/root (readlink: Permission denied) 2024-07-27T03:18:57.577 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35562 mgr-fin ceph txt unknown /proc/33062/task/35562/exe (readlink: Permission denied) 2024-07-27T03:18:57.577 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35562 mgr-fin ceph NOFD /proc/33062/task/35562/fd (opendir: Permission denied) 2024-07-27T03:18:57.577 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35563 mgr-fin ceph cwd unknown /proc/33062/task/35563/cwd (readlink: Permission denied) 2024-07-27T03:18:57.578 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35563 mgr-fin ceph rtd unknown /proc/33062/task/35563/root (readlink: Permission denied) 2024-07-27T03:18:57.578 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35563 mgr-fin ceph txt unknown /proc/33062/task/35563/exe (readlink: Permission denied) 2024-07-27T03:18:57.578 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35563 mgr-fin ceph NOFD /proc/33062/task/35563/fd (opendir: Permission denied) 2024-07-27T03:18:57.578 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35564 mgr-fin ceph cwd unknown /proc/33062/task/35564/cwd (readlink: Permission denied) 2024-07-27T03:18:57.578 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35564 mgr-fin ceph rtd unknown /proc/33062/task/35564/root (readlink: Permission denied) 2024-07-27T03:18:57.578 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35564 mgr-fin ceph txt unknown /proc/33062/task/35564/exe (readlink: Permission denied) 2024-07-27T03:18:57.578 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35564 mgr-fin ceph NOFD /proc/33062/task/35564/fd (opendir: Permission denied) 2024-07-27T03:18:57.578 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35565 mgr-fin ceph cwd unknown /proc/33062/task/35565/cwd (readlink: Permission denied) 2024-07-27T03:18:57.578 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35565 mgr-fin ceph rtd unknown /proc/33062/task/35565/root (readlink: Permission denied) 2024-07-27T03:18:57.578 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35565 mgr-fin ceph txt unknown /proc/33062/task/35565/exe (readlink: Permission denied) 2024-07-27T03:18:57.578 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35565 mgr-fin ceph NOFD /proc/33062/task/35565/fd (opendir: Permission denied) 2024-07-27T03:18:57.578 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35566 mgr-fin ceph cwd unknown /proc/33062/task/35566/cwd (readlink: Permission denied) 2024-07-27T03:18:57.578 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35566 mgr-fin ceph rtd unknown /proc/33062/task/35566/root (readlink: Permission denied) 2024-07-27T03:18:57.579 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35566 mgr-fin ceph txt unknown /proc/33062/task/35566/exe (readlink: Permission denied) 2024-07-27T03:18:57.579 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35566 mgr-fin ceph NOFD /proc/33062/task/35566/fd (opendir: Permission denied) 2024-07-27T03:18:57.579 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35567 mgr-fin ceph cwd unknown /proc/33062/task/35567/cwd (readlink: Permission denied) 2024-07-27T03:18:57.579 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35567 mgr-fin ceph rtd unknown /proc/33062/task/35567/root (readlink: Permission denied) 2024-07-27T03:18:57.579 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35567 mgr-fin ceph txt unknown /proc/33062/task/35567/exe (readlink: Permission denied) 2024-07-27T03:18:57.579 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35567 mgr-fin ceph NOFD /proc/33062/task/35567/fd (opendir: Permission denied) 2024-07-27T03:18:57.579 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35568 mgr-fin ceph cwd unknown /proc/33062/task/35568/cwd (readlink: Permission denied) 2024-07-27T03:18:57.579 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35568 mgr-fin ceph rtd unknown /proc/33062/task/35568/root (readlink: Permission denied) 2024-07-27T03:18:57.579 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35568 mgr-fin ceph txt unknown /proc/33062/task/35568/exe (readlink: Permission denied) 2024-07-27T03:18:57.579 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35568 mgr-fin ceph NOFD /proc/33062/task/35568/fd (opendir: Permission denied) 2024-07-27T03:18:57.579 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35569 mgr-fin ceph cwd unknown /proc/33062/task/35569/cwd (readlink: Permission denied) 2024-07-27T03:18:57.579 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35569 mgr-fin ceph rtd unknown /proc/33062/task/35569/root (readlink: Permission denied) 2024-07-27T03:18:57.579 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35569 mgr-fin ceph txt unknown /proc/33062/task/35569/exe (readlink: Permission denied) 2024-07-27T03:18:57.579 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35569 mgr-fin ceph NOFD /proc/33062/task/35569/fd (opendir: Permission denied) 2024-07-27T03:18:57.580 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35570 mgr-fin ceph cwd unknown /proc/33062/task/35570/cwd (readlink: Permission denied) 2024-07-27T03:18:57.580 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35570 mgr-fin ceph rtd unknown /proc/33062/task/35570/root (readlink: Permission denied) 2024-07-27T03:18:57.580 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35570 mgr-fin ceph txt unknown /proc/33062/task/35570/exe (readlink: Permission denied) 2024-07-27T03:18:57.580 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35570 mgr-fin ceph NOFD /proc/33062/task/35570/fd (opendir: Permission denied) 2024-07-27T03:18:57.580 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35571 mgr-fin ceph cwd unknown /proc/33062/task/35571/cwd (readlink: Permission denied) 2024-07-27T03:18:57.580 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35571 mgr-fin ceph rtd unknown /proc/33062/task/35571/root (readlink: Permission denied) 2024-07-27T03:18:57.580 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35571 mgr-fin ceph txt unknown /proc/33062/task/35571/exe (readlink: Permission denied) 2024-07-27T03:18:57.580 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35571 mgr-fin ceph NOFD /proc/33062/task/35571/fd (opendir: Permission denied) 2024-07-27T03:18:57.580 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35572 mgr-fin ceph cwd unknown /proc/33062/task/35572/cwd (readlink: Permission denied) 2024-07-27T03:18:57.580 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35572 mgr-fin ceph rtd unknown /proc/33062/task/35572/root (readlink: Permission denied) 2024-07-27T03:18:57.580 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35572 mgr-fin ceph txt unknown /proc/33062/task/35572/exe (readlink: Permission denied) 2024-07-27T03:18:57.580 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35572 mgr-fin ceph NOFD /proc/33062/task/35572/fd (opendir: Permission denied) 2024-07-27T03:18:57.580 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35573 mgr-fin ceph cwd unknown /proc/33062/task/35573/cwd (readlink: Permission denied) 2024-07-27T03:18:57.581 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35573 mgr-fin ceph rtd unknown /proc/33062/task/35573/root (readlink: Permission denied) 2024-07-27T03:18:57.581 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35573 mgr-fin ceph txt unknown /proc/33062/task/35573/exe (readlink: Permission denied) 2024-07-27T03:18:57.581 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35573 mgr-fin ceph NOFD /proc/33062/task/35573/fd (opendir: Permission denied) 2024-07-27T03:18:57.581 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35574 mgr-fin ceph cwd unknown /proc/33062/task/35574/cwd (readlink: Permission denied) 2024-07-27T03:18:57.581 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35574 mgr-fin ceph rtd unknown /proc/33062/task/35574/root (readlink: Permission denied) 2024-07-27T03:18:57.581 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35574 mgr-fin ceph txt unknown /proc/33062/task/35574/exe (readlink: Permission denied) 2024-07-27T03:18:57.581 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35574 mgr-fin ceph NOFD /proc/33062/task/35574/fd (opendir: Permission denied) 2024-07-27T03:18:57.581 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35575 mgr-fin ceph cwd unknown /proc/33062/task/35575/cwd (readlink: Permission denied) 2024-07-27T03:18:57.581 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35575 mgr-fin ceph rtd unknown /proc/33062/task/35575/root (readlink: Permission denied) 2024-07-27T03:18:57.581 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35575 mgr-fin ceph txt unknown /proc/33062/task/35575/exe (readlink: Permission denied) 2024-07-27T03:18:57.581 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35575 mgr-fin ceph NOFD /proc/33062/task/35575/fd (opendir: Permission denied) 2024-07-27T03:18:57.581 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35576 cephadm ceph cwd unknown /proc/33062/task/35576/cwd (readlink: Permission denied) 2024-07-27T03:18:57.581 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35576 cephadm ceph rtd unknown /proc/33062/task/35576/root (readlink: Permission denied) 2024-07-27T03:18:57.581 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35576 cephadm ceph txt unknown /proc/33062/task/35576/exe (readlink: Permission denied) 2024-07-27T03:18:57.582 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35576 cephadm ceph NOFD /proc/33062/task/35576/fd (opendir: Permission denied) 2024-07-27T03:18:57.582 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35577 m-fin-cep ceph cwd unknown /proc/33062/task/35577/cwd (readlink: Permission denied) 2024-07-27T03:18:57.582 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35577 m-fin-cep ceph rtd unknown /proc/33062/task/35577/root (readlink: Permission denied) 2024-07-27T03:18:57.582 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35577 m-fin-cep ceph txt unknown /proc/33062/task/35577/exe (readlink: Permission denied) 2024-07-27T03:18:57.582 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35577 m-fin-cep ceph NOFD /proc/33062/task/35577/fd (opendir: Permission denied) 2024-07-27T03:18:57.582 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35578 mgr-fin ceph cwd unknown /proc/33062/task/35578/cwd (readlink: Permission denied) 2024-07-27T03:18:57.582 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35578 mgr-fin ceph rtd unknown /proc/33062/task/35578/root (readlink: Permission denied) 2024-07-27T03:18:57.582 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35578 mgr-fin ceph txt unknown /proc/33062/task/35578/exe (readlink: Permission denied) 2024-07-27T03:18:57.582 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35578 mgr-fin ceph NOFD /proc/33062/task/35578/fd (opendir: Permission denied) 2024-07-27T03:18:57.582 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35579 mgr-fin ceph cwd unknown /proc/33062/task/35579/cwd (readlink: Permission denied) 2024-07-27T03:18:57.582 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35579 mgr-fin ceph rtd unknown /proc/33062/task/35579/root (readlink: Permission denied) 2024-07-27T03:18:57.582 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35579 mgr-fin ceph txt unknown /proc/33062/task/35579/exe (readlink: Permission denied) 2024-07-27T03:18:57.582 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35579 mgr-fin ceph NOFD /proc/33062/task/35579/fd (opendir: Permission denied) 2024-07-27T03:18:57.583 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35580 mgr-fin ceph cwd unknown /proc/33062/task/35580/cwd (readlink: Permission denied) 2024-07-27T03:18:57.583 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35580 mgr-fin ceph rtd unknown /proc/33062/task/35580/root (readlink: Permission denied) 2024-07-27T03:18:57.583 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35580 mgr-fin ceph txt unknown /proc/33062/task/35580/exe (readlink: Permission denied) 2024-07-27T03:18:57.583 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35580 mgr-fin ceph NOFD /proc/33062/task/35580/fd (opendir: Permission denied) 2024-07-27T03:18:57.583 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35581 cephadm ceph cwd unknown /proc/33062/task/35581/cwd (readlink: Permission denied) 2024-07-27T03:18:57.583 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35581 cephadm ceph rtd unknown /proc/33062/task/35581/root (readlink: Permission denied) 2024-07-27T03:18:57.583 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35581 cephadm ceph txt unknown /proc/33062/task/35581/exe (readlink: Permission denied) 2024-07-27T03:18:57.583 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35581 cephadm ceph NOFD /proc/33062/task/35581/fd (opendir: Permission denied) 2024-07-27T03:18:57.583 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35582 mgr-fin ceph cwd unknown /proc/33062/task/35582/cwd (readlink: Permission denied) 2024-07-27T03:18:57.583 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35582 mgr-fin ceph rtd unknown /proc/33062/task/35582/root (readlink: Permission denied) 2024-07-27T03:18:57.583 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35582 mgr-fin ceph txt unknown /proc/33062/task/35582/exe (readlink: Permission denied) 2024-07-27T03:18:57.583 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35582 mgr-fin ceph NOFD /proc/33062/task/35582/fd (opendir: Permission denied) 2024-07-27T03:18:57.583 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35583 crash ceph cwd unknown /proc/33062/task/35583/cwd (readlink: Permission denied) 2024-07-27T03:18:57.583 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35583 crash ceph rtd unknown /proc/33062/task/35583/root (readlink: Permission denied) 2024-07-27T03:18:57.584 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35583 crash ceph txt unknown /proc/33062/task/35583/exe (readlink: Permission denied) 2024-07-27T03:18:57.584 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35583 crash ceph NOFD /proc/33062/task/35583/fd (opendir: Permission denied) 2024-07-27T03:18:57.584 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35584 m-fin-cra ceph cwd unknown /proc/33062/task/35584/cwd (readlink: Permission denied) 2024-07-27T03:18:57.584 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35584 m-fin-cra ceph rtd unknown /proc/33062/task/35584/root (readlink: Permission denied) 2024-07-27T03:18:57.584 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35584 m-fin-cra ceph txt unknown /proc/33062/task/35584/exe (readlink: Permission denied) 2024-07-27T03:18:57.584 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35584 m-fin-cra ceph NOFD /proc/33062/task/35584/fd (opendir: Permission denied) 2024-07-27T03:18:57.584 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35585 mgr-fin ceph cwd unknown /proc/33062/task/35585/cwd (readlink: Permission denied) 2024-07-27T03:18:57.584 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35585 mgr-fin ceph rtd unknown /proc/33062/task/35585/root (readlink: Permission denied) 2024-07-27T03:18:57.584 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35585 mgr-fin ceph txt unknown /proc/33062/task/35585/exe (readlink: Permission denied) 2024-07-27T03:18:57.584 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35585 mgr-fin ceph NOFD /proc/33062/task/35585/fd (opendir: Permission denied) 2024-07-27T03:18:57.584 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35586 mgr-fin ceph cwd unknown /proc/33062/task/35586/cwd (readlink: Permission denied) 2024-07-27T03:18:57.584 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35586 mgr-fin ceph rtd unknown /proc/33062/task/35586/root (readlink: Permission denied) 2024-07-27T03:18:57.584 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35586 mgr-fin ceph txt unknown /proc/33062/task/35586/exe (readlink: Permission denied) 2024-07-27T03:18:57.585 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35586 mgr-fin ceph NOFD /proc/33062/task/35586/fd (opendir: Permission denied) 2024-07-27T03:18:57.585 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35587 mgr-fin ceph cwd unknown /proc/33062/task/35587/cwd (readlink: Permission denied) 2024-07-27T03:18:57.585 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35587 mgr-fin ceph rtd unknown /proc/33062/task/35587/root (readlink: Permission denied) 2024-07-27T03:18:57.585 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35587 mgr-fin ceph txt unknown /proc/33062/task/35587/exe (readlink: Permission denied) 2024-07-27T03:18:57.585 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35587 mgr-fin ceph NOFD /proc/33062/task/35587/fd (opendir: Permission denied) 2024-07-27T03:18:57.585 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35588 mgr-fin ceph cwd unknown /proc/33062/task/35588/cwd (readlink: Permission denied) 2024-07-27T03:18:57.585 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35588 mgr-fin ceph rtd unknown /proc/33062/task/35588/root (readlink: Permission denied) 2024-07-27T03:18:57.585 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35588 mgr-fin ceph txt unknown /proc/33062/task/35588/exe (readlink: Permission denied) 2024-07-27T03:18:57.585 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35588 mgr-fin ceph NOFD /proc/33062/task/35588/fd (opendir: Permission denied) 2024-07-27T03:18:57.585 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35589 dashboard ceph cwd unknown /proc/33062/task/35589/cwd (readlink: Permission denied) 2024-07-27T03:18:57.585 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35589 dashboard ceph rtd unknown /proc/33062/task/35589/root (readlink: Permission denied) 2024-07-27T03:18:57.585 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35589 dashboard ceph txt unknown /proc/33062/task/35589/exe (readlink: Permission denied) 2024-07-27T03:18:57.585 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35589 dashboard ceph NOFD /proc/33062/task/35589/fd (opendir: Permission denied) 2024-07-27T03:18:57.585 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35590 m-fin-das ceph cwd unknown /proc/33062/task/35590/cwd (readlink: Permission denied) 2024-07-27T03:18:57.586 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35590 m-fin-das ceph rtd unknown /proc/33062/task/35590/root (readlink: Permission denied) 2024-07-27T03:18:57.586 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35590 m-fin-das ceph txt unknown /proc/33062/task/35590/exe (readlink: Permission denied) 2024-07-27T03:18:57.586 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35590 m-fin-das ceph NOFD /proc/33062/task/35590/fd (opendir: Permission denied) 2024-07-27T03:18:57.586 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35591 mgr-fin ceph cwd unknown /proc/33062/task/35591/cwd (readlink: Permission denied) 2024-07-27T03:18:57.586 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35591 mgr-fin ceph rtd unknown /proc/33062/task/35591/root (readlink: Permission denied) 2024-07-27T03:18:57.586 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35591 mgr-fin ceph txt unknown /proc/33062/task/35591/exe (readlink: Permission denied) 2024-07-27T03:18:57.586 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35591 mgr-fin ceph NOFD /proc/33062/task/35591/fd (opendir: Permission denied) 2024-07-27T03:18:57.586 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35592 devicehea ceph cwd unknown /proc/33062/task/35592/cwd (readlink: Permission denied) 2024-07-27T03:18:57.586 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35592 devicehea ceph rtd unknown /proc/33062/task/35592/root (readlink: Permission denied) 2024-07-27T03:18:57.586 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35592 devicehea ceph txt unknown /proc/33062/task/35592/exe (readlink: Permission denied) 2024-07-27T03:18:57.586 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35592 devicehea ceph NOFD /proc/33062/task/35592/fd (opendir: Permission denied) 2024-07-27T03:18:57.586 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35593 m-fin-dev ceph cwd unknown /proc/33062/task/35593/cwd (readlink: Permission denied) 2024-07-27T03:18:57.586 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35593 m-fin-dev ceph rtd unknown /proc/33062/task/35593/root (readlink: Permission denied) 2024-07-27T03:18:57.586 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35593 m-fin-dev ceph txt unknown /proc/33062/task/35593/exe (readlink: Permission denied) 2024-07-27T03:18:57.587 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35593 m-fin-dev ceph NOFD /proc/33062/task/35593/fd (opendir: Permission denied) 2024-07-27T03:18:57.587 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35594 mgr-fin ceph cwd unknown /proc/33062/task/35594/cwd (readlink: Permission denied) 2024-07-27T03:18:57.587 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35594 mgr-fin ceph rtd unknown /proc/33062/task/35594/root (readlink: Permission denied) 2024-07-27T03:18:57.587 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35594 mgr-fin ceph txt unknown /proc/33062/task/35594/exe (readlink: Permission denied) 2024-07-27T03:18:57.587 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35594 mgr-fin ceph NOFD /proc/33062/task/35594/fd (opendir: Permission denied) 2024-07-27T03:18:57.587 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35595 mgr-fin ceph cwd unknown /proc/33062/task/35595/cwd (readlink: Permission denied) 2024-07-27T03:18:57.587 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35595 mgr-fin ceph rtd unknown /proc/33062/task/35595/root (readlink: Permission denied) 2024-07-27T03:18:57.587 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35595 mgr-fin ceph txt unknown /proc/33062/task/35595/exe (readlink: Permission denied) 2024-07-27T03:18:57.587 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35595 mgr-fin ceph NOFD /proc/33062/task/35595/fd (opendir: Permission denied) 2024-07-27T03:18:57.587 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35597 m-fin-ios ceph cwd unknown /proc/33062/task/35597/cwd (readlink: Permission denied) 2024-07-27T03:18:57.587 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35597 m-fin-ios ceph rtd unknown /proc/33062/task/35597/root (readlink: Permission denied) 2024-07-27T03:18:57.587 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35597 m-fin-ios ceph txt unknown /proc/33062/task/35597/exe (readlink: Permission denied) 2024-07-27T03:18:57.587 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35597 m-fin-ios ceph NOFD /proc/33062/task/35597/fd (opendir: Permission denied) 2024-07-27T03:18:57.588 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35598 mgr-fin ceph cwd unknown /proc/33062/task/35598/cwd (readlink: Permission denied) 2024-07-27T03:18:57.588 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35598 mgr-fin ceph rtd unknown /proc/33062/task/35598/root (readlink: Permission denied) 2024-07-27T03:18:57.588 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35598 mgr-fin ceph txt unknown /proc/33062/task/35598/exe (readlink: Permission denied) 2024-07-27T03:18:57.588 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35598 mgr-fin ceph NOFD /proc/33062/task/35598/fd (opendir: Permission denied) 2024-07-27T03:18:57.588 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35600 m-fin-nfs ceph cwd unknown /proc/33062/task/35600/cwd (readlink: Permission denied) 2024-07-27T03:18:57.588 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35600 m-fin-nfs ceph rtd unknown /proc/33062/task/35600/root (readlink: Permission denied) 2024-07-27T03:18:57.588 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35600 m-fin-nfs ceph txt unknown /proc/33062/task/35600/exe (readlink: Permission denied) 2024-07-27T03:18:57.588 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35600 m-fin-nfs ceph NOFD /proc/33062/task/35600/fd (opendir: Permission denied) 2024-07-27T03:18:57.588 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35601 mgr-fin ceph cwd unknown /proc/33062/task/35601/cwd (readlink: Permission denied) 2024-07-27T03:18:57.588 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35601 mgr-fin ceph rtd unknown /proc/33062/task/35601/root (readlink: Permission denied) 2024-07-27T03:18:57.588 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35601 mgr-fin ceph txt unknown /proc/33062/task/35601/exe (readlink: Permission denied) 2024-07-27T03:18:57.588 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35601 mgr-fin ceph NOFD /proc/33062/task/35601/fd (opendir: Permission denied) 2024-07-27T03:18:57.588 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35602 cephadm ceph cwd unknown /proc/33062/task/35602/cwd (readlink: Permission denied) 2024-07-27T03:18:57.588 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35602 cephadm ceph rtd unknown /proc/33062/task/35602/root (readlink: Permission denied) 2024-07-27T03:18:57.589 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35602 cephadm ceph txt unknown /proc/33062/task/35602/exe (readlink: Permission denied) 2024-07-27T03:18:57.589 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35602 cephadm ceph NOFD /proc/33062/task/35602/fd (opendir: Permission denied) 2024-07-27T03:18:57.589 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35604 m-fin-orc ceph cwd unknown /proc/33062/task/35604/cwd (readlink: Permission denied) 2024-07-27T03:18:57.589 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35604 m-fin-orc ceph rtd unknown /proc/33062/task/35604/root (readlink: Permission denied) 2024-07-27T03:18:57.589 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35604 m-fin-orc ceph txt unknown /proc/33062/task/35604/exe (readlink: Permission denied) 2024-07-27T03:18:57.589 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35604 m-fin-orc ceph NOFD /proc/33062/task/35604/fd (opendir: Permission denied) 2024-07-27T03:18:57.589 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35605 pg_autosc ceph cwd unknown /proc/33062/task/35605/cwd (readlink: Permission denied) 2024-07-27T03:18:57.589 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35605 pg_autosc ceph rtd unknown /proc/33062/task/35605/root (readlink: Permission denied) 2024-07-27T03:18:57.589 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35605 pg_autosc ceph txt unknown /proc/33062/task/35605/exe (readlink: Permission denied) 2024-07-27T03:18:57.589 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35605 pg_autosc ceph NOFD /proc/33062/task/35605/fd (opendir: Permission denied) 2024-07-27T03:18:57.589 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35606 m-fin-pg_ ceph cwd unknown /proc/33062/task/35606/cwd (readlink: Permission denied) 2024-07-27T03:18:57.589 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35606 m-fin-pg_ ceph rtd unknown /proc/33062/task/35606/root (readlink: Permission denied) 2024-07-27T03:18:57.589 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35606 m-fin-pg_ ceph txt unknown /proc/33062/task/35606/exe (readlink: Permission denied) 2024-07-27T03:18:57.590 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35606 m-fin-pg_ ceph NOFD /proc/33062/task/35606/fd (opendir: Permission denied) 2024-07-27T03:18:57.590 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35607 progress ceph cwd unknown /proc/33062/task/35607/cwd (readlink: Permission denied) 2024-07-27T03:18:57.590 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35607 progress ceph rtd unknown /proc/33062/task/35607/root (readlink: Permission denied) 2024-07-27T03:18:57.590 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35607 progress ceph txt unknown /proc/33062/task/35607/exe (readlink: Permission denied) 2024-07-27T03:18:57.590 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35607 progress ceph NOFD /proc/33062/task/35607/fd (opendir: Permission denied) 2024-07-27T03:18:57.590 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35608 m-fin-pro ceph cwd unknown /proc/33062/task/35608/cwd (readlink: Permission denied) 2024-07-27T03:18:57.590 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35608 m-fin-pro ceph rtd unknown /proc/33062/task/35608/root (readlink: Permission denied) 2024-07-27T03:18:57.590 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35608 m-fin-pro ceph txt unknown /proc/33062/task/35608/exe (readlink: Permission denied) 2024-07-27T03:18:57.590 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35608 m-fin-pro ceph NOFD /proc/33062/task/35608/fd (opendir: Permission denied) 2024-07-27T03:18:57.590 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35612 io_contex ceph cwd unknown /proc/33062/task/35612/cwd (readlink: Permission denied) 2024-07-27T03:18:57.590 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35612 io_contex ceph rtd unknown /proc/33062/task/35612/root (readlink: Permission denied) 2024-07-27T03:18:57.590 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35612 io_contex ceph txt unknown /proc/33062/task/35612/exe (readlink: Permission denied) 2024-07-27T03:18:57.590 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35612 io_contex ceph NOFD /proc/33062/task/35612/fd (opendir: Permission denied) 2024-07-27T03:18:57.590 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35613 io_contex ceph cwd unknown /proc/33062/task/35613/cwd (readlink: Permission denied) 2024-07-27T03:18:57.591 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35613 io_contex ceph rtd unknown /proc/33062/task/35613/root (readlink: Permission denied) 2024-07-27T03:18:57.591 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35613 io_contex ceph txt unknown /proc/33062/task/35613/exe (readlink: Permission denied) 2024-07-27T03:18:57.591 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35613 io_contex ceph NOFD /proc/33062/task/35613/fd (opendir: Permission denied) 2024-07-27T03:18:57.591 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35614 ceph_time ceph cwd unknown /proc/33062/task/35614/cwd (readlink: Permission denied) 2024-07-27T03:18:57.591 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35614 ceph_time ceph rtd unknown /proc/33062/task/35614/root (readlink: Permission denied) 2024-07-27T03:18:57.591 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35614 ceph_time ceph txt unknown /proc/33062/task/35614/exe (readlink: Permission denied) 2024-07-27T03:18:57.591 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35614 ceph_time ceph NOFD /proc/33062/task/35614/fd (opendir: Permission denied) 2024-07-27T03:18:57.591 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35615 ms_dispat ceph cwd unknown /proc/33062/task/35615/cwd (readlink: Permission denied) 2024-07-27T03:18:57.591 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35615 ms_dispat ceph rtd unknown /proc/33062/task/35615/root (readlink: Permission denied) 2024-07-27T03:18:57.591 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35615 ms_dispat ceph txt unknown /proc/33062/task/35615/exe (readlink: Permission denied) 2024-07-27T03:18:57.591 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35615 ms_dispat ceph NOFD /proc/33062/task/35615/fd (opendir: Permission denied) 2024-07-27T03:18:57.591 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35616 ms_local ceph cwd unknown /proc/33062/task/35616/cwd (readlink: Permission denied) 2024-07-27T03:18:57.591 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35616 ms_local ceph rtd unknown /proc/33062/task/35616/root (readlink: Permission denied) 2024-07-27T03:18:57.592 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35616 ms_local ceph txt unknown /proc/33062/task/35616/exe (readlink: Permission denied) 2024-07-27T03:18:57.592 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35616 ms_local ceph NOFD /proc/33062/task/35616/fd (opendir: Permission denied) 2024-07-27T03:18:57.592 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35617 safe_time ceph cwd unknown /proc/33062/task/35617/cwd (readlink: Permission denied) 2024-07-27T03:18:57.592 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35617 safe_time ceph rtd unknown /proc/33062/task/35617/root (readlink: Permission denied) 2024-07-27T03:18:57.592 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35617 safe_time ceph txt unknown /proc/33062/task/35617/exe (readlink: Permission denied) 2024-07-27T03:18:57.592 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35617 safe_time ceph NOFD /proc/33062/task/35617/fd (opendir: Permission denied) 2024-07-27T03:18:57.592 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35618 safe_time ceph cwd unknown /proc/33062/task/35618/cwd (readlink: Permission denied) 2024-07-27T03:18:57.592 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35618 safe_time ceph rtd unknown /proc/33062/task/35618/root (readlink: Permission denied) 2024-07-27T03:18:57.592 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35618 safe_time ceph txt unknown /proc/33062/task/35618/exe (readlink: Permission denied) 2024-07-27T03:18:57.592 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35618 safe_time ceph NOFD /proc/33062/task/35618/fd (opendir: Permission denied) 2024-07-27T03:18:57.592 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35619 mgr-fin ceph cwd unknown /proc/33062/task/35619/cwd (readlink: Permission denied) 2024-07-27T03:18:57.592 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35619 mgr-fin ceph rtd unknown /proc/33062/task/35619/root (readlink: Permission denied) 2024-07-27T03:18:57.592 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35619 mgr-fin ceph txt unknown /proc/33062/task/35619/exe (readlink: Permission denied) 2024-07-27T03:18:57.592 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35619 mgr-fin ceph NOFD /proc/33062/task/35619/fd (opendir: Permission denied) 2024-07-27T03:18:57.593 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35621 m-fin-rbd ceph cwd unknown /proc/33062/task/35621/cwd (readlink: Permission denied) 2024-07-27T03:18:57.593 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35621 m-fin-rbd ceph rtd unknown /proc/33062/task/35621/root (readlink: Permission denied) 2024-07-27T03:18:57.593 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35621 m-fin-rbd ceph txt unknown /proc/33062/task/35621/exe (readlink: Permission denied) 2024-07-27T03:18:57.593 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35621 m-fin-rbd ceph NOFD /proc/33062/task/35621/fd (opendir: Permission denied) 2024-07-27T03:18:57.593 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35622 restful ceph cwd unknown /proc/33062/task/35622/cwd (readlink: Permission denied) 2024-07-27T03:18:57.593 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35622 restful ceph rtd unknown /proc/33062/task/35622/root (readlink: Permission denied) 2024-07-27T03:18:57.593 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35622 restful ceph txt unknown /proc/33062/task/35622/exe (readlink: Permission denied) 2024-07-27T03:18:57.593 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35622 restful ceph NOFD /proc/33062/task/35622/fd (opendir: Permission denied) 2024-07-27T03:18:57.593 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35623 m-fin-res ceph cwd unknown /proc/33062/task/35623/cwd (readlink: Permission denied) 2024-07-27T03:18:57.593 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35623 m-fin-res ceph rtd unknown /proc/33062/task/35623/root (readlink: Permission denied) 2024-07-27T03:18:57.593 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35623 m-fin-res ceph txt unknown /proc/33062/task/35623/exe (readlink: Permission denied) 2024-07-27T03:18:57.593 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35623 m-fin-res ceph NOFD /proc/33062/task/35623/fd (opendir: Permission denied) 2024-07-27T03:18:57.593 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35625 m-fin-sta ceph cwd unknown /proc/33062/task/35625/cwd (readlink: Permission denied) 2024-07-27T03:18:57.593 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35625 m-fin-sta ceph rtd unknown /proc/33062/task/35625/root (readlink: Permission denied) 2024-07-27T03:18:57.594 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35625 m-fin-sta ceph txt unknown /proc/33062/task/35625/exe (readlink: Permission denied) 2024-07-27T03:18:57.594 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35625 m-fin-sta ceph NOFD /proc/33062/task/35625/fd (opendir: Permission denied) 2024-07-27T03:18:57.594 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35626 mgr-fin ceph cwd unknown /proc/33062/task/35626/cwd (readlink: Permission denied) 2024-07-27T03:18:57.594 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35626 mgr-fin ceph rtd unknown /proc/33062/task/35626/root (readlink: Permission denied) 2024-07-27T03:18:57.594 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35626 mgr-fin ceph txt unknown /proc/33062/task/35626/exe (readlink: Permission denied) 2024-07-27T03:18:57.594 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35626 mgr-fin ceph NOFD /proc/33062/task/35626/fd (opendir: Permission denied) 2024-07-27T03:18:57.594 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35627 mgr-fin ceph cwd unknown /proc/33062/task/35627/cwd (readlink: Permission denied) 2024-07-27T03:18:57.594 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35627 mgr-fin ceph rtd unknown /proc/33062/task/35627/root (readlink: Permission denied) 2024-07-27T03:18:57.594 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35627 mgr-fin ceph txt unknown /proc/33062/task/35627/exe (readlink: Permission denied) 2024-07-27T03:18:57.594 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35627 mgr-fin ceph NOFD /proc/33062/task/35627/fd (opendir: Permission denied) 2024-07-27T03:18:57.594 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35628 mgr-fin ceph cwd unknown /proc/33062/task/35628/cwd (readlink: Permission denied) 2024-07-27T03:18:57.594 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35628 mgr-fin ceph rtd unknown /proc/33062/task/35628/root (readlink: Permission denied) 2024-07-27T03:18:57.594 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35628 mgr-fin ceph txt unknown /proc/33062/task/35628/exe (readlink: Permission denied) 2024-07-27T03:18:57.595 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35628 mgr-fin ceph NOFD /proc/33062/task/35628/fd (opendir: Permission denied) 2024-07-27T03:18:57.595 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35629 telemetry ceph cwd unknown /proc/33062/task/35629/cwd (readlink: Permission denied) 2024-07-27T03:18:57.595 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35629 telemetry ceph rtd unknown /proc/33062/task/35629/root (readlink: Permission denied) 2024-07-27T03:18:57.595 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35629 telemetry ceph txt unknown /proc/33062/task/35629/exe (readlink: Permission denied) 2024-07-27T03:18:57.595 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35629 telemetry ceph NOFD /proc/33062/task/35629/fd (opendir: Permission denied) 2024-07-27T03:18:57.595 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35630 m-fin-tel ceph cwd unknown /proc/33062/task/35630/cwd (readlink: Permission denied) 2024-07-27T03:18:57.595 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35630 m-fin-tel ceph rtd unknown /proc/33062/task/35630/root (readlink: Permission denied) 2024-07-27T03:18:57.595 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35630 m-fin-tel ceph txt unknown /proc/33062/task/35630/exe (readlink: Permission denied) 2024-07-27T03:18:57.595 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35630 m-fin-tel ceph NOFD /proc/33062/task/35630/fd (opendir: Permission denied) 2024-07-27T03:18:57.595 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35631 mgr-fin ceph cwd unknown /proc/33062/task/35631/cwd (readlink: Permission denied) 2024-07-27T03:18:57.595 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35631 mgr-fin ceph rtd unknown /proc/33062/task/35631/root (readlink: Permission denied) 2024-07-27T03:18:57.595 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35631 mgr-fin ceph txt unknown /proc/33062/task/35631/exe (readlink: Permission denied) 2024-07-27T03:18:57.595 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35631 mgr-fin ceph NOFD /proc/33062/task/35631/fd (opendir: Permission denied) 2024-07-27T03:18:57.595 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35632 mgr-fin ceph cwd unknown /proc/33062/task/35632/cwd (readlink: Permission denied) 2024-07-27T03:18:57.596 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35632 mgr-fin ceph rtd unknown /proc/33062/task/35632/root (readlink: Permission denied) 2024-07-27T03:18:57.596 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35632 mgr-fin ceph txt unknown /proc/33062/task/35632/exe (readlink: Permission denied) 2024-07-27T03:18:57.596 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35632 mgr-fin ceph NOFD /proc/33062/task/35632/fd (opendir: Permission denied) 2024-07-27T03:18:57.596 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35636 io_contex ceph cwd unknown /proc/33062/task/35636/cwd (readlink: Permission denied) 2024-07-27T03:18:57.596 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35636 io_contex ceph rtd unknown /proc/33062/task/35636/root (readlink: Permission denied) 2024-07-27T03:18:57.596 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35636 io_contex ceph txt unknown /proc/33062/task/35636/exe (readlink: Permission denied) 2024-07-27T03:18:57.596 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35636 io_contex ceph NOFD /proc/33062/task/35636/fd (opendir: Permission denied) 2024-07-27T03:18:57.596 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35637 io_contex ceph cwd unknown /proc/33062/task/35637/cwd (readlink: Permission denied) 2024-07-27T03:18:57.596 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35637 io_contex ceph rtd unknown /proc/33062/task/35637/root (readlink: Permission denied) 2024-07-27T03:18:57.596 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35637 io_contex ceph txt unknown /proc/33062/task/35637/exe (readlink: Permission denied) 2024-07-27T03:18:57.596 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35637 io_contex ceph NOFD /proc/33062/task/35637/fd (opendir: Permission denied) 2024-07-27T03:18:57.596 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35638 ceph_time ceph cwd unknown /proc/33062/task/35638/cwd (readlink: Permission denied) 2024-07-27T03:18:57.596 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35638 ceph_time ceph rtd unknown /proc/33062/task/35638/root (readlink: Permission denied) 2024-07-27T03:18:57.597 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35638 ceph_time ceph txt unknown /proc/33062/task/35638/exe (readlink: Permission denied) 2024-07-27T03:18:57.597 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35638 ceph_time ceph NOFD /proc/33062/task/35638/fd (opendir: Permission denied) 2024-07-27T03:18:57.597 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35639 ms_dispat ceph cwd unknown /proc/33062/task/35639/cwd (readlink: Permission denied) 2024-07-27T03:18:57.597 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35639 ms_dispat ceph rtd unknown /proc/33062/task/35639/root (readlink: Permission denied) 2024-07-27T03:18:57.597 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35639 ms_dispat ceph txt unknown /proc/33062/task/35639/exe (readlink: Permission denied) 2024-07-27T03:18:57.597 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35639 ms_dispat ceph NOFD /proc/33062/task/35639/fd (opendir: Permission denied) 2024-07-27T03:18:57.597 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35640 ms_local ceph cwd unknown /proc/33062/task/35640/cwd (readlink: Permission denied) 2024-07-27T03:18:57.597 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35640 ms_local ceph rtd unknown /proc/33062/task/35640/root (readlink: Permission denied) 2024-07-27T03:18:57.597 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35640 ms_local ceph txt unknown /proc/33062/task/35640/exe (readlink: Permission denied) 2024-07-27T03:18:57.597 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35640 ms_local ceph NOFD /proc/33062/task/35640/fd (opendir: Permission denied) 2024-07-27T03:18:57.597 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35641 safe_time ceph cwd unknown /proc/33062/task/35641/cwd (readlink: Permission denied) 2024-07-27T03:18:57.597 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35641 safe_time ceph rtd unknown /proc/33062/task/35641/root (readlink: Permission denied) 2024-07-27T03:18:57.597 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35641 safe_time ceph txt unknown /proc/33062/task/35641/exe (readlink: Permission denied) 2024-07-27T03:18:57.598 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35641 safe_time ceph NOFD /proc/33062/task/35641/fd (opendir: Permission denied) 2024-07-27T03:18:57.598 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35642 safe_time ceph cwd unknown /proc/33062/task/35642/cwd (readlink: Permission denied) 2024-07-27T03:18:57.598 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35642 safe_time ceph rtd unknown /proc/33062/task/35642/root (readlink: Permission denied) 2024-07-27T03:18:57.598 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35642 safe_time ceph txt unknown /proc/33062/task/35642/exe (readlink: Permission denied) 2024-07-27T03:18:57.598 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35642 safe_time ceph NOFD /proc/33062/task/35642/fd (opendir: Permission denied) 2024-07-27T03:18:57.598 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35643 mgr-fin ceph cwd unknown /proc/33062/task/35643/cwd (readlink: Permission denied) 2024-07-27T03:18:57.598 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35643 mgr-fin ceph rtd unknown /proc/33062/task/35643/root (readlink: Permission denied) 2024-07-27T03:18:57.598 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35643 mgr-fin ceph txt unknown /proc/33062/task/35643/exe (readlink: Permission denied) 2024-07-27T03:18:57.598 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35643 mgr-fin ceph NOFD /proc/33062/task/35643/fd (opendir: Permission denied) 2024-07-27T03:18:57.598 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35644 mgr-fin ceph cwd unknown /proc/33062/task/35644/cwd (readlink: Permission denied) 2024-07-27T03:18:57.598 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35644 mgr-fin ceph rtd unknown /proc/33062/task/35644/root (readlink: Permission denied) 2024-07-27T03:18:57.598 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35644 mgr-fin ceph txt unknown /proc/33062/task/35644/exe (readlink: Permission denied) 2024-07-27T03:18:57.599 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35644 mgr-fin ceph NOFD /proc/33062/task/35644/fd (opendir: Permission denied) 2024-07-27T03:18:57.599 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35645 mgr-fin ceph cwd unknown /proc/33062/task/35645/cwd (readlink: Permission denied) 2024-07-27T03:18:57.599 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35645 mgr-fin ceph rtd unknown /proc/33062/task/35645/root (readlink: Permission denied) 2024-07-27T03:18:57.599 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35645 mgr-fin ceph txt unknown /proc/33062/task/35645/exe (readlink: Permission denied) 2024-07-27T03:18:57.599 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35645 mgr-fin ceph NOFD /proc/33062/task/35645/fd (opendir: Permission denied) 2024-07-27T03:18:57.599 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35646 mgr-fin ceph cwd unknown /proc/33062/task/35646/cwd (readlink: Permission denied) 2024-07-27T03:18:57.599 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35646 mgr-fin ceph rtd unknown /proc/33062/task/35646/root (readlink: Permission denied) 2024-07-27T03:18:57.599 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35646 mgr-fin ceph txt unknown /proc/33062/task/35646/exe (readlink: Permission denied) 2024-07-27T03:18:57.599 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35646 mgr-fin ceph NOFD /proc/33062/task/35646/fd (opendir: Permission denied) 2024-07-27T03:18:57.599 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35647 mgr-fin ceph cwd unknown /proc/33062/task/35647/cwd (readlink: Permission denied) 2024-07-27T03:18:57.599 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35647 mgr-fin ceph rtd unknown /proc/33062/task/35647/root (readlink: Permission denied) 2024-07-27T03:18:57.599 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35647 mgr-fin ceph txt unknown /proc/33062/task/35647/exe (readlink: Permission denied) 2024-07-27T03:18:57.599 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35647 mgr-fin ceph NOFD /proc/33062/task/35647/fd (opendir: Permission denied) 2024-07-27T03:18:57.599 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35648 mgr-fin ceph cwd unknown /proc/33062/task/35648/cwd (readlink: Permission denied) 2024-07-27T03:18:57.600 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35648 mgr-fin ceph rtd unknown /proc/33062/task/35648/root (readlink: Permission denied) 2024-07-27T03:18:57.600 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35648 mgr-fin ceph txt unknown /proc/33062/task/35648/exe (readlink: Permission denied) 2024-07-27T03:18:57.600 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35648 mgr-fin ceph NOFD /proc/33062/task/35648/fd (opendir: Permission denied) 2024-07-27T03:18:57.600 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35649 mgr-fin ceph cwd unknown /proc/33062/task/35649/cwd (readlink: Permission denied) 2024-07-27T03:18:57.600 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35649 mgr-fin ceph rtd unknown /proc/33062/task/35649/root (readlink: Permission denied) 2024-07-27T03:18:57.600 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35649 mgr-fin ceph txt unknown /proc/33062/task/35649/exe (readlink: Permission denied) 2024-07-27T03:18:57.600 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35649 mgr-fin ceph NOFD /proc/33062/task/35649/fd (opendir: Permission denied) 2024-07-27T03:18:57.600 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35650 mgr-fin ceph cwd unknown /proc/33062/task/35650/cwd (readlink: Permission denied) 2024-07-27T03:18:57.600 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35650 mgr-fin ceph rtd unknown /proc/33062/task/35650/root (readlink: Permission denied) 2024-07-27T03:18:57.600 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35650 mgr-fin ceph txt unknown /proc/33062/task/35650/exe (readlink: Permission denied) 2024-07-27T03:18:57.600 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35650 mgr-fin ceph NOFD /proc/33062/task/35650/fd (opendir: Permission denied) 2024-07-27T03:18:57.600 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35651 mgr-fin ceph cwd unknown /proc/33062/task/35651/cwd (readlink: Permission denied) 2024-07-27T03:18:57.600 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35651 mgr-fin ceph rtd unknown /proc/33062/task/35651/root (readlink: Permission denied) 2024-07-27T03:18:57.601 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35651 mgr-fin ceph txt unknown /proc/33062/task/35651/exe (readlink: Permission denied) 2024-07-27T03:18:57.601 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35651 mgr-fin ceph NOFD /proc/33062/task/35651/fd (opendir: Permission denied) 2024-07-27T03:18:57.601 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35652 mgr-fin ceph cwd unknown /proc/33062/task/35652/cwd (readlink: Permission denied) 2024-07-27T03:18:57.601 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35652 mgr-fin ceph rtd unknown /proc/33062/task/35652/root (readlink: Permission denied) 2024-07-27T03:18:57.601 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35652 mgr-fin ceph txt unknown /proc/33062/task/35652/exe (readlink: Permission denied) 2024-07-27T03:18:57.601 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35652 mgr-fin ceph NOFD /proc/33062/task/35652/fd (opendir: Permission denied) 2024-07-27T03:18:57.601 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35653 mgr-fin ceph cwd unknown /proc/33062/task/35653/cwd (readlink: Permission denied) 2024-07-27T03:18:57.601 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35653 mgr-fin ceph rtd unknown /proc/33062/task/35653/root (readlink: Permission denied) 2024-07-27T03:18:57.601 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35653 mgr-fin ceph txt unknown /proc/33062/task/35653/exe (readlink: Permission denied) 2024-07-27T03:18:57.601 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35653 mgr-fin ceph NOFD /proc/33062/task/35653/fd (opendir: Permission denied) 2024-07-27T03:18:57.601 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35654 mgr-fin ceph cwd unknown /proc/33062/task/35654/cwd (readlink: Permission denied) 2024-07-27T03:18:57.601 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35654 mgr-fin ceph rtd unknown /proc/33062/task/35654/root (readlink: Permission denied) 2024-07-27T03:18:57.601 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35654 mgr-fin ceph txt unknown /proc/33062/task/35654/exe (readlink: Permission denied) 2024-07-27T03:18:57.601 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35654 mgr-fin ceph NOFD /proc/33062/task/35654/fd (opendir: Permission denied) 2024-07-27T03:18:57.602 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35656 m-fin-vol ceph cwd unknown /proc/33062/task/35656/cwd (readlink: Permission denied) 2024-07-27T03:18:57.602 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35656 m-fin-vol ceph rtd unknown /proc/33062/task/35656/root (readlink: Permission denied) 2024-07-27T03:18:57.602 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35656 m-fin-vol ceph txt unknown /proc/33062/task/35656/exe (readlink: Permission denied) 2024-07-27T03:18:57.602 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 35656 m-fin-vol ceph NOFD /proc/33062/task/35656/fd (opendir: Permission denied) 2024-07-27T03:18:57.602 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36115 mgr-fin ceph cwd unknown /proc/33062/task/36115/cwd (readlink: Permission denied) 2024-07-27T03:18:57.602 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36115 mgr-fin ceph rtd unknown /proc/33062/task/36115/root (readlink: Permission denied) 2024-07-27T03:18:57.602 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36115 mgr-fin ceph txt unknown /proc/33062/task/36115/exe (readlink: Permission denied) 2024-07-27T03:18:57.602 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36115 mgr-fin ceph NOFD /proc/33062/task/36115/fd (opendir: Permission denied) 2024-07-27T03:18:57.602 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36116 mgr-fin ceph cwd unknown /proc/33062/task/36116/cwd (readlink: Permission denied) 2024-07-27T03:18:57.602 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36116 mgr-fin ceph rtd unknown /proc/33062/task/36116/root (readlink: Permission denied) 2024-07-27T03:18:57.602 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36116 mgr-fin ceph txt unknown /proc/33062/task/36116/exe (readlink: Permission denied) 2024-07-27T03:18:57.602 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36116 mgr-fin ceph NOFD /proc/33062/task/36116/fd (opendir: Permission denied) 2024-07-27T03:18:57.603 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36117 mgr-fin ceph cwd unknown /proc/33062/task/36117/cwd (readlink: Permission denied) 2024-07-27T03:18:57.603 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36117 mgr-fin ceph rtd unknown /proc/33062/task/36117/root (readlink: Permission denied) 2024-07-27T03:18:57.603 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36117 mgr-fin ceph txt unknown /proc/33062/task/36117/exe (readlink: Permission denied) 2024-07-27T03:18:57.603 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36117 mgr-fin ceph NOFD /proc/33062/task/36117/fd (opendir: Permission denied) 2024-07-27T03:18:57.603 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36118 mgr-fin ceph cwd unknown /proc/33062/task/36118/cwd (readlink: Permission denied) 2024-07-27T03:18:57.603 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36118 mgr-fin ceph rtd unknown /proc/33062/task/36118/root (readlink: Permission denied) 2024-07-27T03:18:57.603 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36118 mgr-fin ceph txt unknown /proc/33062/task/36118/exe (readlink: Permission denied) 2024-07-27T03:18:57.603 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36118 mgr-fin ceph NOFD /proc/33062/task/36118/fd (opendir: Permission denied) 2024-07-27T03:18:57.603 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36119 mgr-fin ceph cwd unknown /proc/33062/task/36119/cwd (readlink: Permission denied) 2024-07-27T03:18:57.603 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36119 mgr-fin ceph rtd unknown /proc/33062/task/36119/root (readlink: Permission denied) 2024-07-27T03:18:57.603 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36119 mgr-fin ceph txt unknown /proc/33062/task/36119/exe (readlink: Permission denied) 2024-07-27T03:18:57.603 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36119 mgr-fin ceph NOFD /proc/33062/task/36119/fd (opendir: Permission denied) 2024-07-27T03:18:57.603 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36120 mgr-fin ceph cwd unknown /proc/33062/task/36120/cwd (readlink: Permission denied) 2024-07-27T03:18:57.604 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36120 mgr-fin ceph rtd unknown /proc/33062/task/36120/root (readlink: Permission denied) 2024-07-27T03:18:57.604 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36120 mgr-fin ceph txt unknown /proc/33062/task/36120/exe (readlink: Permission denied) 2024-07-27T03:18:57.604 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36120 mgr-fin ceph NOFD /proc/33062/task/36120/fd (opendir: Permission denied) 2024-07-27T03:18:57.604 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36121 mgr-fin ceph cwd unknown /proc/33062/task/36121/cwd (readlink: Permission denied) 2024-07-27T03:18:57.604 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36121 mgr-fin ceph rtd unknown /proc/33062/task/36121/root (readlink: Permission denied) 2024-07-27T03:18:57.604 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36121 mgr-fin ceph txt unknown /proc/33062/task/36121/exe (readlink: Permission denied) 2024-07-27T03:18:57.604 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36121 mgr-fin ceph NOFD /proc/33062/task/36121/fd (opendir: Permission denied) 2024-07-27T03:18:57.604 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36122 mgr-fin ceph cwd unknown /proc/33062/task/36122/cwd (readlink: Permission denied) 2024-07-27T03:18:57.604 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36122 mgr-fin ceph rtd unknown /proc/33062/task/36122/root (readlink: Permission denied) 2024-07-27T03:18:57.604 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36122 mgr-fin ceph txt unknown /proc/33062/task/36122/exe (readlink: Permission denied) 2024-07-27T03:18:57.604 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36122 mgr-fin ceph NOFD /proc/33062/task/36122/fd (opendir: Permission denied) 2024-07-27T03:18:57.604 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36123 mgr-fin ceph cwd unknown /proc/33062/task/36123/cwd (readlink: Permission denied) 2024-07-27T03:18:57.604 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36123 mgr-fin ceph rtd unknown /proc/33062/task/36123/root (readlink: Permission denied) 2024-07-27T03:18:57.604 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36123 mgr-fin ceph txt unknown /proc/33062/task/36123/exe (readlink: Permission denied) 2024-07-27T03:18:57.605 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36123 mgr-fin ceph NOFD /proc/33062/task/36123/fd (opendir: Permission denied) 2024-07-27T03:18:57.605 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36124 mgr-fin ceph cwd unknown /proc/33062/task/36124/cwd (readlink: Permission denied) 2024-07-27T03:18:57.605 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36124 mgr-fin ceph rtd unknown /proc/33062/task/36124/root (readlink: Permission denied) 2024-07-27T03:18:57.605 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36124 mgr-fin ceph txt unknown /proc/33062/task/36124/exe (readlink: Permission denied) 2024-07-27T03:18:57.605 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36124 mgr-fin ceph NOFD /proc/33062/task/36124/fd (opendir: Permission denied) 2024-07-27T03:18:57.605 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36125 mgr-fin ceph cwd unknown /proc/33062/task/36125/cwd (readlink: Permission denied) 2024-07-27T03:18:57.605 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36125 mgr-fin ceph rtd unknown /proc/33062/task/36125/root (readlink: Permission denied) 2024-07-27T03:18:57.605 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36125 mgr-fin ceph txt unknown /proc/33062/task/36125/exe (readlink: Permission denied) 2024-07-27T03:18:57.605 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36125 mgr-fin ceph NOFD /proc/33062/task/36125/fd (opendir: Permission denied) 2024-07-27T03:18:57.605 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36159 mgr-fin ceph cwd unknown /proc/33062/task/36159/cwd (readlink: Permission denied) 2024-07-27T03:18:57.605 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36159 mgr-fin ceph rtd unknown /proc/33062/task/36159/root (readlink: Permission denied) 2024-07-27T03:18:57.605 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36159 mgr-fin ceph txt unknown /proc/33062/task/36159/exe (readlink: Permission denied) 2024-07-27T03:18:57.605 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36159 mgr-fin ceph NOFD /proc/33062/task/36159/fd (opendir: Permission denied) 2024-07-27T03:18:57.606 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36160 mgr-fin ceph cwd unknown /proc/33062/task/36160/cwd (readlink: Permission denied) 2024-07-27T03:18:57.606 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36160 mgr-fin ceph rtd unknown /proc/33062/task/36160/root (readlink: Permission denied) 2024-07-27T03:18:57.606 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36160 mgr-fin ceph txt unknown /proc/33062/task/36160/exe (readlink: Permission denied) 2024-07-27T03:18:57.606 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36160 mgr-fin ceph NOFD /proc/33062/task/36160/fd (opendir: Permission denied) 2024-07-27T03:18:57.606 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36161 mgr-fin ceph cwd unknown /proc/33062/task/36161/cwd (readlink: Permission denied) 2024-07-27T03:18:57.606 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36161 mgr-fin ceph rtd unknown /proc/33062/task/36161/root (readlink: Permission denied) 2024-07-27T03:18:57.606 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36161 mgr-fin ceph txt unknown /proc/33062/task/36161/exe (readlink: Permission denied) 2024-07-27T03:18:57.606 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36161 mgr-fin ceph NOFD /proc/33062/task/36161/fd (opendir: Permission denied) 2024-07-27T03:18:57.606 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36162 mgr-fin ceph cwd unknown /proc/33062/task/36162/cwd (readlink: Permission denied) 2024-07-27T03:18:57.606 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36162 mgr-fin ceph rtd unknown /proc/33062/task/36162/root (readlink: Permission denied) 2024-07-27T03:18:57.606 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36162 mgr-fin ceph txt unknown /proc/33062/task/36162/exe (readlink: Permission denied) 2024-07-27T03:18:57.606 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36162 mgr-fin ceph NOFD /proc/33062/task/36162/fd (opendir: Permission denied) 2024-07-27T03:18:57.606 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36163 mgr-fin ceph cwd unknown /proc/33062/task/36163/cwd (readlink: Permission denied) 2024-07-27T03:18:57.607 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36163 mgr-fin ceph rtd unknown /proc/33062/task/36163/root (readlink: Permission denied) 2024-07-27T03:18:57.607 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36163 mgr-fin ceph txt unknown /proc/33062/task/36163/exe (readlink: Permission denied) 2024-07-27T03:18:57.607 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36163 mgr-fin ceph NOFD /proc/33062/task/36163/fd (opendir: Permission denied) 2024-07-27T03:18:57.607 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36164 mgr-fin ceph cwd unknown /proc/33062/task/36164/cwd (readlink: Permission denied) 2024-07-27T03:18:57.607 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36164 mgr-fin ceph rtd unknown /proc/33062/task/36164/root (readlink: Permission denied) 2024-07-27T03:18:57.607 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36164 mgr-fin ceph txt unknown /proc/33062/task/36164/exe (readlink: Permission denied) 2024-07-27T03:18:57.607 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36164 mgr-fin ceph NOFD /proc/33062/task/36164/fd (opendir: Permission denied) 2024-07-27T03:18:57.607 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36165 mgr-fin ceph cwd unknown /proc/33062/task/36165/cwd (readlink: Permission denied) 2024-07-27T03:18:57.607 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36165 mgr-fin ceph rtd unknown /proc/33062/task/36165/root (readlink: Permission denied) 2024-07-27T03:18:57.607 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36165 mgr-fin ceph txt unknown /proc/33062/task/36165/exe (readlink: Permission denied) 2024-07-27T03:18:57.607 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36165 mgr-fin ceph NOFD /proc/33062/task/36165/fd (opendir: Permission denied) 2024-07-27T03:18:57.607 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36166 mgr-fin ceph cwd unknown /proc/33062/task/36166/cwd (readlink: Permission denied) 2024-07-27T03:18:57.607 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36166 mgr-fin ceph rtd unknown /proc/33062/task/36166/root (readlink: Permission denied) 2024-07-27T03:18:57.607 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36166 mgr-fin ceph txt unknown /proc/33062/task/36166/exe (readlink: Permission denied) 2024-07-27T03:18:57.608 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36166 mgr-fin ceph NOFD /proc/33062/task/36166/fd (opendir: Permission denied) 2024-07-27T03:18:57.608 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36167 mgr-fin ceph cwd unknown /proc/33062/task/36167/cwd (readlink: Permission denied) 2024-07-27T03:18:57.608 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36167 mgr-fin ceph rtd unknown /proc/33062/task/36167/root (readlink: Permission denied) 2024-07-27T03:18:57.608 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36167 mgr-fin ceph txt unknown /proc/33062/task/36167/exe (readlink: Permission denied) 2024-07-27T03:18:57.608 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36167 mgr-fin ceph NOFD /proc/33062/task/36167/fd (opendir: Permission denied) 2024-07-27T03:18:57.608 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36168 mgr-fin ceph cwd unknown /proc/33062/task/36168/cwd (readlink: Permission denied) 2024-07-27T03:18:57.608 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36168 mgr-fin ceph rtd unknown /proc/33062/task/36168/root (readlink: Permission denied) 2024-07-27T03:18:57.608 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36168 mgr-fin ceph txt unknown /proc/33062/task/36168/exe (readlink: Permission denied) 2024-07-27T03:18:57.608 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36168 mgr-fin ceph NOFD /proc/33062/task/36168/fd (opendir: Permission denied) 2024-07-27T03:18:57.608 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36169 mgr-fin ceph cwd unknown /proc/33062/task/36169/cwd (readlink: Permission denied) 2024-07-27T03:18:57.608 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36169 mgr-fin ceph rtd unknown /proc/33062/task/36169/root (readlink: Permission denied) 2024-07-27T03:18:57.608 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36169 mgr-fin ceph txt unknown /proc/33062/task/36169/exe (readlink: Permission denied) 2024-07-27T03:18:57.608 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36169 mgr-fin ceph NOFD /proc/33062/task/36169/fd (opendir: Permission denied) 2024-07-27T03:18:57.609 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36355 dashboard ceph cwd unknown /proc/33062/task/36355/cwd (readlink: Permission denied) 2024-07-27T03:18:57.609 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36355 dashboard ceph rtd unknown /proc/33062/task/36355/root (readlink: Permission denied) 2024-07-27T03:18:57.609 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36355 dashboard ceph txt unknown /proc/33062/task/36355/exe (readlink: Permission denied) 2024-07-27T03:18:57.609 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36355 dashboard ceph NOFD /proc/33062/task/36355/fd (opendir: Permission denied) 2024-07-27T03:18:57.609 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36356 dashboard ceph cwd unknown /proc/33062/task/36356/cwd (readlink: Permission denied) 2024-07-27T03:18:57.609 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36356 dashboard ceph rtd unknown /proc/33062/task/36356/root (readlink: Permission denied) 2024-07-27T03:18:57.609 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36356 dashboard ceph txt unknown /proc/33062/task/36356/exe (readlink: Permission denied) 2024-07-27T03:18:57.609 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36356 dashboard ceph NOFD /proc/33062/task/36356/fd (opendir: Permission denied) 2024-07-27T03:18:57.609 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36357 dashboard ceph cwd unknown /proc/33062/task/36357/cwd (readlink: Permission denied) 2024-07-27T03:18:57.609 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36357 dashboard ceph rtd unknown /proc/33062/task/36357/root (readlink: Permission denied) 2024-07-27T03:18:57.609 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36357 dashboard ceph txt unknown /proc/33062/task/36357/exe (readlink: Permission denied) 2024-07-27T03:18:57.609 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36357 dashboard ceph NOFD /proc/33062/task/36357/fd (opendir: Permission denied) 2024-07-27T03:18:57.609 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36358 dashboard ceph cwd unknown /proc/33062/task/36358/cwd (readlink: Permission denied) 2024-07-27T03:18:57.609 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36358 dashboard ceph rtd unknown /proc/33062/task/36358/root (readlink: Permission denied) 2024-07-27T03:18:57.610 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36358 dashboard ceph txt unknown /proc/33062/task/36358/exe (readlink: Permission denied) 2024-07-27T03:18:57.610 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36358 dashboard ceph NOFD /proc/33062/task/36358/fd (opendir: Permission denied) 2024-07-27T03:18:57.610 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36359 dashboard ceph cwd unknown /proc/33062/task/36359/cwd (readlink: Permission denied) 2024-07-27T03:18:57.610 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36359 dashboard ceph rtd unknown /proc/33062/task/36359/root (readlink: Permission denied) 2024-07-27T03:18:57.610 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36359 dashboard ceph txt unknown /proc/33062/task/36359/exe (readlink: Permission denied) 2024-07-27T03:18:57.610 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36359 dashboard ceph NOFD /proc/33062/task/36359/fd (opendir: Permission denied) 2024-07-27T03:18:57.610 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36360 dashboard ceph cwd unknown /proc/33062/task/36360/cwd (readlink: Permission denied) 2024-07-27T03:18:57.610 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36360 dashboard ceph rtd unknown /proc/33062/task/36360/root (readlink: Permission denied) 2024-07-27T03:18:57.610 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36360 dashboard ceph txt unknown /proc/33062/task/36360/exe (readlink: Permission denied) 2024-07-27T03:18:57.610 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36360 dashboard ceph NOFD /proc/33062/task/36360/fd (opendir: Permission denied) 2024-07-27T03:18:57.610 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36361 dashboard ceph cwd unknown /proc/33062/task/36361/cwd (readlink: Permission denied) 2024-07-27T03:18:57.610 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36361 dashboard ceph rtd unknown /proc/33062/task/36361/root (readlink: Permission denied) 2024-07-27T03:18:57.610 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36361 dashboard ceph txt unknown /proc/33062/task/36361/exe (readlink: Permission denied) 2024-07-27T03:18:57.611 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36361 dashboard ceph NOFD /proc/33062/task/36361/fd (opendir: Permission denied) 2024-07-27T03:18:57.611 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36362 dashboard ceph cwd unknown /proc/33062/task/36362/cwd (readlink: Permission denied) 2024-07-27T03:18:57.611 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36362 dashboard ceph rtd unknown /proc/33062/task/36362/root (readlink: Permission denied) 2024-07-27T03:18:57.611 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36362 dashboard ceph txt unknown /proc/33062/task/36362/exe (readlink: Permission denied) 2024-07-27T03:18:57.611 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36362 dashboard ceph NOFD /proc/33062/task/36362/fd (opendir: Permission denied) 2024-07-27T03:18:57.611 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36363 dashboard ceph cwd unknown /proc/33062/task/36363/cwd (readlink: Permission denied) 2024-07-27T03:18:57.611 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36363 dashboard ceph rtd unknown /proc/33062/task/36363/root (readlink: Permission denied) 2024-07-27T03:18:57.611 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36363 dashboard ceph txt unknown /proc/33062/task/36363/exe (readlink: Permission denied) 2024-07-27T03:18:57.611 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36363 dashboard ceph NOFD /proc/33062/task/36363/fd (opendir: Permission denied) 2024-07-27T03:18:57.611 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36364 dashboard ceph cwd unknown /proc/33062/task/36364/cwd (readlink: Permission denied) 2024-07-27T03:18:57.611 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36364 dashboard ceph rtd unknown /proc/33062/task/36364/root (readlink: Permission denied) 2024-07-27T03:18:57.611 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36364 dashboard ceph txt unknown /proc/33062/task/36364/exe (readlink: Permission denied) 2024-07-27T03:18:57.611 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36364 dashboard ceph NOFD /proc/33062/task/36364/fd (opendir: Permission denied) 2024-07-27T03:18:57.611 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36365 dashboard ceph cwd unknown /proc/33062/task/36365/cwd (readlink: Permission denied) 2024-07-27T03:18:57.612 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36365 dashboard ceph rtd unknown /proc/33062/task/36365/root (readlink: Permission denied) 2024-07-27T03:18:57.612 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36365 dashboard ceph txt unknown /proc/33062/task/36365/exe (readlink: Permission denied) 2024-07-27T03:18:57.612 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36365 dashboard ceph NOFD /proc/33062/task/36365/fd (opendir: Permission denied) 2024-07-27T03:18:57.612 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36376 dashboard ceph cwd unknown /proc/33062/task/36376/cwd (readlink: Permission denied) 2024-07-27T03:18:57.612 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36376 dashboard ceph rtd unknown /proc/33062/task/36376/root (readlink: Permission denied) 2024-07-27T03:18:57.612 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36376 dashboard ceph txt unknown /proc/33062/task/36376/exe (readlink: Permission denied) 2024-07-27T03:18:57.612 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 36376 dashboard ceph NOFD /proc/33062/task/36376/fd (opendir: Permission denied) 2024-07-27T03:18:57.612 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55359 devicehea ceph cwd unknown /proc/33062/task/55359/cwd (readlink: Permission denied) 2024-07-27T03:18:57.612 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55359 devicehea ceph rtd unknown /proc/33062/task/55359/root (readlink: Permission denied) 2024-07-27T03:18:57.612 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55359 devicehea ceph txt unknown /proc/33062/task/55359/exe (readlink: Permission denied) 2024-07-27T03:18:57.612 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55359 devicehea ceph NOFD /proc/33062/task/55359/fd (opendir: Permission denied) 2024-07-27T03:18:57.612 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55365 io_contex ceph cwd unknown /proc/33062/task/55365/cwd (readlink: Permission denied) 2024-07-27T03:18:57.612 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55365 io_contex ceph rtd unknown /proc/33062/task/55365/root (readlink: Permission denied) 2024-07-27T03:18:57.612 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55365 io_contex ceph txt unknown /proc/33062/task/55365/exe (readlink: Permission denied) 2024-07-27T03:18:57.613 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55365 io_contex ceph NOFD /proc/33062/task/55365/fd (opendir: Permission denied) 2024-07-27T03:18:57.613 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55366 io_contex ceph cwd unknown /proc/33062/task/55366/cwd (readlink: Permission denied) 2024-07-27T03:18:57.613 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55366 io_contex ceph rtd unknown /proc/33062/task/55366/root (readlink: Permission denied) 2024-07-27T03:18:57.613 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55366 io_contex ceph txt unknown /proc/33062/task/55366/exe (readlink: Permission denied) 2024-07-27T03:18:57.613 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55366 io_contex ceph NOFD /proc/33062/task/55366/fd (opendir: Permission denied) 2024-07-27T03:18:57.613 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55367 ceph_time ceph cwd unknown /proc/33062/task/55367/cwd (readlink: Permission denied) 2024-07-27T03:18:57.613 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55367 ceph_time ceph rtd unknown /proc/33062/task/55367/root (readlink: Permission denied) 2024-07-27T03:18:57.613 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55367 ceph_time ceph txt unknown /proc/33062/task/55367/exe (readlink: Permission denied) 2024-07-27T03:18:57.613 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55367 ceph_time ceph NOFD /proc/33062/task/55367/fd (opendir: Permission denied) 2024-07-27T03:18:57.613 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55368 ms_dispat ceph cwd unknown /proc/33062/task/55368/cwd (readlink: Permission denied) 2024-07-27T03:18:57.613 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55368 ms_dispat ceph rtd unknown /proc/33062/task/55368/root (readlink: Permission denied) 2024-07-27T03:18:57.613 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55368 ms_dispat ceph txt unknown /proc/33062/task/55368/exe (readlink: Permission denied) 2024-07-27T03:18:57.613 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55368 ms_dispat ceph NOFD /proc/33062/task/55368/fd (opendir: Permission denied) 2024-07-27T03:18:57.614 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55369 ms_local ceph cwd unknown /proc/33062/task/55369/cwd (readlink: Permission denied) 2024-07-27T03:18:57.614 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55369 ms_local ceph rtd unknown /proc/33062/task/55369/root (readlink: Permission denied) 2024-07-27T03:18:57.614 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55369 ms_local ceph txt unknown /proc/33062/task/55369/exe (readlink: Permission denied) 2024-07-27T03:18:57.614 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55369 ms_local ceph NOFD /proc/33062/task/55369/fd (opendir: Permission denied) 2024-07-27T03:18:57.614 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55370 safe_time ceph cwd unknown /proc/33062/task/55370/cwd (readlink: Permission denied) 2024-07-27T03:18:57.614 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55370 safe_time ceph rtd unknown /proc/33062/task/55370/root (readlink: Permission denied) 2024-07-27T03:18:57.614 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55370 safe_time ceph txt unknown /proc/33062/task/55370/exe (readlink: Permission denied) 2024-07-27T03:18:57.614 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55370 safe_time ceph NOFD /proc/33062/task/55370/fd (opendir: Permission denied) 2024-07-27T03:18:57.614 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55371 safe_time ceph cwd unknown /proc/33062/task/55371/cwd (readlink: Permission denied) 2024-07-27T03:18:57.614 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55371 safe_time ceph rtd unknown /proc/33062/task/55371/root (readlink: Permission denied) 2024-07-27T03:18:57.614 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55371 safe_time ceph txt unknown /proc/33062/task/55371/exe (readlink: Permission denied) 2024-07-27T03:18:57.614 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mgr 33062 55371 safe_time ceph NOFD /proc/33062/task/55371/fd (opendir: Permission denied) 2024-07-27T03:18:57.614 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 34021 root cwd unknown /proc/34021/cwd (readlink: Permission denied) 2024-07-27T03:18:57.615 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 34021 root rtd unknown /proc/34021/root (readlink: Permission denied) 2024-07-27T03:18:57.615 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 34021 root txt unknown /proc/34021/exe (readlink: Permission denied) 2024-07-27T03:18:57.615 INFO:tasks.workunit.client.0.smithi067.stdout:systemd 34021 root NOFD /proc/34021/fd (opendir: Permission denied) 2024-07-27T03:18:57.615 INFO:tasks.workunit.client.0.smithi067.stdout:(sd-pam) 34023 root cwd unknown /proc/34023/cwd (readlink: Permission denied) 2024-07-27T03:18:57.615 INFO:tasks.workunit.client.0.smithi067.stdout:(sd-pam) 34023 root rtd unknown /proc/34023/root (readlink: Permission denied) 2024-07-27T03:18:57.615 INFO:tasks.workunit.client.0.smithi067.stdout:(sd-pam) 34023 root txt unknown /proc/34023/exe (readlink: Permission denied) 2024-07-27T03:18:57.615 INFO:tasks.workunit.client.0.smithi067.stdout:(sd-pam) 34023 root NOFD /proc/34023/fd (opendir: Permission denied) 2024-07-27T03:18:57.615 INFO:tasks.workunit.client.0.smithi067.stdout:sshd 35657 root cwd unknown /proc/35657/cwd (readlink: Permission denied) 2024-07-27T03:18:57.615 INFO:tasks.workunit.client.0.smithi067.stdout:sshd 35657 root rtd unknown /proc/35657/root (readlink: Permission denied) 2024-07-27T03:18:57.615 INFO:tasks.workunit.client.0.smithi067.stdout:sshd 35657 root txt unknown /proc/35657/exe (readlink: Permission denied) 2024-07-27T03:18:57.615 INFO:tasks.workunit.client.0.smithi067.stdout:sshd 35657 root NOFD /proc/35657/fd (opendir: Permission denied) 2024-07-27T03:18:57.615 INFO:tasks.workunit.client.0.smithi067.stdout:sshd 35660 root cwd unknown /proc/35660/cwd (readlink: Permission denied) 2024-07-27T03:18:57.615 INFO:tasks.workunit.client.0.smithi067.stdout:sshd 35660 root rtd unknown /proc/35660/root (readlink: Permission denied) 2024-07-27T03:18:57.615 INFO:tasks.workunit.client.0.smithi067.stdout:sshd 35660 root txt unknown /proc/35660/exe (readlink: Permission denied) 2024-07-27T03:18:57.616 INFO:tasks.workunit.client.0.smithi067.stdout:sshd 35660 root NOFD /proc/35660/fd (opendir: Permission denied) 2024-07-27T03:18:57.616 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 44747 root cwd unknown /proc/44747/cwd (readlink: Permission denied) 2024-07-27T03:18:57.616 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 44747 root rtd unknown /proc/44747/root (readlink: Permission denied) 2024-07-27T03:18:57.616 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 44747 root txt unknown /proc/44747/exe (readlink: Permission denied) 2024-07-27T03:18:57.616 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 44747 root NOFD /proc/44747/fd (opendir: Permission denied) 2024-07-27T03:18:57.616 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 46520 root cwd unknown /proc/46520/cwd (readlink: Permission denied) 2024-07-27T03:18:57.616 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 46520 root rtd unknown /proc/46520/root (readlink: Permission denied) 2024-07-27T03:18:57.616 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 46520 root txt unknown /proc/46520/exe (readlink: Permission denied) 2024-07-27T03:18:57.616 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 46520 root NOFD /proc/46520/fd (opendir: Permission denied) 2024-07-27T03:18:57.616 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 48560 root cwd unknown /proc/48560/cwd (readlink: Permission denied) 2024-07-27T03:18:57.616 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 48560 root rtd unknown /proc/48560/root (readlink: Permission denied) 2024-07-27T03:18:57.616 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 48560 root txt unknown /proc/48560/exe (readlink: Permission denied) 2024-07-27T03:18:57.616 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 48560 root NOFD /proc/48560/fd (opendir: Permission denied) 2024-07-27T03:18:57.616 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 50496 root cwd unknown /proc/50496/cwd (readlink: Permission denied) 2024-07-27T03:18:57.617 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 50496 root rtd unknown /proc/50496/root (readlink: Permission denied) 2024-07-27T03:18:57.617 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 50496 root txt unknown /proc/50496/exe (readlink: Permission denied) 2024-07-27T03:18:57.617 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 50496 root NOFD /proc/50496/fd (opendir: Permission denied) 2024-07-27T03:18:57.617 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 50498 root cwd unknown /proc/50498/cwd (readlink: Permission denied) 2024-07-27T03:18:57.617 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 50498 root rtd unknown /proc/50498/root (readlink: Permission denied) 2024-07-27T03:18:57.617 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 50498 root txt unknown /proc/50498/exe (readlink: Permission denied) 2024-07-27T03:18:57.617 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 50498 root NOFD /proc/50498/fd (opendir: Permission denied) 2024-07-27T03:18:57.617 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 ceph cwd unknown /proc/50500/cwd (readlink: Permission denied) 2024-07-27T03:18:57.617 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 ceph rtd unknown /proc/50500/root (readlink: Permission denied) 2024-07-27T03:18:57.617 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 ceph txt unknown /proc/50500/exe (readlink: Permission denied) 2024-07-27T03:18:57.617 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 ceph NOFD /proc/50500/fd (opendir: Permission denied) 2024-07-27T03:18:57.617 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50501 log ceph cwd unknown /proc/50500/task/50501/cwd (readlink: Permission denied) 2024-07-27T03:18:57.617 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50501 log ceph rtd unknown /proc/50500/task/50501/root (readlink: Permission denied) 2024-07-27T03:18:57.618 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50501 log ceph txt unknown /proc/50500/task/50501/exe (readlink: Permission denied) 2024-07-27T03:18:57.618 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50501 log ceph NOFD /proc/50500/task/50501/fd (opendir: Permission denied) 2024-07-27T03:18:57.618 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50503 msgr-work ceph cwd unknown /proc/50500/task/50503/cwd (readlink: Permission denied) 2024-07-27T03:18:57.618 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50503 msgr-work ceph rtd unknown /proc/50500/task/50503/root (readlink: Permission denied) 2024-07-27T03:18:57.618 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50503 msgr-work ceph txt unknown /proc/50500/task/50503/exe (readlink: Permission denied) 2024-07-27T03:18:57.618 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50503 msgr-work ceph NOFD /proc/50500/task/50503/fd (opendir: Permission denied) 2024-07-27T03:18:57.618 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50504 msgr-work ceph cwd unknown /proc/50500/task/50504/cwd (readlink: Permission denied) 2024-07-27T03:18:57.618 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50504 msgr-work ceph rtd unknown /proc/50500/task/50504/root (readlink: Permission denied) 2024-07-27T03:18:57.618 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50504 msgr-work ceph txt unknown /proc/50500/task/50504/exe (readlink: Permission denied) 2024-07-27T03:18:57.618 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50504 msgr-work ceph NOFD /proc/50500/task/50504/fd (opendir: Permission denied) 2024-07-27T03:18:57.618 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50505 msgr-work ceph cwd unknown /proc/50500/task/50505/cwd (readlink: Permission denied) 2024-07-27T03:18:57.618 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50505 msgr-work ceph rtd unknown /proc/50500/task/50505/root (readlink: Permission denied) 2024-07-27T03:18:57.618 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50505 msgr-work ceph txt unknown /proc/50500/task/50505/exe (readlink: Permission denied) 2024-07-27T03:18:57.619 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50505 msgr-work ceph NOFD /proc/50500/task/50505/fd (opendir: Permission denied) 2024-07-27T03:18:57.619 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50509 service ceph cwd unknown /proc/50500/task/50509/cwd (readlink: Permission denied) 2024-07-27T03:18:57.619 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50509 service ceph rtd unknown /proc/50500/task/50509/root (readlink: Permission denied) 2024-07-27T03:18:57.619 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50509 service ceph txt unknown /proc/50500/task/50509/exe (readlink: Permission denied) 2024-07-27T03:18:57.619 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50509 service ceph NOFD /proc/50500/task/50509/fd (opendir: Permission denied) 2024-07-27T03:18:57.619 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50510 admin_soc ceph cwd unknown /proc/50500/task/50510/cwd (readlink: Permission denied) 2024-07-27T03:18:57.619 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50510 admin_soc ceph rtd unknown /proc/50500/task/50510/root (readlink: Permission denied) 2024-07-27T03:18:57.619 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50510 admin_soc ceph txt unknown /proc/50500/task/50510/exe (readlink: Permission denied) 2024-07-27T03:18:57.619 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50510 admin_soc ceph NOFD /proc/50500/task/50510/fd (opendir: Permission denied) 2024-07-27T03:18:57.619 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50765 signal_ha ceph cwd unknown /proc/50500/task/50765/cwd (readlink: Permission denied) 2024-07-27T03:18:57.619 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50765 signal_ha ceph rtd unknown /proc/50500/task/50765/root (readlink: Permission denied) 2024-07-27T03:18:57.619 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50765 signal_ha ceph txt unknown /proc/50500/task/50765/exe (readlink: Permission denied) 2024-07-27T03:18:57.619 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50765 signal_ha ceph NOFD /proc/50500/task/50765/fd (opendir: Permission denied) 2024-07-27T03:18:57.619 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50766 io_contex ceph cwd unknown /proc/50500/task/50766/cwd (readlink: Permission denied) 2024-07-27T03:18:57.620 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50766 io_contex ceph rtd unknown /proc/50500/task/50766/root (readlink: Permission denied) 2024-07-27T03:18:57.620 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50766 io_contex ceph txt unknown /proc/50500/task/50766/exe (readlink: Permission denied) 2024-07-27T03:18:57.620 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50766 io_contex ceph NOFD /proc/50500/task/50766/fd (opendir: Permission denied) 2024-07-27T03:18:57.620 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50767 io_contex ceph cwd unknown /proc/50500/task/50767/cwd (readlink: Permission denied) 2024-07-27T03:18:57.620 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50767 io_contex ceph rtd unknown /proc/50500/task/50767/root (readlink: Permission denied) 2024-07-27T03:18:57.620 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50767 io_contex ceph txt unknown /proc/50500/task/50767/exe (readlink: Permission denied) 2024-07-27T03:18:57.620 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50767 io_contex ceph NOFD /proc/50500/task/50767/fd (opendir: Permission denied) 2024-07-27T03:18:57.620 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50795 OpHistory ceph cwd unknown /proc/50500/task/50795/cwd (readlink: Permission denied) 2024-07-27T03:18:57.620 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50795 OpHistory ceph rtd unknown /proc/50500/task/50795/root (readlink: Permission denied) 2024-07-27T03:18:57.620 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50795 OpHistory ceph txt unknown /proc/50500/task/50795/exe (readlink: Permission denied) 2024-07-27T03:18:57.620 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50795 OpHistory ceph NOFD /proc/50500/task/50795/fd (opendir: Permission denied) 2024-07-27T03:18:57.620 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50796 ceph_time ceph cwd unknown /proc/50500/task/50796/cwd (readlink: Permission denied) 2024-07-27T03:18:57.620 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50796 ceph_time ceph rtd unknown /proc/50500/task/50796/root (readlink: Permission denied) 2024-07-27T03:18:57.620 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50796 ceph_time ceph txt unknown /proc/50500/task/50796/exe (readlink: Permission denied) 2024-07-27T03:18:57.621 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50796 ceph_time ceph NOFD /proc/50500/task/50796/fd (opendir: Permission denied) 2024-07-27T03:18:57.621 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50816 ceph-osd ceph cwd unknown /proc/50500/task/50816/cwd (readlink: Permission denied) 2024-07-27T03:18:57.621 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50816 ceph-osd ceph rtd unknown /proc/50500/task/50816/root (readlink: Permission denied) 2024-07-27T03:18:57.621 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50816 ceph-osd ceph txt unknown /proc/50500/task/50816/exe (readlink: Permission denied) 2024-07-27T03:18:57.621 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50816 ceph-osd ceph NOFD /proc/50500/task/50816/fd (opendir: Permission denied) 2024-07-27T03:18:57.621 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50823 ceph-osd ceph cwd unknown /proc/50500/task/50823/cwd (readlink: Permission denied) 2024-07-27T03:18:57.621 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50823 ceph-osd ceph rtd unknown /proc/50500/task/50823/root (readlink: Permission denied) 2024-07-27T03:18:57.621 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50823 ceph-osd ceph txt unknown /proc/50500/task/50823/exe (readlink: Permission denied) 2024-07-27T03:18:57.621 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50823 ceph-osd ceph NOFD /proc/50500/task/50823/fd (opendir: Permission denied) 2024-07-27T03:18:57.621 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50854 ceph-osd ceph cwd unknown /proc/50500/task/50854/cwd (readlink: Permission denied) 2024-07-27T03:18:57.621 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50854 ceph-osd ceph rtd unknown /proc/50500/task/50854/root (readlink: Permission denied) 2024-07-27T03:18:57.621 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50854 ceph-osd ceph txt unknown /proc/50500/task/50854/exe (readlink: Permission denied) 2024-07-27T03:18:57.621 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50854 ceph-osd ceph NOFD /proc/50500/task/50854/fd (opendir: Permission denied) 2024-07-27T03:18:57.622 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50875 ceph-osd ceph cwd unknown /proc/50500/task/50875/cwd (readlink: Permission denied) 2024-07-27T03:18:57.622 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50875 ceph-osd ceph rtd unknown /proc/50500/task/50875/root (readlink: Permission denied) 2024-07-27T03:18:57.622 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50875 ceph-osd ceph txt unknown /proc/50500/task/50875/exe (readlink: Permission denied) 2024-07-27T03:18:57.622 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50875 ceph-osd ceph NOFD /proc/50500/task/50875/fd (opendir: Permission denied) 2024-07-27T03:18:57.622 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50880 ceph-osd ceph cwd unknown /proc/50500/task/50880/cwd (readlink: Permission denied) 2024-07-27T03:18:57.622 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50880 ceph-osd ceph rtd unknown /proc/50500/task/50880/root (readlink: Permission denied) 2024-07-27T03:18:57.622 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50880 ceph-osd ceph txt unknown /proc/50500/task/50880/exe (readlink: Permission denied) 2024-07-27T03:18:57.622 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50880 ceph-osd ceph NOFD /proc/50500/task/50880/fd (opendir: Permission denied) 2024-07-27T03:18:57.622 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50902 ceph-osd ceph cwd unknown /proc/50500/task/50902/cwd (readlink: Permission denied) 2024-07-27T03:18:57.622 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50902 ceph-osd ceph rtd unknown /proc/50500/task/50902/root (readlink: Permission denied) 2024-07-27T03:18:57.622 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50902 ceph-osd ceph txt unknown /proc/50500/task/50902/exe (readlink: Permission denied) 2024-07-27T03:18:57.622 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 50902 ceph-osd ceph NOFD /proc/50500/task/50902/fd (opendir: Permission denied) 2024-07-27T03:18:57.622 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51013 ceph-osd ceph cwd unknown /proc/50500/task/51013/cwd (readlink: Permission denied) 2024-07-27T03:18:57.622 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51013 ceph-osd ceph rtd unknown /proc/50500/task/51013/root (readlink: Permission denied) 2024-07-27T03:18:57.623 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51013 ceph-osd ceph txt unknown /proc/50500/task/51013/exe (readlink: Permission denied) 2024-07-27T03:18:57.623 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51013 ceph-osd ceph NOFD /proc/50500/task/51013/fd (opendir: Permission denied) 2024-07-27T03:18:57.623 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51048 ceph-osd ceph cwd unknown /proc/50500/task/51048/cwd (readlink: Permission denied) 2024-07-27T03:18:57.623 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51048 ceph-osd ceph rtd unknown /proc/50500/task/51048/root (readlink: Permission denied) 2024-07-27T03:18:57.623 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51048 ceph-osd ceph txt unknown /proc/50500/task/51048/exe (readlink: Permission denied) 2024-07-27T03:18:57.623 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51048 ceph-osd ceph NOFD /proc/50500/task/51048/fd (opendir: Permission denied) 2024-07-27T03:18:57.623 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51050 ceph-osd ceph cwd unknown /proc/50500/task/51050/cwd (readlink: Permission denied) 2024-07-27T03:18:57.623 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51050 ceph-osd ceph rtd unknown /proc/50500/task/51050/root (readlink: Permission denied) 2024-07-27T03:18:57.623 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51050 ceph-osd ceph txt unknown /proc/50500/task/51050/exe (readlink: Permission denied) 2024-07-27T03:18:57.623 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51050 ceph-osd ceph NOFD /proc/50500/task/51050/fd (opendir: Permission denied) 2024-07-27T03:18:57.623 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51051 safe_time ceph cwd unknown /proc/50500/task/51051/cwd (readlink: Permission denied) 2024-07-27T03:18:57.623 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51051 safe_time ceph rtd unknown /proc/50500/task/51051/root (readlink: Permission denied) 2024-07-27T03:18:57.623 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51051 safe_time ceph txt unknown /proc/50500/task/51051/exe (readlink: Permission denied) 2024-07-27T03:18:57.624 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51051 safe_time ceph NOFD /proc/50500/task/51051/fd (opendir: Permission denied) 2024-07-27T03:18:57.624 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51052 safe_time ceph cwd unknown /proc/50500/task/51052/cwd (readlink: Permission denied) 2024-07-27T03:18:57.624 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51052 safe_time ceph rtd unknown /proc/50500/task/51052/root (readlink: Permission denied) 2024-07-27T03:18:57.624 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51052 safe_time ceph txt unknown /proc/50500/task/51052/exe (readlink: Permission denied) 2024-07-27T03:18:57.624 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51052 safe_time ceph NOFD /proc/50500/task/51052/fd (opendir: Permission denied) 2024-07-27T03:18:57.624 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51053 safe_time ceph cwd unknown /proc/50500/task/51053/cwd (readlink: Permission denied) 2024-07-27T03:18:57.624 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51053 safe_time ceph rtd unknown /proc/50500/task/51053/root (readlink: Permission denied) 2024-07-27T03:18:57.624 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51053 safe_time ceph txt unknown /proc/50500/task/51053/exe (readlink: Permission denied) 2024-07-27T03:18:57.624 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51053 safe_time ceph NOFD /proc/50500/task/51053/fd (opendir: Permission denied) 2024-07-27T03:18:57.624 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51054 safe_time ceph cwd unknown /proc/50500/task/51054/cwd (readlink: Permission denied) 2024-07-27T03:18:57.624 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51054 safe_time ceph rtd unknown /proc/50500/task/51054/root (readlink: Permission denied) 2024-07-27T03:18:57.624 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51054 safe_time ceph txt unknown /proc/50500/task/51054/exe (readlink: Permission denied) 2024-07-27T03:18:57.624 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51054 safe_time ceph NOFD /proc/50500/task/51054/fd (opendir: Permission denied) 2024-07-27T03:18:57.624 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51055 fn_anonym ceph cwd unknown /proc/50500/task/51055/cwd (readlink: Permission denied) 2024-07-27T03:18:57.624 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51055 fn_anonym ceph rtd unknown /proc/50500/task/51055/root (readlink: Permission denied) 2024-07-27T03:18:57.625 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51055 fn_anonym ceph txt unknown /proc/50500/task/51055/exe (readlink: Permission denied) 2024-07-27T03:18:57.625 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51055 fn_anonym ceph NOFD /proc/50500/task/51055/fd (opendir: Permission denied) 2024-07-27T03:18:57.625 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51063 rocksdb:l ceph cwd unknown /proc/50500/task/51063/cwd (readlink: Permission denied) 2024-07-27T03:18:57.625 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51063 rocksdb:l ceph rtd unknown /proc/50500/task/51063/root (readlink: Permission denied) 2024-07-27T03:18:57.625 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51063 rocksdb:l ceph txt unknown /proc/50500/task/51063/exe (readlink: Permission denied) 2024-07-27T03:18:57.625 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51063 rocksdb:l ceph NOFD /proc/50500/task/51063/fd (opendir: Permission denied) 2024-07-27T03:18:57.625 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51064 rocksdb:l ceph cwd unknown /proc/50500/task/51064/cwd (readlink: Permission denied) 2024-07-27T03:18:57.625 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51064 rocksdb:l ceph rtd unknown /proc/50500/task/51064/root (readlink: Permission denied) 2024-07-27T03:18:57.625 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51064 rocksdb:l ceph txt unknown /proc/50500/task/51064/exe (readlink: Permission denied) 2024-07-27T03:18:57.625 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51064 rocksdb:l ceph NOFD /proc/50500/task/51064/fd (opendir: Permission denied) 2024-07-27T03:18:57.625 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51065 rocksdb:l ceph cwd unknown /proc/50500/task/51065/cwd (readlink: Permission denied) 2024-07-27T03:18:57.625 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51065 rocksdb:l ceph rtd unknown /proc/50500/task/51065/root (readlink: Permission denied) 2024-07-27T03:18:57.625 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51065 rocksdb:l ceph txt unknown /proc/50500/task/51065/exe (readlink: Permission denied) 2024-07-27T03:18:57.626 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51065 rocksdb:l ceph NOFD /proc/50500/task/51065/fd (opendir: Permission denied) 2024-07-27T03:18:57.626 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51066 rocksdb:h ceph cwd unknown /proc/50500/task/51066/cwd (readlink: Permission denied) 2024-07-27T03:18:57.626 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51066 rocksdb:h ceph rtd unknown /proc/50500/task/51066/root (readlink: Permission denied) 2024-07-27T03:18:57.626 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51066 rocksdb:h ceph txt unknown /proc/50500/task/51066/exe (readlink: Permission denied) 2024-07-27T03:18:57.626 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51066 rocksdb:h ceph NOFD /proc/50500/task/51066/fd (opendir: Permission denied) 2024-07-27T03:18:57.626 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51567 bstore_ai ceph cwd unknown /proc/50500/task/51567/cwd (readlink: Permission denied) 2024-07-27T03:18:57.626 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51567 bstore_ai ceph rtd unknown /proc/50500/task/51567/root (readlink: Permission denied) 2024-07-27T03:18:57.626 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51567 bstore_ai ceph txt unknown /proc/50500/task/51567/exe (readlink: Permission denied) 2024-07-27T03:18:57.626 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51567 bstore_ai ceph NOFD /proc/50500/task/51567/fd (opendir: Permission denied) 2024-07-27T03:18:57.626 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51771 bstore_ai ceph cwd unknown /proc/50500/task/51771/cwd (readlink: Permission denied) 2024-07-27T03:18:57.626 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51771 bstore_ai ceph rtd unknown /proc/50500/task/51771/root (readlink: Permission denied) 2024-07-27T03:18:57.626 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51771 bstore_ai ceph txt unknown /proc/50500/task/51771/exe (readlink: Permission denied) 2024-07-27T03:18:57.626 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51771 bstore_ai ceph NOFD /proc/50500/task/51771/fd (opendir: Permission denied) 2024-07-27T03:18:57.627 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51960 ceph-osd ceph cwd unknown /proc/50500/task/51960/cwd (readlink: Permission denied) 2024-07-27T03:18:57.627 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51960 ceph-osd ceph rtd unknown /proc/50500/task/51960/root (readlink: Permission denied) 2024-07-27T03:18:57.627 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51960 ceph-osd ceph txt unknown /proc/50500/task/51960/exe (readlink: Permission denied) 2024-07-27T03:18:57.627 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51960 ceph-osd ceph NOFD /proc/50500/task/51960/fd (opendir: Permission denied) 2024-07-27T03:18:57.627 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51961 cfin ceph cwd unknown /proc/50500/task/51961/cwd (readlink: Permission denied) 2024-07-27T03:18:57.627 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51961 cfin ceph rtd unknown /proc/50500/task/51961/root (readlink: Permission denied) 2024-07-27T03:18:57.627 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51961 cfin ceph txt unknown /proc/50500/task/51961/exe (readlink: Permission denied) 2024-07-27T03:18:57.627 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51961 cfin ceph NOFD /proc/50500/task/51961/fd (opendir: Permission denied) 2024-07-27T03:18:57.627 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51962 bstore_kv ceph cwd unknown /proc/50500/task/51962/cwd (readlink: Permission denied) 2024-07-27T03:18:57.627 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51962 bstore_kv ceph rtd unknown /proc/50500/task/51962/root (readlink: Permission denied) 2024-07-27T03:18:57.627 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51962 bstore_kv ceph txt unknown /proc/50500/task/51962/exe (readlink: Permission denied) 2024-07-27T03:18:57.627 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51962 bstore_kv ceph NOFD /proc/50500/task/51962/fd (opendir: Permission denied) 2024-07-27T03:18:57.627 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51963 bstore_kv ceph cwd unknown /proc/50500/task/51963/cwd (readlink: Permission denied) 2024-07-27T03:18:57.627 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51963 bstore_kv ceph rtd unknown /proc/50500/task/51963/root (readlink: Permission denied) 2024-07-27T03:18:57.628 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51963 bstore_kv ceph txt unknown /proc/50500/task/51963/exe (readlink: Permission denied) 2024-07-27T03:18:57.628 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51963 bstore_kv ceph NOFD /proc/50500/task/51963/fd (opendir: Permission denied) 2024-07-27T03:18:57.628 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51964 bstore_me ceph cwd unknown /proc/50500/task/51964/cwd (readlink: Permission denied) 2024-07-27T03:18:57.628 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51964 bstore_me ceph rtd unknown /proc/50500/task/51964/root (readlink: Permission denied) 2024-07-27T03:18:57.628 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51964 bstore_me ceph txt unknown /proc/50500/task/51964/exe (readlink: Permission denied) 2024-07-27T03:18:57.628 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51964 bstore_me ceph NOFD /proc/50500/task/51964/fd (opendir: Permission denied) 2024-07-27T03:18:57.628 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51969 ms_dispat ceph cwd unknown /proc/50500/task/51969/cwd (readlink: Permission denied) 2024-07-27T03:18:57.628 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51969 ms_dispat ceph rtd unknown /proc/50500/task/51969/root (readlink: Permission denied) 2024-07-27T03:18:57.628 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51969 ms_dispat ceph txt unknown /proc/50500/task/51969/exe (readlink: Permission denied) 2024-07-27T03:18:57.628 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51969 ms_dispat ceph NOFD /proc/50500/task/51969/fd (opendir: Permission denied) 2024-07-27T03:18:57.628 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51970 ms_local ceph cwd unknown /proc/50500/task/51970/cwd (readlink: Permission denied) 2024-07-27T03:18:57.628 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51970 ms_local ceph rtd unknown /proc/50500/task/51970/root (readlink: Permission denied) 2024-07-27T03:18:57.628 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51970 ms_local ceph txt unknown /proc/50500/task/51970/exe (readlink: Permission denied) 2024-07-27T03:18:57.628 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51970 ms_local ceph NOFD /proc/50500/task/51970/fd (opendir: Permission denied) 2024-07-27T03:18:57.629 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51971 safe_time ceph cwd unknown /proc/50500/task/51971/cwd (readlink: Permission denied) 2024-07-27T03:18:57.629 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51971 safe_time ceph rtd unknown /proc/50500/task/51971/root (readlink: Permission denied) 2024-07-27T03:18:57.629 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51971 safe_time ceph txt unknown /proc/50500/task/51971/exe (readlink: Permission denied) 2024-07-27T03:18:57.629 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51971 safe_time ceph NOFD /proc/50500/task/51971/fd (opendir: Permission denied) 2024-07-27T03:18:57.629 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51972 safe_time ceph cwd unknown /proc/50500/task/51972/cwd (readlink: Permission denied) 2024-07-27T03:18:57.629 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51972 safe_time ceph rtd unknown /proc/50500/task/51972/root (readlink: Permission denied) 2024-07-27T03:18:57.629 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51972 safe_time ceph txt unknown /proc/50500/task/51972/exe (readlink: Permission denied) 2024-07-27T03:18:57.629 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51972 safe_time ceph NOFD /proc/50500/task/51972/fd (opendir: Permission denied) 2024-07-27T03:18:57.629 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51973 ms_dispat ceph cwd unknown /proc/50500/task/51973/cwd (readlink: Permission denied) 2024-07-27T03:18:57.629 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51973 ms_dispat ceph rtd unknown /proc/50500/task/51973/root (readlink: Permission denied) 2024-07-27T03:18:57.629 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51973 ms_dispat ceph txt unknown /proc/50500/task/51973/exe (readlink: Permission denied) 2024-07-27T03:18:57.629 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51973 ms_dispat ceph NOFD /proc/50500/task/51973/fd (opendir: Permission denied) 2024-07-27T03:18:57.629 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51974 ms_local ceph cwd unknown /proc/50500/task/51974/cwd (readlink: Permission denied) 2024-07-27T03:18:57.630 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51974 ms_local ceph rtd unknown /proc/50500/task/51974/root (readlink: Permission denied) 2024-07-27T03:18:57.630 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51974 ms_local ceph txt unknown /proc/50500/task/51974/exe (readlink: Permission denied) 2024-07-27T03:18:57.630 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51974 ms_local ceph NOFD /proc/50500/task/51974/fd (opendir: Permission denied) 2024-07-27T03:18:57.630 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51975 ms_dispat ceph cwd unknown /proc/50500/task/51975/cwd (readlink: Permission denied) 2024-07-27T03:18:57.630 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51975 ms_dispat ceph rtd unknown /proc/50500/task/51975/root (readlink: Permission denied) 2024-07-27T03:18:57.630 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51975 ms_dispat ceph txt unknown /proc/50500/task/51975/exe (readlink: Permission denied) 2024-07-27T03:18:57.630 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51975 ms_dispat ceph NOFD /proc/50500/task/51975/fd (opendir: Permission denied) 2024-07-27T03:18:57.630 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51976 ms_local ceph cwd unknown /proc/50500/task/51976/cwd (readlink: Permission denied) 2024-07-27T03:18:57.630 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51976 ms_local ceph rtd unknown /proc/50500/task/51976/root (readlink: Permission denied) 2024-07-27T03:18:57.630 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51976 ms_local ceph txt unknown /proc/50500/task/51976/exe (readlink: Permission denied) 2024-07-27T03:18:57.630 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51976 ms_local ceph NOFD /proc/50500/task/51976/fd (opendir: Permission denied) 2024-07-27T03:18:57.630 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51977 ms_dispat ceph cwd unknown /proc/50500/task/51977/cwd (readlink: Permission denied) 2024-07-27T03:18:57.630 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51977 ms_dispat ceph rtd unknown /proc/50500/task/51977/root (readlink: Permission denied) 2024-07-27T03:18:57.630 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51977 ms_dispat ceph txt unknown /proc/50500/task/51977/exe (readlink: Permission denied) 2024-07-27T03:18:57.631 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51977 ms_dispat ceph NOFD /proc/50500/task/51977/fd (opendir: Permission denied) 2024-07-27T03:18:57.631 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51978 ms_local ceph cwd unknown /proc/50500/task/51978/cwd (readlink: Permission denied) 2024-07-27T03:18:57.631 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51978 ms_local ceph rtd unknown /proc/50500/task/51978/root (readlink: Permission denied) 2024-07-27T03:18:57.631 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51978 ms_local ceph txt unknown /proc/50500/task/51978/exe (readlink: Permission denied) 2024-07-27T03:18:57.631 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51978 ms_local ceph NOFD /proc/50500/task/51978/fd (opendir: Permission denied) 2024-07-27T03:18:57.631 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51979 ms_dispat ceph cwd unknown /proc/50500/task/51979/cwd (readlink: Permission denied) 2024-07-27T03:18:57.631 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51979 ms_dispat ceph rtd unknown /proc/50500/task/51979/root (readlink: Permission denied) 2024-07-27T03:18:57.631 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51979 ms_dispat ceph txt unknown /proc/50500/task/51979/exe (readlink: Permission denied) 2024-07-27T03:18:57.631 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51979 ms_dispat ceph NOFD /proc/50500/task/51979/fd (opendir: Permission denied) 2024-07-27T03:18:57.631 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51980 ms_local ceph cwd unknown /proc/50500/task/51980/cwd (readlink: Permission denied) 2024-07-27T03:18:57.631 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51980 ms_local ceph rtd unknown /proc/50500/task/51980/root (readlink: Permission denied) 2024-07-27T03:18:57.631 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51980 ms_local ceph txt unknown /proc/50500/task/51980/exe (readlink: Permission denied) 2024-07-27T03:18:57.631 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51980 ms_local ceph NOFD /proc/50500/task/51980/fd (opendir: Permission denied) 2024-07-27T03:18:57.631 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51981 ms_dispat ceph cwd unknown /proc/50500/task/51981/cwd (readlink: Permission denied) 2024-07-27T03:18:57.632 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51981 ms_dispat ceph rtd unknown /proc/50500/task/51981/root (readlink: Permission denied) 2024-07-27T03:18:57.632 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51981 ms_dispat ceph txt unknown /proc/50500/task/51981/exe (readlink: Permission denied) 2024-07-27T03:18:57.632 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51981 ms_dispat ceph NOFD /proc/50500/task/51981/fd (opendir: Permission denied) 2024-07-27T03:18:57.632 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51982 ms_local ceph cwd unknown /proc/50500/task/51982/cwd (readlink: Permission denied) 2024-07-27T03:18:57.632 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51982 ms_local ceph rtd unknown /proc/50500/task/51982/root (readlink: Permission denied) 2024-07-27T03:18:57.632 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51982 ms_local ceph txt unknown /proc/50500/task/51982/exe (readlink: Permission denied) 2024-07-27T03:18:57.632 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51982 ms_local ceph NOFD /proc/50500/task/51982/fd (opendir: Permission denied) 2024-07-27T03:18:57.632 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51983 ms_dispat ceph cwd unknown /proc/50500/task/51983/cwd (readlink: Permission denied) 2024-07-27T03:18:57.632 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51983 ms_dispat ceph rtd unknown /proc/50500/task/51983/root (readlink: Permission denied) 2024-07-27T03:18:57.632 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51983 ms_dispat ceph txt unknown /proc/50500/task/51983/exe (readlink: Permission denied) 2024-07-27T03:18:57.632 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51983 ms_dispat ceph NOFD /proc/50500/task/51983/fd (opendir: Permission denied) 2024-07-27T03:18:57.632 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51984 ms_local ceph cwd unknown /proc/50500/task/51984/cwd (readlink: Permission denied) 2024-07-27T03:18:57.632 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51984 ms_local ceph rtd unknown /proc/50500/task/51984/root (readlink: Permission denied) 2024-07-27T03:18:57.632 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51984 ms_local ceph txt unknown /proc/50500/task/51984/exe (readlink: Permission denied) 2024-07-27T03:18:57.633 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51984 ms_local ceph NOFD /proc/50500/task/51984/fd (opendir: Permission denied) 2024-07-27T03:18:57.633 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51985 fn_anonym ceph cwd unknown /proc/50500/task/51985/cwd (readlink: Permission denied) 2024-07-27T03:18:57.633 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51985 fn_anonym ceph rtd unknown /proc/50500/task/51985/root (readlink: Permission denied) 2024-07-27T03:18:57.633 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51985 fn_anonym ceph txt unknown /proc/50500/task/51985/exe (readlink: Permission denied) 2024-07-27T03:18:57.633 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51985 fn_anonym ceph NOFD /proc/50500/task/51985/fd (opendir: Permission denied) 2024-07-27T03:18:57.633 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51986 finisher ceph cwd unknown /proc/50500/task/51986/cwd (readlink: Permission denied) 2024-07-27T03:18:57.633 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51986 finisher ceph rtd unknown /proc/50500/task/51986/root (readlink: Permission denied) 2024-07-27T03:18:57.633 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51986 finisher ceph txt unknown /proc/50500/task/51986/exe (readlink: Permission denied) 2024-07-27T03:18:57.633 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51986 finisher ceph NOFD /proc/50500/task/51986/fd (opendir: Permission denied) 2024-07-27T03:18:57.633 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51987 safe_time ceph cwd unknown /proc/50500/task/51987/cwd (readlink: Permission denied) 2024-07-27T03:18:57.633 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51987 safe_time ceph rtd unknown /proc/50500/task/51987/root (readlink: Permission denied) 2024-07-27T03:18:57.633 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51987 safe_time ceph txt unknown /proc/50500/task/51987/exe (readlink: Permission denied) 2024-07-27T03:18:57.633 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51987 safe_time ceph NOFD /proc/50500/task/51987/fd (opendir: Permission denied) 2024-07-27T03:18:57.633 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51988 safe_time ceph cwd unknown /proc/50500/task/51988/cwd (readlink: Permission denied) 2024-07-27T03:18:57.634 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51988 safe_time ceph rtd unknown /proc/50500/task/51988/root (readlink: Permission denied) 2024-07-27T03:18:57.634 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51988 safe_time ceph txt unknown /proc/50500/task/51988/exe (readlink: Permission denied) 2024-07-27T03:18:57.634 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51988 safe_time ceph NOFD /proc/50500/task/51988/fd (opendir: Permission denied) 2024-07-27T03:18:57.634 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51989 ceph-osd ceph cwd unknown /proc/50500/task/51989/cwd (readlink: Permission denied) 2024-07-27T03:18:57.634 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51989 ceph-osd ceph rtd unknown /proc/50500/task/51989/root (readlink: Permission denied) 2024-07-27T03:18:57.634 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51989 ceph-osd ceph txt unknown /proc/50500/task/51989/exe (readlink: Permission denied) 2024-07-27T03:18:57.634 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51989 ceph-osd ceph NOFD /proc/50500/task/51989/fd (opendir: Permission denied) 2024-07-27T03:18:57.634 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51990 osd_srv_a ceph cwd unknown /proc/50500/task/51990/cwd (readlink: Permission denied) 2024-07-27T03:18:57.634 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51990 osd_srv_a ceph rtd unknown /proc/50500/task/51990/root (readlink: Permission denied) 2024-07-27T03:18:57.634 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51990 osd_srv_a ceph txt unknown /proc/50500/task/51990/exe (readlink: Permission denied) 2024-07-27T03:18:57.634 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51990 osd_srv_a ceph NOFD /proc/50500/task/51990/fd (opendir: Permission denied) 2024-07-27T03:18:57.634 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51991 tp_osd_tp ceph cwd unknown /proc/50500/task/51991/cwd (readlink: Permission denied) 2024-07-27T03:18:57.634 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51991 tp_osd_tp ceph rtd unknown /proc/50500/task/51991/root (readlink: Permission denied) 2024-07-27T03:18:57.635 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51991 tp_osd_tp ceph txt unknown /proc/50500/task/51991/exe (readlink: Permission denied) 2024-07-27T03:18:57.635 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51991 tp_osd_tp ceph NOFD /proc/50500/task/51991/fd (opendir: Permission denied) 2024-07-27T03:18:57.635 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51992 tp_osd_tp ceph cwd unknown /proc/50500/task/51992/cwd (readlink: Permission denied) 2024-07-27T03:18:57.635 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51992 tp_osd_tp ceph rtd unknown /proc/50500/task/51992/root (readlink: Permission denied) 2024-07-27T03:18:57.635 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51992 tp_osd_tp ceph txt unknown /proc/50500/task/51992/exe (readlink: Permission denied) 2024-07-27T03:18:57.635 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51992 tp_osd_tp ceph NOFD /proc/50500/task/51992/fd (opendir: Permission denied) 2024-07-27T03:18:57.635 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51993 tp_osd_tp ceph cwd unknown /proc/50500/task/51993/cwd (readlink: Permission denied) 2024-07-27T03:18:57.635 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51993 tp_osd_tp ceph rtd unknown /proc/50500/task/51993/root (readlink: Permission denied) 2024-07-27T03:18:57.635 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51993 tp_osd_tp ceph txt unknown /proc/50500/task/51993/exe (readlink: Permission denied) 2024-07-27T03:18:57.635 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51993 tp_osd_tp ceph NOFD /proc/50500/task/51993/fd (opendir: Permission denied) 2024-07-27T03:18:57.635 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51994 tp_osd_tp ceph cwd unknown /proc/50500/task/51994/cwd (readlink: Permission denied) 2024-07-27T03:18:57.635 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51994 tp_osd_tp ceph rtd unknown /proc/50500/task/51994/root (readlink: Permission denied) 2024-07-27T03:18:57.635 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51994 tp_osd_tp ceph txt unknown /proc/50500/task/51994/exe (readlink: Permission denied) 2024-07-27T03:18:57.635 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51994 tp_osd_tp ceph NOFD /proc/50500/task/51994/fd (opendir: Permission denied) 2024-07-27T03:18:57.636 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51995 tp_osd_tp ceph cwd unknown /proc/50500/task/51995/cwd (readlink: Permission denied) 2024-07-27T03:18:57.636 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51995 tp_osd_tp ceph rtd unknown /proc/50500/task/51995/root (readlink: Permission denied) 2024-07-27T03:18:57.636 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51995 tp_osd_tp ceph txt unknown /proc/50500/task/51995/exe (readlink: Permission denied) 2024-07-27T03:18:57.636 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51995 tp_osd_tp ceph NOFD /proc/50500/task/51995/fd (opendir: Permission denied) 2024-07-27T03:18:57.636 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51996 tp_osd_tp ceph cwd unknown /proc/50500/task/51996/cwd (readlink: Permission denied) 2024-07-27T03:18:57.636 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51996 tp_osd_tp ceph rtd unknown /proc/50500/task/51996/root (readlink: Permission denied) 2024-07-27T03:18:57.636 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51996 tp_osd_tp ceph txt unknown /proc/50500/task/51996/exe (readlink: Permission denied) 2024-07-27T03:18:57.636 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51996 tp_osd_tp ceph NOFD /proc/50500/task/51996/fd (opendir: Permission denied) 2024-07-27T03:18:57.636 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51997 tp_osd_tp ceph cwd unknown /proc/50500/task/51997/cwd (readlink: Permission denied) 2024-07-27T03:18:57.636 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51997 tp_osd_tp ceph rtd unknown /proc/50500/task/51997/root (readlink: Permission denied) 2024-07-27T03:18:57.636 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51997 tp_osd_tp ceph txt unknown /proc/50500/task/51997/exe (readlink: Permission denied) 2024-07-27T03:18:57.636 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51997 tp_osd_tp ceph NOFD /proc/50500/task/51997/fd (opendir: Permission denied) 2024-07-27T03:18:57.636 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51998 tp_osd_tp ceph cwd unknown /proc/50500/task/51998/cwd (readlink: Permission denied) 2024-07-27T03:18:57.637 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51998 tp_osd_tp ceph rtd unknown /proc/50500/task/51998/root (readlink: Permission denied) 2024-07-27T03:18:57.637 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51998 tp_osd_tp ceph txt unknown /proc/50500/task/51998/exe (readlink: Permission denied) 2024-07-27T03:18:57.637 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51998 tp_osd_tp ceph NOFD /proc/50500/task/51998/fd (opendir: Permission denied) 2024-07-27T03:18:57.637 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51999 tp_osd_tp ceph cwd unknown /proc/50500/task/51999/cwd (readlink: Permission denied) 2024-07-27T03:18:57.637 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51999 tp_osd_tp ceph rtd unknown /proc/50500/task/51999/root (readlink: Permission denied) 2024-07-27T03:18:57.637 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51999 tp_osd_tp ceph txt unknown /proc/50500/task/51999/exe (readlink: Permission denied) 2024-07-27T03:18:57.637 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 51999 tp_osd_tp ceph NOFD /proc/50500/task/51999/fd (opendir: Permission denied) 2024-07-27T03:18:57.637 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52000 tp_osd_tp ceph cwd unknown /proc/50500/task/52000/cwd (readlink: Permission denied) 2024-07-27T03:18:57.637 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52000 tp_osd_tp ceph rtd unknown /proc/50500/task/52000/root (readlink: Permission denied) 2024-07-27T03:18:57.637 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52000 tp_osd_tp ceph txt unknown /proc/50500/task/52000/exe (readlink: Permission denied) 2024-07-27T03:18:57.637 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52000 tp_osd_tp ceph NOFD /proc/50500/task/52000/fd (opendir: Permission denied) 2024-07-27T03:18:57.637 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52001 tp_osd_tp ceph cwd unknown /proc/50500/task/52001/cwd (readlink: Permission denied) 2024-07-27T03:18:57.637 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52001 tp_osd_tp ceph rtd unknown /proc/50500/task/52001/root (readlink: Permission denied) 2024-07-27T03:18:57.638 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52001 tp_osd_tp ceph txt unknown /proc/50500/task/52001/exe (readlink: Permission denied) 2024-07-27T03:18:57.638 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52001 tp_osd_tp ceph NOFD /proc/50500/task/52001/fd (opendir: Permission denied) 2024-07-27T03:18:57.638 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52002 tp_osd_tp ceph cwd unknown /proc/50500/task/52002/cwd (readlink: Permission denied) 2024-07-27T03:18:57.638 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52002 tp_osd_tp ceph rtd unknown /proc/50500/task/52002/root (readlink: Permission denied) 2024-07-27T03:18:57.638 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52002 tp_osd_tp ceph txt unknown /proc/50500/task/52002/exe (readlink: Permission denied) 2024-07-27T03:18:57.638 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52002 tp_osd_tp ceph NOFD /proc/50500/task/52002/fd (opendir: Permission denied) 2024-07-27T03:18:57.638 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52003 tp_osd_tp ceph cwd unknown /proc/50500/task/52003/cwd (readlink: Permission denied) 2024-07-27T03:18:57.638 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52003 tp_osd_tp ceph rtd unknown /proc/50500/task/52003/root (readlink: Permission denied) 2024-07-27T03:18:57.638 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52003 tp_osd_tp ceph txt unknown /proc/50500/task/52003/exe (readlink: Permission denied) 2024-07-27T03:18:57.638 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52003 tp_osd_tp ceph NOFD /proc/50500/task/52003/fd (opendir: Permission denied) 2024-07-27T03:18:57.638 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52004 tp_osd_tp ceph cwd unknown /proc/50500/task/52004/cwd (readlink: Permission denied) 2024-07-27T03:18:57.638 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52004 tp_osd_tp ceph rtd unknown /proc/50500/task/52004/root (readlink: Permission denied) 2024-07-27T03:18:57.638 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52004 tp_osd_tp ceph txt unknown /proc/50500/task/52004/exe (readlink: Permission denied) 2024-07-27T03:18:57.639 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52004 tp_osd_tp ceph NOFD /proc/50500/task/52004/fd (opendir: Permission denied) 2024-07-27T03:18:57.639 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52005 tp_osd_tp ceph cwd unknown /proc/50500/task/52005/cwd (readlink: Permission denied) 2024-07-27T03:18:57.639 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52005 tp_osd_tp ceph rtd unknown /proc/50500/task/52005/root (readlink: Permission denied) 2024-07-27T03:18:57.639 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52005 tp_osd_tp ceph txt unknown /proc/50500/task/52005/exe (readlink: Permission denied) 2024-07-27T03:18:57.639 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52005 tp_osd_tp ceph NOFD /proc/50500/task/52005/fd (opendir: Permission denied) 2024-07-27T03:18:57.639 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52006 tp_osd_tp ceph cwd unknown /proc/50500/task/52006/cwd (readlink: Permission denied) 2024-07-27T03:18:57.639 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52006 tp_osd_tp ceph rtd unknown /proc/50500/task/52006/root (readlink: Permission denied) 2024-07-27T03:18:57.639 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52006 tp_osd_tp ceph txt unknown /proc/50500/task/52006/exe (readlink: Permission denied) 2024-07-27T03:18:57.639 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52006 tp_osd_tp ceph NOFD /proc/50500/task/52006/fd (opendir: Permission denied) 2024-07-27T03:18:57.639 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52007 osd_srv_h ceph cwd unknown /proc/50500/task/52007/cwd (readlink: Permission denied) 2024-07-27T03:18:57.639 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52007 osd_srv_h ceph rtd unknown /proc/50500/task/52007/root (readlink: Permission denied) 2024-07-27T03:18:57.639 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52007 osd_srv_h ceph txt unknown /proc/50500/task/52007/exe (readlink: Permission denied) 2024-07-27T03:18:57.639 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 50500 52007 osd_srv_h ceph NOFD /proc/50500/task/52007/fd (opendir: Permission denied) 2024-07-27T03:18:57.639 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 51753 root cwd unknown /proc/51753/cwd (readlink: Permission denied) 2024-07-27T03:18:57.640 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 51753 root rtd unknown /proc/51753/root (readlink: Permission denied) 2024-07-27T03:18:57.640 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 51753 root txt unknown /proc/51753/exe (readlink: Permission denied) 2024-07-27T03:18:57.640 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 51753 root NOFD /proc/51753/fd (opendir: Permission denied) 2024-07-27T03:18:57.640 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 51770 root cwd unknown /proc/51770/cwd (readlink: Permission denied) 2024-07-27T03:18:57.640 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 51770 root rtd unknown /proc/51770/root (readlink: Permission denied) 2024-07-27T03:18:57.640 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 51770 root txt unknown /proc/51770/exe (readlink: Permission denied) 2024-07-27T03:18:57.640 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 51770 root NOFD /proc/51770/fd (opendir: Permission denied) 2024-07-27T03:18:57.640 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/4 52823 root cwd unknown /proc/52823/cwd (readlink: Permission denied) 2024-07-27T03:18:57.640 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/4 52823 root rtd unknown /proc/52823/root (readlink: Permission denied) 2024-07-27T03:18:57.640 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/4 52823 root txt unknown /proc/52823/exe (readlink: Permission denied) 2024-07-27T03:18:57.640 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/4 52823 root NOFD /proc/52823/fd (opendir: Permission denied) 2024-07-27T03:18:57.640 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 56732 root cwd unknown /proc/56732/cwd (readlink: Permission denied) 2024-07-27T03:18:57.640 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 56732 root rtd unknown /proc/56732/root (readlink: Permission denied) 2024-07-27T03:18:57.640 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 56732 root txt unknown /proc/56732/exe (readlink: Permission denied) 2024-07-27T03:18:57.641 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 56732 root NOFD /proc/56732/fd (opendir: Permission denied) 2024-07-27T03:18:57.641 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 57396 root cwd unknown /proc/57396/cwd (readlink: Permission denied) 2024-07-27T03:18:57.641 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 57396 root rtd unknown /proc/57396/root (readlink: Permission denied) 2024-07-27T03:18:57.641 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 57396 root txt unknown /proc/57396/exe (readlink: Permission denied) 2024-07-27T03:18:57.641 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 57396 root NOFD /proc/57396/fd (opendir: Permission denied) 2024-07-27T03:18:57.641 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 57398 root cwd unknown /proc/57398/cwd (readlink: Permission denied) 2024-07-27T03:18:57.641 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 57398 root rtd unknown /proc/57398/root (readlink: Permission denied) 2024-07-27T03:18:57.641 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 57398 root txt unknown /proc/57398/exe (readlink: Permission denied) 2024-07-27T03:18:57.641 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 57398 root NOFD /proc/57398/fd (opendir: Permission denied) 2024-07-27T03:18:57.641 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 ceph cwd unknown /proc/57400/cwd (readlink: Permission denied) 2024-07-27T03:18:57.641 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 ceph rtd unknown /proc/57400/root (readlink: Permission denied) 2024-07-27T03:18:57.641 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 ceph txt unknown /proc/57400/exe (readlink: Permission denied) 2024-07-27T03:18:57.641 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 ceph NOFD /proc/57400/fd (opendir: Permission denied) 2024-07-27T03:18:57.641 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57401 log ceph cwd unknown /proc/57400/task/57401/cwd (readlink: Permission denied) 2024-07-27T03:18:57.642 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57401 log ceph rtd unknown /proc/57400/task/57401/root (readlink: Permission denied) 2024-07-27T03:18:57.642 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57401 log ceph txt unknown /proc/57400/task/57401/exe (readlink: Permission denied) 2024-07-27T03:18:57.642 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57401 log ceph NOFD /proc/57400/task/57401/fd (opendir: Permission denied) 2024-07-27T03:18:57.642 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57403 msgr-work ceph cwd unknown /proc/57400/task/57403/cwd (readlink: Permission denied) 2024-07-27T03:18:57.642 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57403 msgr-work ceph rtd unknown /proc/57400/task/57403/root (readlink: Permission denied) 2024-07-27T03:18:57.642 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57403 msgr-work ceph txt unknown /proc/57400/task/57403/exe (readlink: Permission denied) 2024-07-27T03:18:57.642 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57403 msgr-work ceph NOFD /proc/57400/task/57403/fd (opendir: Permission denied) 2024-07-27T03:18:57.642 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57404 msgr-work ceph cwd unknown /proc/57400/task/57404/cwd (readlink: Permission denied) 2024-07-27T03:18:57.642 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57404 msgr-work ceph rtd unknown /proc/57400/task/57404/root (readlink: Permission denied) 2024-07-27T03:18:57.642 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57404 msgr-work ceph txt unknown /proc/57400/task/57404/exe (readlink: Permission denied) 2024-07-27T03:18:57.642 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57404 msgr-work ceph NOFD /proc/57400/task/57404/fd (opendir: Permission denied) 2024-07-27T03:18:57.642 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57405 msgr-work ceph cwd unknown /proc/57400/task/57405/cwd (readlink: Permission denied) 2024-07-27T03:18:57.642 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57405 msgr-work ceph rtd unknown /proc/57400/task/57405/root (readlink: Permission denied) 2024-07-27T03:18:57.643 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57405 msgr-work ceph txt unknown /proc/57400/task/57405/exe (readlink: Permission denied) 2024-07-27T03:18:57.643 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57405 msgr-work ceph NOFD /proc/57400/task/57405/fd (opendir: Permission denied) 2024-07-27T03:18:57.643 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57409 service ceph cwd unknown /proc/57400/task/57409/cwd (readlink: Permission denied) 2024-07-27T03:18:57.643 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57409 service ceph rtd unknown /proc/57400/task/57409/root (readlink: Permission denied) 2024-07-27T03:18:57.643 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57409 service ceph txt unknown /proc/57400/task/57409/exe (readlink: Permission denied) 2024-07-27T03:18:57.643 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57409 service ceph NOFD /proc/57400/task/57409/fd (opendir: Permission denied) 2024-07-27T03:18:57.643 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57410 admin_soc ceph cwd unknown /proc/57400/task/57410/cwd (readlink: Permission denied) 2024-07-27T03:18:57.643 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57410 admin_soc ceph rtd unknown /proc/57400/task/57410/root (readlink: Permission denied) 2024-07-27T03:18:57.643 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57410 admin_soc ceph txt unknown /proc/57400/task/57410/exe (readlink: Permission denied) 2024-07-27T03:18:57.643 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57410 admin_soc ceph NOFD /proc/57400/task/57410/fd (opendir: Permission denied) 2024-07-27T03:18:57.643 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57675 signal_ha ceph cwd unknown /proc/57400/task/57675/cwd (readlink: Permission denied) 2024-07-27T03:18:57.643 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57675 signal_ha ceph rtd unknown /proc/57400/task/57675/root (readlink: Permission denied) 2024-07-27T03:18:57.643 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57675 signal_ha ceph txt unknown /proc/57400/task/57675/exe (readlink: Permission denied) 2024-07-27T03:18:57.643 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57675 signal_ha ceph NOFD /proc/57400/task/57675/fd (opendir: Permission denied) 2024-07-27T03:18:57.644 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57676 io_contex ceph cwd unknown /proc/57400/task/57676/cwd (readlink: Permission denied) 2024-07-27T03:18:57.644 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57676 io_contex ceph rtd unknown /proc/57400/task/57676/root (readlink: Permission denied) 2024-07-27T03:18:57.644 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57676 io_contex ceph txt unknown /proc/57400/task/57676/exe (readlink: Permission denied) 2024-07-27T03:18:57.644 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57676 io_contex ceph NOFD /proc/57400/task/57676/fd (opendir: Permission denied) 2024-07-27T03:18:57.644 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57677 io_contex ceph cwd unknown /proc/57400/task/57677/cwd (readlink: Permission denied) 2024-07-27T03:18:57.644 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57677 io_contex ceph rtd unknown /proc/57400/task/57677/root (readlink: Permission denied) 2024-07-27T03:18:57.644 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57677 io_contex ceph txt unknown /proc/57400/task/57677/exe (readlink: Permission denied) 2024-07-27T03:18:57.644 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57677 io_contex ceph NOFD /proc/57400/task/57677/fd (opendir: Permission denied) 2024-07-27T03:18:57.644 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57703 OpHistory ceph cwd unknown /proc/57400/task/57703/cwd (readlink: Permission denied) 2024-07-27T03:18:57.644 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57703 OpHistory ceph rtd unknown /proc/57400/task/57703/root (readlink: Permission denied) 2024-07-27T03:18:57.644 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57703 OpHistory ceph txt unknown /proc/57400/task/57703/exe (readlink: Permission denied) 2024-07-27T03:18:57.644 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57703 OpHistory ceph NOFD /proc/57400/task/57703/fd (opendir: Permission denied) 2024-07-27T03:18:57.644 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57704 ceph_time ceph cwd unknown /proc/57400/task/57704/cwd (readlink: Permission denied) 2024-07-27T03:18:57.645 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57704 ceph_time ceph rtd unknown /proc/57400/task/57704/root (readlink: Permission denied) 2024-07-27T03:18:57.645 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57704 ceph_time ceph txt unknown /proc/57400/task/57704/exe (readlink: Permission denied) 2024-07-27T03:18:57.645 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57704 ceph_time ceph NOFD /proc/57400/task/57704/fd (opendir: Permission denied) 2024-07-27T03:18:57.645 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57930 ceph-osd ceph cwd unknown /proc/57400/task/57930/cwd (readlink: Permission denied) 2024-07-27T03:18:57.645 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57930 ceph-osd ceph rtd unknown /proc/57400/task/57930/root (readlink: Permission denied) 2024-07-27T03:18:57.645 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57930 ceph-osd ceph txt unknown /proc/57400/task/57930/exe (readlink: Permission denied) 2024-07-27T03:18:57.645 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57930 ceph-osd ceph NOFD /proc/57400/task/57930/fd (opendir: Permission denied) 2024-07-27T03:18:57.645 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57931 safe_time ceph cwd unknown /proc/57400/task/57931/cwd (readlink: Permission denied) 2024-07-27T03:18:57.645 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57931 safe_time ceph rtd unknown /proc/57400/task/57931/root (readlink: Permission denied) 2024-07-27T03:18:57.645 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57931 safe_time ceph txt unknown /proc/57400/task/57931/exe (readlink: Permission denied) 2024-07-27T03:18:57.645 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57931 safe_time ceph NOFD /proc/57400/task/57931/fd (opendir: Permission denied) 2024-07-27T03:18:57.645 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57932 safe_time ceph cwd unknown /proc/57400/task/57932/cwd (readlink: Permission denied) 2024-07-27T03:18:57.645 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57932 safe_time ceph rtd unknown /proc/57400/task/57932/root (readlink: Permission denied) 2024-07-27T03:18:57.645 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57932 safe_time ceph txt unknown /proc/57400/task/57932/exe (readlink: Permission denied) 2024-07-27T03:18:57.646 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57932 safe_time ceph NOFD /proc/57400/task/57932/fd (opendir: Permission denied) 2024-07-27T03:18:57.646 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57933 safe_time ceph cwd unknown /proc/57400/task/57933/cwd (readlink: Permission denied) 2024-07-27T03:18:57.646 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57933 safe_time ceph rtd unknown /proc/57400/task/57933/root (readlink: Permission denied) 2024-07-27T03:18:57.646 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57933 safe_time ceph txt unknown /proc/57400/task/57933/exe (readlink: Permission denied) 2024-07-27T03:18:57.646 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57933 safe_time ceph NOFD /proc/57400/task/57933/fd (opendir: Permission denied) 2024-07-27T03:18:57.646 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57934 safe_time ceph cwd unknown /proc/57400/task/57934/cwd (readlink: Permission denied) 2024-07-27T03:18:57.646 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57934 safe_time ceph rtd unknown /proc/57400/task/57934/root (readlink: Permission denied) 2024-07-27T03:18:57.646 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57934 safe_time ceph txt unknown /proc/57400/task/57934/exe (readlink: Permission denied) 2024-07-27T03:18:57.646 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57934 safe_time ceph NOFD /proc/57400/task/57934/fd (opendir: Permission denied) 2024-07-27T03:18:57.646 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57935 fn_anonym ceph cwd unknown /proc/57400/task/57935/cwd (readlink: Permission denied) 2024-07-27T03:18:57.646 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57935 fn_anonym ceph rtd unknown /proc/57400/task/57935/root (readlink: Permission denied) 2024-07-27T03:18:57.646 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57935 fn_anonym ceph txt unknown /proc/57400/task/57935/exe (readlink: Permission denied) 2024-07-27T03:18:57.647 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57935 fn_anonym ceph NOFD /proc/57400/task/57935/fd (opendir: Permission denied) 2024-07-27T03:18:57.647 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57938 rocksdb:l ceph cwd unknown /proc/57400/task/57938/cwd (readlink: Permission denied) 2024-07-27T03:18:57.647 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57938 rocksdb:l ceph rtd unknown /proc/57400/task/57938/root (readlink: Permission denied) 2024-07-27T03:18:57.647 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57938 rocksdb:l ceph txt unknown /proc/57400/task/57938/exe (readlink: Permission denied) 2024-07-27T03:18:57.647 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57938 rocksdb:l ceph NOFD /proc/57400/task/57938/fd (opendir: Permission denied) 2024-07-27T03:18:57.647 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57939 rocksdb:l ceph cwd unknown /proc/57400/task/57939/cwd (readlink: Permission denied) 2024-07-27T03:18:57.647 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57939 rocksdb:l ceph rtd unknown /proc/57400/task/57939/root (readlink: Permission denied) 2024-07-27T03:18:57.647 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57939 rocksdb:l ceph txt unknown /proc/57400/task/57939/exe (readlink: Permission denied) 2024-07-27T03:18:57.647 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57939 rocksdb:l ceph NOFD /proc/57400/task/57939/fd (opendir: Permission denied) 2024-07-27T03:18:57.647 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57940 rocksdb:l ceph cwd unknown /proc/57400/task/57940/cwd (readlink: Permission denied) 2024-07-27T03:18:57.647 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57940 rocksdb:l ceph rtd unknown /proc/57400/task/57940/root (readlink: Permission denied) 2024-07-27T03:18:57.647 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57940 rocksdb:l ceph txt unknown /proc/57400/task/57940/exe (readlink: Permission denied) 2024-07-27T03:18:57.647 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57940 rocksdb:l ceph NOFD /proc/57400/task/57940/fd (opendir: Permission denied) 2024-07-27T03:18:57.648 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57941 rocksdb:h ceph cwd unknown /proc/57400/task/57941/cwd (readlink: Permission denied) 2024-07-27T03:18:57.648 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57941 rocksdb:h ceph rtd unknown /proc/57400/task/57941/root (readlink: Permission denied) 2024-07-27T03:18:57.648 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57941 rocksdb:h ceph txt unknown /proc/57400/task/57941/exe (readlink: Permission denied) 2024-07-27T03:18:57.648 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 57941 rocksdb:h ceph NOFD /proc/57400/task/57941/fd (opendir: Permission denied) 2024-07-27T03:18:57.648 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58470 bstore_ai ceph cwd unknown /proc/57400/task/58470/cwd (readlink: Permission denied) 2024-07-27T03:18:57.648 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58470 bstore_ai ceph rtd unknown /proc/57400/task/58470/root (readlink: Permission denied) 2024-07-27T03:18:57.648 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58470 bstore_ai ceph txt unknown /proc/57400/task/58470/exe (readlink: Permission denied) 2024-07-27T03:18:57.648 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58470 bstore_ai ceph NOFD /proc/57400/task/58470/fd (opendir: Permission denied) 2024-07-27T03:18:57.648 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58678 bstore_ai ceph cwd unknown /proc/57400/task/58678/cwd (readlink: Permission denied) 2024-07-27T03:18:57.648 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58678 bstore_ai ceph rtd unknown /proc/57400/task/58678/root (readlink: Permission denied) 2024-07-27T03:18:57.648 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58678 bstore_ai ceph txt unknown /proc/57400/task/58678/exe (readlink: Permission denied) 2024-07-27T03:18:57.648 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58678 bstore_ai ceph NOFD /proc/57400/task/58678/fd (opendir: Permission denied) 2024-07-27T03:18:57.648 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58860 ceph-osd ceph cwd unknown /proc/57400/task/58860/cwd (readlink: Permission denied) 2024-07-27T03:18:57.648 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58860 ceph-osd ceph rtd unknown /proc/57400/task/58860/root (readlink: Permission denied) 2024-07-27T03:18:57.649 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58860 ceph-osd ceph txt unknown /proc/57400/task/58860/exe (readlink: Permission denied) 2024-07-27T03:18:57.649 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58860 ceph-osd ceph NOFD /proc/57400/task/58860/fd (opendir: Permission denied) 2024-07-27T03:18:57.649 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58861 cfin ceph cwd unknown /proc/57400/task/58861/cwd (readlink: Permission denied) 2024-07-27T03:18:57.649 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58861 cfin ceph rtd unknown /proc/57400/task/58861/root (readlink: Permission denied) 2024-07-27T03:18:57.649 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58861 cfin ceph txt unknown /proc/57400/task/58861/exe (readlink: Permission denied) 2024-07-27T03:18:57.649 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58861 cfin ceph NOFD /proc/57400/task/58861/fd (opendir: Permission denied) 2024-07-27T03:18:57.649 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58862 bstore_kv ceph cwd unknown /proc/57400/task/58862/cwd (readlink: Permission denied) 2024-07-27T03:18:57.649 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58862 bstore_kv ceph rtd unknown /proc/57400/task/58862/root (readlink: Permission denied) 2024-07-27T03:18:57.649 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58862 bstore_kv ceph txt unknown /proc/57400/task/58862/exe (readlink: Permission denied) 2024-07-27T03:18:57.649 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58862 bstore_kv ceph NOFD /proc/57400/task/58862/fd (opendir: Permission denied) 2024-07-27T03:18:57.649 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58863 bstore_kv ceph cwd unknown /proc/57400/task/58863/cwd (readlink: Permission denied) 2024-07-27T03:18:57.649 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58863 bstore_kv ceph rtd unknown /proc/57400/task/58863/root (readlink: Permission denied) 2024-07-27T03:18:57.649 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58863 bstore_kv ceph txt unknown /proc/57400/task/58863/exe (readlink: Permission denied) 2024-07-27T03:18:57.650 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58863 bstore_kv ceph NOFD /proc/57400/task/58863/fd (opendir: Permission denied) 2024-07-27T03:18:57.650 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58864 bstore_me ceph cwd unknown /proc/57400/task/58864/cwd (readlink: Permission denied) 2024-07-27T03:18:57.650 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58864 bstore_me ceph rtd unknown /proc/57400/task/58864/root (readlink: Permission denied) 2024-07-27T03:18:57.650 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58864 bstore_me ceph txt unknown /proc/57400/task/58864/exe (readlink: Permission denied) 2024-07-27T03:18:57.650 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58864 bstore_me ceph NOFD /proc/57400/task/58864/fd (opendir: Permission denied) 2024-07-27T03:18:57.650 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58865 ms_dispat ceph cwd unknown /proc/57400/task/58865/cwd (readlink: Permission denied) 2024-07-27T03:18:57.650 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58865 ms_dispat ceph rtd unknown /proc/57400/task/58865/root (readlink: Permission denied) 2024-07-27T03:18:57.650 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58865 ms_dispat ceph txt unknown /proc/57400/task/58865/exe (readlink: Permission denied) 2024-07-27T03:18:57.650 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58865 ms_dispat ceph NOFD /proc/57400/task/58865/fd (opendir: Permission denied) 2024-07-27T03:18:57.650 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58866 ms_local ceph cwd unknown /proc/57400/task/58866/cwd (readlink: Permission denied) 2024-07-27T03:18:57.650 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58866 ms_local ceph rtd unknown /proc/57400/task/58866/root (readlink: Permission denied) 2024-07-27T03:18:57.650 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58866 ms_local ceph txt unknown /proc/57400/task/58866/exe (readlink: Permission denied) 2024-07-27T03:18:57.650 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58866 ms_local ceph NOFD /proc/57400/task/58866/fd (opendir: Permission denied) 2024-07-27T03:18:57.650 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58867 safe_time ceph cwd unknown /proc/57400/task/58867/cwd (readlink: Permission denied) 2024-07-27T03:18:57.651 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58867 safe_time ceph rtd unknown /proc/57400/task/58867/root (readlink: Permission denied) 2024-07-27T03:18:57.651 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58867 safe_time ceph txt unknown /proc/57400/task/58867/exe (readlink: Permission denied) 2024-07-27T03:18:57.651 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58867 safe_time ceph NOFD /proc/57400/task/58867/fd (opendir: Permission denied) 2024-07-27T03:18:57.651 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58868 safe_time ceph cwd unknown /proc/57400/task/58868/cwd (readlink: Permission denied) 2024-07-27T03:18:57.651 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58868 safe_time ceph rtd unknown /proc/57400/task/58868/root (readlink: Permission denied) 2024-07-27T03:18:57.651 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58868 safe_time ceph txt unknown /proc/57400/task/58868/exe (readlink: Permission denied) 2024-07-27T03:18:57.651 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58868 safe_time ceph NOFD /proc/57400/task/58868/fd (opendir: Permission denied) 2024-07-27T03:18:57.651 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58869 ms_dispat ceph cwd unknown /proc/57400/task/58869/cwd (readlink: Permission denied) 2024-07-27T03:18:57.651 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58869 ms_dispat ceph rtd unknown /proc/57400/task/58869/root (readlink: Permission denied) 2024-07-27T03:18:57.651 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58869 ms_dispat ceph txt unknown /proc/57400/task/58869/exe (readlink: Permission denied) 2024-07-27T03:18:57.651 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58869 ms_dispat ceph NOFD /proc/57400/task/58869/fd (opendir: Permission denied) 2024-07-27T03:18:57.651 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58870 ms_local ceph cwd unknown /proc/57400/task/58870/cwd (readlink: Permission denied) 2024-07-27T03:18:57.651 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58870 ms_local ceph rtd unknown /proc/57400/task/58870/root (readlink: Permission denied) 2024-07-27T03:18:57.652 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58870 ms_local ceph txt unknown /proc/57400/task/58870/exe (readlink: Permission denied) 2024-07-27T03:18:57.652 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58870 ms_local ceph NOFD /proc/57400/task/58870/fd (opendir: Permission denied) 2024-07-27T03:18:57.652 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58871 ms_dispat ceph cwd unknown /proc/57400/task/58871/cwd (readlink: Permission denied) 2024-07-27T03:18:57.652 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58871 ms_dispat ceph rtd unknown /proc/57400/task/58871/root (readlink: Permission denied) 2024-07-27T03:18:57.652 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58871 ms_dispat ceph txt unknown /proc/57400/task/58871/exe (readlink: Permission denied) 2024-07-27T03:18:57.652 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58871 ms_dispat ceph NOFD /proc/57400/task/58871/fd (opendir: Permission denied) 2024-07-27T03:18:57.652 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58872 ms_local ceph cwd unknown /proc/57400/task/58872/cwd (readlink: Permission denied) 2024-07-27T03:18:57.652 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58872 ms_local ceph rtd unknown /proc/57400/task/58872/root (readlink: Permission denied) 2024-07-27T03:18:57.652 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58872 ms_local ceph txt unknown /proc/57400/task/58872/exe (readlink: Permission denied) 2024-07-27T03:18:57.652 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58872 ms_local ceph NOFD /proc/57400/task/58872/fd (opendir: Permission denied) 2024-07-27T03:18:57.652 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58873 ms_dispat ceph cwd unknown /proc/57400/task/58873/cwd (readlink: Permission denied) 2024-07-27T03:18:57.652 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58873 ms_dispat ceph rtd unknown /proc/57400/task/58873/root (readlink: Permission denied) 2024-07-27T03:18:57.652 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58873 ms_dispat ceph txt unknown /proc/57400/task/58873/exe (readlink: Permission denied) 2024-07-27T03:18:57.652 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58873 ms_dispat ceph NOFD /proc/57400/task/58873/fd (opendir: Permission denied) 2024-07-27T03:18:57.653 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58874 ms_local ceph cwd unknown /proc/57400/task/58874/cwd (readlink: Permission denied) 2024-07-27T03:18:57.653 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58874 ms_local ceph rtd unknown /proc/57400/task/58874/root (readlink: Permission denied) 2024-07-27T03:18:57.653 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58874 ms_local ceph txt unknown /proc/57400/task/58874/exe (readlink: Permission denied) 2024-07-27T03:18:57.653 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58874 ms_local ceph NOFD /proc/57400/task/58874/fd (opendir: Permission denied) 2024-07-27T03:18:57.653 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58875 ms_dispat ceph cwd unknown /proc/57400/task/58875/cwd (readlink: Permission denied) 2024-07-27T03:18:57.653 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58875 ms_dispat ceph rtd unknown /proc/57400/task/58875/root (readlink: Permission denied) 2024-07-27T03:18:57.653 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58875 ms_dispat ceph txt unknown /proc/57400/task/58875/exe (readlink: Permission denied) 2024-07-27T03:18:57.653 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58875 ms_dispat ceph NOFD /proc/57400/task/58875/fd (opendir: Permission denied) 2024-07-27T03:18:57.653 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58876 ms_local ceph cwd unknown /proc/57400/task/58876/cwd (readlink: Permission denied) 2024-07-27T03:18:57.653 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58876 ms_local ceph rtd unknown /proc/57400/task/58876/root (readlink: Permission denied) 2024-07-27T03:18:57.653 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58876 ms_local ceph txt unknown /proc/57400/task/58876/exe (readlink: Permission denied) 2024-07-27T03:18:57.653 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58876 ms_local ceph NOFD /proc/57400/task/58876/fd (opendir: Permission denied) 2024-07-27T03:18:57.653 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58877 ms_dispat ceph cwd unknown /proc/57400/task/58877/cwd (readlink: Permission denied) 2024-07-27T03:18:57.653 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58877 ms_dispat ceph rtd unknown /proc/57400/task/58877/root (readlink: Permission denied) 2024-07-27T03:18:57.654 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58877 ms_dispat ceph txt unknown /proc/57400/task/58877/exe (readlink: Permission denied) 2024-07-27T03:18:57.654 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58877 ms_dispat ceph NOFD /proc/57400/task/58877/fd (opendir: Permission denied) 2024-07-27T03:18:57.654 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58878 ms_local ceph cwd unknown /proc/57400/task/58878/cwd (readlink: Permission denied) 2024-07-27T03:18:57.654 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58878 ms_local ceph rtd unknown /proc/57400/task/58878/root (readlink: Permission denied) 2024-07-27T03:18:57.654 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58878 ms_local ceph txt unknown /proc/57400/task/58878/exe (readlink: Permission denied) 2024-07-27T03:18:57.654 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58878 ms_local ceph NOFD /proc/57400/task/58878/fd (opendir: Permission denied) 2024-07-27T03:18:57.654 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58879 ms_dispat ceph cwd unknown /proc/57400/task/58879/cwd (readlink: Permission denied) 2024-07-27T03:18:57.654 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58879 ms_dispat ceph rtd unknown /proc/57400/task/58879/root (readlink: Permission denied) 2024-07-27T03:18:57.654 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58879 ms_dispat ceph txt unknown /proc/57400/task/58879/exe (readlink: Permission denied) 2024-07-27T03:18:57.654 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58879 ms_dispat ceph NOFD /proc/57400/task/58879/fd (opendir: Permission denied) 2024-07-27T03:18:57.654 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58880 ms_local ceph cwd unknown /proc/57400/task/58880/cwd (readlink: Permission denied) 2024-07-27T03:18:57.654 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58880 ms_local ceph rtd unknown /proc/57400/task/58880/root (readlink: Permission denied) 2024-07-27T03:18:57.654 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58880 ms_local ceph txt unknown /proc/57400/task/58880/exe (readlink: Permission denied) 2024-07-27T03:18:57.654 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58880 ms_local ceph NOFD /proc/57400/task/58880/fd (opendir: Permission denied) 2024-07-27T03:18:57.655 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58881 fn_anonym ceph cwd unknown /proc/57400/task/58881/cwd (readlink: Permission denied) 2024-07-27T03:18:57.655 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58881 fn_anonym ceph rtd unknown /proc/57400/task/58881/root (readlink: Permission denied) 2024-07-27T03:18:57.655 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58881 fn_anonym ceph txt unknown /proc/57400/task/58881/exe (readlink: Permission denied) 2024-07-27T03:18:57.655 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58881 fn_anonym ceph NOFD /proc/57400/task/58881/fd (opendir: Permission denied) 2024-07-27T03:18:57.655 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58882 finisher ceph cwd unknown /proc/57400/task/58882/cwd (readlink: Permission denied) 2024-07-27T03:18:57.655 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58882 finisher ceph rtd unknown /proc/57400/task/58882/root (readlink: Permission denied) 2024-07-27T03:18:57.655 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58882 finisher ceph txt unknown /proc/57400/task/58882/exe (readlink: Permission denied) 2024-07-27T03:18:57.655 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58882 finisher ceph NOFD /proc/57400/task/58882/fd (opendir: Permission denied) 2024-07-27T03:18:57.655 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58883 safe_time ceph cwd unknown /proc/57400/task/58883/cwd (readlink: Permission denied) 2024-07-27T03:18:57.655 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58883 safe_time ceph rtd unknown /proc/57400/task/58883/root (readlink: Permission denied) 2024-07-27T03:18:57.655 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58883 safe_time ceph txt unknown /proc/57400/task/58883/exe (readlink: Permission denied) 2024-07-27T03:18:57.655 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58883 safe_time ceph NOFD /proc/57400/task/58883/fd (opendir: Permission denied) 2024-07-27T03:18:57.655 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58884 safe_time ceph cwd unknown /proc/57400/task/58884/cwd (readlink: Permission denied) 2024-07-27T03:18:57.656 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58884 safe_time ceph rtd unknown /proc/57400/task/58884/root (readlink: Permission denied) 2024-07-27T03:18:57.656 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58884 safe_time ceph txt unknown /proc/57400/task/58884/exe (readlink: Permission denied) 2024-07-27T03:18:57.656 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58884 safe_time ceph NOFD /proc/57400/task/58884/fd (opendir: Permission denied) 2024-07-27T03:18:57.656 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58885 ceph-osd ceph cwd unknown /proc/57400/task/58885/cwd (readlink: Permission denied) 2024-07-27T03:18:57.656 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58885 ceph-osd ceph rtd unknown /proc/57400/task/58885/root (readlink: Permission denied) 2024-07-27T03:18:57.656 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58885 ceph-osd ceph txt unknown /proc/57400/task/58885/exe (readlink: Permission denied) 2024-07-27T03:18:57.656 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58885 ceph-osd ceph NOFD /proc/57400/task/58885/fd (opendir: Permission denied) 2024-07-27T03:18:57.656 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58886 osd_srv_a ceph cwd unknown /proc/57400/task/58886/cwd (readlink: Permission denied) 2024-07-27T03:18:57.656 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58886 osd_srv_a ceph rtd unknown /proc/57400/task/58886/root (readlink: Permission denied) 2024-07-27T03:18:57.656 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58886 osd_srv_a ceph txt unknown /proc/57400/task/58886/exe (readlink: Permission denied) 2024-07-27T03:18:57.656 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58886 osd_srv_a ceph NOFD /proc/57400/task/58886/fd (opendir: Permission denied) 2024-07-27T03:18:57.656 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58887 tp_osd_tp ceph cwd unknown /proc/57400/task/58887/cwd (readlink: Permission denied) 2024-07-27T03:18:57.656 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58887 tp_osd_tp ceph rtd unknown /proc/57400/task/58887/root (readlink: Permission denied) 2024-07-27T03:18:57.656 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58887 tp_osd_tp ceph txt unknown /proc/57400/task/58887/exe (readlink: Permission denied) 2024-07-27T03:18:57.657 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58887 tp_osd_tp ceph NOFD /proc/57400/task/58887/fd (opendir: Permission denied) 2024-07-27T03:18:57.657 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58888 tp_osd_tp ceph cwd unknown /proc/57400/task/58888/cwd (readlink: Permission denied) 2024-07-27T03:18:57.657 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58888 tp_osd_tp ceph rtd unknown /proc/57400/task/58888/root (readlink: Permission denied) 2024-07-27T03:18:57.657 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58888 tp_osd_tp ceph txt unknown /proc/57400/task/58888/exe (readlink: Permission denied) 2024-07-27T03:18:57.657 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58888 tp_osd_tp ceph NOFD /proc/57400/task/58888/fd (opendir: Permission denied) 2024-07-27T03:18:57.657 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58889 tp_osd_tp ceph cwd unknown /proc/57400/task/58889/cwd (readlink: Permission denied) 2024-07-27T03:18:57.657 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58889 tp_osd_tp ceph rtd unknown /proc/57400/task/58889/root (readlink: Permission denied) 2024-07-27T03:18:57.657 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58889 tp_osd_tp ceph txt unknown /proc/57400/task/58889/exe (readlink: Permission denied) 2024-07-27T03:18:57.657 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58889 tp_osd_tp ceph NOFD /proc/57400/task/58889/fd (opendir: Permission denied) 2024-07-27T03:18:57.657 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58890 tp_osd_tp ceph cwd unknown /proc/57400/task/58890/cwd (readlink: Permission denied) 2024-07-27T03:18:57.657 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58890 tp_osd_tp ceph rtd unknown /proc/57400/task/58890/root (readlink: Permission denied) 2024-07-27T03:18:57.657 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58890 tp_osd_tp ceph txt unknown /proc/57400/task/58890/exe (readlink: Permission denied) 2024-07-27T03:18:57.657 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58890 tp_osd_tp ceph NOFD /proc/57400/task/58890/fd (opendir: Permission denied) 2024-07-27T03:18:57.657 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58891 tp_osd_tp ceph cwd unknown /proc/57400/task/58891/cwd (readlink: Permission denied) 2024-07-27T03:18:57.658 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58891 tp_osd_tp ceph rtd unknown /proc/57400/task/58891/root (readlink: Permission denied) 2024-07-27T03:18:57.658 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58891 tp_osd_tp ceph txt unknown /proc/57400/task/58891/exe (readlink: Permission denied) 2024-07-27T03:18:57.658 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58891 tp_osd_tp ceph NOFD /proc/57400/task/58891/fd (opendir: Permission denied) 2024-07-27T03:18:57.658 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58892 tp_osd_tp ceph cwd unknown /proc/57400/task/58892/cwd (readlink: Permission denied) 2024-07-27T03:18:57.658 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58892 tp_osd_tp ceph rtd unknown /proc/57400/task/58892/root (readlink: Permission denied) 2024-07-27T03:18:57.658 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58892 tp_osd_tp ceph txt unknown /proc/57400/task/58892/exe (readlink: Permission denied) 2024-07-27T03:18:57.658 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58892 tp_osd_tp ceph NOFD /proc/57400/task/58892/fd (opendir: Permission denied) 2024-07-27T03:18:57.658 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58893 tp_osd_tp ceph cwd unknown /proc/57400/task/58893/cwd (readlink: Permission denied) 2024-07-27T03:18:57.658 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58893 tp_osd_tp ceph rtd unknown /proc/57400/task/58893/root (readlink: Permission denied) 2024-07-27T03:18:57.658 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58893 tp_osd_tp ceph txt unknown /proc/57400/task/58893/exe (readlink: Permission denied) 2024-07-27T03:18:57.658 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58893 tp_osd_tp ceph NOFD /proc/57400/task/58893/fd (opendir: Permission denied) 2024-07-27T03:18:57.658 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58894 tp_osd_tp ceph cwd unknown /proc/57400/task/58894/cwd (readlink: Permission denied) 2024-07-27T03:18:57.659 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58894 tp_osd_tp ceph rtd unknown /proc/57400/task/58894/root (readlink: Permission denied) 2024-07-27T03:18:57.659 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58894 tp_osd_tp ceph txt unknown /proc/57400/task/58894/exe (readlink: Permission denied) 2024-07-27T03:18:57.659 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58894 tp_osd_tp ceph NOFD /proc/57400/task/58894/fd (opendir: Permission denied) 2024-07-27T03:18:57.659 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58895 tp_osd_tp ceph cwd unknown /proc/57400/task/58895/cwd (readlink: Permission denied) 2024-07-27T03:18:57.659 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58895 tp_osd_tp ceph rtd unknown /proc/57400/task/58895/root (readlink: Permission denied) 2024-07-27T03:18:57.659 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58895 tp_osd_tp ceph txt unknown /proc/57400/task/58895/exe (readlink: Permission denied) 2024-07-27T03:18:57.659 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58895 tp_osd_tp ceph NOFD /proc/57400/task/58895/fd (opendir: Permission denied) 2024-07-27T03:18:57.659 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58896 tp_osd_tp ceph cwd unknown /proc/57400/task/58896/cwd (readlink: Permission denied) 2024-07-27T03:18:57.659 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58896 tp_osd_tp ceph rtd unknown /proc/57400/task/58896/root (readlink: Permission denied) 2024-07-27T03:18:57.659 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58896 tp_osd_tp ceph txt unknown /proc/57400/task/58896/exe (readlink: Permission denied) 2024-07-27T03:18:57.659 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58896 tp_osd_tp ceph NOFD /proc/57400/task/58896/fd (opendir: Permission denied) 2024-07-27T03:18:57.659 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58897 tp_osd_tp ceph cwd unknown /proc/57400/task/58897/cwd (readlink: Permission denied) 2024-07-27T03:18:57.659 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58897 tp_osd_tp ceph rtd unknown /proc/57400/task/58897/root (readlink: Permission denied) 2024-07-27T03:18:57.659 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58897 tp_osd_tp ceph txt unknown /proc/57400/task/58897/exe (readlink: Permission denied) 2024-07-27T03:18:57.660 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58897 tp_osd_tp ceph NOFD /proc/57400/task/58897/fd (opendir: Permission denied) 2024-07-27T03:18:57.660 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58898 tp_osd_tp ceph cwd unknown /proc/57400/task/58898/cwd (readlink: Permission denied) 2024-07-27T03:18:57.660 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58898 tp_osd_tp ceph rtd unknown /proc/57400/task/58898/root (readlink: Permission denied) 2024-07-27T03:18:57.660 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58898 tp_osd_tp ceph txt unknown /proc/57400/task/58898/exe (readlink: Permission denied) 2024-07-27T03:18:57.660 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58898 tp_osd_tp ceph NOFD /proc/57400/task/58898/fd (opendir: Permission denied) 2024-07-27T03:18:57.660 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58899 tp_osd_tp ceph cwd unknown /proc/57400/task/58899/cwd (readlink: Permission denied) 2024-07-27T03:18:57.660 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58899 tp_osd_tp ceph rtd unknown /proc/57400/task/58899/root (readlink: Permission denied) 2024-07-27T03:18:57.660 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58899 tp_osd_tp ceph txt unknown /proc/57400/task/58899/exe (readlink: Permission denied) 2024-07-27T03:18:57.660 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58899 tp_osd_tp ceph NOFD /proc/57400/task/58899/fd (opendir: Permission denied) 2024-07-27T03:18:57.660 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58900 tp_osd_tp ceph cwd unknown /proc/57400/task/58900/cwd (readlink: Permission denied) 2024-07-27T03:18:57.660 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58900 tp_osd_tp ceph rtd unknown /proc/57400/task/58900/root (readlink: Permission denied) 2024-07-27T03:18:57.660 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58900 tp_osd_tp ceph txt unknown /proc/57400/task/58900/exe (readlink: Permission denied) 2024-07-27T03:18:57.660 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58900 tp_osd_tp ceph NOFD /proc/57400/task/58900/fd (opendir: Permission denied) 2024-07-27T03:18:57.660 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58901 tp_osd_tp ceph cwd unknown /proc/57400/task/58901/cwd (readlink: Permission denied) 2024-07-27T03:18:57.661 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58901 tp_osd_tp ceph rtd unknown /proc/57400/task/58901/root (readlink: Permission denied) 2024-07-27T03:18:57.661 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58901 tp_osd_tp ceph txt unknown /proc/57400/task/58901/exe (readlink: Permission denied) 2024-07-27T03:18:57.661 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58901 tp_osd_tp ceph NOFD /proc/57400/task/58901/fd (opendir: Permission denied) 2024-07-27T03:18:57.661 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58902 tp_osd_tp ceph cwd unknown /proc/57400/task/58902/cwd (readlink: Permission denied) 2024-07-27T03:18:57.661 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58902 tp_osd_tp ceph rtd unknown /proc/57400/task/58902/root (readlink: Permission denied) 2024-07-27T03:18:57.661 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58902 tp_osd_tp ceph txt unknown /proc/57400/task/58902/exe (readlink: Permission denied) 2024-07-27T03:18:57.661 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58902 tp_osd_tp ceph NOFD /proc/57400/task/58902/fd (opendir: Permission denied) 2024-07-27T03:18:57.661 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58903 osd_srv_h ceph cwd unknown /proc/57400/task/58903/cwd (readlink: Permission denied) 2024-07-27T03:18:57.661 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58903 osd_srv_h ceph rtd unknown /proc/57400/task/58903/root (readlink: Permission denied) 2024-07-27T03:18:57.661 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58903 osd_srv_h ceph txt unknown /proc/57400/task/58903/exe (readlink: Permission denied) 2024-07-27T03:18:57.661 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 57400 58903 osd_srv_h ceph NOFD /proc/57400/task/58903/fd (opendir: Permission denied) 2024-07-27T03:18:57.661 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 58907 root cwd unknown /proc/58907/cwd (readlink: Permission denied) 2024-07-27T03:18:57.661 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 58907 root rtd unknown /proc/58907/root (readlink: Permission denied) 2024-07-27T03:18:57.661 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 58907 root txt unknown /proc/58907/exe (readlink: Permission denied) 2024-07-27T03:18:57.662 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 58907 root NOFD /proc/58907/fd (opendir: Permission denied) 2024-07-27T03:18:57.662 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 58924 root cwd unknown /proc/58924/cwd (readlink: Permission denied) 2024-07-27T03:18:57.662 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 58924 root rtd unknown /proc/58924/root (readlink: Permission denied) 2024-07-27T03:18:57.662 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 58924 root txt unknown /proc/58924/exe (readlink: Permission denied) 2024-07-27T03:18:57.662 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 58924 root NOFD /proc/58924/fd (opendir: Permission denied) 2024-07-27T03:18:57.662 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/2 61561 root cwd unknown /proc/61561/cwd (readlink: Permission denied) 2024-07-27T03:18:57.662 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/2 61561 root rtd unknown /proc/61561/root (readlink: Permission denied) 2024-07-27T03:18:57.662 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/2 61561 root txt unknown /proc/61561/exe (readlink: Permission denied) 2024-07-27T03:18:57.662 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/2 61561 root NOFD /proc/61561/fd (opendir: Permission denied) 2024-07-27T03:18:57.662 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 64302 root cwd unknown /proc/64302/cwd (readlink: Permission denied) 2024-07-27T03:18:57.662 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 64302 root rtd unknown /proc/64302/root (readlink: Permission denied) 2024-07-27T03:18:57.662 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 64302 root txt unknown /proc/64302/exe (readlink: Permission denied) 2024-07-27T03:18:57.662 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 64302 root NOFD /proc/64302/fd (opendir: Permission denied) 2024-07-27T03:18:57.662 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 64304 root cwd unknown /proc/64304/cwd (readlink: Permission denied) 2024-07-27T03:18:57.663 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 64304 root rtd unknown /proc/64304/root (readlink: Permission denied) 2024-07-27T03:18:57.663 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 64304 root txt unknown /proc/64304/exe (readlink: Permission denied) 2024-07-27T03:18:57.663 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 64304 root NOFD /proc/64304/fd (opendir: Permission denied) 2024-07-27T03:18:57.663 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 ceph cwd unknown /proc/64306/cwd (readlink: Permission denied) 2024-07-27T03:18:57.663 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 ceph rtd unknown /proc/64306/root (readlink: Permission denied) 2024-07-27T03:18:57.663 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 ceph txt unknown /proc/64306/exe (readlink: Permission denied) 2024-07-27T03:18:57.663 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 ceph NOFD /proc/64306/fd (opendir: Permission denied) 2024-07-27T03:18:57.663 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64307 log ceph cwd unknown /proc/64306/task/64307/cwd (readlink: Permission denied) 2024-07-27T03:18:57.663 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64307 log ceph rtd unknown /proc/64306/task/64307/root (readlink: Permission denied) 2024-07-27T03:18:57.663 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64307 log ceph txt unknown /proc/64306/task/64307/exe (readlink: Permission denied) 2024-07-27T03:18:57.663 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64307 log ceph NOFD /proc/64306/task/64307/fd (opendir: Permission denied) 2024-07-27T03:18:57.663 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64309 msgr-work ceph cwd unknown /proc/64306/task/64309/cwd (readlink: Permission denied) 2024-07-27T03:18:57.663 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64309 msgr-work ceph rtd unknown /proc/64306/task/64309/root (readlink: Permission denied) 2024-07-27T03:18:57.664 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64309 msgr-work ceph txt unknown /proc/64306/task/64309/exe (readlink: Permission denied) 2024-07-27T03:18:57.664 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64309 msgr-work ceph NOFD /proc/64306/task/64309/fd (opendir: Permission denied) 2024-07-27T03:18:57.664 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64310 msgr-work ceph cwd unknown /proc/64306/task/64310/cwd (readlink: Permission denied) 2024-07-27T03:18:57.664 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64310 msgr-work ceph rtd unknown /proc/64306/task/64310/root (readlink: Permission denied) 2024-07-27T03:18:57.664 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64310 msgr-work ceph txt unknown /proc/64306/task/64310/exe (readlink: Permission denied) 2024-07-27T03:18:57.664 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64310 msgr-work ceph NOFD /proc/64306/task/64310/fd (opendir: Permission denied) 2024-07-27T03:18:57.664 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64311 msgr-work ceph cwd unknown /proc/64306/task/64311/cwd (readlink: Permission denied) 2024-07-27T03:18:57.664 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64311 msgr-work ceph rtd unknown /proc/64306/task/64311/root (readlink: Permission denied) 2024-07-27T03:18:57.664 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64311 msgr-work ceph txt unknown /proc/64306/task/64311/exe (readlink: Permission denied) 2024-07-27T03:18:57.664 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64311 msgr-work ceph NOFD /proc/64306/task/64311/fd (opendir: Permission denied) 2024-07-27T03:18:57.664 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64315 service ceph cwd unknown /proc/64306/task/64315/cwd (readlink: Permission denied) 2024-07-27T03:18:57.664 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64315 service ceph rtd unknown /proc/64306/task/64315/root (readlink: Permission denied) 2024-07-27T03:18:57.664 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64315 service ceph txt unknown /proc/64306/task/64315/exe (readlink: Permission denied) 2024-07-27T03:18:57.664 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64315 service ceph NOFD /proc/64306/task/64315/fd (opendir: Permission denied) 2024-07-27T03:18:57.665 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64316 admin_soc ceph cwd unknown /proc/64306/task/64316/cwd (readlink: Permission denied) 2024-07-27T03:18:57.665 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64316 admin_soc ceph rtd unknown /proc/64306/task/64316/root (readlink: Permission denied) 2024-07-27T03:18:57.665 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64316 admin_soc ceph txt unknown /proc/64306/task/64316/exe (readlink: Permission denied) 2024-07-27T03:18:57.665 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64316 admin_soc ceph NOFD /proc/64306/task/64316/fd (opendir: Permission denied) 2024-07-27T03:18:57.665 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64569 signal_ha ceph cwd unknown /proc/64306/task/64569/cwd (readlink: Permission denied) 2024-07-27T03:18:57.665 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64569 signal_ha ceph rtd unknown /proc/64306/task/64569/root (readlink: Permission denied) 2024-07-27T03:18:57.665 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64569 signal_ha ceph txt unknown /proc/64306/task/64569/exe (readlink: Permission denied) 2024-07-27T03:18:57.665 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64569 signal_ha ceph NOFD /proc/64306/task/64569/fd (opendir: Permission denied) 2024-07-27T03:18:57.665 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64570 io_contex ceph cwd unknown /proc/64306/task/64570/cwd (readlink: Permission denied) 2024-07-27T03:18:57.665 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64570 io_contex ceph rtd unknown /proc/64306/task/64570/root (readlink: Permission denied) 2024-07-27T03:18:57.665 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64570 io_contex ceph txt unknown /proc/64306/task/64570/exe (readlink: Permission denied) 2024-07-27T03:18:57.665 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64570 io_contex ceph NOFD /proc/64306/task/64570/fd (opendir: Permission denied) 2024-07-27T03:18:57.665 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64571 io_contex ceph cwd unknown /proc/64306/task/64571/cwd (readlink: Permission denied) 2024-07-27T03:18:57.665 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64571 io_contex ceph rtd unknown /proc/64306/task/64571/root (readlink: Permission denied) 2024-07-27T03:18:57.666 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64571 io_contex ceph txt unknown /proc/64306/task/64571/exe (readlink: Permission denied) 2024-07-27T03:18:57.666 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64571 io_contex ceph NOFD /proc/64306/task/64571/fd (opendir: Permission denied) 2024-07-27T03:18:57.666 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64590 OpHistory ceph cwd unknown /proc/64306/task/64590/cwd (readlink: Permission denied) 2024-07-27T03:18:57.666 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64590 OpHistory ceph rtd unknown /proc/64306/task/64590/root (readlink: Permission denied) 2024-07-27T03:18:57.666 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64590 OpHistory ceph txt unknown /proc/64306/task/64590/exe (readlink: Permission denied) 2024-07-27T03:18:57.666 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64590 OpHistory ceph NOFD /proc/64306/task/64590/fd (opendir: Permission denied) 2024-07-27T03:18:57.666 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64591 ceph_time ceph cwd unknown /proc/64306/task/64591/cwd (readlink: Permission denied) 2024-07-27T03:18:57.666 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64591 ceph_time ceph rtd unknown /proc/64306/task/64591/root (readlink: Permission denied) 2024-07-27T03:18:57.666 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64591 ceph_time ceph txt unknown /proc/64306/task/64591/exe (readlink: Permission denied) 2024-07-27T03:18:57.666 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64591 ceph_time ceph NOFD /proc/64306/task/64591/fd (opendir: Permission denied) 2024-07-27T03:18:57.666 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64837 ceph-osd ceph cwd unknown /proc/64306/task/64837/cwd (readlink: Permission denied) 2024-07-27T03:18:57.666 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64837 ceph-osd ceph rtd unknown /proc/64306/task/64837/root (readlink: Permission denied) 2024-07-27T03:18:57.666 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64837 ceph-osd ceph txt unknown /proc/64306/task/64837/exe (readlink: Permission denied) 2024-07-27T03:18:57.667 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64837 ceph-osd ceph NOFD /proc/64306/task/64837/fd (opendir: Permission denied) 2024-07-27T03:18:57.667 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64838 safe_time ceph cwd unknown /proc/64306/task/64838/cwd (readlink: Permission denied) 2024-07-27T03:18:57.667 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64838 safe_time ceph rtd unknown /proc/64306/task/64838/root (readlink: Permission denied) 2024-07-27T03:18:57.667 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64838 safe_time ceph txt unknown /proc/64306/task/64838/exe (readlink: Permission denied) 2024-07-27T03:18:57.667 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64838 safe_time ceph NOFD /proc/64306/task/64838/fd (opendir: Permission denied) 2024-07-27T03:18:57.667 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64839 safe_time ceph cwd unknown /proc/64306/task/64839/cwd (readlink: Permission denied) 2024-07-27T03:18:57.667 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64839 safe_time ceph rtd unknown /proc/64306/task/64839/root (readlink: Permission denied) 2024-07-27T03:18:57.667 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64839 safe_time ceph txt unknown /proc/64306/task/64839/exe (readlink: Permission denied) 2024-07-27T03:18:57.667 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64839 safe_time ceph NOFD /proc/64306/task/64839/fd (opendir: Permission denied) 2024-07-27T03:18:57.667 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64840 safe_time ceph cwd unknown /proc/64306/task/64840/cwd (readlink: Permission denied) 2024-07-27T03:18:57.667 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64840 safe_time ceph rtd unknown /proc/64306/task/64840/root (readlink: Permission denied) 2024-07-27T03:18:57.667 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64840 safe_time ceph txt unknown /proc/64306/task/64840/exe (readlink: Permission denied) 2024-07-27T03:18:57.667 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64840 safe_time ceph NOFD /proc/64306/task/64840/fd (opendir: Permission denied) 2024-07-27T03:18:57.668 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64841 safe_time ceph cwd unknown /proc/64306/task/64841/cwd (readlink: Permission denied) 2024-07-27T03:18:57.668 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64841 safe_time ceph rtd unknown /proc/64306/task/64841/root (readlink: Permission denied) 2024-07-27T03:18:57.668 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64841 safe_time ceph txt unknown /proc/64306/task/64841/exe (readlink: Permission denied) 2024-07-27T03:18:57.668 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64841 safe_time ceph NOFD /proc/64306/task/64841/fd (opendir: Permission denied) 2024-07-27T03:18:57.668 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64842 fn_anonym ceph cwd unknown /proc/64306/task/64842/cwd (readlink: Permission denied) 2024-07-27T03:18:57.668 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64842 fn_anonym ceph rtd unknown /proc/64306/task/64842/root (readlink: Permission denied) 2024-07-27T03:18:57.668 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64842 fn_anonym ceph txt unknown /proc/64306/task/64842/exe (readlink: Permission denied) 2024-07-27T03:18:57.668 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64842 fn_anonym ceph NOFD /proc/64306/task/64842/fd (opendir: Permission denied) 2024-07-27T03:18:57.668 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64846 rocksdb:l ceph cwd unknown /proc/64306/task/64846/cwd (readlink: Permission denied) 2024-07-27T03:18:57.668 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64846 rocksdb:l ceph rtd unknown /proc/64306/task/64846/root (readlink: Permission denied) 2024-07-27T03:18:57.668 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64846 rocksdb:l ceph txt unknown /proc/64306/task/64846/exe (readlink: Permission denied) 2024-07-27T03:18:57.668 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64846 rocksdb:l ceph NOFD /proc/64306/task/64846/fd (opendir: Permission denied) 2024-07-27T03:18:57.668 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64847 rocksdb:l ceph cwd unknown /proc/64306/task/64847/cwd (readlink: Permission denied) 2024-07-27T03:18:57.668 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64847 rocksdb:l ceph rtd unknown /proc/64306/task/64847/root (readlink: Permission denied) 2024-07-27T03:18:57.669 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64847 rocksdb:l ceph txt unknown /proc/64306/task/64847/exe (readlink: Permission denied) 2024-07-27T03:18:57.669 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64847 rocksdb:l ceph NOFD /proc/64306/task/64847/fd (opendir: Permission denied) 2024-07-27T03:18:57.669 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64848 rocksdb:l ceph cwd unknown /proc/64306/task/64848/cwd (readlink: Permission denied) 2024-07-27T03:18:57.669 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64848 rocksdb:l ceph rtd unknown /proc/64306/task/64848/root (readlink: Permission denied) 2024-07-27T03:18:57.669 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64848 rocksdb:l ceph txt unknown /proc/64306/task/64848/exe (readlink: Permission denied) 2024-07-27T03:18:57.669 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64848 rocksdb:l ceph NOFD /proc/64306/task/64848/fd (opendir: Permission denied) 2024-07-27T03:18:57.669 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64849 rocksdb:h ceph cwd unknown /proc/64306/task/64849/cwd (readlink: Permission denied) 2024-07-27T03:18:57.669 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64849 rocksdb:h ceph rtd unknown /proc/64306/task/64849/root (readlink: Permission denied) 2024-07-27T03:18:57.669 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64849 rocksdb:h ceph txt unknown /proc/64306/task/64849/exe (readlink: Permission denied) 2024-07-27T03:18:57.669 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 64849 rocksdb:h ceph NOFD /proc/64306/task/64849/fd (opendir: Permission denied) 2024-07-27T03:18:57.669 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65310 bstore_ai ceph cwd unknown /proc/64306/task/65310/cwd (readlink: Permission denied) 2024-07-27T03:18:57.669 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65310 bstore_ai ceph rtd unknown /proc/64306/task/65310/root (readlink: Permission denied) 2024-07-27T03:18:57.669 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65310 bstore_ai ceph txt unknown /proc/64306/task/65310/exe (readlink: Permission denied) 2024-07-27T03:18:57.670 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65310 bstore_ai ceph NOFD /proc/64306/task/65310/fd (opendir: Permission denied) 2024-07-27T03:18:57.670 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65572 bstore_ai ceph cwd unknown /proc/64306/task/65572/cwd (readlink: Permission denied) 2024-07-27T03:18:57.670 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65572 bstore_ai ceph rtd unknown /proc/64306/task/65572/root (readlink: Permission denied) 2024-07-27T03:18:57.670 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65572 bstore_ai ceph txt unknown /proc/64306/task/65572/exe (readlink: Permission denied) 2024-07-27T03:18:57.670 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65572 bstore_ai ceph NOFD /proc/64306/task/65572/fd (opendir: Permission denied) 2024-07-27T03:18:57.670 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65754 ceph-osd ceph cwd unknown /proc/64306/task/65754/cwd (readlink: Permission denied) 2024-07-27T03:18:57.670 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65754 ceph-osd ceph rtd unknown /proc/64306/task/65754/root (readlink: Permission denied) 2024-07-27T03:18:57.670 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65754 ceph-osd ceph txt unknown /proc/64306/task/65754/exe (readlink: Permission denied) 2024-07-27T03:18:57.670 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65754 ceph-osd ceph NOFD /proc/64306/task/65754/fd (opendir: Permission denied) 2024-07-27T03:18:57.670 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65755 cfin ceph cwd unknown /proc/64306/task/65755/cwd (readlink: Permission denied) 2024-07-27T03:18:57.670 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65755 cfin ceph rtd unknown /proc/64306/task/65755/root (readlink: Permission denied) 2024-07-27T03:18:57.670 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65755 cfin ceph txt unknown /proc/64306/task/65755/exe (readlink: Permission denied) 2024-07-27T03:18:57.670 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65755 cfin ceph NOFD /proc/64306/task/65755/fd (opendir: Permission denied) 2024-07-27T03:18:57.671 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65756 bstore_kv ceph cwd unknown /proc/64306/task/65756/cwd (readlink: Permission denied) 2024-07-27T03:18:57.671 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65756 bstore_kv ceph rtd unknown /proc/64306/task/65756/root (readlink: Permission denied) 2024-07-27T03:18:57.671 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65756 bstore_kv ceph txt unknown /proc/64306/task/65756/exe (readlink: Permission denied) 2024-07-27T03:18:57.671 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65756 bstore_kv ceph NOFD /proc/64306/task/65756/fd (opendir: Permission denied) 2024-07-27T03:18:57.671 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65757 bstore_kv ceph cwd unknown /proc/64306/task/65757/cwd (readlink: Permission denied) 2024-07-27T03:18:57.671 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65757 bstore_kv ceph rtd unknown /proc/64306/task/65757/root (readlink: Permission denied) 2024-07-27T03:18:57.671 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65757 bstore_kv ceph txt unknown /proc/64306/task/65757/exe (readlink: Permission denied) 2024-07-27T03:18:57.671 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65757 bstore_kv ceph NOFD /proc/64306/task/65757/fd (opendir: Permission denied) 2024-07-27T03:18:57.671 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65758 bstore_me ceph cwd unknown /proc/64306/task/65758/cwd (readlink: Permission denied) 2024-07-27T03:18:57.671 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65758 bstore_me ceph rtd unknown /proc/64306/task/65758/root (readlink: Permission denied) 2024-07-27T03:18:57.671 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65758 bstore_me ceph txt unknown /proc/64306/task/65758/exe (readlink: Permission denied) 2024-07-27T03:18:57.671 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65758 bstore_me ceph NOFD /proc/64306/task/65758/fd (opendir: Permission denied) 2024-07-27T03:18:57.671 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65760 ms_dispat ceph cwd unknown /proc/64306/task/65760/cwd (readlink: Permission denied) 2024-07-27T03:18:57.671 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65760 ms_dispat ceph rtd unknown /proc/64306/task/65760/root (readlink: Permission denied) 2024-07-27T03:18:57.672 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65760 ms_dispat ceph txt unknown /proc/64306/task/65760/exe (readlink: Permission denied) 2024-07-27T03:18:57.672 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65760 ms_dispat ceph NOFD /proc/64306/task/65760/fd (opendir: Permission denied) 2024-07-27T03:18:57.672 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65761 ms_local ceph cwd unknown /proc/64306/task/65761/cwd (readlink: Permission denied) 2024-07-27T03:18:57.672 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65761 ms_local ceph rtd unknown /proc/64306/task/65761/root (readlink: Permission denied) 2024-07-27T03:18:57.672 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65761 ms_local ceph txt unknown /proc/64306/task/65761/exe (readlink: Permission denied) 2024-07-27T03:18:57.672 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65761 ms_local ceph NOFD /proc/64306/task/65761/fd (opendir: Permission denied) 2024-07-27T03:18:57.672 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65762 safe_time ceph cwd unknown /proc/64306/task/65762/cwd (readlink: Permission denied) 2024-07-27T03:18:57.672 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65762 safe_time ceph rtd unknown /proc/64306/task/65762/root (readlink: Permission denied) 2024-07-27T03:18:57.672 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65762 safe_time ceph txt unknown /proc/64306/task/65762/exe (readlink: Permission denied) 2024-07-27T03:18:57.672 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65762 safe_time ceph NOFD /proc/64306/task/65762/fd (opendir: Permission denied) 2024-07-27T03:18:57.672 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65763 safe_time ceph cwd unknown /proc/64306/task/65763/cwd (readlink: Permission denied) 2024-07-27T03:18:57.672 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65763 safe_time ceph rtd unknown /proc/64306/task/65763/root (readlink: Permission denied) 2024-07-27T03:18:57.672 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65763 safe_time ceph txt unknown /proc/64306/task/65763/exe (readlink: Permission denied) 2024-07-27T03:18:57.673 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65763 safe_time ceph NOFD /proc/64306/task/65763/fd (opendir: Permission denied) 2024-07-27T03:18:57.673 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65764 ms_dispat ceph cwd unknown /proc/64306/task/65764/cwd (readlink: Permission denied) 2024-07-27T03:18:57.673 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65764 ms_dispat ceph rtd unknown /proc/64306/task/65764/root (readlink: Permission denied) 2024-07-27T03:18:57.673 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65764 ms_dispat ceph txt unknown /proc/64306/task/65764/exe (readlink: Permission denied) 2024-07-27T03:18:57.673 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65764 ms_dispat ceph NOFD /proc/64306/task/65764/fd (opendir: Permission denied) 2024-07-27T03:18:57.673 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65765 ms_local ceph cwd unknown /proc/64306/task/65765/cwd (readlink: Permission denied) 2024-07-27T03:18:57.673 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65765 ms_local ceph rtd unknown /proc/64306/task/65765/root (readlink: Permission denied) 2024-07-27T03:18:57.673 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65765 ms_local ceph txt unknown /proc/64306/task/65765/exe (readlink: Permission denied) 2024-07-27T03:18:57.673 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65765 ms_local ceph NOFD /proc/64306/task/65765/fd (opendir: Permission denied) 2024-07-27T03:18:57.673 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65766 ms_dispat ceph cwd unknown /proc/64306/task/65766/cwd (readlink: Permission denied) 2024-07-27T03:18:57.673 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65766 ms_dispat ceph rtd unknown /proc/64306/task/65766/root (readlink: Permission denied) 2024-07-27T03:18:57.673 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65766 ms_dispat ceph txt unknown /proc/64306/task/65766/exe (readlink: Permission denied) 2024-07-27T03:18:57.673 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65766 ms_dispat ceph NOFD /proc/64306/task/65766/fd (opendir: Permission denied) 2024-07-27T03:18:57.673 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65767 ms_local ceph cwd unknown /proc/64306/task/65767/cwd (readlink: Permission denied) 2024-07-27T03:18:57.674 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65767 ms_local ceph rtd unknown /proc/64306/task/65767/root (readlink: Permission denied) 2024-07-27T03:18:57.674 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65767 ms_local ceph txt unknown /proc/64306/task/65767/exe (readlink: Permission denied) 2024-07-27T03:18:57.674 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65767 ms_local ceph NOFD /proc/64306/task/65767/fd (opendir: Permission denied) 2024-07-27T03:18:57.674 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65768 ms_dispat ceph cwd unknown /proc/64306/task/65768/cwd (readlink: Permission denied) 2024-07-27T03:18:57.674 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65768 ms_dispat ceph rtd unknown /proc/64306/task/65768/root (readlink: Permission denied) 2024-07-27T03:18:57.674 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65768 ms_dispat ceph txt unknown /proc/64306/task/65768/exe (readlink: Permission denied) 2024-07-27T03:18:57.674 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65768 ms_dispat ceph NOFD /proc/64306/task/65768/fd (opendir: Permission denied) 2024-07-27T03:18:57.674 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65769 ms_local ceph cwd unknown /proc/64306/task/65769/cwd (readlink: Permission denied) 2024-07-27T03:18:57.674 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65769 ms_local ceph rtd unknown /proc/64306/task/65769/root (readlink: Permission denied) 2024-07-27T03:18:57.674 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65769 ms_local ceph txt unknown /proc/64306/task/65769/exe (readlink: Permission denied) 2024-07-27T03:18:57.674 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65769 ms_local ceph NOFD /proc/64306/task/65769/fd (opendir: Permission denied) 2024-07-27T03:18:57.674 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65770 ms_dispat ceph cwd unknown /proc/64306/task/65770/cwd (readlink: Permission denied) 2024-07-27T03:18:57.674 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65770 ms_dispat ceph rtd unknown /proc/64306/task/65770/root (readlink: Permission denied) 2024-07-27T03:18:57.675 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65770 ms_dispat ceph txt unknown /proc/64306/task/65770/exe (readlink: Permission denied) 2024-07-27T03:18:57.675 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65770 ms_dispat ceph NOFD /proc/64306/task/65770/fd (opendir: Permission denied) 2024-07-27T03:18:57.675 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65771 ms_local ceph cwd unknown /proc/64306/task/65771/cwd (readlink: Permission denied) 2024-07-27T03:18:57.675 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65771 ms_local ceph rtd unknown /proc/64306/task/65771/root (readlink: Permission denied) 2024-07-27T03:18:57.675 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65771 ms_local ceph txt unknown /proc/64306/task/65771/exe (readlink: Permission denied) 2024-07-27T03:18:57.675 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65771 ms_local ceph NOFD /proc/64306/task/65771/fd (opendir: Permission denied) 2024-07-27T03:18:57.675 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65772 ms_dispat ceph cwd unknown /proc/64306/task/65772/cwd (readlink: Permission denied) 2024-07-27T03:18:57.675 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65772 ms_dispat ceph rtd unknown /proc/64306/task/65772/root (readlink: Permission denied) 2024-07-27T03:18:57.675 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65772 ms_dispat ceph txt unknown /proc/64306/task/65772/exe (readlink: Permission denied) 2024-07-27T03:18:57.675 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65772 ms_dispat ceph NOFD /proc/64306/task/65772/fd (opendir: Permission denied) 2024-07-27T03:18:57.675 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65773 ms_local ceph cwd unknown /proc/64306/task/65773/cwd (readlink: Permission denied) 2024-07-27T03:18:57.675 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65773 ms_local ceph rtd unknown /proc/64306/task/65773/root (readlink: Permission denied) 2024-07-27T03:18:57.675 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65773 ms_local ceph txt unknown /proc/64306/task/65773/exe (readlink: Permission denied) 2024-07-27T03:18:57.675 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65773 ms_local ceph NOFD /proc/64306/task/65773/fd (opendir: Permission denied) 2024-07-27T03:18:57.676 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65774 ms_dispat ceph cwd unknown /proc/64306/task/65774/cwd (readlink: Permission denied) 2024-07-27T03:18:57.676 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65774 ms_dispat ceph rtd unknown /proc/64306/task/65774/root (readlink: Permission denied) 2024-07-27T03:18:57.676 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65774 ms_dispat ceph txt unknown /proc/64306/task/65774/exe (readlink: Permission denied) 2024-07-27T03:18:57.676 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65774 ms_dispat ceph NOFD /proc/64306/task/65774/fd (opendir: Permission denied) 2024-07-27T03:18:57.676 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65775 ms_local ceph cwd unknown /proc/64306/task/65775/cwd (readlink: Permission denied) 2024-07-27T03:18:57.676 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65775 ms_local ceph rtd unknown /proc/64306/task/65775/root (readlink: Permission denied) 2024-07-27T03:18:57.676 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65775 ms_local ceph txt unknown /proc/64306/task/65775/exe (readlink: Permission denied) 2024-07-27T03:18:57.676 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65775 ms_local ceph NOFD /proc/64306/task/65775/fd (opendir: Permission denied) 2024-07-27T03:18:57.676 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65776 fn_anonym ceph cwd unknown /proc/64306/task/65776/cwd (readlink: Permission denied) 2024-07-27T03:18:57.676 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65776 fn_anonym ceph rtd unknown /proc/64306/task/65776/root (readlink: Permission denied) 2024-07-27T03:18:57.676 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65776 fn_anonym ceph txt unknown /proc/64306/task/65776/exe (readlink: Permission denied) 2024-07-27T03:18:57.676 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65776 fn_anonym ceph NOFD /proc/64306/task/65776/fd (opendir: Permission denied) 2024-07-27T03:18:57.676 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65777 finisher ceph cwd unknown /proc/64306/task/65777/cwd (readlink: Permission denied) 2024-07-27T03:18:57.676 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65777 finisher ceph rtd unknown /proc/64306/task/65777/root (readlink: Permission denied) 2024-07-27T03:18:57.676 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65777 finisher ceph txt unknown /proc/64306/task/65777/exe (readlink: Permission denied) 2024-07-27T03:18:57.677 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65777 finisher ceph NOFD /proc/64306/task/65777/fd (opendir: Permission denied) 2024-07-27T03:18:57.677 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65778 safe_time ceph cwd unknown /proc/64306/task/65778/cwd (readlink: Permission denied) 2024-07-27T03:18:57.677 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65778 safe_time ceph rtd unknown /proc/64306/task/65778/root (readlink: Permission denied) 2024-07-27T03:18:57.677 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65778 safe_time ceph txt unknown /proc/64306/task/65778/exe (readlink: Permission denied) 2024-07-27T03:18:57.677 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65778 safe_time ceph NOFD /proc/64306/task/65778/fd (opendir: Permission denied) 2024-07-27T03:18:57.677 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65779 safe_time ceph cwd unknown /proc/64306/task/65779/cwd (readlink: Permission denied) 2024-07-27T03:18:57.677 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65779 safe_time ceph rtd unknown /proc/64306/task/65779/root (readlink: Permission denied) 2024-07-27T03:18:57.677 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65779 safe_time ceph txt unknown /proc/64306/task/65779/exe (readlink: Permission denied) 2024-07-27T03:18:57.677 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65779 safe_time ceph NOFD /proc/64306/task/65779/fd (opendir: Permission denied) 2024-07-27T03:18:57.677 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65780 ceph-osd ceph cwd unknown /proc/64306/task/65780/cwd (readlink: Permission denied) 2024-07-27T03:18:57.677 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65780 ceph-osd ceph rtd unknown /proc/64306/task/65780/root (readlink: Permission denied) 2024-07-27T03:18:57.677 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65780 ceph-osd ceph txt unknown /proc/64306/task/65780/exe (readlink: Permission denied) 2024-07-27T03:18:57.677 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65780 ceph-osd ceph NOFD /proc/64306/task/65780/fd (opendir: Permission denied) 2024-07-27T03:18:57.678 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65781 osd_srv_a ceph cwd unknown /proc/64306/task/65781/cwd (readlink: Permission denied) 2024-07-27T03:18:57.678 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65781 osd_srv_a ceph rtd unknown /proc/64306/task/65781/root (readlink: Permission denied) 2024-07-27T03:18:57.678 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65781 osd_srv_a ceph txt unknown /proc/64306/task/65781/exe (readlink: Permission denied) 2024-07-27T03:18:57.678 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65781 osd_srv_a ceph NOFD /proc/64306/task/65781/fd (opendir: Permission denied) 2024-07-27T03:18:57.678 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65782 tp_osd_tp ceph cwd unknown /proc/64306/task/65782/cwd (readlink: Permission denied) 2024-07-27T03:18:57.678 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65782 tp_osd_tp ceph rtd unknown /proc/64306/task/65782/root (readlink: Permission denied) 2024-07-27T03:18:57.678 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65782 tp_osd_tp ceph txt unknown /proc/64306/task/65782/exe (readlink: Permission denied) 2024-07-27T03:18:57.678 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65782 tp_osd_tp ceph NOFD /proc/64306/task/65782/fd (opendir: Permission denied) 2024-07-27T03:18:57.678 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65783 tp_osd_tp ceph cwd unknown /proc/64306/task/65783/cwd (readlink: Permission denied) 2024-07-27T03:18:57.678 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65783 tp_osd_tp ceph rtd unknown /proc/64306/task/65783/root (readlink: Permission denied) 2024-07-27T03:18:57.678 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65783 tp_osd_tp ceph txt unknown /proc/64306/task/65783/exe (readlink: Permission denied) 2024-07-27T03:18:57.678 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65783 tp_osd_tp ceph NOFD /proc/64306/task/65783/fd (opendir: Permission denied) 2024-07-27T03:18:57.678 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65784 tp_osd_tp ceph cwd unknown /proc/64306/task/65784/cwd (readlink: Permission denied) 2024-07-27T03:18:57.679 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65784 tp_osd_tp ceph rtd unknown /proc/64306/task/65784/root (readlink: Permission denied) 2024-07-27T03:18:57.679 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65784 tp_osd_tp ceph txt unknown /proc/64306/task/65784/exe (readlink: Permission denied) 2024-07-27T03:18:57.679 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65784 tp_osd_tp ceph NOFD /proc/64306/task/65784/fd (opendir: Permission denied) 2024-07-27T03:18:57.679 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65785 tp_osd_tp ceph cwd unknown /proc/64306/task/65785/cwd (readlink: Permission denied) 2024-07-27T03:18:57.679 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65785 tp_osd_tp ceph rtd unknown /proc/64306/task/65785/root (readlink: Permission denied) 2024-07-27T03:18:57.679 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65785 tp_osd_tp ceph txt unknown /proc/64306/task/65785/exe (readlink: Permission denied) 2024-07-27T03:18:57.679 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65785 tp_osd_tp ceph NOFD /proc/64306/task/65785/fd (opendir: Permission denied) 2024-07-27T03:18:57.679 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65786 tp_osd_tp ceph cwd unknown /proc/64306/task/65786/cwd (readlink: Permission denied) 2024-07-27T03:18:57.679 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65786 tp_osd_tp ceph rtd unknown /proc/64306/task/65786/root (readlink: Permission denied) 2024-07-27T03:18:57.679 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65786 tp_osd_tp ceph txt unknown /proc/64306/task/65786/exe (readlink: Permission denied) 2024-07-27T03:18:57.679 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65786 tp_osd_tp ceph NOFD /proc/64306/task/65786/fd (opendir: Permission denied) 2024-07-27T03:18:57.679 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65787 tp_osd_tp ceph cwd unknown /proc/64306/task/65787/cwd (readlink: Permission denied) 2024-07-27T03:18:57.679 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65787 tp_osd_tp ceph rtd unknown /proc/64306/task/65787/root (readlink: Permission denied) 2024-07-27T03:18:57.679 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65787 tp_osd_tp ceph txt unknown /proc/64306/task/65787/exe (readlink: Permission denied) 2024-07-27T03:18:57.680 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65787 tp_osd_tp ceph NOFD /proc/64306/task/65787/fd (opendir: Permission denied) 2024-07-27T03:18:57.680 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65788 tp_osd_tp ceph cwd unknown /proc/64306/task/65788/cwd (readlink: Permission denied) 2024-07-27T03:18:57.680 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65788 tp_osd_tp ceph rtd unknown /proc/64306/task/65788/root (readlink: Permission denied) 2024-07-27T03:18:57.680 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65788 tp_osd_tp ceph txt unknown /proc/64306/task/65788/exe (readlink: Permission denied) 2024-07-27T03:18:57.680 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65788 tp_osd_tp ceph NOFD /proc/64306/task/65788/fd (opendir: Permission denied) 2024-07-27T03:18:57.680 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65789 tp_osd_tp ceph cwd unknown /proc/64306/task/65789/cwd (readlink: Permission denied) 2024-07-27T03:18:57.680 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65789 tp_osd_tp ceph rtd unknown /proc/64306/task/65789/root (readlink: Permission denied) 2024-07-27T03:18:57.680 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65789 tp_osd_tp ceph txt unknown /proc/64306/task/65789/exe (readlink: Permission denied) 2024-07-27T03:18:57.680 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65789 tp_osd_tp ceph NOFD /proc/64306/task/65789/fd (opendir: Permission denied) 2024-07-27T03:18:57.680 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65790 tp_osd_tp ceph cwd unknown /proc/64306/task/65790/cwd (readlink: Permission denied) 2024-07-27T03:18:57.680 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65790 tp_osd_tp ceph rtd unknown /proc/64306/task/65790/root (readlink: Permission denied) 2024-07-27T03:18:57.680 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65790 tp_osd_tp ceph txt unknown /proc/64306/task/65790/exe (readlink: Permission denied) 2024-07-27T03:18:57.680 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65790 tp_osd_tp ceph NOFD /proc/64306/task/65790/fd (opendir: Permission denied) 2024-07-27T03:18:57.680 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65791 tp_osd_tp ceph cwd unknown /proc/64306/task/65791/cwd (readlink: Permission denied) 2024-07-27T03:18:57.680 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65791 tp_osd_tp ceph rtd unknown /proc/64306/task/65791/root (readlink: Permission denied) 2024-07-27T03:18:57.681 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65791 tp_osd_tp ceph txt unknown /proc/64306/task/65791/exe (readlink: Permission denied) 2024-07-27T03:18:57.681 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65791 tp_osd_tp ceph NOFD /proc/64306/task/65791/fd (opendir: Permission denied) 2024-07-27T03:18:57.681 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65792 tp_osd_tp ceph cwd unknown /proc/64306/task/65792/cwd (readlink: Permission denied) 2024-07-27T03:18:57.681 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65792 tp_osd_tp ceph rtd unknown /proc/64306/task/65792/root (readlink: Permission denied) 2024-07-27T03:18:57.681 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65792 tp_osd_tp ceph txt unknown /proc/64306/task/65792/exe (readlink: Permission denied) 2024-07-27T03:18:57.681 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65792 tp_osd_tp ceph NOFD /proc/64306/task/65792/fd (opendir: Permission denied) 2024-07-27T03:18:57.681 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65793 tp_osd_tp ceph cwd unknown /proc/64306/task/65793/cwd (readlink: Permission denied) 2024-07-27T03:18:57.681 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65793 tp_osd_tp ceph rtd unknown /proc/64306/task/65793/root (readlink: Permission denied) 2024-07-27T03:18:57.681 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65793 tp_osd_tp ceph txt unknown /proc/64306/task/65793/exe (readlink: Permission denied) 2024-07-27T03:18:57.681 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65793 tp_osd_tp ceph NOFD /proc/64306/task/65793/fd (opendir: Permission denied) 2024-07-27T03:18:57.681 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65794 tp_osd_tp ceph cwd unknown /proc/64306/task/65794/cwd (readlink: Permission denied) 2024-07-27T03:18:57.681 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65794 tp_osd_tp ceph rtd unknown /proc/64306/task/65794/root (readlink: Permission denied) 2024-07-27T03:18:57.681 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65794 tp_osd_tp ceph txt unknown /proc/64306/task/65794/exe (readlink: Permission denied) 2024-07-27T03:18:57.681 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65794 tp_osd_tp ceph NOFD /proc/64306/task/65794/fd (opendir: Permission denied) 2024-07-27T03:18:57.682 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65795 tp_osd_tp ceph cwd unknown /proc/64306/task/65795/cwd (readlink: Permission denied) 2024-07-27T03:18:57.682 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65795 tp_osd_tp ceph rtd unknown /proc/64306/task/65795/root (readlink: Permission denied) 2024-07-27T03:18:57.682 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65795 tp_osd_tp ceph txt unknown /proc/64306/task/65795/exe (readlink: Permission denied) 2024-07-27T03:18:57.682 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65795 tp_osd_tp ceph NOFD /proc/64306/task/65795/fd (opendir: Permission denied) 2024-07-27T03:18:57.682 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65796 tp_osd_tp ceph cwd unknown /proc/64306/task/65796/cwd (readlink: Permission denied) 2024-07-27T03:18:57.682 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65796 tp_osd_tp ceph rtd unknown /proc/64306/task/65796/root (readlink: Permission denied) 2024-07-27T03:18:57.682 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65796 tp_osd_tp ceph txt unknown /proc/64306/task/65796/exe (readlink: Permission denied) 2024-07-27T03:18:57.682 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65796 tp_osd_tp ceph NOFD /proc/64306/task/65796/fd (opendir: Permission denied) 2024-07-27T03:18:57.682 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65797 tp_osd_tp ceph cwd unknown /proc/64306/task/65797/cwd (readlink: Permission denied) 2024-07-27T03:18:57.682 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65797 tp_osd_tp ceph rtd unknown /proc/64306/task/65797/root (readlink: Permission denied) 2024-07-27T03:18:57.682 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65797 tp_osd_tp ceph txt unknown /proc/64306/task/65797/exe (readlink: Permission denied) 2024-07-27T03:18:57.682 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65797 tp_osd_tp ceph NOFD /proc/64306/task/65797/fd (opendir: Permission denied) 2024-07-27T03:18:57.682 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65798 osd_srv_h ceph cwd unknown /proc/64306/task/65798/cwd (readlink: Permission denied) 2024-07-27T03:18:57.683 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65798 osd_srv_h ceph rtd unknown /proc/64306/task/65798/root (readlink: Permission denied) 2024-07-27T03:18:57.683 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65798 osd_srv_h ceph txt unknown /proc/64306/task/65798/exe (readlink: Permission denied) 2024-07-27T03:18:57.683 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 64306 65798 osd_srv_h ceph NOFD /proc/64306/task/65798/fd (opendir: Permission denied) 2024-07-27T03:18:57.683 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 65815 root cwd unknown /proc/65815/cwd (readlink: Permission denied) 2024-07-27T03:18:57.683 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 65815 root rtd unknown /proc/65815/root (readlink: Permission denied) 2024-07-27T03:18:57.683 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 65815 root txt unknown /proc/65815/exe (readlink: Permission denied) 2024-07-27T03:18:57.683 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 65815 root NOFD /proc/65815/fd (opendir: Permission denied) 2024-07-27T03:18:57.683 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 65832 root cwd unknown /proc/65832/cwd (readlink: Permission denied) 2024-07-27T03:18:57.683 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 65832 root rtd unknown /proc/65832/root (readlink: Permission denied) 2024-07-27T03:18:57.683 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 65832 root txt unknown /proc/65832/exe (readlink: Permission denied) 2024-07-27T03:18:57.683 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 65832 root NOFD /proc/65832/fd (opendir: Permission denied) 2024-07-27T03:18:57.683 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 68421 root cwd unknown /proc/68421/cwd (readlink: Permission denied) 2024-07-27T03:18:57.683 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 68421 root rtd unknown /proc/68421/root (readlink: Permission denied) 2024-07-27T03:18:57.683 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 68421 root txt unknown /proc/68421/exe (readlink: Permission denied) 2024-07-27T03:18:57.684 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 68421 root NOFD /proc/68421/fd (opendir: Permission denied) 2024-07-27T03:18:57.684 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 71093 root cwd unknown /proc/71093/cwd (readlink: Permission denied) 2024-07-27T03:18:57.684 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 71093 root rtd unknown /proc/71093/root (readlink: Permission denied) 2024-07-27T03:18:57.684 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 71093 root txt unknown /proc/71093/exe (readlink: Permission denied) 2024-07-27T03:18:57.684 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 71093 root NOFD /proc/71093/fd (opendir: Permission denied) 2024-07-27T03:18:57.684 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 71095 root cwd unknown /proc/71095/cwd (readlink: Permission denied) 2024-07-27T03:18:57.684 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 71095 root rtd unknown /proc/71095/root (readlink: Permission denied) 2024-07-27T03:18:57.684 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 71095 root txt unknown /proc/71095/exe (readlink: Permission denied) 2024-07-27T03:18:57.684 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 71095 root NOFD /proc/71095/fd (opendir: Permission denied) 2024-07-27T03:18:57.684 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 ceph cwd unknown /proc/71097/cwd (readlink: Permission denied) 2024-07-27T03:18:57.684 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 ceph rtd unknown /proc/71097/root (readlink: Permission denied) 2024-07-27T03:18:57.684 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 ceph txt unknown /proc/71097/exe (readlink: Permission denied) 2024-07-27T03:18:57.684 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 ceph NOFD /proc/71097/fd (opendir: Permission denied) 2024-07-27T03:18:57.684 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71098 log ceph cwd unknown /proc/71097/task/71098/cwd (readlink: Permission denied) 2024-07-27T03:18:57.685 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71098 log ceph rtd unknown /proc/71097/task/71098/root (readlink: Permission denied) 2024-07-27T03:18:57.685 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71098 log ceph txt unknown /proc/71097/task/71098/exe (readlink: Permission denied) 2024-07-27T03:18:57.685 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71098 log ceph NOFD /proc/71097/task/71098/fd (opendir: Permission denied) 2024-07-27T03:18:57.685 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71100 msgr-work ceph cwd unknown /proc/71097/task/71100/cwd (readlink: Permission denied) 2024-07-27T03:18:57.685 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71100 msgr-work ceph rtd unknown /proc/71097/task/71100/root (readlink: Permission denied) 2024-07-27T03:18:57.685 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71100 msgr-work ceph txt unknown /proc/71097/task/71100/exe (readlink: Permission denied) 2024-07-27T03:18:57.685 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71100 msgr-work ceph NOFD /proc/71097/task/71100/fd (opendir: Permission denied) 2024-07-27T03:18:57.685 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71101 msgr-work ceph cwd unknown /proc/71097/task/71101/cwd (readlink: Permission denied) 2024-07-27T03:18:57.685 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71101 msgr-work ceph rtd unknown /proc/71097/task/71101/root (readlink: Permission denied) 2024-07-27T03:18:57.685 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71101 msgr-work ceph txt unknown /proc/71097/task/71101/exe (readlink: Permission denied) 2024-07-27T03:18:57.685 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71101 msgr-work ceph NOFD /proc/71097/task/71101/fd (opendir: Permission denied) 2024-07-27T03:18:57.685 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71102 msgr-work ceph cwd unknown /proc/71097/task/71102/cwd (readlink: Permission denied) 2024-07-27T03:18:57.685 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71102 msgr-work ceph rtd unknown /proc/71097/task/71102/root (readlink: Permission denied) 2024-07-27T03:18:57.685 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71102 msgr-work ceph txt unknown /proc/71097/task/71102/exe (readlink: Permission denied) 2024-07-27T03:18:57.686 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71102 msgr-work ceph NOFD /proc/71097/task/71102/fd (opendir: Permission denied) 2024-07-27T03:18:57.686 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71106 service ceph cwd unknown /proc/71097/task/71106/cwd (readlink: Permission denied) 2024-07-27T03:18:57.686 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71106 service ceph rtd unknown /proc/71097/task/71106/root (readlink: Permission denied) 2024-07-27T03:18:57.686 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71106 service ceph txt unknown /proc/71097/task/71106/exe (readlink: Permission denied) 2024-07-27T03:18:57.686 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71106 service ceph NOFD /proc/71097/task/71106/fd (opendir: Permission denied) 2024-07-27T03:18:57.686 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71107 admin_soc ceph cwd unknown /proc/71097/task/71107/cwd (readlink: Permission denied) 2024-07-27T03:18:57.686 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71107 admin_soc ceph rtd unknown /proc/71097/task/71107/root (readlink: Permission denied) 2024-07-27T03:18:57.686 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71107 admin_soc ceph txt unknown /proc/71097/task/71107/exe (readlink: Permission denied) 2024-07-27T03:18:57.686 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71107 admin_soc ceph NOFD /proc/71097/task/71107/fd (opendir: Permission denied) 2024-07-27T03:18:57.686 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71364 signal_ha ceph cwd unknown /proc/71097/task/71364/cwd (readlink: Permission denied) 2024-07-27T03:18:57.686 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71364 signal_ha ceph rtd unknown /proc/71097/task/71364/root (readlink: Permission denied) 2024-07-27T03:18:57.686 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71364 signal_ha ceph txt unknown /proc/71097/task/71364/exe (readlink: Permission denied) 2024-07-27T03:18:57.686 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71364 signal_ha ceph NOFD /proc/71097/task/71364/fd (opendir: Permission denied) 2024-07-27T03:18:57.687 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71365 io_contex ceph cwd unknown /proc/71097/task/71365/cwd (readlink: Permission denied) 2024-07-27T03:18:57.687 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71365 io_contex ceph rtd unknown /proc/71097/task/71365/root (readlink: Permission denied) 2024-07-27T03:18:57.687 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71365 io_contex ceph txt unknown /proc/71097/task/71365/exe (readlink: Permission denied) 2024-07-27T03:18:57.687 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71365 io_contex ceph NOFD /proc/71097/task/71365/fd (opendir: Permission denied) 2024-07-27T03:18:57.687 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71366 io_contex ceph cwd unknown /proc/71097/task/71366/cwd (readlink: Permission denied) 2024-07-27T03:18:57.687 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71366 io_contex ceph rtd unknown /proc/71097/task/71366/root (readlink: Permission denied) 2024-07-27T03:18:57.687 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71366 io_contex ceph txt unknown /proc/71097/task/71366/exe (readlink: Permission denied) 2024-07-27T03:18:57.687 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71366 io_contex ceph NOFD /proc/71097/task/71366/fd (opendir: Permission denied) 2024-07-27T03:18:57.687 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71384 OpHistory ceph cwd unknown /proc/71097/task/71384/cwd (readlink: Permission denied) 2024-07-27T03:18:57.687 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71384 OpHistory ceph rtd unknown /proc/71097/task/71384/root (readlink: Permission denied) 2024-07-27T03:18:57.687 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71384 OpHistory ceph txt unknown /proc/71097/task/71384/exe (readlink: Permission denied) 2024-07-27T03:18:57.687 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71384 OpHistory ceph NOFD /proc/71097/task/71384/fd (opendir: Permission denied) 2024-07-27T03:18:57.687 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71385 ceph_time ceph cwd unknown /proc/71097/task/71385/cwd (readlink: Permission denied) 2024-07-27T03:18:57.687 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71385 ceph_time ceph rtd unknown /proc/71097/task/71385/root (readlink: Permission denied) 2024-07-27T03:18:57.687 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71385 ceph_time ceph txt unknown /proc/71097/task/71385/exe (readlink: Permission denied) 2024-07-27T03:18:57.688 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71385 ceph_time ceph NOFD /proc/71097/task/71385/fd (opendir: Permission denied) 2024-07-27T03:18:57.688 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71400 ceph-osd ceph cwd unknown /proc/71097/task/71400/cwd (readlink: Permission denied) 2024-07-27T03:18:57.688 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71400 ceph-osd ceph rtd unknown /proc/71097/task/71400/root (readlink: Permission denied) 2024-07-27T03:18:57.688 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71400 ceph-osd ceph txt unknown /proc/71097/task/71400/exe (readlink: Permission denied) 2024-07-27T03:18:57.688 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71400 ceph-osd ceph NOFD /proc/71097/task/71400/fd (opendir: Permission denied) 2024-07-27T03:18:57.688 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71410 ceph-osd ceph cwd unknown /proc/71097/task/71410/cwd (readlink: Permission denied) 2024-07-27T03:18:57.688 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71410 ceph-osd ceph rtd unknown /proc/71097/task/71410/root (readlink: Permission denied) 2024-07-27T03:18:57.688 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71410 ceph-osd ceph txt unknown /proc/71097/task/71410/exe (readlink: Permission denied) 2024-07-27T03:18:57.688 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71410 ceph-osd ceph NOFD /proc/71097/task/71410/fd (opendir: Permission denied) 2024-07-27T03:18:57.688 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71448 ceph-osd ceph cwd unknown /proc/71097/task/71448/cwd (readlink: Permission denied) 2024-07-27T03:18:57.688 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71448 ceph-osd ceph rtd unknown /proc/71097/task/71448/root (readlink: Permission denied) 2024-07-27T03:18:57.688 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71448 ceph-osd ceph txt unknown /proc/71097/task/71448/exe (readlink: Permission denied) 2024-07-27T03:18:57.688 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71448 ceph-osd ceph NOFD /proc/71097/task/71448/fd (opendir: Permission denied) 2024-07-27T03:18:57.689 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71468 ceph-osd ceph cwd unknown /proc/71097/task/71468/cwd (readlink: Permission denied) 2024-07-27T03:18:57.689 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71468 ceph-osd ceph rtd unknown /proc/71097/task/71468/root (readlink: Permission denied) 2024-07-27T03:18:57.689 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71468 ceph-osd ceph txt unknown /proc/71097/task/71468/exe (readlink: Permission denied) 2024-07-27T03:18:57.689 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71468 ceph-osd ceph NOFD /proc/71097/task/71468/fd (opendir: Permission denied) 2024-07-27T03:18:57.689 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71472 ceph-osd ceph cwd unknown /proc/71097/task/71472/cwd (readlink: Permission denied) 2024-07-27T03:18:57.689 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71472 ceph-osd ceph rtd unknown /proc/71097/task/71472/root (readlink: Permission denied) 2024-07-27T03:18:57.689 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71472 ceph-osd ceph txt unknown /proc/71097/task/71472/exe (readlink: Permission denied) 2024-07-27T03:18:57.689 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71472 ceph-osd ceph NOFD /proc/71097/task/71472/fd (opendir: Permission denied) 2024-07-27T03:18:57.689 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71497 ceph-osd ceph cwd unknown /proc/71097/task/71497/cwd (readlink: Permission denied) 2024-07-27T03:18:57.689 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71497 ceph-osd ceph rtd unknown /proc/71097/task/71497/root (readlink: Permission denied) 2024-07-27T03:18:57.689 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71497 ceph-osd ceph txt unknown /proc/71097/task/71497/exe (readlink: Permission denied) 2024-07-27T03:18:57.689 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71497 ceph-osd ceph NOFD /proc/71097/task/71497/fd (opendir: Permission denied) 2024-07-27T03:18:57.689 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71525 ceph-osd ceph cwd unknown /proc/71097/task/71525/cwd (readlink: Permission denied) 2024-07-27T03:18:57.689 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71525 ceph-osd ceph rtd unknown /proc/71097/task/71525/root (readlink: Permission denied) 2024-07-27T03:18:57.690 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71525 ceph-osd ceph txt unknown /proc/71097/task/71525/exe (readlink: Permission denied) 2024-07-27T03:18:57.690 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71525 ceph-osd ceph NOFD /proc/71097/task/71525/fd (opendir: Permission denied) 2024-07-27T03:18:57.690 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71653 ceph-osd ceph cwd unknown /proc/71097/task/71653/cwd (readlink: Permission denied) 2024-07-27T03:18:57.690 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71653 ceph-osd ceph rtd unknown /proc/71097/task/71653/root (readlink: Permission denied) 2024-07-27T03:18:57.690 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71653 ceph-osd ceph txt unknown /proc/71097/task/71653/exe (readlink: Permission denied) 2024-07-27T03:18:57.690 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71653 ceph-osd ceph NOFD /proc/71097/task/71653/fd (opendir: Permission denied) 2024-07-27T03:18:57.690 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71655 ceph-osd ceph cwd unknown /proc/71097/task/71655/cwd (readlink: Permission denied) 2024-07-27T03:18:57.690 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71655 ceph-osd ceph rtd unknown /proc/71097/task/71655/root (readlink: Permission denied) 2024-07-27T03:18:57.690 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71655 ceph-osd ceph txt unknown /proc/71097/task/71655/exe (readlink: Permission denied) 2024-07-27T03:18:57.690 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71655 ceph-osd ceph NOFD /proc/71097/task/71655/fd (opendir: Permission denied) 2024-07-27T03:18:57.690 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71657 safe_time ceph cwd unknown /proc/71097/task/71657/cwd (readlink: Permission denied) 2024-07-27T03:18:57.690 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71657 safe_time ceph rtd unknown /proc/71097/task/71657/root (readlink: Permission denied) 2024-07-27T03:18:57.690 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71657 safe_time ceph txt unknown /proc/71097/task/71657/exe (readlink: Permission denied) 2024-07-27T03:18:57.691 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71657 safe_time ceph NOFD /proc/71097/task/71657/fd (opendir: Permission denied) 2024-07-27T03:18:57.691 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71658 safe_time ceph cwd unknown /proc/71097/task/71658/cwd (readlink: Permission denied) 2024-07-27T03:18:57.691 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71658 safe_time ceph rtd unknown /proc/71097/task/71658/root (readlink: Permission denied) 2024-07-27T03:18:57.691 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71658 safe_time ceph txt unknown /proc/71097/task/71658/exe (readlink: Permission denied) 2024-07-27T03:18:57.691 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71658 safe_time ceph NOFD /proc/71097/task/71658/fd (opendir: Permission denied) 2024-07-27T03:18:57.691 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71659 safe_time ceph cwd unknown /proc/71097/task/71659/cwd (readlink: Permission denied) 2024-07-27T03:18:57.691 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71659 safe_time ceph rtd unknown /proc/71097/task/71659/root (readlink: Permission denied) 2024-07-27T03:18:57.691 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71659 safe_time ceph txt unknown /proc/71097/task/71659/exe (readlink: Permission denied) 2024-07-27T03:18:57.691 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71659 safe_time ceph NOFD /proc/71097/task/71659/fd (opendir: Permission denied) 2024-07-27T03:18:57.691 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71660 safe_time ceph cwd unknown /proc/71097/task/71660/cwd (readlink: Permission denied) 2024-07-27T03:18:57.691 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71660 safe_time ceph rtd unknown /proc/71097/task/71660/root (readlink: Permission denied) 2024-07-27T03:18:57.691 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71660 safe_time ceph txt unknown /proc/71097/task/71660/exe (readlink: Permission denied) 2024-07-27T03:18:57.691 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71660 safe_time ceph NOFD /proc/71097/task/71660/fd (opendir: Permission denied) 2024-07-27T03:18:57.691 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71661 fn_anonym ceph cwd unknown /proc/71097/task/71661/cwd (readlink: Permission denied) 2024-07-27T03:18:57.692 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71661 fn_anonym ceph rtd unknown /proc/71097/task/71661/root (readlink: Permission denied) 2024-07-27T03:18:57.692 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71661 fn_anonym ceph txt unknown /proc/71097/task/71661/exe (readlink: Permission denied) 2024-07-27T03:18:57.692 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71661 fn_anonym ceph NOFD /proc/71097/task/71661/fd (opendir: Permission denied) 2024-07-27T03:18:57.692 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71667 rocksdb:l ceph cwd unknown /proc/71097/task/71667/cwd (readlink: Permission denied) 2024-07-27T03:18:57.692 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71667 rocksdb:l ceph rtd unknown /proc/71097/task/71667/root (readlink: Permission denied) 2024-07-27T03:18:57.692 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71667 rocksdb:l ceph txt unknown /proc/71097/task/71667/exe (readlink: Permission denied) 2024-07-27T03:18:57.692 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71667 rocksdb:l ceph NOFD /proc/71097/task/71667/fd (opendir: Permission denied) 2024-07-27T03:18:57.692 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71669 rocksdb:l ceph cwd unknown /proc/71097/task/71669/cwd (readlink: Permission denied) 2024-07-27T03:18:57.692 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71669 rocksdb:l ceph rtd unknown /proc/71097/task/71669/root (readlink: Permission denied) 2024-07-27T03:18:57.692 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71669 rocksdb:l ceph txt unknown /proc/71097/task/71669/exe (readlink: Permission denied) 2024-07-27T03:18:57.692 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71669 rocksdb:l ceph NOFD /proc/71097/task/71669/fd (opendir: Permission denied) 2024-07-27T03:18:57.692 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71670 rocksdb:l ceph cwd unknown /proc/71097/task/71670/cwd (readlink: Permission denied) 2024-07-27T03:18:57.692 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71670 rocksdb:l ceph rtd unknown /proc/71097/task/71670/root (readlink: Permission denied) 2024-07-27T03:18:57.693 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71670 rocksdb:l ceph txt unknown /proc/71097/task/71670/exe (readlink: Permission denied) 2024-07-27T03:18:57.693 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71670 rocksdb:l ceph NOFD /proc/71097/task/71670/fd (opendir: Permission denied) 2024-07-27T03:18:57.693 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71671 rocksdb:h ceph cwd unknown /proc/71097/task/71671/cwd (readlink: Permission denied) 2024-07-27T03:18:57.693 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71671 rocksdb:h ceph rtd unknown /proc/71097/task/71671/root (readlink: Permission denied) 2024-07-27T03:18:57.693 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71671 rocksdb:h ceph txt unknown /proc/71097/task/71671/exe (readlink: Permission denied) 2024-07-27T03:18:57.693 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 71671 rocksdb:h ceph NOFD /proc/71097/task/71671/fd (opendir: Permission denied) 2024-07-27T03:18:57.693 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72175 bstore_ai ceph cwd unknown /proc/71097/task/72175/cwd (readlink: Permission denied) 2024-07-27T03:18:57.693 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72175 bstore_ai ceph rtd unknown /proc/71097/task/72175/root (readlink: Permission denied) 2024-07-27T03:18:57.693 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72175 bstore_ai ceph txt unknown /proc/71097/task/72175/exe (readlink: Permission denied) 2024-07-27T03:18:57.693 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72175 bstore_ai ceph NOFD /proc/71097/task/72175/fd (opendir: Permission denied) 2024-07-27T03:18:57.693 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72404 bstore_ai ceph cwd unknown /proc/71097/task/72404/cwd (readlink: Permission denied) 2024-07-27T03:18:57.693 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72404 bstore_ai ceph rtd unknown /proc/71097/task/72404/root (readlink: Permission denied) 2024-07-27T03:18:57.693 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72404 bstore_ai ceph txt unknown /proc/71097/task/72404/exe (readlink: Permission denied) 2024-07-27T03:18:57.693 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72404 bstore_ai ceph NOFD /proc/71097/task/72404/fd (opendir: Permission denied) 2024-07-27T03:18:57.694 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72588 ceph-osd ceph cwd unknown /proc/71097/task/72588/cwd (readlink: Permission denied) 2024-07-27T03:18:57.694 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72588 ceph-osd ceph rtd unknown /proc/71097/task/72588/root (readlink: Permission denied) 2024-07-27T03:18:57.694 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72588 ceph-osd ceph txt unknown /proc/71097/task/72588/exe (readlink: Permission denied) 2024-07-27T03:18:57.694 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72588 ceph-osd ceph NOFD /proc/71097/task/72588/fd (opendir: Permission denied) 2024-07-27T03:18:57.694 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72589 cfin ceph cwd unknown /proc/71097/task/72589/cwd (readlink: Permission denied) 2024-07-27T03:18:57.694 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72589 cfin ceph rtd unknown /proc/71097/task/72589/root (readlink: Permission denied) 2024-07-27T03:18:57.694 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72589 cfin ceph txt unknown /proc/71097/task/72589/exe (readlink: Permission denied) 2024-07-27T03:18:57.694 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72589 cfin ceph NOFD /proc/71097/task/72589/fd (opendir: Permission denied) 2024-07-27T03:18:57.694 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72590 bstore_kv ceph cwd unknown /proc/71097/task/72590/cwd (readlink: Permission denied) 2024-07-27T03:18:57.694 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72590 bstore_kv ceph rtd unknown /proc/71097/task/72590/root (readlink: Permission denied) 2024-07-27T03:18:57.694 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72590 bstore_kv ceph txt unknown /proc/71097/task/72590/exe (readlink: Permission denied) 2024-07-27T03:18:57.694 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72590 bstore_kv ceph NOFD /proc/71097/task/72590/fd (opendir: Permission denied) 2024-07-27T03:18:57.694 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72591 bstore_kv ceph cwd unknown /proc/71097/task/72591/cwd (readlink: Permission denied) 2024-07-27T03:18:57.695 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72591 bstore_kv ceph rtd unknown /proc/71097/task/72591/root (readlink: Permission denied) 2024-07-27T03:18:57.695 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72591 bstore_kv ceph txt unknown /proc/71097/task/72591/exe (readlink: Permission denied) 2024-07-27T03:18:57.695 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72591 bstore_kv ceph NOFD /proc/71097/task/72591/fd (opendir: Permission denied) 2024-07-27T03:18:57.695 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72592 bstore_me ceph cwd unknown /proc/71097/task/72592/cwd (readlink: Permission denied) 2024-07-27T03:18:57.695 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72592 bstore_me ceph rtd unknown /proc/71097/task/72592/root (readlink: Permission denied) 2024-07-27T03:18:57.695 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72592 bstore_me ceph txt unknown /proc/71097/task/72592/exe (readlink: Permission denied) 2024-07-27T03:18:57.695 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72592 bstore_me ceph NOFD /proc/71097/task/72592/fd (opendir: Permission denied) 2024-07-27T03:18:57.695 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72593 ms_dispat ceph cwd unknown /proc/71097/task/72593/cwd (readlink: Permission denied) 2024-07-27T03:18:57.695 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72593 ms_dispat ceph rtd unknown /proc/71097/task/72593/root (readlink: Permission denied) 2024-07-27T03:18:57.695 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72593 ms_dispat ceph txt unknown /proc/71097/task/72593/exe (readlink: Permission denied) 2024-07-27T03:18:57.695 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72593 ms_dispat ceph NOFD /proc/71097/task/72593/fd (opendir: Permission denied) 2024-07-27T03:18:57.695 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72594 ms_local ceph cwd unknown /proc/71097/task/72594/cwd (readlink: Permission denied) 2024-07-27T03:18:57.695 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72594 ms_local ceph rtd unknown /proc/71097/task/72594/root (readlink: Permission denied) 2024-07-27T03:18:57.695 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72594 ms_local ceph txt unknown /proc/71097/task/72594/exe (readlink: Permission denied) 2024-07-27T03:18:57.696 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72594 ms_local ceph NOFD /proc/71097/task/72594/fd (opendir: Permission denied) 2024-07-27T03:18:57.696 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72595 safe_time ceph cwd unknown /proc/71097/task/72595/cwd (readlink: Permission denied) 2024-07-27T03:18:57.696 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72595 safe_time ceph rtd unknown /proc/71097/task/72595/root (readlink: Permission denied) 2024-07-27T03:18:57.696 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72595 safe_time ceph txt unknown /proc/71097/task/72595/exe (readlink: Permission denied) 2024-07-27T03:18:57.696 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72595 safe_time ceph NOFD /proc/71097/task/72595/fd (opendir: Permission denied) 2024-07-27T03:18:57.696 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72596 safe_time ceph cwd unknown /proc/71097/task/72596/cwd (readlink: Permission denied) 2024-07-27T03:18:57.696 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72596 safe_time ceph rtd unknown /proc/71097/task/72596/root (readlink: Permission denied) 2024-07-27T03:18:57.696 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72596 safe_time ceph txt unknown /proc/71097/task/72596/exe (readlink: Permission denied) 2024-07-27T03:18:57.696 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72596 safe_time ceph NOFD /proc/71097/task/72596/fd (opendir: Permission denied) 2024-07-27T03:18:57.696 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72597 ms_dispat ceph cwd unknown /proc/71097/task/72597/cwd (readlink: Permission denied) 2024-07-27T03:18:57.696 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72597 ms_dispat ceph rtd unknown /proc/71097/task/72597/root (readlink: Permission denied) 2024-07-27T03:18:57.696 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72597 ms_dispat ceph txt unknown /proc/71097/task/72597/exe (readlink: Permission denied) 2024-07-27T03:18:57.696 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72597 ms_dispat ceph NOFD /proc/71097/task/72597/fd (opendir: Permission denied) 2024-07-27T03:18:57.697 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72598 ms_local ceph cwd unknown /proc/71097/task/72598/cwd (readlink: Permission denied) 2024-07-27T03:18:57.697 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72598 ms_local ceph rtd unknown /proc/71097/task/72598/root (readlink: Permission denied) 2024-07-27T03:18:57.697 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72598 ms_local ceph txt unknown /proc/71097/task/72598/exe (readlink: Permission denied) 2024-07-27T03:18:57.697 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72598 ms_local ceph NOFD /proc/71097/task/72598/fd (opendir: Permission denied) 2024-07-27T03:18:57.697 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72599 ms_dispat ceph cwd unknown /proc/71097/task/72599/cwd (readlink: Permission denied) 2024-07-27T03:18:57.697 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72599 ms_dispat ceph rtd unknown /proc/71097/task/72599/root (readlink: Permission denied) 2024-07-27T03:18:57.697 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72599 ms_dispat ceph txt unknown /proc/71097/task/72599/exe (readlink: Permission denied) 2024-07-27T03:18:57.697 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72599 ms_dispat ceph NOFD /proc/71097/task/72599/fd (opendir: Permission denied) 2024-07-27T03:18:57.697 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72600 ms_local ceph cwd unknown /proc/71097/task/72600/cwd (readlink: Permission denied) 2024-07-27T03:18:57.697 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72600 ms_local ceph rtd unknown /proc/71097/task/72600/root (readlink: Permission denied) 2024-07-27T03:18:57.697 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72600 ms_local ceph txt unknown /proc/71097/task/72600/exe (readlink: Permission denied) 2024-07-27T03:18:57.697 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72600 ms_local ceph NOFD /proc/71097/task/72600/fd (opendir: Permission denied) 2024-07-27T03:18:57.697 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72601 ms_dispat ceph cwd unknown /proc/71097/task/72601/cwd (readlink: Permission denied) 2024-07-27T03:18:57.698 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72601 ms_dispat ceph rtd unknown /proc/71097/task/72601/root (readlink: Permission denied) 2024-07-27T03:18:57.698 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72601 ms_dispat ceph txt unknown /proc/71097/task/72601/exe (readlink: Permission denied) 2024-07-27T03:18:57.698 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72601 ms_dispat ceph NOFD /proc/71097/task/72601/fd (opendir: Permission denied) 2024-07-27T03:18:57.698 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72602 ms_local ceph cwd unknown /proc/71097/task/72602/cwd (readlink: Permission denied) 2024-07-27T03:18:57.698 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72602 ms_local ceph rtd unknown /proc/71097/task/72602/root (readlink: Permission denied) 2024-07-27T03:18:57.698 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72602 ms_local ceph txt unknown /proc/71097/task/72602/exe (readlink: Permission denied) 2024-07-27T03:18:57.698 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72602 ms_local ceph NOFD /proc/71097/task/72602/fd (opendir: Permission denied) 2024-07-27T03:18:57.698 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72603 ms_dispat ceph cwd unknown /proc/71097/task/72603/cwd (readlink: Permission denied) 2024-07-27T03:18:57.698 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72603 ms_dispat ceph rtd unknown /proc/71097/task/72603/root (readlink: Permission denied) 2024-07-27T03:18:57.698 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72603 ms_dispat ceph txt unknown /proc/71097/task/72603/exe (readlink: Permission denied) 2024-07-27T03:18:57.698 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72603 ms_dispat ceph NOFD /proc/71097/task/72603/fd (opendir: Permission denied) 2024-07-27T03:18:57.698 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72604 ms_local ceph cwd unknown /proc/71097/task/72604/cwd (readlink: Permission denied) 2024-07-27T03:18:57.698 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72604 ms_local ceph rtd unknown /proc/71097/task/72604/root (readlink: Permission denied) 2024-07-27T03:18:57.698 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72604 ms_local ceph txt unknown /proc/71097/task/72604/exe (readlink: Permission denied) 2024-07-27T03:18:57.699 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72604 ms_local ceph NOFD /proc/71097/task/72604/fd (opendir: Permission denied) 2024-07-27T03:18:57.699 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72605 ms_dispat ceph cwd unknown /proc/71097/task/72605/cwd (readlink: Permission denied) 2024-07-27T03:18:57.699 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72605 ms_dispat ceph rtd unknown /proc/71097/task/72605/root (readlink: Permission denied) 2024-07-27T03:18:57.699 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72605 ms_dispat ceph txt unknown /proc/71097/task/72605/exe (readlink: Permission denied) 2024-07-27T03:18:57.699 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72605 ms_dispat ceph NOFD /proc/71097/task/72605/fd (opendir: Permission denied) 2024-07-27T03:18:57.699 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72606 ms_local ceph cwd unknown /proc/71097/task/72606/cwd (readlink: Permission denied) 2024-07-27T03:18:57.699 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72606 ms_local ceph rtd unknown /proc/71097/task/72606/root (readlink: Permission denied) 2024-07-27T03:18:57.699 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72606 ms_local ceph txt unknown /proc/71097/task/72606/exe (readlink: Permission denied) 2024-07-27T03:18:57.699 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72606 ms_local ceph NOFD /proc/71097/task/72606/fd (opendir: Permission denied) 2024-07-27T03:18:57.699 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72607 ms_dispat ceph cwd unknown /proc/71097/task/72607/cwd (readlink: Permission denied) 2024-07-27T03:18:57.699 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72607 ms_dispat ceph rtd unknown /proc/71097/task/72607/root (readlink: Permission denied) 2024-07-27T03:18:57.699 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72607 ms_dispat ceph txt unknown /proc/71097/task/72607/exe (readlink: Permission denied) 2024-07-27T03:18:57.699 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72607 ms_dispat ceph NOFD /proc/71097/task/72607/fd (opendir: Permission denied) 2024-07-27T03:18:57.700 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72608 ms_local ceph cwd unknown /proc/71097/task/72608/cwd (readlink: Permission denied) 2024-07-27T03:18:57.700 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72608 ms_local ceph rtd unknown /proc/71097/task/72608/root (readlink: Permission denied) 2024-07-27T03:18:57.700 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72608 ms_local ceph txt unknown /proc/71097/task/72608/exe (readlink: Permission denied) 2024-07-27T03:18:57.700 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72608 ms_local ceph NOFD /proc/71097/task/72608/fd (opendir: Permission denied) 2024-07-27T03:18:57.700 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72609 fn_anonym ceph cwd unknown /proc/71097/task/72609/cwd (readlink: Permission denied) 2024-07-27T03:18:57.700 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72609 fn_anonym ceph rtd unknown /proc/71097/task/72609/root (readlink: Permission denied) 2024-07-27T03:18:57.700 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72609 fn_anonym ceph txt unknown /proc/71097/task/72609/exe (readlink: Permission denied) 2024-07-27T03:18:57.700 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72609 fn_anonym ceph NOFD /proc/71097/task/72609/fd (opendir: Permission denied) 2024-07-27T03:18:57.700 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72610 finisher ceph cwd unknown /proc/71097/task/72610/cwd (readlink: Permission denied) 2024-07-27T03:18:57.700 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72610 finisher ceph rtd unknown /proc/71097/task/72610/root (readlink: Permission denied) 2024-07-27T03:18:57.700 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72610 finisher ceph txt unknown /proc/71097/task/72610/exe (readlink: Permission denied) 2024-07-27T03:18:57.700 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72610 finisher ceph NOFD /proc/71097/task/72610/fd (opendir: Permission denied) 2024-07-27T03:18:57.700 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72611 safe_time ceph cwd unknown /proc/71097/task/72611/cwd (readlink: Permission denied) 2024-07-27T03:18:57.700 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72611 safe_time ceph rtd unknown /proc/71097/task/72611/root (readlink: Permission denied) 2024-07-27T03:18:57.701 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72611 safe_time ceph txt unknown /proc/71097/task/72611/exe (readlink: Permission denied) 2024-07-27T03:18:57.701 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72611 safe_time ceph NOFD /proc/71097/task/72611/fd (opendir: Permission denied) 2024-07-27T03:18:57.701 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72612 safe_time ceph cwd unknown /proc/71097/task/72612/cwd (readlink: Permission denied) 2024-07-27T03:18:57.701 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72612 safe_time ceph rtd unknown /proc/71097/task/72612/root (readlink: Permission denied) 2024-07-27T03:18:57.701 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72612 safe_time ceph txt unknown /proc/71097/task/72612/exe (readlink: Permission denied) 2024-07-27T03:18:57.701 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72612 safe_time ceph NOFD /proc/71097/task/72612/fd (opendir: Permission denied) 2024-07-27T03:18:57.701 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72613 ceph-osd ceph cwd unknown /proc/71097/task/72613/cwd (readlink: Permission denied) 2024-07-27T03:18:57.701 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72613 ceph-osd ceph rtd unknown /proc/71097/task/72613/root (readlink: Permission denied) 2024-07-27T03:18:57.701 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72613 ceph-osd ceph txt unknown /proc/71097/task/72613/exe (readlink: Permission denied) 2024-07-27T03:18:57.701 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72613 ceph-osd ceph NOFD /proc/71097/task/72613/fd (opendir: Permission denied) 2024-07-27T03:18:57.701 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72614 osd_srv_a ceph cwd unknown /proc/71097/task/72614/cwd (readlink: Permission denied) 2024-07-27T03:18:57.701 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72614 osd_srv_a ceph rtd unknown /proc/71097/task/72614/root (readlink: Permission denied) 2024-07-27T03:18:57.701 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72614 osd_srv_a ceph txt unknown /proc/71097/task/72614/exe (readlink: Permission denied) 2024-07-27T03:18:57.701 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72614 osd_srv_a ceph NOFD /proc/71097/task/72614/fd (opendir: Permission denied) 2024-07-27T03:18:57.702 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72615 tp_osd_tp ceph cwd unknown /proc/71097/task/72615/cwd (readlink: Permission denied) 2024-07-27T03:18:57.702 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72615 tp_osd_tp ceph rtd unknown /proc/71097/task/72615/root (readlink: Permission denied) 2024-07-27T03:18:57.702 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72615 tp_osd_tp ceph txt unknown /proc/71097/task/72615/exe (readlink: Permission denied) 2024-07-27T03:18:57.702 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72615 tp_osd_tp ceph NOFD /proc/71097/task/72615/fd (opendir: Permission denied) 2024-07-27T03:18:57.702 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72616 tp_osd_tp ceph cwd unknown /proc/71097/task/72616/cwd (readlink: Permission denied) 2024-07-27T03:18:57.702 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72616 tp_osd_tp ceph rtd unknown /proc/71097/task/72616/root (readlink: Permission denied) 2024-07-27T03:18:57.702 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72616 tp_osd_tp ceph txt unknown /proc/71097/task/72616/exe (readlink: Permission denied) 2024-07-27T03:18:57.702 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72616 tp_osd_tp ceph NOFD /proc/71097/task/72616/fd (opendir: Permission denied) 2024-07-27T03:18:57.702 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72617 tp_osd_tp ceph cwd unknown /proc/71097/task/72617/cwd (readlink: Permission denied) 2024-07-27T03:18:57.702 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72617 tp_osd_tp ceph rtd unknown /proc/71097/task/72617/root (readlink: Permission denied) 2024-07-27T03:18:57.702 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72617 tp_osd_tp ceph txt unknown /proc/71097/task/72617/exe (readlink: Permission denied) 2024-07-27T03:18:57.702 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72617 tp_osd_tp ceph NOFD /proc/71097/task/72617/fd (opendir: Permission denied) 2024-07-27T03:18:57.702 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72618 tp_osd_tp ceph cwd unknown /proc/71097/task/72618/cwd (readlink: Permission denied) 2024-07-27T03:18:57.702 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72618 tp_osd_tp ceph rtd unknown /proc/71097/task/72618/root (readlink: Permission denied) 2024-07-27T03:18:57.703 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72618 tp_osd_tp ceph txt unknown /proc/71097/task/72618/exe (readlink: Permission denied) 2024-07-27T03:18:57.703 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72618 tp_osd_tp ceph NOFD /proc/71097/task/72618/fd (opendir: Permission denied) 2024-07-27T03:18:57.703 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72619 tp_osd_tp ceph cwd unknown /proc/71097/task/72619/cwd (readlink: Permission denied) 2024-07-27T03:18:57.703 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72619 tp_osd_tp ceph rtd unknown /proc/71097/task/72619/root (readlink: Permission denied) 2024-07-27T03:18:57.703 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72619 tp_osd_tp ceph txt unknown /proc/71097/task/72619/exe (readlink: Permission denied) 2024-07-27T03:18:57.703 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72619 tp_osd_tp ceph NOFD /proc/71097/task/72619/fd (opendir: Permission denied) 2024-07-27T03:18:57.703 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72620 tp_osd_tp ceph cwd unknown /proc/71097/task/72620/cwd (readlink: Permission denied) 2024-07-27T03:18:57.703 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72620 tp_osd_tp ceph rtd unknown /proc/71097/task/72620/root (readlink: Permission denied) 2024-07-27T03:18:57.703 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72620 tp_osd_tp ceph txt unknown /proc/71097/task/72620/exe (readlink: Permission denied) 2024-07-27T03:18:57.703 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72620 tp_osd_tp ceph NOFD /proc/71097/task/72620/fd (opendir: Permission denied) 2024-07-27T03:18:57.703 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72621 tp_osd_tp ceph cwd unknown /proc/71097/task/72621/cwd (readlink: Permission denied) 2024-07-27T03:18:57.703 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72621 tp_osd_tp ceph rtd unknown /proc/71097/task/72621/root (readlink: Permission denied) 2024-07-27T03:18:57.703 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72621 tp_osd_tp ceph txt unknown /proc/71097/task/72621/exe (readlink: Permission denied) 2024-07-27T03:18:57.703 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72621 tp_osd_tp ceph NOFD /proc/71097/task/72621/fd (opendir: Permission denied) 2024-07-27T03:18:57.704 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72622 tp_osd_tp ceph cwd unknown /proc/71097/task/72622/cwd (readlink: Permission denied) 2024-07-27T03:18:57.704 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72622 tp_osd_tp ceph rtd unknown /proc/71097/task/72622/root (readlink: Permission denied) 2024-07-27T03:18:57.704 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72622 tp_osd_tp ceph txt unknown /proc/71097/task/72622/exe (readlink: Permission denied) 2024-07-27T03:18:57.704 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72622 tp_osd_tp ceph NOFD /proc/71097/task/72622/fd (opendir: Permission denied) 2024-07-27T03:18:57.704 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72623 tp_osd_tp ceph cwd unknown /proc/71097/task/72623/cwd (readlink: Permission denied) 2024-07-27T03:18:57.704 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72623 tp_osd_tp ceph rtd unknown /proc/71097/task/72623/root (readlink: Permission denied) 2024-07-27T03:18:57.704 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72623 tp_osd_tp ceph txt unknown /proc/71097/task/72623/exe (readlink: Permission denied) 2024-07-27T03:18:57.704 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72623 tp_osd_tp ceph NOFD /proc/71097/task/72623/fd (opendir: Permission denied) 2024-07-27T03:18:57.704 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72624 tp_osd_tp ceph cwd unknown /proc/71097/task/72624/cwd (readlink: Permission denied) 2024-07-27T03:18:57.704 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72624 tp_osd_tp ceph rtd unknown /proc/71097/task/72624/root (readlink: Permission denied) 2024-07-27T03:18:57.704 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72624 tp_osd_tp ceph txt unknown /proc/71097/task/72624/exe (readlink: Permission denied) 2024-07-27T03:18:57.704 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72624 tp_osd_tp ceph NOFD /proc/71097/task/72624/fd (opendir: Permission denied) 2024-07-27T03:18:57.704 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72625 tp_osd_tp ceph cwd unknown /proc/71097/task/72625/cwd (readlink: Permission denied) 2024-07-27T03:18:57.705 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72625 tp_osd_tp ceph rtd unknown /proc/71097/task/72625/root (readlink: Permission denied) 2024-07-27T03:18:57.705 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72625 tp_osd_tp ceph txt unknown /proc/71097/task/72625/exe (readlink: Permission denied) 2024-07-27T03:18:57.705 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72625 tp_osd_tp ceph NOFD /proc/71097/task/72625/fd (opendir: Permission denied) 2024-07-27T03:18:57.705 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72626 tp_osd_tp ceph cwd unknown /proc/71097/task/72626/cwd (readlink: Permission denied) 2024-07-27T03:18:57.705 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72626 tp_osd_tp ceph rtd unknown /proc/71097/task/72626/root (readlink: Permission denied) 2024-07-27T03:18:57.705 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72626 tp_osd_tp ceph txt unknown /proc/71097/task/72626/exe (readlink: Permission denied) 2024-07-27T03:18:57.705 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72626 tp_osd_tp ceph NOFD /proc/71097/task/72626/fd (opendir: Permission denied) 2024-07-27T03:18:57.705 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72627 tp_osd_tp ceph cwd unknown /proc/71097/task/72627/cwd (readlink: Permission denied) 2024-07-27T03:18:57.705 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72627 tp_osd_tp ceph rtd unknown /proc/71097/task/72627/root (readlink: Permission denied) 2024-07-27T03:18:57.705 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72627 tp_osd_tp ceph txt unknown /proc/71097/task/72627/exe (readlink: Permission denied) 2024-07-27T03:18:57.705 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72627 tp_osd_tp ceph NOFD /proc/71097/task/72627/fd (opendir: Permission denied) 2024-07-27T03:18:57.705 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72628 tp_osd_tp ceph cwd unknown /proc/71097/task/72628/cwd (readlink: Permission denied) 2024-07-27T03:18:57.705 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72628 tp_osd_tp ceph rtd unknown /proc/71097/task/72628/root (readlink: Permission denied) 2024-07-27T03:18:57.705 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72628 tp_osd_tp ceph txt unknown /proc/71097/task/72628/exe (readlink: Permission denied) 2024-07-27T03:18:57.706 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72628 tp_osd_tp ceph NOFD /proc/71097/task/72628/fd (opendir: Permission denied) 2024-07-27T03:18:57.706 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72629 tp_osd_tp ceph cwd unknown /proc/71097/task/72629/cwd (readlink: Permission denied) 2024-07-27T03:18:57.706 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72629 tp_osd_tp ceph rtd unknown /proc/71097/task/72629/root (readlink: Permission denied) 2024-07-27T03:18:57.706 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72629 tp_osd_tp ceph txt unknown /proc/71097/task/72629/exe (readlink: Permission denied) 2024-07-27T03:18:57.706 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72629 tp_osd_tp ceph NOFD /proc/71097/task/72629/fd (opendir: Permission denied) 2024-07-27T03:18:57.706 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72630 tp_osd_tp ceph cwd unknown /proc/71097/task/72630/cwd (readlink: Permission denied) 2024-07-27T03:18:57.706 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72630 tp_osd_tp ceph rtd unknown /proc/71097/task/72630/root (readlink: Permission denied) 2024-07-27T03:18:57.706 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72630 tp_osd_tp ceph txt unknown /proc/71097/task/72630/exe (readlink: Permission denied) 2024-07-27T03:18:57.706 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72630 tp_osd_tp ceph NOFD /proc/71097/task/72630/fd (opendir: Permission denied) 2024-07-27T03:18:57.706 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72631 osd_srv_h ceph cwd unknown /proc/71097/task/72631/cwd (readlink: Permission denied) 2024-07-27T03:18:57.706 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72631 osd_srv_h ceph rtd unknown /proc/71097/task/72631/root (readlink: Permission denied) 2024-07-27T03:18:57.706 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72631 osd_srv_h ceph txt unknown /proc/71097/task/72631/exe (readlink: Permission denied) 2024-07-27T03:18:57.706 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-osd 71097 72631 osd_srv_h ceph NOFD /proc/71097/task/72631/fd (opendir: Permission denied) 2024-07-27T03:18:57.707 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 72636 root cwd unknown /proc/72636/cwd (readlink: Permission denied) 2024-07-27T03:18:57.707 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 72636 root rtd unknown /proc/72636/root (readlink: Permission denied) 2024-07-27T03:18:57.707 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 72636 root txt unknown /proc/72636/exe (readlink: Permission denied) 2024-07-27T03:18:57.707 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 72636 root NOFD /proc/72636/fd (opendir: Permission denied) 2024-07-27T03:18:57.707 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 72653 root cwd unknown /proc/72653/cwd (readlink: Permission denied) 2024-07-27T03:18:57.707 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 72653 root rtd unknown /proc/72653/root (readlink: Permission denied) 2024-07-27T03:18:57.707 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 72653 root txt unknown /proc/72653/exe (readlink: Permission denied) 2024-07-27T03:18:57.707 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 72653 root NOFD /proc/72653/fd (opendir: Permission denied) 2024-07-27T03:18:57.707 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 75370 root cwd unknown /proc/75370/cwd (readlink: Permission denied) 2024-07-27T03:18:57.707 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 75370 root rtd unknown /proc/75370/root (readlink: Permission denied) 2024-07-27T03:18:57.707 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 75370 root txt unknown /proc/75370/exe (readlink: Permission denied) 2024-07-27T03:18:57.707 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 75370 root NOFD /proc/75370/fd (opendir: Permission denied) 2024-07-27T03:18:57.707 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 75371 root cwd unknown /proc/75371/cwd (readlink: Permission denied) 2024-07-27T03:18:57.707 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 75371 root rtd unknown /proc/75371/root (readlink: Permission denied) 2024-07-27T03:18:57.708 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 75371 root txt unknown /proc/75371/exe (readlink: Permission denied) 2024-07-27T03:18:57.708 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 75371 root NOFD /proc/75371/fd (opendir: Permission denied) 2024-07-27T03:18:57.708 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 76154 root cwd unknown /proc/76154/cwd (readlink: Permission denied) 2024-07-27T03:18:57.708 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 76154 root rtd unknown /proc/76154/root (readlink: Permission denied) 2024-07-27T03:18:57.708 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 76154 root txt unknown /proc/76154/exe (readlink: Permission denied) 2024-07-27T03:18:57.708 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 76154 root NOFD /proc/76154/fd (opendir: Permission denied) 2024-07-27T03:18:57.708 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 76263 root cwd unknown /proc/76263/cwd (readlink: Permission denied) 2024-07-27T03:18:57.708 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 76263 root rtd unknown /proc/76263/root (readlink: Permission denied) 2024-07-27T03:18:57.708 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 76263 root txt unknown /proc/76263/exe (readlink: Permission denied) 2024-07-27T03:18:57.708 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 76263 root NOFD /proc/76263/fd (opendir: Permission denied) 2024-07-27T03:18:57.708 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 76280 root cwd unknown /proc/76280/cwd (readlink: Permission denied) 2024-07-27T03:18:57.708 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 76280 root rtd unknown /proc/76280/root (readlink: Permission denied) 2024-07-27T03:18:57.708 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 76280 root txt unknown /proc/76280/exe (readlink: Permission denied) 2024-07-27T03:18:57.709 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 76280 root NOFD /proc/76280/fd (opendir: Permission denied) 2024-07-27T03:18:57.709 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 76281 root cwd unknown /proc/76281/cwd (readlink: Permission denied) 2024-07-27T03:18:57.709 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 76281 root rtd unknown /proc/76281/root (readlink: Permission denied) 2024-07-27T03:18:57.709 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 76281 root txt unknown /proc/76281/exe (readlink: Permission denied) 2024-07-27T03:18:57.709 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 76281 root NOFD /proc/76281/fd (opendir: Permission denied) 2024-07-27T03:18:57.709 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 76298 root cwd unknown /proc/76298/cwd (readlink: Permission denied) 2024-07-27T03:18:57.709 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 76298 root rtd unknown /proc/76298/root (readlink: Permission denied) 2024-07-27T03:18:57.709 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 76298 root txt unknown /proc/76298/exe (readlink: Permission denied) 2024-07-27T03:18:57.709 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 76298 root NOFD /proc/76298/fd (opendir: Permission denied) 2024-07-27T03:18:57.709 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 76299 root cwd unknown /proc/76299/cwd (readlink: Permission denied) 2024-07-27T03:18:57.709 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 76299 root rtd unknown /proc/76299/root (readlink: Permission denied) 2024-07-27T03:18:57.709 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 76299 root txt unknown /proc/76299/exe (readlink: Permission denied) 2024-07-27T03:18:57.709 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 76299 root NOFD /proc/76299/fd (opendir: Permission denied) 2024-07-27T03:18:57.709 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 76316 root cwd unknown /proc/76316/cwd (readlink: Permission denied) 2024-07-27T03:18:57.710 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 76316 root rtd unknown /proc/76316/root (readlink: Permission denied) 2024-07-27T03:18:57.710 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 76316 root txt unknown /proc/76316/exe (readlink: Permission denied) 2024-07-27T03:18:57.710 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 76316 root NOFD /proc/76316/fd (opendir: Permission denied) 2024-07-27T03:18:57.710 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 76317 root cwd unknown /proc/76317/cwd (readlink: Permission denied) 2024-07-27T03:18:57.710 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 76317 root rtd unknown /proc/76317/root (readlink: Permission denied) 2024-07-27T03:18:57.710 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 76317 root txt unknown /proc/76317/exe (readlink: Permission denied) 2024-07-27T03:18:57.710 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 76317 root NOFD /proc/76317/fd (opendir: Permission denied) 2024-07-27T03:18:57.710 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 76334 root cwd unknown /proc/76334/cwd (readlink: Permission denied) 2024-07-27T03:18:57.710 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 76334 root rtd unknown /proc/76334/root (readlink: Permission denied) 2024-07-27T03:18:57.710 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 76334 root txt unknown /proc/76334/exe (readlink: Permission denied) 2024-07-27T03:18:57.710 INFO:tasks.workunit.client.0.smithi067.stdout:journalct 76334 root NOFD /proc/76334/fd (opendir: Permission denied) 2024-07-27T03:18:57.710 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 76646 root cwd unknown /proc/76646/cwd (readlink: Permission denied) 2024-07-27T03:18:57.710 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 76646 root rtd unknown /proc/76646/root (readlink: Permission denied) 2024-07-27T03:18:57.711 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 76646 root txt unknown /proc/76646/exe (readlink: Permission denied) 2024-07-27T03:18:57.711 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 76646 root NOFD /proc/76646/fd (opendir: Permission denied) 2024-07-27T03:18:57.711 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 76886 root cwd unknown /proc/76886/cwd (readlink: Permission denied) 2024-07-27T03:18:57.711 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 76886 root rtd unknown /proc/76886/root (readlink: Permission denied) 2024-07-27T03:18:57.711 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 76886 root txt unknown /proc/76886/exe (readlink: Permission denied) 2024-07-27T03:18:57.711 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 76886 root NOFD /proc/76886/fd (opendir: Permission denied) 2024-07-27T03:18:57.711 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 76888 root cwd unknown /proc/76888/cwd (readlink: Permission denied) 2024-07-27T03:18:57.711 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 76888 root rtd unknown /proc/76888/root (readlink: Permission denied) 2024-07-27T03:18:57.711 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 76888 root txt unknown /proc/76888/exe (readlink: Permission denied) 2024-07-27T03:18:57.711 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 76888 root NOFD /proc/76888/fd (opendir: Permission denied) 2024-07-27T03:18:57.711 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 ceph cwd unknown /proc/76911/cwd (readlink: Permission denied) 2024-07-27T03:18:57.711 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 ceph rtd unknown /proc/76911/root (readlink: Permission denied) 2024-07-27T03:18:57.711 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 ceph txt unknown /proc/76911/exe (readlink: Permission denied) 2024-07-27T03:18:57.711 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 ceph NOFD /proc/76911/fd (opendir: Permission denied) 2024-07-27T03:18:57.712 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76912 log ceph cwd unknown /proc/76911/task/76912/cwd (readlink: Permission denied) 2024-07-27T03:18:57.712 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76912 log ceph rtd unknown /proc/76911/task/76912/root (readlink: Permission denied) 2024-07-27T03:18:57.712 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76912 log ceph txt unknown /proc/76911/task/76912/exe (readlink: Permission denied) 2024-07-27T03:18:57.712 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76912 log ceph NOFD /proc/76911/task/76912/fd (opendir: Permission denied) 2024-07-27T03:18:57.712 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76914 msgr-work ceph cwd unknown /proc/76911/task/76914/cwd (readlink: Permission denied) 2024-07-27T03:18:57.712 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76914 msgr-work ceph rtd unknown /proc/76911/task/76914/root (readlink: Permission denied) 2024-07-27T03:18:57.712 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76914 msgr-work ceph txt unknown /proc/76911/task/76914/exe (readlink: Permission denied) 2024-07-27T03:18:57.712 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76914 msgr-work ceph NOFD /proc/76911/task/76914/fd (opendir: Permission denied) 2024-07-27T03:18:57.712 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76915 msgr-work ceph cwd unknown /proc/76911/task/76915/cwd (readlink: Permission denied) 2024-07-27T03:18:57.712 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76915 msgr-work ceph rtd unknown /proc/76911/task/76915/root (readlink: Permission denied) 2024-07-27T03:18:57.712 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76915 msgr-work ceph txt unknown /proc/76911/task/76915/exe (readlink: Permission denied) 2024-07-27T03:18:57.712 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76915 msgr-work ceph NOFD /proc/76911/task/76915/fd (opendir: Permission denied) 2024-07-27T03:18:57.712 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76916 msgr-work ceph cwd unknown /proc/76911/task/76916/cwd (readlink: Permission denied) 2024-07-27T03:18:57.712 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76916 msgr-work ceph rtd unknown /proc/76911/task/76916/root (readlink: Permission denied) 2024-07-27T03:18:57.713 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76916 msgr-work ceph txt unknown /proc/76911/task/76916/exe (readlink: Permission denied) 2024-07-27T03:18:57.713 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76916 msgr-work ceph NOFD /proc/76911/task/76916/fd (opendir: Permission denied) 2024-07-27T03:18:57.713 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76920 service ceph cwd unknown /proc/76911/task/76920/cwd (readlink: Permission denied) 2024-07-27T03:18:57.713 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76920 service ceph rtd unknown /proc/76911/task/76920/root (readlink: Permission denied) 2024-07-27T03:18:57.713 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76920 service ceph txt unknown /proc/76911/task/76920/exe (readlink: Permission denied) 2024-07-27T03:18:57.713 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76920 service ceph NOFD /proc/76911/task/76920/fd (opendir: Permission denied) 2024-07-27T03:18:57.713 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76921 admin_soc ceph cwd unknown /proc/76911/task/76921/cwd (readlink: Permission denied) 2024-07-27T03:18:57.713 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76921 admin_soc ceph rtd unknown /proc/76911/task/76921/root (readlink: Permission denied) 2024-07-27T03:18:57.713 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76921 admin_soc ceph txt unknown /proc/76911/task/76921/exe (readlink: Permission denied) 2024-07-27T03:18:57.713 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76921 admin_soc ceph NOFD /proc/76911/task/76921/fd (opendir: Permission denied) 2024-07-27T03:18:57.713 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76922 signal_ha ceph cwd unknown /proc/76911/task/76922/cwd (readlink: Permission denied) 2024-07-27T03:18:57.713 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76922 signal_ha ceph rtd unknown /proc/76911/task/76922/root (readlink: Permission denied) 2024-07-27T03:18:57.713 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76922 signal_ha ceph txt unknown /proc/76911/task/76922/exe (readlink: Permission denied) 2024-07-27T03:18:57.713 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76922 signal_ha ceph NOFD /proc/76911/task/76922/fd (opendir: Permission denied) 2024-07-27T03:18:57.714 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76923 io_contex ceph cwd unknown /proc/76911/task/76923/cwd (readlink: Permission denied) 2024-07-27T03:18:57.714 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76923 io_contex ceph rtd unknown /proc/76911/task/76923/root (readlink: Permission denied) 2024-07-27T03:18:57.714 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76923 io_contex ceph txt unknown /proc/76911/task/76923/exe (readlink: Permission denied) 2024-07-27T03:18:57.714 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76923 io_contex ceph NOFD /proc/76911/task/76923/fd (opendir: Permission denied) 2024-07-27T03:18:57.714 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76924 io_contex ceph cwd unknown /proc/76911/task/76924/cwd (readlink: Permission denied) 2024-07-27T03:18:57.714 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76924 io_contex ceph rtd unknown /proc/76911/task/76924/root (readlink: Permission denied) 2024-07-27T03:18:57.714 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76924 io_contex ceph txt unknown /proc/76911/task/76924/exe (readlink: Permission denied) 2024-07-27T03:18:57.714 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76924 io_contex ceph NOFD /proc/76911/task/76924/fd (opendir: Permission denied) 2024-07-27T03:18:57.714 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76925 ms_dispat ceph cwd unknown /proc/76911/task/76925/cwd (readlink: Permission denied) 2024-07-27T03:18:57.714 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76925 ms_dispat ceph rtd unknown /proc/76911/task/76925/root (readlink: Permission denied) 2024-07-27T03:18:57.714 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76925 ms_dispat ceph txt unknown /proc/76911/task/76925/exe (readlink: Permission denied) 2024-07-27T03:18:57.714 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76925 ms_dispat ceph NOFD /proc/76911/task/76925/fd (opendir: Permission denied) 2024-07-27T03:18:57.714 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76926 ms_local ceph cwd unknown /proc/76911/task/76926/cwd (readlink: Permission denied) 2024-07-27T03:18:57.715 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76926 ms_local ceph rtd unknown /proc/76911/task/76926/root (readlink: Permission denied) 2024-07-27T03:18:57.715 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76926 ms_local ceph txt unknown /proc/76911/task/76926/exe (readlink: Permission denied) 2024-07-27T03:18:57.715 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76926 ms_local ceph NOFD /proc/76911/task/76926/fd (opendir: Permission denied) 2024-07-27T03:18:57.715 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76927 safe_time ceph cwd unknown /proc/76911/task/76927/cwd (readlink: Permission denied) 2024-07-27T03:18:57.715 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76927 safe_time ceph rtd unknown /proc/76911/task/76927/root (readlink: Permission denied) 2024-07-27T03:18:57.715 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76927 safe_time ceph txt unknown /proc/76911/task/76927/exe (readlink: Permission denied) 2024-07-27T03:18:57.715 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76927 safe_time ceph NOFD /proc/76911/task/76927/fd (opendir: Permission denied) 2024-07-27T03:18:57.715 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76928 safe_time ceph cwd unknown /proc/76911/task/76928/cwd (readlink: Permission denied) 2024-07-27T03:18:57.715 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76928 safe_time ceph rtd unknown /proc/76911/task/76928/root (readlink: Permission denied) 2024-07-27T03:18:57.715 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76928 safe_time ceph txt unknown /proc/76911/task/76928/exe (readlink: Permission denied) 2024-07-27T03:18:57.715 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76928 safe_time ceph NOFD /proc/76911/task/76928/fd (opendir: Permission denied) 2024-07-27T03:18:57.715 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76929 ceph-mds ceph cwd unknown /proc/76911/task/76929/cwd (readlink: Permission denied) 2024-07-27T03:18:57.715 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76929 ceph-mds ceph rtd unknown /proc/76911/task/76929/root (readlink: Permission denied) 2024-07-27T03:18:57.715 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76929 ceph-mds ceph txt unknown /proc/76911/task/76929/exe (readlink: Permission denied) 2024-07-27T03:18:57.716 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76929 ceph-mds ceph NOFD /proc/76911/task/76929/fd (opendir: Permission denied) 2024-07-27T03:18:57.716 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76954 safe_time ceph cwd unknown /proc/76911/task/76954/cwd (readlink: Permission denied) 2024-07-27T03:18:57.716 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76954 safe_time ceph rtd unknown /proc/76911/task/76954/root (readlink: Permission denied) 2024-07-27T03:18:57.716 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76954 safe_time ceph txt unknown /proc/76911/task/76954/exe (readlink: Permission denied) 2024-07-27T03:18:57.716 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 76911 76954 safe_time ceph NOFD /proc/76911/task/76954/fd (opendir: Permission denied) 2024-07-27T03:18:57.716 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 77071 root cwd unknown /proc/77071/cwd (readlink: Permission denied) 2024-07-27T03:18:57.716 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 77071 root rtd unknown /proc/77071/root (readlink: Permission denied) 2024-07-27T03:18:57.716 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 77071 root txt unknown /proc/77071/exe (readlink: Permission denied) 2024-07-27T03:18:57.716 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 77071 root NOFD /proc/77071/fd (opendir: Permission denied) 2024-07-27T03:18:57.716 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 77248 root cwd unknown /proc/77248/cwd (readlink: Permission denied) 2024-07-27T03:18:57.716 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 77248 root rtd unknown /proc/77248/root (readlink: Permission denied) 2024-07-27T03:18:57.716 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 77248 root txt unknown /proc/77248/exe (readlink: Permission denied) 2024-07-27T03:18:57.716 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 77248 root NOFD /proc/77248/fd (opendir: Permission denied) 2024-07-27T03:18:57.717 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 77250 root cwd unknown /proc/77250/cwd (readlink: Permission denied) 2024-07-27T03:18:57.717 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 77250 root rtd unknown /proc/77250/root (readlink: Permission denied) 2024-07-27T03:18:57.717 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 77250 root txt unknown /proc/77250/exe (readlink: Permission denied) 2024-07-27T03:18:57.717 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 77250 root NOFD /proc/77250/fd (opendir: Permission denied) 2024-07-27T03:18:57.717 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 ceph cwd unknown /proc/77252/cwd (readlink: Permission denied) 2024-07-27T03:18:57.717 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 ceph rtd unknown /proc/77252/root (readlink: Permission denied) 2024-07-27T03:18:57.717 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 ceph txt unknown /proc/77252/exe (readlink: Permission denied) 2024-07-27T03:18:57.717 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 ceph NOFD /proc/77252/fd (opendir: Permission denied) 2024-07-27T03:18:57.717 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77253 log ceph cwd unknown /proc/77252/task/77253/cwd (readlink: Permission denied) 2024-07-27T03:18:57.717 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77253 log ceph rtd unknown /proc/77252/task/77253/root (readlink: Permission denied) 2024-07-27T03:18:57.717 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77253 log ceph txt unknown /proc/77252/task/77253/exe (readlink: Permission denied) 2024-07-27T03:18:57.717 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77253 log ceph NOFD /proc/77252/task/77253/fd (opendir: Permission denied) 2024-07-27T03:18:57.717 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77256 msgr-work ceph cwd unknown /proc/77252/task/77256/cwd (readlink: Permission denied) 2024-07-27T03:18:57.717 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77256 msgr-work ceph rtd unknown /proc/77252/task/77256/root (readlink: Permission denied) 2024-07-27T03:18:57.718 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77256 msgr-work ceph txt unknown /proc/77252/task/77256/exe (readlink: Permission denied) 2024-07-27T03:18:57.718 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77256 msgr-work ceph NOFD /proc/77252/task/77256/fd (opendir: Permission denied) 2024-07-27T03:18:57.718 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77257 msgr-work ceph cwd unknown /proc/77252/task/77257/cwd (readlink: Permission denied) 2024-07-27T03:18:57.718 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77257 msgr-work ceph rtd unknown /proc/77252/task/77257/root (readlink: Permission denied) 2024-07-27T03:18:57.718 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77257 msgr-work ceph txt unknown /proc/77252/task/77257/exe (readlink: Permission denied) 2024-07-27T03:18:57.718 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77257 msgr-work ceph NOFD /proc/77252/task/77257/fd (opendir: Permission denied) 2024-07-27T03:18:57.718 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77258 msgr-work ceph cwd unknown /proc/77252/task/77258/cwd (readlink: Permission denied) 2024-07-27T03:18:57.718 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77258 msgr-work ceph rtd unknown /proc/77252/task/77258/root (readlink: Permission denied) 2024-07-27T03:18:57.718 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77258 msgr-work ceph txt unknown /proc/77252/task/77258/exe (readlink: Permission denied) 2024-07-27T03:18:57.718 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77258 msgr-work ceph NOFD /proc/77252/task/77258/fd (opendir: Permission denied) 2024-07-27T03:18:57.718 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77263 service ceph cwd unknown /proc/77252/task/77263/cwd (readlink: Permission denied) 2024-07-27T03:18:57.718 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77263 service ceph rtd unknown /proc/77252/task/77263/root (readlink: Permission denied) 2024-07-27T03:18:57.718 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77263 service ceph txt unknown /proc/77252/task/77263/exe (readlink: Permission denied) 2024-07-27T03:18:57.719 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77263 service ceph NOFD /proc/77252/task/77263/fd (opendir: Permission denied) 2024-07-27T03:18:57.719 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77264 admin_soc ceph cwd unknown /proc/77252/task/77264/cwd (readlink: Permission denied) 2024-07-27T03:18:57.719 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77264 admin_soc ceph rtd unknown /proc/77252/task/77264/root (readlink: Permission denied) 2024-07-27T03:18:57.719 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77264 admin_soc ceph txt unknown /proc/77252/task/77264/exe (readlink: Permission denied) 2024-07-27T03:18:57.719 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77264 admin_soc ceph NOFD /proc/77252/task/77264/fd (opendir: Permission denied) 2024-07-27T03:18:57.719 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77269 signal_ha ceph cwd unknown /proc/77252/task/77269/cwd (readlink: Permission denied) 2024-07-27T03:18:57.719 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77269 signal_ha ceph rtd unknown /proc/77252/task/77269/root (readlink: Permission denied) 2024-07-27T03:18:57.719 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77269 signal_ha ceph txt unknown /proc/77252/task/77269/exe (readlink: Permission denied) 2024-07-27T03:18:57.719 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77269 signal_ha ceph NOFD /proc/77252/task/77269/fd (opendir: Permission denied) 2024-07-27T03:18:57.719 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77271 io_contex ceph cwd unknown /proc/77252/task/77271/cwd (readlink: Permission denied) 2024-07-27T03:18:57.719 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77271 io_contex ceph rtd unknown /proc/77252/task/77271/root (readlink: Permission denied) 2024-07-27T03:18:57.719 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77271 io_contex ceph txt unknown /proc/77252/task/77271/exe (readlink: Permission denied) 2024-07-27T03:18:57.719 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77271 io_contex ceph NOFD /proc/77252/task/77271/fd (opendir: Permission denied) 2024-07-27T03:18:57.719 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77272 io_contex ceph cwd unknown /proc/77252/task/77272/cwd (readlink: Permission denied) 2024-07-27T03:18:57.720 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77272 io_contex ceph rtd unknown /proc/77252/task/77272/root (readlink: Permission denied) 2024-07-27T03:18:57.720 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77272 io_contex ceph txt unknown /proc/77252/task/77272/exe (readlink: Permission denied) 2024-07-27T03:18:57.720 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77272 io_contex ceph NOFD /proc/77252/task/77272/fd (opendir: Permission denied) 2024-07-27T03:18:57.720 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77273 ms_dispat ceph cwd unknown /proc/77252/task/77273/cwd (readlink: Permission denied) 2024-07-27T03:18:57.720 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77273 ms_dispat ceph rtd unknown /proc/77252/task/77273/root (readlink: Permission denied) 2024-07-27T03:18:57.720 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77273 ms_dispat ceph txt unknown /proc/77252/task/77273/exe (readlink: Permission denied) 2024-07-27T03:18:57.720 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77273 ms_dispat ceph NOFD /proc/77252/task/77273/fd (opendir: Permission denied) 2024-07-27T03:18:57.720 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77274 ms_local ceph cwd unknown /proc/77252/task/77274/cwd (readlink: Permission denied) 2024-07-27T03:18:57.720 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77274 ms_local ceph rtd unknown /proc/77252/task/77274/root (readlink: Permission denied) 2024-07-27T03:18:57.720 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77274 ms_local ceph txt unknown /proc/77252/task/77274/exe (readlink: Permission denied) 2024-07-27T03:18:57.720 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77274 ms_local ceph NOFD /proc/77252/task/77274/fd (opendir: Permission denied) 2024-07-27T03:18:57.720 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77275 safe_time ceph cwd unknown /proc/77252/task/77275/cwd (readlink: Permission denied) 2024-07-27T03:18:57.720 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77275 safe_time ceph rtd unknown /proc/77252/task/77275/root (readlink: Permission denied) 2024-07-27T03:18:57.721 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77275 safe_time ceph txt unknown /proc/77252/task/77275/exe (readlink: Permission denied) 2024-07-27T03:18:57.721 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77275 safe_time ceph NOFD /proc/77252/task/77275/fd (opendir: Permission denied) 2024-07-27T03:18:57.721 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77276 safe_time ceph cwd unknown /proc/77252/task/77276/cwd (readlink: Permission denied) 2024-07-27T03:18:57.721 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77276 safe_time ceph rtd unknown /proc/77252/task/77276/root (readlink: Permission denied) 2024-07-27T03:18:57.721 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77276 safe_time ceph txt unknown /proc/77252/task/77276/exe (readlink: Permission denied) 2024-07-27T03:18:57.721 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77276 safe_time ceph NOFD /proc/77252/task/77276/fd (opendir: Permission denied) 2024-07-27T03:18:57.721 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77277 ceph-mds ceph cwd unknown /proc/77252/task/77277/cwd (readlink: Permission denied) 2024-07-27T03:18:57.721 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77277 ceph-mds ceph rtd unknown /proc/77252/task/77277/root (readlink: Permission denied) 2024-07-27T03:18:57.721 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77277 ceph-mds ceph txt unknown /proc/77252/task/77277/exe (readlink: Permission denied) 2024-07-27T03:18:57.721 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77277 ceph-mds ceph NOFD /proc/77252/task/77277/fd (opendir: Permission denied) 2024-07-27T03:18:57.721 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77329 safe_time ceph cwd unknown /proc/77252/task/77329/cwd (readlink: Permission denied) 2024-07-27T03:18:57.721 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77329 safe_time ceph rtd unknown /proc/77252/task/77329/root (readlink: Permission denied) 2024-07-27T03:18:57.721 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77329 safe_time ceph txt unknown /proc/77252/task/77329/exe (readlink: Permission denied) 2024-07-27T03:18:57.722 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 77252 77329 safe_time ceph NOFD /proc/77252/task/77329/fd (opendir: Permission denied) 2024-07-27T03:18:57.722 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 78048 root cwd unknown /proc/78048/cwd (readlink: Permission denied) 2024-07-27T03:18:57.722 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 78048 root rtd unknown /proc/78048/root (readlink: Permission denied) 2024-07-27T03:18:57.722 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 78048 root txt unknown /proc/78048/exe (readlink: Permission denied) 2024-07-27T03:18:57.722 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 78048 root NOFD /proc/78048/fd (opendir: Permission denied) 2024-07-27T03:18:57.722 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 78050 root cwd unknown /proc/78050/cwd (readlink: Permission denied) 2024-07-27T03:18:57.722 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 78050 root rtd unknown /proc/78050/root (readlink: Permission denied) 2024-07-27T03:18:57.722 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 78050 root txt unknown /proc/78050/exe (readlink: Permission denied) 2024-07-27T03:18:57.722 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 78050 root NOFD /proc/78050/fd (opendir: Permission denied) 2024-07-27T03:18:57.722 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 ceph cwd unknown /proc/78058/cwd (readlink: Permission denied) 2024-07-27T03:18:57.722 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 ceph rtd unknown /proc/78058/root (readlink: Permission denied) 2024-07-27T03:18:57.722 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 ceph txt unknown /proc/78058/exe (readlink: Permission denied) 2024-07-27T03:18:57.722 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 ceph NOFD /proc/78058/fd (opendir: Permission denied) 2024-07-27T03:18:57.723 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78059 log ceph cwd unknown /proc/78058/task/78059/cwd (readlink: Permission denied) 2024-07-27T03:18:57.723 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78059 log ceph rtd unknown /proc/78058/task/78059/root (readlink: Permission denied) 2024-07-27T03:18:57.723 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78059 log ceph txt unknown /proc/78058/task/78059/exe (readlink: Permission denied) 2024-07-27T03:18:57.723 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78059 log ceph NOFD /proc/78058/task/78059/fd (opendir: Permission denied) 2024-07-27T03:18:57.723 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78061 msgr-work ceph cwd unknown /proc/78058/task/78061/cwd (readlink: Permission denied) 2024-07-27T03:18:57.723 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78061 msgr-work ceph rtd unknown /proc/78058/task/78061/root (readlink: Permission denied) 2024-07-27T03:18:57.723 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78061 msgr-work ceph txt unknown /proc/78058/task/78061/exe (readlink: Permission denied) 2024-07-27T03:18:57.723 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78061 msgr-work ceph NOFD /proc/78058/task/78061/fd (opendir: Permission denied) 2024-07-27T03:18:57.723 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78062 msgr-work ceph cwd unknown /proc/78058/task/78062/cwd (readlink: Permission denied) 2024-07-27T03:18:57.723 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78062 msgr-work ceph rtd unknown /proc/78058/task/78062/root (readlink: Permission denied) 2024-07-27T03:18:57.723 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78062 msgr-work ceph txt unknown /proc/78058/task/78062/exe (readlink: Permission denied) 2024-07-27T03:18:57.723 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78062 msgr-work ceph NOFD /proc/78058/task/78062/fd (opendir: Permission denied) 2024-07-27T03:18:57.723 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78063 msgr-work ceph cwd unknown /proc/78058/task/78063/cwd (readlink: Permission denied) 2024-07-27T03:18:57.723 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78063 msgr-work ceph rtd unknown /proc/78058/task/78063/root (readlink: Permission denied) 2024-07-27T03:18:57.724 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78063 msgr-work ceph txt unknown /proc/78058/task/78063/exe (readlink: Permission denied) 2024-07-27T03:18:57.724 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78063 msgr-work ceph NOFD /proc/78058/task/78063/fd (opendir: Permission denied) 2024-07-27T03:18:57.724 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78067 service ceph cwd unknown /proc/78058/task/78067/cwd (readlink: Permission denied) 2024-07-27T03:18:57.724 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78067 service ceph rtd unknown /proc/78058/task/78067/root (readlink: Permission denied) 2024-07-27T03:18:57.724 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78067 service ceph txt unknown /proc/78058/task/78067/exe (readlink: Permission denied) 2024-07-27T03:18:57.724 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78067 service ceph NOFD /proc/78058/task/78067/fd (opendir: Permission denied) 2024-07-27T03:18:57.724 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78068 admin_soc ceph cwd unknown /proc/78058/task/78068/cwd (readlink: Permission denied) 2024-07-27T03:18:57.724 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78068 admin_soc ceph rtd unknown /proc/78058/task/78068/root (readlink: Permission denied) 2024-07-27T03:18:57.724 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78068 admin_soc ceph txt unknown /proc/78058/task/78068/exe (readlink: Permission denied) 2024-07-27T03:18:57.724 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78068 admin_soc ceph NOFD /proc/78058/task/78068/fd (opendir: Permission denied) 2024-07-27T03:18:57.724 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78069 signal_ha ceph cwd unknown /proc/78058/task/78069/cwd (readlink: Permission denied) 2024-07-27T03:18:57.724 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78069 signal_ha ceph rtd unknown /proc/78058/task/78069/root (readlink: Permission denied) 2024-07-27T03:18:57.724 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78069 signal_ha ceph txt unknown /proc/78058/task/78069/exe (readlink: Permission denied) 2024-07-27T03:18:57.725 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78069 signal_ha ceph NOFD /proc/78058/task/78069/fd (opendir: Permission denied) 2024-07-27T03:18:57.725 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78070 io_contex ceph cwd unknown /proc/78058/task/78070/cwd (readlink: Permission denied) 2024-07-27T03:18:57.725 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78070 io_contex ceph rtd unknown /proc/78058/task/78070/root (readlink: Permission denied) 2024-07-27T03:18:57.725 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78070 io_contex ceph txt unknown /proc/78058/task/78070/exe (readlink: Permission denied) 2024-07-27T03:18:57.725 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78070 io_contex ceph NOFD /proc/78058/task/78070/fd (opendir: Permission denied) 2024-07-27T03:18:57.725 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78071 io_contex ceph cwd unknown /proc/78058/task/78071/cwd (readlink: Permission denied) 2024-07-27T03:18:57.725 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78071 io_contex ceph rtd unknown /proc/78058/task/78071/root (readlink: Permission denied) 2024-07-27T03:18:57.725 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78071 io_contex ceph txt unknown /proc/78058/task/78071/exe (readlink: Permission denied) 2024-07-27T03:18:57.725 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78071 io_contex ceph NOFD /proc/78058/task/78071/fd (opendir: Permission denied) 2024-07-27T03:18:57.725 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78072 ms_dispat ceph cwd unknown /proc/78058/task/78072/cwd (readlink: Permission denied) 2024-07-27T03:18:57.725 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78072 ms_dispat ceph rtd unknown /proc/78058/task/78072/root (readlink: Permission denied) 2024-07-27T03:18:57.725 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78072 ms_dispat ceph txt unknown /proc/78058/task/78072/exe (readlink: Permission denied) 2024-07-27T03:18:57.725 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78072 ms_dispat ceph NOFD /proc/78058/task/78072/fd (opendir: Permission denied) 2024-07-27T03:18:57.725 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78073 ms_local ceph cwd unknown /proc/78058/task/78073/cwd (readlink: Permission denied) 2024-07-27T03:18:57.726 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78073 ms_local ceph rtd unknown /proc/78058/task/78073/root (readlink: Permission denied) 2024-07-27T03:18:57.726 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78073 ms_local ceph txt unknown /proc/78058/task/78073/exe (readlink: Permission denied) 2024-07-27T03:18:57.726 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78073 ms_local ceph NOFD /proc/78058/task/78073/fd (opendir: Permission denied) 2024-07-27T03:18:57.726 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78074 safe_time ceph cwd unknown /proc/78058/task/78074/cwd (readlink: Permission denied) 2024-07-27T03:18:57.726 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78074 safe_time ceph rtd unknown /proc/78058/task/78074/root (readlink: Permission denied) 2024-07-27T03:18:57.726 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78074 safe_time ceph txt unknown /proc/78058/task/78074/exe (readlink: Permission denied) 2024-07-27T03:18:57.726 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78074 safe_time ceph NOFD /proc/78058/task/78074/fd (opendir: Permission denied) 2024-07-27T03:18:57.726 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78075 safe_time ceph cwd unknown /proc/78058/task/78075/cwd (readlink: Permission denied) 2024-07-27T03:18:57.726 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78075 safe_time ceph rtd unknown /proc/78058/task/78075/root (readlink: Permission denied) 2024-07-27T03:18:57.726 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78075 safe_time ceph txt unknown /proc/78058/task/78075/exe (readlink: Permission denied) 2024-07-27T03:18:57.726 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78075 safe_time ceph NOFD /proc/78058/task/78075/fd (opendir: Permission denied) 2024-07-27T03:18:57.726 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78076 ceph-mds ceph cwd unknown /proc/78058/task/78076/cwd (readlink: Permission denied) 2024-07-27T03:18:57.727 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78076 ceph-mds ceph rtd unknown /proc/78058/task/78076/root (readlink: Permission denied) 2024-07-27T03:18:57.727 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78076 ceph-mds ceph txt unknown /proc/78058/task/78076/exe (readlink: Permission denied) 2024-07-27T03:18:57.727 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78076 ceph-mds ceph NOFD /proc/78058/task/78076/fd (opendir: Permission denied) 2024-07-27T03:18:57.727 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78223 safe_time ceph cwd unknown /proc/78058/task/78223/cwd (readlink: Permission denied) 2024-07-27T03:18:57.727 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78223 safe_time ceph rtd unknown /proc/78058/task/78223/root (readlink: Permission denied) 2024-07-27T03:18:57.727 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78223 safe_time ceph txt unknown /proc/78058/task/78223/exe (readlink: Permission denied) 2024-07-27T03:18:57.727 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 78223 safe_time ceph NOFD /proc/78058/task/78223/fd (opendir: Permission denied) 2024-07-27T03:18:57.727 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84128 ceph_time ceph cwd unknown /proc/78058/task/84128/cwd (readlink: Permission denied) 2024-07-27T03:18:57.727 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84128 ceph_time ceph rtd unknown /proc/78058/task/84128/root (readlink: Permission denied) 2024-07-27T03:18:57.727 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84128 ceph_time ceph txt unknown /proc/78058/task/84128/exe (readlink: Permission denied) 2024-07-27T03:18:57.727 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84128 ceph_time ceph NOFD /proc/78058/task/84128/fd (opendir: Permission denied) 2024-07-27T03:18:57.727 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84129 OpHistory ceph cwd unknown /proc/78058/task/84129/cwd (readlink: Permission denied) 2024-07-27T03:18:57.727 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84129 OpHistory ceph rtd unknown /proc/78058/task/84129/root (readlink: Permission denied) 2024-07-27T03:18:57.727 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84129 OpHistory ceph txt unknown /proc/78058/task/84129/exe (readlink: Permission denied) 2024-07-27T03:18:57.728 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84129 OpHistory ceph NOFD /proc/78058/task/84129/fd (opendir: Permission denied) 2024-07-27T03:18:57.728 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84131 ms_dispat ceph cwd unknown /proc/78058/task/84131/cwd (readlink: Permission denied) 2024-07-27T03:18:57.728 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84131 ms_dispat ceph rtd unknown /proc/78058/task/84131/root (readlink: Permission denied) 2024-07-27T03:18:57.728 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84131 ms_dispat ceph txt unknown /proc/78058/task/84131/exe (readlink: Permission denied) 2024-07-27T03:18:57.728 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84131 ms_dispat ceph NOFD /proc/78058/task/84131/fd (opendir: Permission denied) 2024-07-27T03:18:57.728 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84133 ms_dispat ceph cwd unknown /proc/78058/task/84133/cwd (readlink: Permission denied) 2024-07-27T03:18:57.728 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84133 ms_dispat ceph rtd unknown /proc/78058/task/84133/root (readlink: Permission denied) 2024-07-27T03:18:57.728 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84133 ms_dispat ceph txt unknown /proc/78058/task/84133/exe (readlink: Permission denied) 2024-07-27T03:18:57.728 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84133 ms_dispat ceph NOFD /proc/78058/task/84133/fd (opendir: Permission denied) 2024-07-27T03:18:57.728 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84135 mds_rank_ ceph cwd unknown /proc/78058/task/84135/cwd (readlink: Permission denied) 2024-07-27T03:18:57.728 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84135 mds_rank_ ceph rtd unknown /proc/78058/task/84135/root (readlink: Permission denied) 2024-07-27T03:18:57.728 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84135 mds_rank_ ceph txt unknown /proc/78058/task/84135/exe (readlink: Permission denied) 2024-07-27T03:18:57.728 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84135 mds_rank_ ceph NOFD /proc/78058/task/84135/fd (opendir: Permission denied) 2024-07-27T03:18:57.728 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84137 PQ_Finish ceph cwd unknown /proc/78058/task/84137/cwd (readlink: Permission denied) 2024-07-27T03:18:57.729 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84137 PQ_Finish ceph rtd unknown /proc/78058/task/84137/root (readlink: Permission denied) 2024-07-27T03:18:57.729 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84137 PQ_Finish ceph txt unknown /proc/78058/task/84137/exe (readlink: Permission denied) 2024-07-27T03:18:57.729 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84137 PQ_Finish ceph NOFD /proc/78058/task/84137/fd (opendir: Permission denied) 2024-07-27T03:18:57.729 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84139 safe_time ceph cwd unknown /proc/78058/task/84139/cwd (readlink: Permission denied) 2024-07-27T03:18:57.729 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84139 safe_time ceph rtd unknown /proc/78058/task/84139/root (readlink: Permission denied) 2024-07-27T03:18:57.729 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84139 safe_time ceph txt unknown /proc/78058/task/84139/exe (readlink: Permission denied) 2024-07-27T03:18:57.729 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84139 safe_time ceph NOFD /proc/78058/task/84139/fd (opendir: Permission denied) 2024-07-27T03:18:57.729 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84141 MR_Finish ceph cwd unknown /proc/78058/task/84141/cwd (readlink: Permission denied) 2024-07-27T03:18:57.729 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84141 MR_Finish ceph rtd unknown /proc/78058/task/84141/root (readlink: Permission denied) 2024-07-27T03:18:57.729 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84141 MR_Finish ceph txt unknown /proc/78058/task/84141/exe (readlink: Permission denied) 2024-07-27T03:18:57.729 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84141 MR_Finish ceph NOFD /proc/78058/task/84141/fd (opendir: Permission denied) 2024-07-27T03:18:57.729 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84146 md_submit ceph cwd unknown /proc/78058/task/84146/cwd (readlink: Permission denied) 2024-07-27T03:18:57.729 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84146 md_submit ceph rtd unknown /proc/78058/task/84146/root (readlink: Permission denied) 2024-07-27T03:18:57.729 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84146 md_submit ceph txt unknown /proc/78058/task/84146/exe (readlink: Permission denied) 2024-07-27T03:18:57.730 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78058 84146 md_submit ceph NOFD /proc/78058/task/84146/fd (opendir: Permission denied) 2024-07-27T03:18:57.730 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 78470 root cwd unknown /proc/78470/cwd (readlink: Permission denied) 2024-07-27T03:18:57.730 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 78470 root rtd unknown /proc/78470/root (readlink: Permission denied) 2024-07-27T03:18:57.730 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 78470 root txt unknown /proc/78470/exe (readlink: Permission denied) 2024-07-27T03:18:57.730 INFO:tasks.workunit.client.0.smithi067.stdout:conmon 78470 root NOFD /proc/78470/fd (opendir: Permission denied) 2024-07-27T03:18:57.730 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 78472 root cwd unknown /proc/78472/cwd (readlink: Permission denied) 2024-07-27T03:18:57.730 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 78472 root rtd unknown /proc/78472/root (readlink: Permission denied) 2024-07-27T03:18:57.730 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 78472 root txt unknown /proc/78472/exe (readlink: Permission denied) 2024-07-27T03:18:57.730 INFO:tasks.workunit.client.0.smithi067.stdout:podman-in 78472 root NOFD /proc/78472/fd (opendir: Permission denied) 2024-07-27T03:18:57.730 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 ceph cwd unknown /proc/78474/cwd (readlink: Permission denied) 2024-07-27T03:18:57.730 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 ceph rtd unknown /proc/78474/root (readlink: Permission denied) 2024-07-27T03:18:57.730 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 ceph txt unknown /proc/78474/exe (readlink: Permission denied) 2024-07-27T03:18:57.730 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 ceph NOFD /proc/78474/fd (opendir: Permission denied) 2024-07-27T03:18:57.731 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78475 log ceph cwd unknown /proc/78474/task/78475/cwd (readlink: Permission denied) 2024-07-27T03:18:57.731 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78475 log ceph rtd unknown /proc/78474/task/78475/root (readlink: Permission denied) 2024-07-27T03:18:57.731 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78475 log ceph txt unknown /proc/78474/task/78475/exe (readlink: Permission denied) 2024-07-27T03:18:57.731 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78475 log ceph NOFD /proc/78474/task/78475/fd (opendir: Permission denied) 2024-07-27T03:18:57.731 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78477 msgr-work ceph cwd unknown /proc/78474/task/78477/cwd (readlink: Permission denied) 2024-07-27T03:18:57.731 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78477 msgr-work ceph rtd unknown /proc/78474/task/78477/root (readlink: Permission denied) 2024-07-27T03:18:57.731 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78477 msgr-work ceph txt unknown /proc/78474/task/78477/exe (readlink: Permission denied) 2024-07-27T03:18:57.731 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78477 msgr-work ceph NOFD /proc/78474/task/78477/fd (opendir: Permission denied) 2024-07-27T03:18:57.731 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78478 msgr-work ceph cwd unknown /proc/78474/task/78478/cwd (readlink: Permission denied) 2024-07-27T03:18:57.731 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78478 msgr-work ceph rtd unknown /proc/78474/task/78478/root (readlink: Permission denied) 2024-07-27T03:18:57.731 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78478 msgr-work ceph txt unknown /proc/78474/task/78478/exe (readlink: Permission denied) 2024-07-27T03:18:57.731 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78478 msgr-work ceph NOFD /proc/78474/task/78478/fd (opendir: Permission denied) 2024-07-27T03:18:57.731 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78479 msgr-work ceph cwd unknown /proc/78474/task/78479/cwd (readlink: Permission denied) 2024-07-27T03:18:57.731 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78479 msgr-work ceph rtd unknown /proc/78474/task/78479/root (readlink: Permission denied) 2024-07-27T03:18:57.732 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78479 msgr-work ceph txt unknown /proc/78474/task/78479/exe (readlink: Permission denied) 2024-07-27T03:18:57.732 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78479 msgr-work ceph NOFD /proc/78474/task/78479/fd (opendir: Permission denied) 2024-07-27T03:18:57.732 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78483 service ceph cwd unknown /proc/78474/task/78483/cwd (readlink: Permission denied) 2024-07-27T03:18:57.732 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78483 service ceph rtd unknown /proc/78474/task/78483/root (readlink: Permission denied) 2024-07-27T03:18:57.732 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78483 service ceph txt unknown /proc/78474/task/78483/exe (readlink: Permission denied) 2024-07-27T03:18:57.732 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78483 service ceph NOFD /proc/78474/task/78483/fd (opendir: Permission denied) 2024-07-27T03:18:57.732 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78484 admin_soc ceph cwd unknown /proc/78474/task/78484/cwd (readlink: Permission denied) 2024-07-27T03:18:57.732 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78484 admin_soc ceph rtd unknown /proc/78474/task/78484/root (readlink: Permission denied) 2024-07-27T03:18:57.732 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78484 admin_soc ceph txt unknown /proc/78474/task/78484/exe (readlink: Permission denied) 2024-07-27T03:18:57.732 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78484 admin_soc ceph NOFD /proc/78474/task/78484/fd (opendir: Permission denied) 2024-07-27T03:18:57.732 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78485 signal_ha ceph cwd unknown /proc/78474/task/78485/cwd (readlink: Permission denied) 2024-07-27T03:18:57.732 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78485 signal_ha ceph rtd unknown /proc/78474/task/78485/root (readlink: Permission denied) 2024-07-27T03:18:57.732 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78485 signal_ha ceph txt unknown /proc/78474/task/78485/exe (readlink: Permission denied) 2024-07-27T03:18:57.732 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78485 signal_ha ceph NOFD /proc/78474/task/78485/fd (opendir: Permission denied) 2024-07-27T03:18:57.733 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78486 io_contex ceph cwd unknown /proc/78474/task/78486/cwd (readlink: Permission denied) 2024-07-27T03:18:57.733 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78486 io_contex ceph rtd unknown /proc/78474/task/78486/root (readlink: Permission denied) 2024-07-27T03:18:57.733 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78486 io_contex ceph txt unknown /proc/78474/task/78486/exe (readlink: Permission denied) 2024-07-27T03:18:57.733 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78486 io_contex ceph NOFD /proc/78474/task/78486/fd (opendir: Permission denied) 2024-07-27T03:18:57.733 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78487 io_contex ceph cwd unknown /proc/78474/task/78487/cwd (readlink: Permission denied) 2024-07-27T03:18:57.733 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78487 io_contex ceph rtd unknown /proc/78474/task/78487/root (readlink: Permission denied) 2024-07-27T03:18:57.733 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78487 io_contex ceph txt unknown /proc/78474/task/78487/exe (readlink: Permission denied) 2024-07-27T03:18:57.733 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78487 io_contex ceph NOFD /proc/78474/task/78487/fd (opendir: Permission denied) 2024-07-27T03:18:57.733 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78488 ms_dispat ceph cwd unknown /proc/78474/task/78488/cwd (readlink: Permission denied) 2024-07-27T03:18:57.733 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78488 ms_dispat ceph rtd unknown /proc/78474/task/78488/root (readlink: Permission denied) 2024-07-27T03:18:57.733 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78488 ms_dispat ceph txt unknown /proc/78474/task/78488/exe (readlink: Permission denied) 2024-07-27T03:18:57.733 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78488 ms_dispat ceph NOFD /proc/78474/task/78488/fd (opendir: Permission denied) 2024-07-27T03:18:57.733 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78489 ms_local ceph cwd unknown /proc/78474/task/78489/cwd (readlink: Permission denied) 2024-07-27T03:18:57.734 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78489 ms_local ceph rtd unknown /proc/78474/task/78489/root (readlink: Permission denied) 2024-07-27T03:18:57.734 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78489 ms_local ceph txt unknown /proc/78474/task/78489/exe (readlink: Permission denied) 2024-07-27T03:18:57.734 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78489 ms_local ceph NOFD /proc/78474/task/78489/fd (opendir: Permission denied) 2024-07-27T03:18:57.734 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78490 safe_time ceph cwd unknown /proc/78474/task/78490/cwd (readlink: Permission denied) 2024-07-27T03:18:57.734 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78490 safe_time ceph rtd unknown /proc/78474/task/78490/root (readlink: Permission denied) 2024-07-27T03:18:57.734 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78490 safe_time ceph txt unknown /proc/78474/task/78490/exe (readlink: Permission denied) 2024-07-27T03:18:57.734 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78490 safe_time ceph NOFD /proc/78474/task/78490/fd (opendir: Permission denied) 2024-07-27T03:18:57.734 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78491 safe_time ceph cwd unknown /proc/78474/task/78491/cwd (readlink: Permission denied) 2024-07-27T03:18:57.734 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78491 safe_time ceph rtd unknown /proc/78474/task/78491/root (readlink: Permission denied) 2024-07-27T03:18:57.734 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78491 safe_time ceph txt unknown /proc/78474/task/78491/exe (readlink: Permission denied) 2024-07-27T03:18:57.734 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78491 safe_time ceph NOFD /proc/78474/task/78491/fd (opendir: Permission denied) 2024-07-27T03:18:57.734 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78492 ceph-mds ceph cwd unknown /proc/78474/task/78492/cwd (readlink: Permission denied) 2024-07-27T03:18:57.734 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78492 ceph-mds ceph rtd unknown /proc/78474/task/78492/root (readlink: Permission denied) 2024-07-27T03:18:57.734 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78492 ceph-mds ceph txt unknown /proc/78474/task/78492/exe (readlink: Permission denied) 2024-07-27T03:18:57.735 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78492 ceph-mds ceph NOFD /proc/78474/task/78492/fd (opendir: Permission denied) 2024-07-27T03:18:57.735 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78515 safe_time ceph cwd unknown /proc/78474/task/78515/cwd (readlink: Permission denied) 2024-07-27T03:18:57.735 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78515 safe_time ceph rtd unknown /proc/78474/task/78515/root (readlink: Permission denied) 2024-07-27T03:18:57.735 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78515 safe_time ceph txt unknown /proc/78474/task/78515/exe (readlink: Permission denied) 2024-07-27T03:18:57.735 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 78515 safe_time ceph NOFD /proc/78474/task/78515/fd (opendir: Permission denied) 2024-07-27T03:18:57.735 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84127 ceph_time ceph cwd unknown /proc/78474/task/84127/cwd (readlink: Permission denied) 2024-07-27T03:18:57.735 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84127 ceph_time ceph rtd unknown /proc/78474/task/84127/root (readlink: Permission denied) 2024-07-27T03:18:57.735 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84127 ceph_time ceph txt unknown /proc/78474/task/84127/exe (readlink: Permission denied) 2024-07-27T03:18:57.735 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84127 ceph_time ceph NOFD /proc/78474/task/84127/fd (opendir: Permission denied) 2024-07-27T03:18:57.735 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84130 OpHistory ceph cwd unknown /proc/78474/task/84130/cwd (readlink: Permission denied) 2024-07-27T03:18:57.735 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84130 OpHistory ceph rtd unknown /proc/78474/task/84130/root (readlink: Permission denied) 2024-07-27T03:18:57.735 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84130 OpHistory ceph txt unknown /proc/78474/task/84130/exe (readlink: Permission denied) 2024-07-27T03:18:57.735 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84130 OpHistory ceph NOFD /proc/78474/task/84130/fd (opendir: Permission denied) 2024-07-27T03:18:57.735 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84132 ms_dispat ceph cwd unknown /proc/78474/task/84132/cwd (readlink: Permission denied) 2024-07-27T03:18:57.736 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84132 ms_dispat ceph rtd unknown /proc/78474/task/84132/root (readlink: Permission denied) 2024-07-27T03:18:57.736 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84132 ms_dispat ceph txt unknown /proc/78474/task/84132/exe (readlink: Permission denied) 2024-07-27T03:18:57.736 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84132 ms_dispat ceph NOFD /proc/78474/task/84132/fd (opendir: Permission denied) 2024-07-27T03:18:57.736 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84134 ms_dispat ceph cwd unknown /proc/78474/task/84134/cwd (readlink: Permission denied) 2024-07-27T03:18:57.736 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84134 ms_dispat ceph rtd unknown /proc/78474/task/84134/root (readlink: Permission denied) 2024-07-27T03:18:57.736 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84134 ms_dispat ceph txt unknown /proc/78474/task/84134/exe (readlink: Permission denied) 2024-07-27T03:18:57.736 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84134 ms_dispat ceph NOFD /proc/78474/task/84134/fd (opendir: Permission denied) 2024-07-27T03:18:57.736 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84136 mds_rank_ ceph cwd unknown /proc/78474/task/84136/cwd (readlink: Permission denied) 2024-07-27T03:18:57.736 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84136 mds_rank_ ceph rtd unknown /proc/78474/task/84136/root (readlink: Permission denied) 2024-07-27T03:18:57.736 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84136 mds_rank_ ceph txt unknown /proc/78474/task/84136/exe (readlink: Permission denied) 2024-07-27T03:18:57.736 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84136 mds_rank_ ceph NOFD /proc/78474/task/84136/fd (opendir: Permission denied) 2024-07-27T03:18:57.736 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84138 PQ_Finish ceph cwd unknown /proc/78474/task/84138/cwd (readlink: Permission denied) 2024-07-27T03:18:57.736 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84138 PQ_Finish ceph rtd unknown /proc/78474/task/84138/root (readlink: Permission denied) 2024-07-27T03:18:57.736 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84138 PQ_Finish ceph txt unknown /proc/78474/task/84138/exe (readlink: Permission denied) 2024-07-27T03:18:57.737 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84138 PQ_Finish ceph NOFD /proc/78474/task/84138/fd (opendir: Permission denied) 2024-07-27T03:18:57.737 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84140 safe_time ceph cwd unknown /proc/78474/task/84140/cwd (readlink: Permission denied) 2024-07-27T03:18:57.737 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84140 safe_time ceph rtd unknown /proc/78474/task/84140/root (readlink: Permission denied) 2024-07-27T03:18:57.737 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84140 safe_time ceph txt unknown /proc/78474/task/84140/exe (readlink: Permission denied) 2024-07-27T03:18:57.737 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84140 safe_time ceph NOFD /proc/78474/task/84140/fd (opendir: Permission denied) 2024-07-27T03:18:57.737 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84142 MR_Finish ceph cwd unknown /proc/78474/task/84142/cwd (readlink: Permission denied) 2024-07-27T03:18:57.737 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84142 MR_Finish ceph rtd unknown /proc/78474/task/84142/root (readlink: Permission denied) 2024-07-27T03:18:57.737 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84142 MR_Finish ceph txt unknown /proc/78474/task/84142/exe (readlink: Permission denied) 2024-07-27T03:18:57.737 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84142 MR_Finish ceph NOFD /proc/78474/task/84142/fd (opendir: Permission denied) 2024-07-27T03:18:57.737 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84145 md_submit ceph cwd unknown /proc/78474/task/84145/cwd (readlink: Permission denied) 2024-07-27T03:18:57.737 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84145 md_submit ceph rtd unknown /proc/78474/task/84145/root (readlink: Permission denied) 2024-07-27T03:18:57.737 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84145 md_submit ceph txt unknown /proc/78474/task/84145/exe (readlink: Permission denied) 2024-07-27T03:18:57.737 INFO:tasks.workunit.client.0.smithi067.stdout:ceph-mds 78474 84145 md_submit ceph NOFD /proc/78474/task/84145/fd (opendir: Permission denied) 2024-07-27T03:18:57.737 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 80378 root cwd unknown /proc/80378/cwd (readlink: Permission denied) 2024-07-27T03:18:57.738 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 80378 root rtd unknown /proc/80378/root (readlink: Permission denied) 2024-07-27T03:18:57.738 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 80378 root txt unknown /proc/80378/exe (readlink: Permission denied) 2024-07-27T03:18:57.738 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 80378 root NOFD /proc/80378/fd (opendir: Permission denied) 2024-07-27T03:18:57.738 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 80522 root cwd unknown /proc/80522/cwd (readlink: Permission denied) 2024-07-27T03:18:57.738 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 80522 root rtd unknown /proc/80522/root (readlink: Permission denied) 2024-07-27T03:18:57.738 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 80522 root txt unknown /proc/80522/exe (readlink: Permission denied) 2024-07-27T03:18:57.738 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/u 80522 root NOFD /proc/80522/fd (opendir: Permission denied) 2024-07-27T03:18:57.738 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 84553 root cwd unknown /proc/84553/cwd (readlink: Permission denied) 2024-07-27T03:18:57.738 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 84553 root rtd unknown /proc/84553/root (readlink: Permission denied) 2024-07-27T03:18:57.738 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 84553 root txt unknown /proc/84553/exe (readlink: Permission denied) 2024-07-27T03:18:57.738 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 84553 root NOFD /proc/84553/fd (opendir: Permission denied) 2024-07-27T03:18:57.738 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/2 87765 root cwd unknown /proc/87765/cwd (readlink: Permission denied) 2024-07-27T03:18:57.738 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/2 87765 root rtd unknown /proc/87765/root (readlink: Permission denied) 2024-07-27T03:18:57.739 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/2 87765 root txt unknown /proc/87765/exe (readlink: Permission denied) 2024-07-27T03:18:57.739 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/2 87765 root NOFD /proc/87765/fd (opendir: Permission denied) 2024-07-27T03:18:57.739 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 89399 root cwd unknown /proc/89399/cwd (readlink: Permission denied) 2024-07-27T03:18:57.739 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 89399 root rtd unknown /proc/89399/root (readlink: Permission denied) 2024-07-27T03:18:57.739 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 89399 root txt unknown /proc/89399/exe (readlink: Permission denied) 2024-07-27T03:18:57.739 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/1 89399 root NOFD /proc/89399/fd (opendir: Permission denied) 2024-07-27T03:18:57.739 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 90098 root cwd unknown /proc/90098/cwd (readlink: Permission denied) 2024-07-27T03:18:57.739 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 90098 root rtd unknown /proc/90098/root (readlink: Permission denied) 2024-07-27T03:18:57.739 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 90098 root txt unknown /proc/90098/exe (readlink: Permission denied) 2024-07-27T03:18:57.739 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/5 90098 root NOFD /proc/90098/fd (opendir: Permission denied) 2024-07-27T03:18:57.739 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 90980 root cwd unknown /proc/90980/cwd (readlink: Permission denied) 2024-07-27T03:18:57.739 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 90980 root rtd unknown /proc/90980/root (readlink: Permission denied) 2024-07-27T03:18:57.739 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 90980 root txt unknown /proc/90980/exe (readlink: Permission denied) 2024-07-27T03:18:57.739 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 90980 root NOFD /proc/90980/fd (opendir: Permission denied) 2024-07-27T03:18:57.740 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 90981 root cwd unknown /proc/90981/cwd (readlink: Permission denied) 2024-07-27T03:18:57.740 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 90981 root rtd unknown /proc/90981/root (readlink: Permission denied) 2024-07-27T03:18:57.740 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 90981 root txt unknown /proc/90981/exe (readlink: Permission denied) 2024-07-27T03:18:57.740 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/7 90981 root NOFD /proc/90981/fd (opendir: Permission denied) 2024-07-27T03:18:57.740 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 95319 root cwd unknown /proc/95319/cwd (readlink: Permission denied) 2024-07-27T03:18:57.740 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 95319 root rtd unknown /proc/95319/root (readlink: Permission denied) 2024-07-27T03:18:57.740 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 95319 root txt unknown /proc/95319/exe (readlink: Permission denied) 2024-07-27T03:18:57.740 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 95319 root NOFD /proc/95319/fd (opendir: Permission denied) 2024-07-27T03:18:57.740 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 95320 root cwd unknown /proc/95320/cwd (readlink: Permission denied) 2024-07-27T03:18:57.740 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 95320 root rtd unknown /proc/95320/root (readlink: Permission denied) 2024-07-27T03:18:57.740 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 95320 root txt unknown /proc/95320/exe (readlink: Permission denied) 2024-07-27T03:18:57.740 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/3 95320 root NOFD /proc/95320/fd (opendir: Permission denied) 2024-07-27T03:18:57.740 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 95811 root cwd unknown /proc/95811/cwd (readlink: Permission denied) 2024-07-27T03:18:57.740 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 95811 root rtd unknown /proc/95811/root (readlink: Permission denied) 2024-07-27T03:18:57.740 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 95811 root txt unknown /proc/95811/exe (readlink: Permission denied) 2024-07-27T03:18:57.741 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 95811 root NOFD /proc/95811/fd (opendir: Permission denied) 2024-07-27T03:18:57.741 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 95812 root cwd unknown /proc/95812/cwd (readlink: Permission denied) 2024-07-27T03:18:57.741 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 95812 root rtd unknown /proc/95812/root (readlink: Permission denied) 2024-07-27T03:18:57.741 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 95812 root txt unknown /proc/95812/exe (readlink: Permission denied) 2024-07-27T03:18:57.741 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 95812 root NOFD /proc/95812/fd (opendir: Permission denied) 2024-07-27T03:18:57.741 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 95813 root cwd unknown /proc/95813/cwd (readlink: Permission denied) 2024-07-27T03:18:57.741 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 95813 root rtd unknown /proc/95813/root (readlink: Permission denied) 2024-07-27T03:18:57.741 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 95813 root txt unknown /proc/95813/exe (readlink: Permission denied) 2024-07-27T03:18:57.741 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/R 95813 root NOFD /proc/95813/fd (opendir: Permission denied) 2024-07-27T03:18:57.741 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 97334 root cwd unknown /proc/97334/cwd (readlink: Permission denied) 2024-07-27T03:18:57.741 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 97334 root rtd unknown /proc/97334/root (readlink: Permission denied) 2024-07-27T03:18:57.741 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 97334 root txt unknown /proc/97334/exe (readlink: Permission denied) 2024-07-27T03:18:57.741 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/6 97334 root NOFD /proc/97334/fd (opendir: Permission denied) 2024-07-27T03:18:57.741 INFO:tasks.workunit.client.0.smithi067.stdout:timeout 98917 ubuntu cwd DIR 0,119 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-07-27T03:18:57.742 INFO:tasks.workunit.client.0.smithi067.stdout:timeout 98917 ubuntu rtd DIR 8,1 4096 2 / 2024-07-27T03:18:57.742 INFO:tasks.workunit.client.0.smithi067.stdout:timeout 98917 ubuntu txt REG 8,1 41144 7580 /usr/bin/timeout 2024-07-27T03:18:57.742 INFO:tasks.workunit.client.0.smithi067.stdout:timeout 98917 ubuntu mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-07-27T03:18:57.742 INFO:tasks.workunit.client.0.smithi067.stdout:timeout 98917 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-07-27T03:18:57.742 INFO:tasks.workunit.client.0.smithi067.stdout:timeout 98917 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-07-27T03:18:57.742 INFO:tasks.workunit.client.0.smithi067.stdout:timeout 98917 ubuntu mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-07-27T03:18:57.742 INFO:tasks.workunit.client.0.smithi067.stdout:timeout 98917 ubuntu mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-07-27T03:18:57.742 INFO:tasks.workunit.client.0.smithi067.stdout:timeout 98917 ubuntu mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-07-27T03:18:57.742 INFO:tasks.workunit.client.0.smithi067.stdout:timeout 98917 ubuntu mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-07-27T03:18:57.742 INFO:tasks.workunit.client.0.smithi067.stdout:timeout 98917 ubuntu mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-07-27T03:18:57.742 INFO:tasks.workunit.client.0.smithi067.stdout:timeout 98917 ubuntu mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-07-27T03:18:57.742 INFO:tasks.workunit.client.0.smithi067.stdout:timeout 98917 ubuntu mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-07-27T03:18:57.742 INFO:tasks.workunit.client.0.smithi067.stdout:timeout 98917 ubuntu mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-07-27T03:18:57.743 INFO:tasks.workunit.client.0.smithi067.stdout:timeout 98917 ubuntu mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-07-27T03:18:57.743 INFO:tasks.workunit.client.0.smithi067.stdout:timeout 98917 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-07-27T03:18:57.743 INFO:tasks.workunit.client.0.smithi067.stdout:timeout 98917 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-07-27T03:18:57.743 INFO:tasks.workunit.client.0.smithi067.stdout:timeout 98917 ubuntu mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-07-27T03:18:57.743 INFO:tasks.workunit.client.0.smithi067.stdout:timeout 98917 ubuntu 0r FIFO 0,13 0t0 276219 pipe 2024-07-27T03:18:57.743 INFO:tasks.workunit.client.0.smithi067.stdout:timeout 98917 ubuntu 1w FIFO 0,13 0t0 276220 pipe 2024-07-27T03:18:57.743 INFO:tasks.workunit.client.0.smithi067.stdout:timeout 98917 ubuntu 2w FIFO 0,13 0t0 276221 pipe 2024-07-27T03:18:57.743 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu cwd DIR 0,119 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-07-27T03:18:57.743 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu rtd DIR 8,1 4096 2 / 2024-07-27T03:18:57.743 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu txt REG 8,1 1389072 7149 /usr/bin/bash 2024-07-27T03:18:57.743 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-07-27T03:18:57.743 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-07-27T03:18:57.743 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-07-27T03:18:57.743 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-07-27T03:18:57.744 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-07-27T03:18:57.744 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-07-27T03:18:57.744 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-07-27T03:18:57.744 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-07-27T03:18:57.744 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-07-27T03:18:57.744 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-07-27T03:18:57.744 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-07-27T03:18:57.744 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu mem REG 8,1 195080 7147 /usr/lib64/libtinfo.so.6.2 2024-07-27T03:18:57.744 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-07-27T03:18:57.744 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-07-27T03:18:57.744 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-07-27T03:18:57.744 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-07-27T03:18:57.744 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu 0r FIFO 0,13 0t0 276219 pipe 2024-07-27T03:18:57.744 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu 1w FIFO 0,13 0t0 276220 pipe 2024-07-27T03:18:57.745 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu 2w FIFO 0,13 0t0 276221 pipe 2024-07-27T03:18:57.745 INFO:tasks.workunit.client.0.smithi067.stdout:fsync-tes 98935 ubuntu 255r REG 8,1 207 526795 /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-07-27T03:18:57.745 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 100799 root cwd unknown /proc/100799/cwd (readlink: Permission denied) 2024-07-27T03:18:57.745 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 100799 root rtd unknown /proc/100799/root (readlink: Permission denied) 2024-07-27T03:18:57.745 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 100799 root txt unknown /proc/100799/exe (readlink: Permission denied) 2024-07-27T03:18:57.745 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/0 100799 root NOFD /proc/100799/fd (opendir: Permission denied) 2024-07-27T03:18:57.745 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/4 102147 root cwd unknown /proc/102147/cwd (readlink: Permission denied) 2024-07-27T03:18:57.745 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/4 102147 root rtd unknown /proc/102147/root (readlink: Permission denied) 2024-07-27T03:18:57.745 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/4 102147 root txt unknown /proc/102147/exe (readlink: Permission denied) 2024-07-27T03:18:57.745 INFO:tasks.workunit.client.0.smithi067.stdout:kworker/4 102147 root NOFD /proc/102147/fd (opendir: Permission denied) 2024-07-27T03:18:57.745 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 102511 root cwd unknown /proc/102511/cwd (readlink: Permission denied) 2024-07-27T03:18:57.745 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 102511 root rtd unknown /proc/102511/root (readlink: Permission denied) 2024-07-27T03:18:57.745 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 102511 root txt unknown /proc/102511/exe (readlink: Permission denied) 2024-07-27T03:18:57.745 INFO:tasks.workunit.client.0.smithi067.stdout:sudo 102511 root NOFD /proc/102511/fd (opendir: Permission denied) 2024-07-27T03:18:57.746 INFO:tasks.workunit.client.0.smithi067.stdout:cephadm 102528 root cwd unknown /proc/102528/cwd (readlink: Permission denied) 2024-07-27T03:18:57.746 INFO:tasks.workunit.client.0.smithi067.stdout:cephadm 102528 root rtd unknown /proc/102528/root (readlink: Permission denied) 2024-07-27T03:18:57.746 INFO:tasks.workunit.client.0.smithi067.stdout:cephadm 102528 root txt unknown /proc/102528/exe (readlink: Permission denied) 2024-07-27T03:18:57.746 INFO:tasks.workunit.client.0.smithi067.stdout:cephadm 102528 root NOFD /proc/102528/fd (opendir: Permission denied) 2024-07-27T03:18:57.746 INFO:tasks.workunit.client.0.smithi067.stdout:cephadm 102528 102574 cephadm root cwd unknown /proc/102528/task/102574/cwd (readlink: Permission denied) 2024-07-27T03:18:57.746 INFO:tasks.workunit.client.0.smithi067.stdout:cephadm 102528 102574 cephadm root rtd unknown /proc/102528/task/102574/root (readlink: Permission denied) 2024-07-27T03:18:57.746 INFO:tasks.workunit.client.0.smithi067.stdout:cephadm 102528 102574 cephadm root txt unknown /proc/102528/task/102574/exe (readlink: Permission denied) 2024-07-27T03:18:57.746 INFO:tasks.workunit.client.0.smithi067.stdout:cephadm 102528 102574 cephadm root NOFD /proc/102528/task/102574/fd (opendir: Permission denied) 2024-07-27T03:18:57.746 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu cwd DIR 0,119 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-07-27T03:18:57.746 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu rtd DIR 8,1 4096 2 / 2024-07-27T03:18:57.746 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu txt REG 8,1 179232 9797 /usr/bin/lsof 2024-07-27T03:18:57.746 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu mem REG 8,1 5435176 24521 /usr/lib64/libcrypto.so.3.2.2 2024-07-27T03:18:57.746 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-07-27T03:18:57.746 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-07-27T03:18:57.747 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-07-27T03:18:57.747 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-07-27T03:18:57.747 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-07-27T03:18:57.747 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-07-27T03:18:57.747 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-07-27T03:18:57.747 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-07-27T03:18:57.747 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-07-27T03:18:57.747 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-07-27T03:18:57.747 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-07-27T03:18:57.747 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-07-27T03:18:57.747 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-07-27T03:18:57.747 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-07-27T03:18:57.747 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-07-27T03:18:57.747 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu 0r FIFO 0,13 0t0 276219 pipe 2024-07-27T03:18:57.748 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu 1w FIFO 0,13 0t0 276220 pipe 2024-07-27T03:18:57.748 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu 2w FIFO 0,13 0t0 276221 pipe 2024-07-27T03:18:57.748 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu 3r DIR 0,20 0 1 /proc 2024-07-27T03:18:57.748 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu 4r DIR 0,20 0 294627 /proc/102566/fd 2024-07-27T03:18:57.748 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu 5w FIFO 0,13 0t0 294632 pipe 2024-07-27T03:18:57.748 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102566 ubuntu 6r FIFO 0,13 0t0 294633 pipe 2024-07-27T03:18:57.748 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102572 ubuntu cwd DIR 0,119 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-07-27T03:18:57.748 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102572 ubuntu rtd DIR 8,1 4096 2 / 2024-07-27T03:18:57.748 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102572 ubuntu txt REG 8,1 179232 9797 /usr/bin/lsof 2024-07-27T03:18:57.748 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102572 ubuntu mem REG 8,1 5435176 24521 /usr/lib64/libcrypto.so.3.2.2 2024-07-27T03:18:57.748 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102572 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-07-27T03:18:57.748 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102572 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-07-27T03:18:57.748 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102572 ubuntu mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-07-27T03:18:57.748 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102572 ubuntu mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-07-27T03:18:57.749 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102572 ubuntu mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-07-27T03:18:57.749 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102572 ubuntu mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-07-27T03:18:57.749 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102572 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-07-27T03:18:57.749 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102572 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-07-27T03:18:57.749 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102572 ubuntu mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-07-27T03:18:57.749 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102572 ubuntu mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-07-27T03:18:57.749 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102572 ubuntu mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-07-27T03:18:57.749 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102572 ubuntu mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-07-27T03:18:57.749 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102572 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-07-27T03:18:57.749 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102572 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-07-27T03:18:57.749 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102572 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-07-27T03:18:57.749 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102572 ubuntu 4r FIFO 0,13 0t0 294632 pipe 2024-07-27T03:18:57.749 INFO:tasks.workunit.client.0.smithi067.stdout:lsof 102572 ubuntu 7w FIFO 0,13 0t0 294633 pipe 2024-07-27T03:18:57.750 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 root cwd unknown /proc/102573/cwd (readlink: Permission denied) 2024-07-27T03:18:57.750 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 root rtd unknown /proc/102573/root (readlink: Permission denied) 2024-07-27T03:18:57.750 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 root txt unknown /proc/102573/exe (readlink: Permission denied) 2024-07-27T03:18:57.750 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 root NOFD /proc/102573/fd (opendir: Permission denied) 2024-07-27T03:18:57.750 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102575 podman root cwd unknown /proc/102573/task/102575/cwd (readlink: Permission denied) 2024-07-27T03:18:57.750 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102575 podman root rtd unknown /proc/102573/task/102575/root (readlink: Permission denied) 2024-07-27T03:18:57.750 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102575 podman root txt unknown /proc/102573/task/102575/exe (readlink: Permission denied) 2024-07-27T03:18:57.750 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102575 podman root NOFD /proc/102573/task/102575/fd (opendir: Permission denied) 2024-07-27T03:18:57.750 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102576 podman root cwd unknown /proc/102573/task/102576/cwd (readlink: Permission denied) 2024-07-27T03:18:57.750 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102576 podman root rtd unknown /proc/102573/task/102576/root (readlink: Permission denied) 2024-07-27T03:18:57.750 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102576 podman root txt unknown /proc/102573/task/102576/exe (readlink: Permission denied) 2024-07-27T03:18:57.750 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102576 podman root NOFD /proc/102573/task/102576/fd (opendir: Permission denied) 2024-07-27T03:18:57.750 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102577 podman root cwd unknown /proc/102573/task/102577/cwd (readlink: Permission denied) 2024-07-27T03:18:57.751 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102577 podman root rtd unknown /proc/102573/task/102577/root (readlink: Permission denied) 2024-07-27T03:18:57.751 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102577 podman root txt unknown /proc/102573/task/102577/exe (readlink: Permission denied) 2024-07-27T03:18:57.751 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102577 podman root NOFD /proc/102573/task/102577/fd (opendir: Permission denied) 2024-07-27T03:18:57.751 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102578 podman root cwd unknown /proc/102573/task/102578/cwd (readlink: Permission denied) 2024-07-27T03:18:57.751 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102578 podman root rtd unknown /proc/102573/task/102578/root (readlink: Permission denied) 2024-07-27T03:18:57.751 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102578 podman root txt unknown /proc/102573/task/102578/exe (readlink: Permission denied) 2024-07-27T03:18:57.751 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102578 podman root NOFD /proc/102573/task/102578/fd (opendir: Permission denied) 2024-07-27T03:18:57.751 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102579 podman root cwd unknown /proc/102573/task/102579/cwd (readlink: Permission denied) 2024-07-27T03:18:57.751 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102579 podman root rtd unknown /proc/102573/task/102579/root (readlink: Permission denied) 2024-07-27T03:18:57.751 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102579 podman root txt unknown /proc/102573/task/102579/exe (readlink: Permission denied) 2024-07-27T03:18:57.751 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102579 podman root NOFD /proc/102573/task/102579/fd (opendir: Permission denied) 2024-07-27T03:18:57.751 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102580 podman root cwd unknown /proc/102573/task/102580/cwd (readlink: Permission denied) 2024-07-27T03:18:57.751 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102580 podman root rtd unknown /proc/102573/task/102580/root (readlink: Permission denied) 2024-07-27T03:18:57.751 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102580 podman root txt unknown /proc/102573/task/102580/exe (readlink: Permission denied) 2024-07-27T03:18:57.752 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102580 podman root NOFD /proc/102573/task/102580/fd (opendir: Permission denied) 2024-07-27T03:18:57.752 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102581 podman root cwd unknown /proc/102573/task/102581/cwd (readlink: Permission denied) 2024-07-27T03:18:57.752 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102581 podman root rtd unknown /proc/102573/task/102581/root (readlink: Permission denied) 2024-07-27T03:18:57.752 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102581 podman root txt unknown /proc/102573/task/102581/exe (readlink: Permission denied) 2024-07-27T03:18:57.752 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102581 podman root NOFD /proc/102573/task/102581/fd (opendir: Permission denied) 2024-07-27T03:18:57.752 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102582 podman root cwd unknown /proc/102573/task/102582/cwd (readlink: Permission denied) 2024-07-27T03:18:57.752 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102582 podman root rtd unknown /proc/102573/task/102582/root (readlink: Permission denied) 2024-07-27T03:18:57.752 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102582 podman root txt unknown /proc/102573/task/102582/exe (readlink: Permission denied) 2024-07-27T03:18:57.752 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102582 podman root NOFD /proc/102573/task/102582/fd (opendir: Permission denied) 2024-07-27T03:18:57.752 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102583 podman root cwd unknown /proc/102573/task/102583/cwd (readlink: Permission denied) 2024-07-27T03:18:57.752 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102583 podman root rtd unknown /proc/102573/task/102583/root (readlink: Permission denied) 2024-07-27T03:18:57.752 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102583 podman root txt unknown /proc/102573/task/102583/exe (readlink: Permission denied) 2024-07-27T03:18:57.752 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102583 podman root NOFD /proc/102573/task/102583/fd (opendir: Permission denied) 2024-07-27T03:18:57.752 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102584 podman root cwd unknown /proc/102573/task/102584/cwd (readlink: Permission denied) 2024-07-27T03:18:57.753 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102584 podman root rtd unknown /proc/102573/task/102584/root (readlink: Permission denied) 2024-07-27T03:18:57.753 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102584 podman root txt unknown /proc/102573/task/102584/exe (readlink: Permission denied) 2024-07-27T03:18:57.753 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102584 podman root NOFD /proc/102573/task/102584/fd (opendir: Permission denied) 2024-07-27T03:18:57.753 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102585 podman root cwd unknown /proc/102573/task/102585/cwd (readlink: Permission denied) 2024-07-27T03:18:57.753 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102585 podman root rtd unknown /proc/102573/task/102585/root (readlink: Permission denied) 2024-07-27T03:18:57.753 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102585 podman root txt unknown /proc/102573/task/102585/exe (readlink: Permission denied) 2024-07-27T03:18:57.753 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102585 podman root NOFD /proc/102573/task/102585/fd (opendir: Permission denied) 2024-07-27T03:18:57.753 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102586 podman root cwd unknown /proc/102573/task/102586/cwd (readlink: Permission denied) 2024-07-27T03:18:57.753 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102586 podman root rtd unknown /proc/102573/task/102586/root (readlink: Permission denied) 2024-07-27T03:18:57.753 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102586 podman root txt unknown /proc/102573/task/102586/exe (readlink: Permission denied) 2024-07-27T03:18:57.753 INFO:tasks.workunit.client.0.smithi067.stdout:podman 102573 102586 podman root NOFD /proc/102573/task/102586/fd (opendir: Permission denied) 2024-07-27T03:18:57.754 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-07-27T03:18:57.754 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-07-27T03:18:57.800 INFO:tasks.workunit:Stopping ['suites/fsync-tester.sh'] on client.0... 2024-07-27T03:18:57.800 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-07-27T03:18:58.111 DEBUG:teuthology.parallel:result is None 2024-07-27T03:18:58.112 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-07-27T03:18:58.149 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-07-27T03:18:58.149 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-07-27T03:18:58.163 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-07-27T03:18:58.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:18:58 smithi067 ceph-mon[32814]: pgmap v502: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 859 KiB/s wr, 12 op/s 2024-07-27T03:18:58.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:18:58 smithi149 ceph-mon[37173]: pgmap v502: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 859 KiB/s wr, 12 op/s 2024-07-27T03:18:59.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:18:58 smithi175 ceph-mon[37125]: pgmap v502: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 859 KiB/s wr, 12 op/s 2024-07-27T03:18:59.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.352+0000 7fb0728d2640 1 -- 172.21.15.67:0/306888907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb06c108760 msgr2=0x7fb06c10ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:59.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.352+0000 7fb0728d2640 1 --2- 172.21.15.67:0/306888907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb06c108760 0x7fb06c10ef90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fb064009ca0 tx=0x7fb06402a8c0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.352+0000 7fb0728d2640 1 -- 172.21.15.67:0/306888907 shutdown_connections 2024-07-27T03:18:59.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.352+0000 7fb0728d2640 1 --2- 172.21.15.67:0/306888907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb06c108760 0x7fb06c10ef90 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.352+0000 7fb0728d2640 1 --2- 172.21.15.67:0/306888907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb06c107e10 0x7fb06c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.352+0000 7fb0728d2640 1 --2- 172.21.15.67:0/306888907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb06c1024d0 0x7fb06c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.352+0000 7fb0728d2640 1 -- 172.21.15.67:0/306888907 >> 172.21.15.67:0/306888907 conn(0x7fb06c0791a0 msgr2=0x7fb06c0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:59.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.352+0000 7fb0728d2640 1 -- 172.21.15.67:0/306888907 shutdown_connections 2024-07-27T03:18:59.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.352+0000 7fb0728d2640 1 -- 172.21.15.67:0/306888907 wait complete. 2024-07-27T03:18:59.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.353+0000 7fb0728d2640 1 Processor -- start 2024-07-27T03:18:59.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.353+0000 7fb0728d2640 1 -- start start 2024-07-27T03:18:59.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.354+0000 7fb0728d2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb06c1024d0 0x7fb06c10fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:59.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.354+0000 7fb0728d2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb06c107e10 0x7fb06c10ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:59.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.354+0000 7fb0728d2640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb06c108760 0x7fb06c1104a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:59.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.354+0000 7fb0728d2640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c113520 con 0x7fb06c1024d0 2024-07-27T03:18:59.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.354+0000 7fb0728d2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1133a0 con 0x7fb06c107e10 2024-07-27T03:18:59.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.354+0000 7fb0728d2640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1136a0 con 0x7fb06c108760 2024-07-27T03:18:59.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.354+0000 7fb06bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb06c1024d0 0x7fb06c10fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:59.369 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.354+0000 7fb06bfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb06c1024d0 0x7fb06c10fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40428/0 (socket says 172.21.15.67:40428) 2024-07-27T03:18:59.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.354+0000 7fb06bfff640 1 -- 172.21.15.67:0/1875487839 learned_addr learned my addr 172.21.15.67:0/1875487839 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:59.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.354+0000 7fb070e6f640 1 --2- 172.21.15.67:0/1875487839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb06c108760 0x7fb06c1104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:59.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.354+0000 7fb06bfff640 1 -- 172.21.15.67:0/1875487839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb06c108760 msgr2=0x7fb06c1104a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:59.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.354+0000 7fb06bfff640 1 --2- 172.21.15.67:0/1875487839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb06c108760 0x7fb06c1104a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.354+0000 7fb06bfff640 1 -- 172.21.15.67:0/1875487839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb06c107e10 msgr2=0x7fb06c10ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:59.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.354+0000 7fb06bfff640 1 --2- 172.21.15.67:0/1875487839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb06c107e10 0x7fb06c10ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.354+0000 7fb06bfff640 1 -- 172.21.15.67:0/1875487839 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06c1a8210 con 0x7fb06c1024d0 2024-07-27T03:18:59.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.354+0000 7fb06bfff640 1 --2- 172.21.15.67:0/1875487839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb06c1024d0 0x7fb06c10fa20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00d950 tx=0x7fb05c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:59.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.354+0000 7fb0697fa640 1 -- 172.21.15.67:0/1875487839 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c017070 con 0x7fb06c1024d0 2024-07-27T03:18:59.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.354+0000 7fb0728d2640 1 -- 172.21.15.67:0/1875487839 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06c1a8500 con 0x7fb06c1024d0 2024-07-27T03:18:59.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.354+0000 7fb0697fa640 1 -- 172.21.15.67:0/1875487839 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb05c009c30 con 0x7fb06c1024d0 2024-07-27T03:18:59.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.354+0000 7fb0697fa640 1 -- 172.21.15.67:0/1875487839 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c00f620 con 0x7fb06c1024d0 2024-07-27T03:18:59.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.354+0000 7fb0728d2640 1 -- 172.21.15.67:0/1875487839 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb06c1a8a90 con 0x7fb06c1024d0 2024-07-27T03:18:59.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.357+0000 7fb078e57740 1 Processor -- start 2024-07-27T03:18:59.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.357+0000 7fb0697fa640 1 -- 172.21.15.67:0/1875487839 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7fb05c010460 con 0x7fb06c1024d0 2024-07-27T03:18:59.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.357+0000 7fb078e57740 1 -- start start 2024-07-27T03:18:59.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.357+0000 7fb078e57740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e58b425070 0x55e58b7c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:59.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.357+0000 7fb078e57740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e58b3fc1b0 0x55e58b7ca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:59.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.357+0000 7fb078e57740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:59.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.357+0000 7fb078e57740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55e58b6b2400 con 0x55e58b404390 2024-07-27T03:18:59.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.357+0000 7fb078e57740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e58b740b20 con 0x55e58b3fc1b0 2024-07-27T03:18:59.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.357+0000 7fb078e57740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e58b730020 con 0x55e58b425070 2024-07-27T03:18:59.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.358+0000 7fb06b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:59.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.358+0000 7fb06b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40438/0 (socket says 172.21.15.67:40438) 2024-07-27T03:18:59.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.358+0000 7fb06b7fe640 1 -- 172.21.15.67:0/307047955 learned_addr learned my addr 172.21.15.67:0/307047955 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:59.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.358+0000 7fb070e6f640 1 --2- 172.21.15.67:0/307047955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e58b425070 0x55e58b7c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:59.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.358+0000 7fb06bfff640 1 --2- 172.21.15.67:0/307047955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e58b3fc1b0 0x55e58b7ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:59.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.358+0000 7fb06b7fe640 1 -- 172.21.15.67:0/307047955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e58b425070 msgr2=0x55e58b7c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:59.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.358+0000 7fb06b7fe640 1 --2- 172.21.15.67:0/307047955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e58b425070 0x55e58b7c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.358+0000 7fb06b7fe640 1 -- 172.21.15.67:0/307047955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e58b3fc1b0 msgr2=0x55e58b7ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:59.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.358+0000 7fb06b7fe640 1 --2- 172.21.15.67:0/307047955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e58b3fc1b0 0x55e58b7ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.358+0000 7fb06b7fe640 1 -- 172.21.15.67:0/307047955 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e58b6c5770 con 0x55e58b404390 2024-07-27T03:18:59.373 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.358+0000 7fb0697fa640 1 --2- 172.21.15.67:0/1875487839 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb0540b3bd0 0x7fb0540b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:59.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.358+0000 7fb06b7fe640 1 --2- 172.21.15.67:0/307047955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7ca7c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb06000ba20 tx=0x7fb06000bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36f3a1668e36550 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:59.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.358+0000 7fb070e6f640 1 --2- 172.21.15.67:0/1875487839 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb0540b3bd0 0x7fb0540b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:59.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.358+0000 7fb0697fa640 1 -- 172.21.15.67:0/1875487839 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fb05c0e3c90 con 0x7fb06c1024d0 2024-07-27T03:18:59.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.358+0000 7fb0527fc640 1 -- 172.21.15.67:0/307047955 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060016070 con 0x55e58b404390 2024-07-27T03:18:59.374 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.358+0000 7fb0527fc640 1 -- 172.21.15.67:0/307047955 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0600054d0 con 0x55e58b404390 2024-07-27T03:18:59.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.358+0000 7fb0527fc640 1 -- 172.21.15.67:0/307047955 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060004d40 con 0x55e58b404390 2024-07-27T03:18:59.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.360+0000 7fb078e57740 1 -- 172.21.15.67:0/307047955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 msgr2=0x55e58b7ca7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:59.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.360+0000 7fb070e6f640 1 --2- 172.21.15.67:0/1875487839 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb0540b3bd0 0x7fb0540b60e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb064004af0 tx=0x7fb064005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:59.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.360+0000 7fb078e57740 1 --2- 172.21.15.67:0/307047955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7ca7c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb06000ba20 tx=0x7fb06000bee0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.375 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.360+0000 7fb078e57740 1 -- 172.21.15.67:0/307047955 shutdown_connections 2024-07-27T03:18:59.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.360+0000 7fb078e57740 1 --2- 172.21.15.67:0/307047955 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7ca7c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.360+0000 7fb078e57740 1 --2- 172.21.15.67:0/307047955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e58b3fc1b0 0x55e58b7ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.360+0000 7fb078e57740 1 --2- 172.21.15.67:0/307047955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e58b425070 0x55e58b7c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.360+0000 7fb078e57740 1 -- 172.21.15.67:0/307047955 >> 172.21.15.67:0/307047955 conn(0x55e58b419460 msgr2=0x55e58b7c7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:59.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.360+0000 7fb078e57740 1 -- 172.21.15.67:0/307047955 shutdown_connections 2024-07-27T03:18:59.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.360+0000 7fb078e57740 1 -- 172.21.15.67:0/307047955 wait complete. 2024-07-27T03:18:59.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.360+0000 7fb078e57740 1 -- start start 2024-07-27T03:18:59.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.360+0000 7fb078e57740 1 Processor -- start 2024-07-27T03:18:59.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.361+0000 7fb078e57740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e58b3fc1b0 0x55e58b7d5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:59.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.361+0000 7fb078e57740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7d5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:59.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.361+0000 7fb078e57740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e58b425070 0x55e58b7e1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:59.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.361+0000 7fb078e57740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55e58b6d2900 con 0x55e58b404390 2024-07-27T03:18:59.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.361+0000 7fb078e57740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55e58b734060 con 0x55e58b3fc1b0 2024-07-27T03:18:59.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.361+0000 7fb078e57740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55e58b593700 con 0x55e58b425070 2024-07-27T03:18:59.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.361+0000 7fb078e57740 10 client.0 _ll_register_callbacks cb 0x55e58b6dad90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:59.378 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.361+0000 7fb06b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7d5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:59.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.361+0000 7fb06b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7d5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40442/0 (socket says 172.21.15.67:40442) 2024-07-27T03:18:59.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.361+0000 7fb06b7fe640 1 -- 172.21.15.67:0/834569176 learned_addr learned my addr 172.21.15.67:0/834569176 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:59.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.361+0000 7fb06bfff640 1 --2- 172.21.15.67:0/834569176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e58b3fc1b0 0x55e58b7d5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:59.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.361+0000 7fb070e6f640 1 --2- 172.21.15.67:0/834569176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e58b425070 0x55e58b7e1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:59.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.361+0000 7fb06b7fe640 1 -- 172.21.15.67:0/834569176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e58b425070 msgr2=0x55e58b7e1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:59.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.361+0000 7fb06b7fe640 1 --2- 172.21.15.67:0/834569176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e58b425070 0x55e58b7e1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.361+0000 7fb06b7fe640 1 -- 172.21.15.67:0/834569176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e58b3fc1b0 msgr2=0x55e58b7d5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:59.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.361+0000 7fb06b7fe640 1 --2- 172.21.15.67:0/834569176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e58b3fc1b0 0x55e58b7d5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.361+0000 7fb06b7fe640 1 -- 172.21.15.67:0/834569176 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e58b6d6bf0 con 0x55e58b404390 2024-07-27T03:18:59.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.361+0000 7fb070e6f640 1 --2- 172.21.15.67:0/834569176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e58b425070 0x55e58b7e1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:18:59.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.361+0000 7fb06b7fe640 1 --2- 172.21.15.67:0/834569176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7d5ec0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb060000c00 tx=0x7fb060005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23a090f976c4093 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:59.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.362+0000 7fb04b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:59.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.362+0000 7fb04b7fe640 1 -- 172.21.15.67:0/834569176 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060016030 con 0x55e58b404390 2024-07-27T03:18:59.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.362+0000 7fb04b7fe640 1 -- 172.21.15.67:0/834569176 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fb06000da40 con 0x55e58b404390 2024-07-27T03:18:59.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.362+0000 7fb078e57740 1 -- 172.21.15.67:0/834569176 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0600054d0 con 0x55e58b404390 2024-07-27T03:18:59.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.362+0000 7fb04b7fe640 1 -- 172.21.15.67:0/834569176 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb06000dc40 con 0x55e58b404390 2024-07-27T03:18:59.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.362+0000 7fb078e57740 10 client.15999 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:59.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.362+0000 7fb078e57740 1 -- 172.21.15.67:0/834569176 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e58b490650 con 0x55e58b404390 2024-07-27T03:18:59.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.362+0000 7fb04b7fe640 1 -- 172.21.15.67:0/834569176 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0600054d0 con 0x55e58b404390 2024-07-27T03:18:59.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.362+0000 7fb04b7fe640 1 -- 172.21.15.67:0/834569176 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb060022070 con 0x55e58b404390 2024-07-27T03:18:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.362+0000 7fb078e57740 10 client.15999 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.362+0000 7fb078e57740 10 client.15999 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-27T03:18:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.362+0000 7fb078e57740 20 client.15999 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.362+0000 7fb078e57740 1 --2- 172.21.15.67:0/834569176 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55e58b3e39f0 0x55e58b7e75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:59.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.362+0000 7fb078e57740 4 client.15999 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:18:59.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.362+0000 7fb078e57740 1 -- 172.21.15.67:0/834569176 --> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e58b742a70 con 0x55e58b3e39f0 2024-07-27T03:18:59.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.363+0000 7fb06bfff640 1 --2- 172.21.15.67:0/834569176 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55e58b3e39f0 0x55e58b7e75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:59.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.363+0000 7fb06bfff640 1 --2- 172.21.15.67:0/834569176 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55e58b3e39f0 0x55e58b7e75f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d50466d353e75ea1 server_cookie=a067691ec8b1b00a in_seq=0 out_seq=0 2024-07-27T03:18:59.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.363+0000 7fb04b7fe640 10 client.15999 ms_handle_connect on v2:172.21.15.175:6838/1670568853 2024-07-27T03:18:59.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.364+0000 7fb04b7fe640 1 -- 172.21.15.67:0/834569176 <== mds.2 v2:172.21.15.175:6838/1670568853 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e58b742a70 con 0x55e58b3e39f0 2024-07-27T03:18:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.364+0000 7fb04b7fe640 10 client.15999 handle_command_reply: tid=0 2024-07-27T03:18:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.364+0000 7fb078e57740 1 client.15999 shutdown 2024-07-27T03:18:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.364+0000 7fb078e57740 1 -- 172.21.15.67:0/834569176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 msgr2=0x55e58b7d5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.364+0000 7fb078e57740 1 --2- 172.21.15.67:0/834569176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7d5ec0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb060000c00 tx=0x7fb060005250 comp rx=0 tx=0).stop 2024-07-27T03:18:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.365+0000 7fb078e57740 1 -- 172.21.15.67:0/834569176 shutdown_connections 2024-07-27T03:18:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.365+0000 7fb078e57740 1 --2- 172.21.15.67:0/834569176 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55e58b3e39f0 0x55e58b7e75f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.365+0000 7fb078e57740 1 --2- 172.21.15.67:0/834569176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e58b425070 0x55e58b7e1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.365+0000 7fb078e57740 1 --2- 172.21.15.67:0/834569176 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7d5ec0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.365+0000 7fb078e57740 1 --2- 172.21.15.67:0/834569176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e58b3fc1b0 0x55e58b7d5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.385 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.365+0000 7fb078e57740 1 -- 172.21.15.67:0/834569176 >> 172.21.15.67:0/834569176 conn(0x55e58b419460 msgr2=0x55e58b7cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:59.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.365+0000 7fb04b7fe640 0 client.15999 ms_handle_reset on v2:172.21.15.175:6838/1670568853 2024-07-27T03:18:59.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.365+0000 7fb078e57740 1 -- 172.21.15.67:0/834569176 shutdown_connections 2024-07-27T03:18:59.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.365+0000 7fb078e57740 1 -- 172.21.15.67:0/834569176 wait complete. 2024-07-27T03:18:59.386 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.365+0000 7fb078e57740 20 client.15999 trim_cache size 0 max 16384 2024-07-27T03:18:59.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.378+0000 7fb078e57740 1 Processor -- start 2024-07-27T03:18:59.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.378+0000 7fb078e57740 1 -- start start 2024-07-27T03:18:59.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.378+0000 7fb078e57740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e58b3fc1b0 0x55e58b7d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:59.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.378+0000 7fb078e57740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7d7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:59.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.378+0000 7fb078e57740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e58b425070 0x55e58b7e1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:59.392 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.378+0000 7fb078e57740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1b2890 con 0x55e58b404390 2024-07-27T03:18:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.378+0000 7fb078e57740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1b43d0 con 0x55e58b3fc1b0 2024-07-27T03:18:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.378+0000 7fb078e57740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1b2f60 con 0x55e58b425070 2024-07-27T03:18:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.379+0000 7fb06b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7d7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.379+0000 7fb06b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7d7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40454/0 (socket says 172.21.15.67:40454) 2024-07-27T03:18:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.379+0000 7fb06b7fe640 1 -- 172.21.15.67:0/2845577957 learned_addr learned my addr 172.21.15.67:0/2845577957 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.379+0000 7fb070e6f640 1 --2- 172.21.15.67:0/2845577957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e58b425070 0x55e58b7e1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.379+0000 7fb06bfff640 1 --2- 172.21.15.67:0/2845577957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e58b3fc1b0 0x55e58b7d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.379+0000 7fb06b7fe640 1 -- 172.21.15.67:0/2845577957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e58b425070 msgr2=0x55e58b7e1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.379+0000 7fb06b7fe640 1 --2- 172.21.15.67:0/2845577957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e58b425070 0x55e58b7e1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.379+0000 7fb06b7fe640 1 -- 172.21.15.67:0/2845577957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e58b3fc1b0 msgr2=0x55e58b7d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.379+0000 7fb06b7fe640 1 --2- 172.21.15.67:0/2845577957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e58b3fc1b0 0x55e58b7d0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.379+0000 7fb06b7fe640 1 -- 172.21.15.67:0/2845577957 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e58b7cc030 con 0x55e58b404390 2024-07-27T03:18:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.379+0000 7fb06b7fe640 1 --2- 172.21.15.67:0/2845577957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7d7620 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb06000b7b0 tx=0x7fb0600035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50c143463d72984 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.379+0000 7fb051ffb640 1 -- 172.21.15.67:0/2845577957 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060016040 con 0x55e58b404390 2024-07-27T03:18:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.379+0000 7fb051ffb640 1 -- 172.21.15.67:0/2845577957 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb06000b870 con 0x55e58b404390 2024-07-27T03:18:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.379+0000 7fb051ffb640 1 -- 172.21.15.67:0/2845577957 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060003e50 con 0x55e58b404390 2024-07-27T03:18:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.380+0000 7fb078e57740 1 -- 172.21.15.67:0/2845577957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 msgr2=0x55e58b7d7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.380+0000 7fb078e57740 1 --2- 172.21.15.67:0/2845577957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7d7620 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb06000b7b0 tx=0x7fb0600035f0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.380+0000 7fb078e57740 1 -- 172.21.15.67:0/2845577957 shutdown_connections 2024-07-27T03:18:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.380+0000 7fb078e57740 1 --2- 172.21.15.67:0/2845577957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e58b425070 0x55e58b7e1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.380+0000 7fb078e57740 1 --2- 172.21.15.67:0/2845577957 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7d7620 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.380+0000 7fb078e57740 1 --2- 172.21.15.67:0/2845577957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e58b3fc1b0 0x55e58b7d0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.380+0000 7fb078e57740 1 -- 172.21.15.67:0/2845577957 >> 172.21.15.67:0/2845577957 conn(0x55e58b419460 msgr2=0x55e58b7cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.380+0000 7fb078e57740 1 -- 172.21.15.67:0/2845577957 shutdown_connections 2024-07-27T03:18:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.380+0000 7fb078e57740 1 -- 172.21.15.67:0/2845577957 wait complete. 2024-07-27T03:18:59.394 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.381+0000 7fb078e57740 1 -- start start 2024-07-27T03:18:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.381+0000 7fb078e57740 1 Processor -- start 2024-07-27T03:18:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.381+0000 7fb078e57740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e58b3fc1b0 0x55e58b7ddc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.381+0000 7fb078e57740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.381+0000 7fb078e57740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e58b425070 0x55e58b7cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.381+0000 7fb078e57740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1b3d00 con 0x55e58b404390 2024-07-27T03:18:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.381+0000 7fb078e57740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1b3630 con 0x55e58b425070 2024-07-27T03:18:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.381+0000 7fb078e57740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c1b0870 con 0x55e58b3fc1b0 2024-07-27T03:18:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.381+0000 7fb078e57740 10 client.0 _ll_register_callbacks cb 0x7fb064009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:18:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.381+0000 7fb06b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.382+0000 7fb06b7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40464/0 (socket says 172.21.15.67:40464) 2024-07-27T03:18:59.395 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.382+0000 7fb06b7fe640 1 -- 172.21.15.67:0/1590924936 learned_addr learned my addr 172.21.15.67:0/1590924936 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:18:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.382+0000 7fb06bfff640 1 --2- 172.21.15.67:0/1590924936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e58b3fc1b0 0x55e58b7ddc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.382+0000 7fb070e6f640 1 --2- 172.21.15.67:0/1590924936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e58b425070 0x55e58b7cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.382+0000 7fb06b7fe640 1 -- 172.21.15.67:0/1590924936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e58b3fc1b0 msgr2=0x55e58b7ddc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.382+0000 7fb06b7fe640 1 --2- 172.21.15.67:0/1590924936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e58b3fc1b0 0x55e58b7ddc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.382+0000 7fb06b7fe640 1 -- 172.21.15.67:0/1590924936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e58b425070 msgr2=0x55e58b7cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:18:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.382+0000 7fb06b7fe640 1 --2- 172.21.15.67:0/1590924936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e58b425070 0x55e58b7cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.382+0000 7fb06b7fe640 1 -- 172.21.15.67:0/1590924936 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e58b7cf2e0 con 0x55e58b404390 2024-07-27T03:18:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.382+0000 7fb06b7fe640 1 --2- 172.21.15.67:0/1590924936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7ce6b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb06000ba20 tx=0x7fb0600048e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381af6d6841bda5a server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:18:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.382+0000 7fb04b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:18:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.382+0000 7fb04b7fe640 1 -- 172.21.15.67:0/1590924936 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06003f060 con 0x55e58b404390 2024-07-27T03:18:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.382+0000 7fb04b7fe640 1 -- 172.21.15.67:0/1590924936 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fb060005230 con 0x55e58b404390 2024-07-27T03:18:59.396 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.382+0000 7fb078e57740 1 -- 172.21.15.67:0/1590924936 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb06000b870 con 0x55e58b404390 2024-07-27T03:18:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.382+0000 7fb04b7fe640 1 -- 172.21.15.67:0/1590924936 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb060046050 con 0x55e58b404390 2024-07-27T03:18:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.382+0000 7fb078e57740 10 client.16005 fetch_fsmap learned FSMap version 30 2024-07-27T03:18:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.382+0000 7fb078e57740 1 -- 172.21.15.67:0/1590924936 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e58b84fc20 con 0x55e58b404390 2024-07-27T03:18:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.383+0000 7fb04b7fe640 1 -- 172.21.15.67:0/1590924936 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb06000b870 con 0x55e58b404390 2024-07-27T03:18:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.383+0000 7fb04b7fe640 1 -- 172.21.15.67:0/1590924936 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb060022070 con 0x55e58b404390 2024-07-27T03:18:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.383+0000 7fb078e57740 10 client.16005 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:18:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.383+0000 7fb078e57740 10 client.16005 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-07-27T03:18:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.383+0000 7fb078e57740 20 client.16005 populate_metadata read hostname 'smithi067' 2024-07-27T03:18:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.383+0000 7fb078e57740 1 --2- 172.21.15.67:0/1590924936 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55e58b3b8da0 0x55e58b8521a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:18:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.383+0000 7fb078e57740 4 client.16005 mds_command: new command op to 24583 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-27T03:18:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.383+0000 7fb078e57740 1 -- 172.21.15.67:0/1590924936 --> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e58b73e510 con 0x55e58b3b8da0 2024-07-27T03:18:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.383+0000 7fb06bfff640 1 --2- 172.21.15.67:0/1590924936 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55e58b3b8da0 0x55e58b8521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:18:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.384+0000 7fb06bfff640 1 --2- 172.21.15.67:0/1590924936 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55e58b3b8da0 0x55e58b8521a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4246ea3f728ce4 server_cookie=4328878a27141760 in_seq=0 out_seq=0 2024-07-27T03:18:59.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.384+0000 7fb04b7fe640 10 client.16005 ms_handle_connect on v2:172.21.15.175:6838/1670568853 2024-07-27T03:18:59.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.384+0000 7fb04b7fe640 1 -- 172.21.15.67:0/1590924936 <== mds.2 v2:172.21.15.175:6838/1670568853 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e58b73e510 con 0x55e58b3b8da0 2024-07-27T03:18:59.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.384+0000 7fb04b7fe640 10 client.16005 handle_command_reply: tid=0 2024-07-27T03:18:59.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.384+0000 7fb078e57740 1 client.16005 shutdown 2024-07-27T03:18:59.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.384+0000 7fb078e57740 1 -- 172.21.15.67:0/1590924936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 msgr2=0x55e58b7ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:59.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.384+0000 7fb078e57740 1 --2- 172.21.15.67:0/1590924936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7ce6b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb06000ba20 tx=0x7fb0600048e0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.384+0000 7fb078e57740 1 -- 172.21.15.67:0/1590924936 shutdown_connections 2024-07-27T03:18:59.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.384+0000 7fb078e57740 1 --2- 172.21.15.67:0/1590924936 >> [v2:172.21.15.175:6838/1670568853,v1:172.21.15.175:6839/1670568853] conn(0x55e58b3b8da0 0x55e58b8521a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.384+0000 7fb078e57740 1 --2- 172.21.15.67:0/1590924936 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e58b425070 0x55e58b7cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.384+0000 7fb078e57740 1 --2- 172.21.15.67:0/1590924936 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55e58b404390 0x55e58b7ce6b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.384+0000 7fb04b7fe640 0 client.16005 ms_handle_reset on v2:172.21.15.175:6838/1670568853 2024-07-27T03:18:59.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.384+0000 7fb078e57740 1 --2- 172.21.15.67:0/1590924936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55e58b3fc1b0 0x55e58b7ddc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.384+0000 7fb078e57740 1 -- 172.21.15.67:0/1590924936 >> 172.21.15.67:0/1590924936 conn(0x55e58b419460 msgr2=0x55e58b7d75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:59.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.384+0000 7fb078e57740 1 -- 172.21.15.67:0/1590924936 shutdown_connections 2024-07-27T03:18:59.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.384+0000 7fb078e57740 1 -- 172.21.15.67:0/1590924936 wait complete. 2024-07-27T03:18:59.399 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.384+0000 7fb078e57740 20 client.16005 trim_cache size 0 max 16384 2024-07-27T03:18:59.399 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-27T03:18:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.385+0000 7fb0728d2640 1 -- 172.21.15.67:0/1875487839 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb0540b3bd0 msgr2=0x7fb0540b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.385+0000 7fb0728d2640 1 --2- 172.21.15.67:0/1875487839 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb0540b3bd0 0x7fb0540b60e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb064004af0 tx=0x7fb064005da0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.385+0000 7fb0728d2640 1 -- 172.21.15.67:0/1875487839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb06c1024d0 msgr2=0x7fb06c10fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:18:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.385+0000 7fb0728d2640 1 --2- 172.21.15.67:0/1875487839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb06c1024d0 0x7fb06c10fa20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00d950 tx=0x7fb05c00de10 comp rx=0 tx=0).stop 2024-07-27T03:18:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.385+0000 7fb0728d2640 1 -- 172.21.15.67:0/1875487839 shutdown_connections 2024-07-27T03:18:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.385+0000 7fb0728d2640 1 --2- 172.21.15.67:0/1875487839 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fb0540b3bd0 0x7fb0540b60e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.385+0000 7fb0728d2640 1 --2- 172.21.15.67:0/1875487839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb06c108760 0x7fb06c1104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.385+0000 7fb0728d2640 1 --2- 172.21.15.67:0/1875487839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb06c107e10 0x7fb06c10ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.385+0000 7fb0728d2640 1 --2- 172.21.15.67:0/1875487839 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb06c1024d0 0x7fb06c10fa20 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:18:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.385+0000 7fb0728d2640 1 -- 172.21.15.67:0/1875487839 >> 172.21.15.67:0/1875487839 conn(0x7fb06c0791a0 msgr2=0x7fb06c103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:18:59.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.385+0000 7fb0728d2640 1 -- 172.21.15.67:0/1875487839 shutdown_connections 2024-07-27T03:18:59.401 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:18:59.385+0000 7fb0728d2640 1 -- 172.21.15.67:0/1875487839 wait complete. 2024-07-27T03:19:00.186 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-07-27T03:19:00.452 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:19:00.851 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:00 smithi067 ceph-mon[32814]: pgmap v503: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.1 MiB/s wr, 11 op/s 2024-07-27T03:19:01.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:00 smithi149 ceph-mon[37173]: pgmap v503: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.1 MiB/s wr, 11 op/s 2024-07-27T03:19:01.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:00 smithi175 ceph-mon[37125]: pgmap v503: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.1 MiB/s wr, 11 op/s 2024-07-27T03:19:02.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.220+0000 7fa8f204c640 1 -- 172.21.15.67:0/1370835008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8ec108780 msgr2=0x7fa8ec10efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:02.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.220+0000 7fa8f204c640 1 --2- 172.21.15.67:0/1370835008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8ec108780 0x7fa8ec10efb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009ca0 tx=0x7fa8e402a8c0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.220+0000 7fa8f204c640 1 -- 172.21.15.67:0/1370835008 shutdown_connections 2024-07-27T03:19:02.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.220+0000 7fa8f204c640 1 --2- 172.21.15.67:0/1370835008 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8ec108780 0x7fa8ec10efb0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.220+0000 7fa8f204c640 1 --2- 172.21.15.67:0/1370835008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8ec107e30 0x7fa8ec108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.220+0000 7fa8f204c640 1 --2- 172.21.15.67:0/1370835008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8ec1024f0 0x7fa8ec102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.220+0000 7fa8f204c640 1 -- 172.21.15.67:0/1370835008 >> 172.21.15.67:0/1370835008 conn(0x7fa8ec0791b0 msgr2=0x7fa8ec0795c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:02.223 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.220+0000 7fa8f204c640 1 -- 172.21.15.67:0/1370835008 shutdown_connections 2024-07-27T03:19:02.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.220+0000 7fa8f204c640 1 -- 172.21.15.67:0/1370835008 wait complete. 2024-07-27T03:19:02.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.221+0000 7fa8f204c640 1 Processor -- start 2024-07-27T03:19:02.224 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.221+0000 7fa8f204c640 1 -- start start 2024-07-27T03:19:02.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.221+0000 7fa8f204c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8ec1024f0 0x7fa8ec10fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:02.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.221+0000 7fa8f204c640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8ec107e30 0x7fa8ec10ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:02.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.221+0000 7fa8f204c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8ec108780 0x7fa8ec1104c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:02.225 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.221+0000 7fa8f204c640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec113510 con 0x7fa8ec1024f0 2024-07-27T03:19:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.221+0000 7fa8f204c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec113390 con 0x7fa8ec108780 2024-07-27T03:19:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.221+0000 7fa8f204c640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec113690 con 0x7fa8ec107e30 2024-07-27T03:19:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.221+0000 7fa8eb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8ec1024f0 0x7fa8ec10fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.221+0000 7fa8eb7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8ec1024f0 0x7fa8ec10fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40476/0 (socket says 172.21.15.67:40476) 2024-07-27T03:19:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.221+0000 7fa8eb7fe640 1 -- 172.21.15.67:0/1303165450 learned_addr learned my addr 172.21.15.67:0/1303165450 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:19:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.222+0000 7fa8ebfff640 1 --2- 172.21.15.67:0/1303165450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8ec108780 0x7fa8ec1104c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.222+0000 7fa8eaffd640 1 --2- 172.21.15.67:0/1303165450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8ec107e30 0x7fa8ec10ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.222+0000 7fa8eb7fe640 1 -- 172.21.15.67:0/1303165450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8ec107e30 msgr2=0x7fa8ec10ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.222+0000 7fa8eb7fe640 1 --2- 172.21.15.67:0/1303165450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8ec107e30 0x7fa8ec10ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.222+0000 7fa8eb7fe640 1 -- 172.21.15.67:0/1303165450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8ec108780 msgr2=0x7fa8ec1104c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.222+0000 7fa8eb7fe640 1 --2- 172.21.15.67:0/1303165450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8ec108780 0x7fa8ec1104c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.222+0000 7fa8eb7fe640 1 -- 172.21.15.67:0/1303165450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8ec1a81e0 con 0x7fa8ec1024f0 2024-07-27T03:19:02.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.222+0000 7fa8eaffd640 1 --2- 172.21.15.67:0/1303165450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8ec107e30 0x7fa8ec10ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:19:02.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.222+0000 7fa8eb7fe640 1 --2- 172.21.15.67:0/1303165450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8ec1024f0 0x7fa8ec10fa40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00d770 tx=0x7fa8dc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:02.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.222+0000 7fa8e8ff9640 1 -- 172.21.15.67:0/1303165450 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc017070 con 0x7fa8ec1024f0 2024-07-27T03:19:02.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.222+0000 7fa8e8ff9640 1 -- 172.21.15.67:0/1303165450 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8dc0054d0 con 0x7fa8ec1024f0 2024-07-27T03:19:02.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.222+0000 7fa8f204c640 1 -- 172.21.15.67:0/1303165450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8ec1a84d0 con 0x7fa8ec1024f0 2024-07-27T03:19:02.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.222+0000 7fa8e8ff9640 1 -- 172.21.15.67:0/1303165450 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc004c30 con 0x7fa8ec1024f0 2024-07-27T03:19:02.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.222+0000 7fa8f204c640 1 -- 172.21.15.67:0/1303165450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8ec1a89c0 con 0x7fa8ec1024f0 2024-07-27T03:19:02.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.225+0000 7fa8f85d1740 1 Processor -- start 2024-07-27T03:19:02.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.225+0000 7fa8e8ff9640 1 -- 172.21.15.67:0/1303165450 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7fa8dc004e10 con 0x7fa8ec1024f0 2024-07-27T03:19:02.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.225+0000 7fa8f85d1740 1 -- start start 2024-07-27T03:19:02.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.225+0000 7fa8f85d1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780b9070 0x56267845dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:02.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.225+0000 7fa8f85d1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626780901b0 0x56267845e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:02.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.226+0000 7fa8f85d1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562678098390 0x56267845e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:02.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.226+0000 7fa8f85d1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562678346400 con 0x5626780b9070 2024-07-27T03:19:02.229 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.226+0000 7fa8f85d1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5626783d4b20 con 0x5626780901b0 2024-07-27T03:19:02.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.226+0000 7fa8f85d1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5626783c4020 con 0x562678098390 2024-07-27T03:19:02.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.226+0000 7fa8ebfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780b9070 0x56267845dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:02.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.226+0000 7fa8ebfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780b9070 0x56267845dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40486/0 (socket says 172.21.15.67:40486) 2024-07-27T03:19:02.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.226+0000 7fa8ebfff640 1 -- 172.21.15.67:0/4135722745 learned_addr learned my addr 172.21.15.67:0/4135722745 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:19:02.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.226+0000 7fa8eb7fe640 1 --2- 172.21.15.67:0/4135722745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626780901b0 0x56267845e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:02.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.226+0000 7fa8eaffd640 1 --2- 172.21.15.67:0/4135722745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562678098390 0x56267845e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:02.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.226+0000 7fa8ebfff640 1 -- 172.21.15.67:0/4135722745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562678098390 msgr2=0x56267845e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:19:02.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.226+0000 7fa8ebfff640 1 --2- 172.21.15.67:0/4135722745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562678098390 0x56267845e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.226+0000 7fa8ebfff640 1 -- 172.21.15.67:0/4135722745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626780901b0 msgr2=0x56267845e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:19:02.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.226+0000 7fa8ebfff640 1 --2- 172.21.15.67:0/4135722745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626780901b0 0x56267845e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.226+0000 7fa8ebfff640 1 -- 172.21.15.67:0/4135722745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562678359770 con 0x5626780b9070 2024-07-27T03:19:02.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.226+0000 7fa8eb7fe640 1 --2- 172.21.15.67:0/4135722745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626780901b0 0x56267845e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:19:02.230 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.226+0000 7fa8e8ff9640 1 --2- 172.21.15.67:0/1303165450 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa8d40b3bd0 0x7fa8d40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:02.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.226+0000 7fa8ebfff640 1 --2- 172.21.15.67:0/4135722745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780b9070 0x56267845dd40 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40098b0 tx=0x7fa8e40098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49b5b1594870ac0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:02.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.226+0000 7fa8eb7fe640 1 --2- 172.21.15.67:0/1303165450 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa8d40b3bd0 0x7fa8d40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:02.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.227+0000 7fa8e8ff9640 1 -- 172.21.15.67:0/1303165450 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fa8dc0e3b20 con 0x7fa8ec1024f0 2024-07-27T03:19:02.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.227+0000 7fa8d1ffb640 1 -- 172.21.15.67:0/4135722745 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4031cf0 con 0x5626780b9070 2024-07-27T03:19:02.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.227+0000 7fa8d1ffb640 1 -- 172.21.15.67:0/4135722745 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8e4036030 con 0x5626780b9070 2024-07-27T03:19:02.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.227+0000 7fa8d1ffb640 1 -- 172.21.15.67:0/4135722745 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e40042a0 con 0x5626780b9070 2024-07-27T03:19:02.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.228+0000 7fa8f85d1740 1 -- 172.21.15.67:0/4135722745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780b9070 msgr2=0x56267845dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:02.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.228+0000 7fa8f85d1740 1 --2- 172.21.15.67:0/4135722745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780b9070 0x56267845dd40 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40098b0 tx=0x7fa8e40098e0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.228+0000 7fa8eb7fe640 1 --2- 172.21.15.67:0/1303165450 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa8d40b3bd0 0x7fa8d40b60e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0aa360 tx=0x7fa8dc009f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:02.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.228+0000 7fa8f85d1740 1 -- 172.21.15.67:0/4135722745 shutdown_connections 2024-07-27T03:19:02.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.228+0000 7fa8f85d1740 1 --2- 172.21.15.67:0/4135722745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562678098390 0x56267845e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.228+0000 7fa8f85d1740 1 --2- 172.21.15.67:0/4135722745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626780901b0 0x56267845e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.228+0000 7fa8f85d1740 1 --2- 172.21.15.67:0/4135722745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780b9070 0x56267845dd40 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.228+0000 7fa8f85d1740 1 -- 172.21.15.67:0/4135722745 >> 172.21.15.67:0/4135722745 conn(0x5626780ad460 msgr2=0x56267845bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:02.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.228+0000 7fa8f85d1740 1 -- 172.21.15.67:0/4135722745 shutdown_connections 2024-07-27T03:19:02.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.228+0000 7fa8f85d1740 1 -- 172.21.15.67:0/4135722745 wait complete. 2024-07-27T03:19:02.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.228+0000 7fa8f85d1740 1 -- start start 2024-07-27T03:19:02.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.228+0000 7fa8f85d1740 1 Processor -- start 2024-07-27T03:19:02.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.229+0000 7fa8f85d1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780901b0 0x562678469970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:02.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.229+0000 7fa8f85d1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562678098390 0x562678469eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:02.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.229+0000 7fa8f85d1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626780b9070 0x5626784754b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:02.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.229+0000 7fa8f85d1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x562678366900 con 0x5626780901b0 2024-07-27T03:19:02.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.229+0000 7fa8f85d1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x5626783c8060 con 0x5626780b9070 2024-07-27T03:19:02.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.229+0000 7fa8f85d1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x562678227700 con 0x562678098390 2024-07-27T03:19:02.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.229+0000 7fa8f85d1740 10 client.0 _ll_register_callbacks cb 0x56267836ed90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:19:02.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.229+0000 7fa8ebfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780901b0 0x562678469970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:02.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.229+0000 7fa8ebfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780901b0 0x562678469970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40502/0 (socket says 172.21.15.67:40502) 2024-07-27T03:19:02.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.229+0000 7fa8ebfff640 1 -- 172.21.15.67:0/3923350671 learned_addr learned my addr 172.21.15.67:0/3923350671 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:19:02.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.229+0000 7fa8eaffd640 1 --2- 172.21.15.67:0/3923350671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562678098390 0x562678469eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:02.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.229+0000 7fa8ebfff640 1 --2- 172.21.15.67:0/3923350671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626780b9070 0x5626784754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:02.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.229+0000 7fa8ebfff640 1 -- 172.21.15.67:0/3923350671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562678098390 msgr2=0x562678469eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:19:02.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.229+0000 7fa8ebfff640 1 --2- 172.21.15.67:0/3923350671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562678098390 0x562678469eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.229+0000 7fa8ebfff640 1 -- 172.21.15.67:0/3923350671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626780b9070 msgr2=0x5626784754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:19:02.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.229+0000 7fa8ebfff640 1 --2- 172.21.15.67:0/3923350671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626780b9070 0x5626784754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.229+0000 7fa8ebfff640 1 -- 172.21.15.67:0/3923350671 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56267836abf0 con 0x5626780901b0 2024-07-27T03:19:02.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.230+0000 7fa8ebfff640 1 --2- 172.21.15.67:0/3923350671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780901b0 0x562678469970 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009620 tx=0x7fa8e4003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d827f0daece621 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:02.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.230+0000 7fa8caffd640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:19:02.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.230+0000 7fa8caffd640 1 -- 172.21.15.67:0/3923350671 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e402f660 con 0x5626780901b0 2024-07-27T03:19:02.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.230+0000 7fa8caffd640 1 -- 172.21.15.67:0/3923350671 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fa8e402f800 con 0x5626780901b0 2024-07-27T03:19:02.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.230+0000 7fa8f85d1740 1 -- 172.21.15.67:0/3923350671 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8e4036030 con 0x5626780901b0 2024-07-27T03:19:02.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.230+0000 7fa8caffd640 1 -- 172.21.15.67:0/3923350671 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8e405f050 con 0x5626780901b0 2024-07-27T03:19:02.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.230+0000 7fa8f85d1740 10 client.16017 fetch_fsmap learned FSMap version 30 2024-07-27T03:19:02.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.230+0000 7fa8f85d1740 1 -- 172.21.15.67:0/3923350671 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562678124650 con 0x5626780901b0 2024-07-27T03:19:02.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.230+0000 7fa8caffd640 1 -- 172.21.15.67:0/3923350671 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8e4036030 con 0x5626780901b0 2024-07-27T03:19:02.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.230+0000 7fa8caffd640 1 -- 172.21.15.67:0/3923350671 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa8e4043070 con 0x5626780901b0 2024-07-27T03:19:02.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.230+0000 7fa8f85d1740 10 client.16017 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:19:02.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.230+0000 7fa8f85d1740 10 client.16017 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-07-27T03:19:02.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.230+0000 7fa8f85d1740 20 client.16017 populate_metadata read hostname 'smithi067' 2024-07-27T03:19:02.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.231+0000 7fa8f85d1740 1 --2- 172.21.15.67:0/3923350671 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x5626780779f0 0x56267847b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:02.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.231+0000 7fa8f85d1740 4 client.16017 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:19:02.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.231+0000 7fa8f85d1740 1 -- 172.21.15.67:0/3923350671 --> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626783d6a70 con 0x5626780779f0 2024-07-27T03:19:02.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.231+0000 7fa8eaffd640 1 --2- 172.21.15.67:0/3923350671 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x5626780779f0 0x56267847b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:02.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.231+0000 7fa8eaffd640 1 --2- 172.21.15.67:0/3923350671 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x5626780779f0 0x56267847b370 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d4ca28e6756130e3 server_cookie=8e25e49f7af411f5 in_seq=0 out_seq=0 2024-07-27T03:19:02.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.231+0000 7fa8caffd640 10 client.16017 ms_handle_connect on v2:172.21.15.149:6836/2932951845 2024-07-27T03:19:02.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.232+0000 7fa8caffd640 1 -- 172.21.15.67:0/3923350671 <== mds.1 v2:172.21.15.149:6836/2932951845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5626783d6a70 con 0x5626780779f0 2024-07-27T03:19:02.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.232+0000 7fa8caffd640 10 client.16017 handle_command_reply: tid=0 2024-07-27T03:19:02.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.232+0000 7fa8f85d1740 1 client.16017 shutdown 2024-07-27T03:19:02.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.232+0000 7fa8f85d1740 1 -- 172.21.15.67:0/3923350671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780901b0 msgr2=0x562678469970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:02.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.232+0000 7fa8f85d1740 1 --2- 172.21.15.67:0/3923350671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780901b0 0x562678469970 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009620 tx=0x7fa8e4003de0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.232+0000 7fa8f85d1740 1 -- 172.21.15.67:0/3923350671 shutdown_connections 2024-07-27T03:19:02.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.232+0000 7fa8f85d1740 1 --2- 172.21.15.67:0/3923350671 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x5626780779f0 0x56267847b370 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.232+0000 7fa8f85d1740 1 --2- 172.21.15.67:0/3923350671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626780b9070 0x5626784754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.232+0000 7fa8f85d1740 1 --2- 172.21.15.67:0/3923350671 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x562678098390 0x562678469eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.232+0000 7fa8f85d1740 1 --2- 172.21.15.67:0/3923350671 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780901b0 0x562678469970 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.232+0000 7fa8f85d1740 1 -- 172.21.15.67:0/3923350671 >> 172.21.15.67:0/3923350671 conn(0x5626780ad460 msgr2=0x562678460f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:02.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.232+0000 7fa8caffd640 0 client.16017 ms_handle_reset on v2:172.21.15.149:6836/2932951845 2024-07-27T03:19:02.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.232+0000 7fa8f85d1740 1 -- 172.21.15.67:0/3923350671 shutdown_connections 2024-07-27T03:19:02.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.232+0000 7fa8f85d1740 1 -- 172.21.15.67:0/3923350671 wait complete. 2024-07-27T03:19:02.239 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.233+0000 7fa8f85d1740 20 client.16017 trim_cache size 0 max 16384 2024-07-27T03:19:02.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.245+0000 7fa8f85d1740 1 Processor -- start 2024-07-27T03:19:02.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.246+0000 7fa8f85d1740 1 -- start start 2024-07-27T03:19:02.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.246+0000 7fa8f85d1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626780901b0 0x562678475420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:02.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.246+0000 7fa8f85d1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562678098390 0x562678475960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:02.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.246+0000 7fa8f85d1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780b9070 0x562678462e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:02.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.246+0000 7fa8f85d1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec1b2750 con 0x5626780b9070 2024-07-27T03:19:02.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.246+0000 7fa8f85d1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec1b4290 con 0x562678098390 2024-07-27T03:19:02.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.246+0000 7fa8f85d1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec1b2e20 con 0x5626780901b0 2024-07-27T03:19:02.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.246+0000 7fa8ebfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780b9070 0x562678462e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:02.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.246+0000 7fa8ebfff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780b9070 0x562678462e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40508/0 (socket says 172.21.15.67:40508) 2024-07-27T03:19:02.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.246+0000 7fa8ebfff640 1 -- 172.21.15.67:0/1241758450 learned_addr learned my addr 172.21.15.67:0/1241758450 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:19:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.246+0000 7fa8ebfff640 1 --2- 172.21.15.67:0/1241758450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626780901b0 0x562678475420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.246+0000 7fa8eaffd640 1 --2- 172.21.15.67:0/1241758450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562678098390 0x562678475960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.247+0000 7fa8ebfff640 1 -- 172.21.15.67:0/1241758450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626780901b0 msgr2=0x562678475420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:19:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.247+0000 7fa8ebfff640 1 --2- 172.21.15.67:0/1241758450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626780901b0 0x562678475420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.247+0000 7fa8ebfff640 1 -- 172.21.15.67:0/1241758450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562678098390 msgr2=0x562678475960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:19:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.247+0000 7fa8ebfff640 1 --2- 172.21.15.67:0/1241758450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562678098390 0x562678475960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.247+0000 7fa8ebfff640 1 -- 172.21.15.67:0/1241758450 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56267846aa30 con 0x5626780b9070 2024-07-27T03:19:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.247+0000 7fa8ebfff640 1 --2- 172.21.15.67:0/1241758450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780b9070 0x562678462e50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa8e405d040 tx=0x7fa8e402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7623e71885a41802 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:02.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.247+0000 7fa8d17fa640 1 -- 172.21.15.67:0/1241758450 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4031920 con 0x5626780b9070 2024-07-27T03:19:02.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.247+0000 7fa8d17fa640 1 -- 172.21.15.67:0/1241758450 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8e4036040 con 0x5626780b9070 2024-07-27T03:19:02.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.247+0000 7fa8d17fa640 1 -- 172.21.15.67:0/1241758450 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e402fb60 con 0x5626780b9070 2024-07-27T03:19:02.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.248+0000 7fa8f85d1740 1 -- 172.21.15.67:0/1241758450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780b9070 msgr2=0x562678462e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:02.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.248+0000 7fa8f85d1740 1 --2- 172.21.15.67:0/1241758450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780b9070 0x562678462e50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa8e405d040 tx=0x7fa8e402a8c0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.248+0000 7fa8f85d1740 1 -- 172.21.15.67:0/1241758450 shutdown_connections 2024-07-27T03:19:02.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.248+0000 7fa8f85d1740 1 --2- 172.21.15.67:0/1241758450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780b9070 0x562678462e50 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.248+0000 7fa8f85d1740 1 --2- 172.21.15.67:0/1241758450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562678098390 0x562678475960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.248+0000 7fa8f85d1740 1 --2- 172.21.15.67:0/1241758450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626780901b0 0x562678475420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.248+0000 7fa8f85d1740 1 -- 172.21.15.67:0/1241758450 >> 172.21.15.67:0/1241758450 conn(0x5626780ad460 msgr2=0x562678467ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:02.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.248+0000 7fa8f85d1740 1 -- 172.21.15.67:0/1241758450 shutdown_connections 2024-07-27T03:19:02.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.248+0000 7fa8f85d1740 1 -- 172.21.15.67:0/1241758450 wait complete. 2024-07-27T03:19:02.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.249+0000 7fa8f85d1740 1 -- start start 2024-07-27T03:19:02.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.249+0000 7fa8f85d1740 1 Processor -- start 2024-07-27T03:19:02.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.249+0000 7fa8f85d1740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626780901b0 0x56267847ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:02.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.249+0000 7fa8f85d1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562678098390 0x56267847bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:02.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.249+0000 7fa8f85d1740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780b9070 0x5626784e18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:02.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.249+0000 7fa8f85d1740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec1b3bc0 con 0x5626780b9070 2024-07-27T03:19:02.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.249+0000 7fa8f85d1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec1b34f0 con 0x562678098390 2024-07-27T03:19:02.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.249+0000 7fa8f85d1740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec1b0730 con 0x5626780901b0 2024-07-27T03:19:02.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.250+0000 7fa8f85d1740 10 client.0 _ll_register_callbacks cb 0x7fa8e4009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:19:02.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.250+0000 7fa8ebfff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626780901b0 0x56267847ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:02.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.250+0000 7fa8ebfff640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626780901b0 0x56267847ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.67:34796/0 (socket says 172.21.15.67:34796) 2024-07-27T03:19:02.254 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.250+0000 7fa8ebfff640 1 -- 172.21.15.67:0/4176962504 learned_addr learned my addr 172.21.15.67:0/4176962504 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:19:02.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.250+0000 7fa8ebfff640 1 --2- 172.21.15.67:0/4176962504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780b9070 0x5626784e18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:02.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.250+0000 7fa8ebfff640 1 -- 172.21.15.67:0/4176962504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562678098390 msgr2=0x56267847bf50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-07-27T03:19:02.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.250+0000 7fa8ebfff640 1 --2- 172.21.15.67:0/4176962504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562678098390 0x56267847bf50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.250+0000 7fa8ebfff640 1 -- 172.21.15.67:0/4176962504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780b9070 msgr2=0x5626784e18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:19:02.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.250+0000 7fa8ebfff640 1 --2- 172.21.15.67:0/4176962504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780b9070 0x5626784e18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.250+0000 7fa8ebfff640 1 -- 172.21.15.67:0/4176962504 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626784e1e40 con 0x5626780901b0 2024-07-27T03:19:02.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.250+0000 7fa8ebfff640 1 --2- 172.21.15.67:0/4176962504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626780901b0 0x56267847ba10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa8e402f590 tx=0x7fa8e402f310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43b5041d336f536e server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:02.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.250+0000 7fa8caffd640 10 client.0 ms_handle_connect on v2:172.21.15.175:3300/0 2024-07-27T03:19:02.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.251+0000 7fa8caffd640 1 -- 172.21.15.67:0/4176962504 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4067070 con 0x5626780901b0 2024-07-27T03:19:02.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.251+0000 7fa8caffd640 1 -- 172.21.15.67:0/4176962504 <== mon.1 v2:172.21.15.175:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fa8e4003de0 con 0x5626780901b0 2024-07-27T03:19:02.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.251+0000 7fa8f85d1740 1 -- 172.21.15.67:0/4176962504 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8e4036040 con 0x5626780901b0 2024-07-27T03:19:02.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.251+0000 7fa8caffd640 1 -- 172.21.15.67:0/4176962504 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8e406a040 con 0x5626780901b0 2024-07-27T03:19:02.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.251+0000 7fa8f85d1740 10 client.25606 fetch_fsmap learned FSMap version 30 2024-07-27T03:19:02.255 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.251+0000 7fa8f85d1740 1 -- 172.21.15.67:0/4176962504 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5626784e30c0 con 0x5626780901b0 2024-07-27T03:19:02.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.251+0000 7fa8caffd640 1 -- 172.21.15.67:0/4176962504 <== mon.1 v2:172.21.15.175:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8e4056070 con 0x5626780901b0 2024-07-27T03:19:02.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.251+0000 7fa8caffd640 1 -- 172.21.15.67:0/4176962504 <== mon.1 v2:172.21.15.175:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa8e4043040 con 0x5626780901b0 2024-07-27T03:19:02.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.251+0000 7fa8f85d1740 10 client.25606 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:19:02.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.251+0000 7fa8f85d1740 10 client.25606 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-07-27T03:19:02.259 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.252+0000 7fa8f85d1740 20 client.25606 populate_metadata read hostname 'smithi067' 2024-07-27T03:19:02.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.252+0000 7fa8f85d1740 1 --2- 172.21.15.67:0/4176962504 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x56267804cda0 0x5626784e5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:02.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.252+0000 7fa8f85d1740 4 client.25606 mds_command: new command op to 24587 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-27T03:19:02.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.252+0000 7fa8f85d1740 1 -- 172.21.15.67:0/4176962504 --> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa8e4036040 con 0x56267804cda0 2024-07-27T03:19:02.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.252+0000 7fa8eaffd640 1 --2- 172.21.15.67:0/4176962504 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x56267804cda0 0x5626784e5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:02.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.252+0000 7fa8caffd640 10 client.25606 ms_handle_connect on v2:172.21.15.149:6836/2932951845 2024-07-27T03:19:02.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.252+0000 7fa8eaffd640 1 --2- 172.21.15.67:0/4176962504 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x56267804cda0 0x5626784e5790 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3fc60a90c6e27d8 server_cookie=2fa9b7c5a019e3c7 in_seq=0 out_seq=0 2024-07-27T03:19:02.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.253+0000 7fa8caffd640 1 -- 172.21.15.67:0/4176962504 <== mds.1 v2:172.21.15.149:6836/2932951845 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa8e4036040 con 0x56267804cda0 2024-07-27T03:19:02.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.253+0000 7fa8caffd640 10 client.25606 handle_command_reply: tid=0 2024-07-27T03:19:02.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.253+0000 7fa8f85d1740 1 client.25606 shutdown 2024-07-27T03:19:02.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.253+0000 7fa8f85d1740 1 -- 172.21.15.67:0/4176962504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626780901b0 msgr2=0x56267847ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:02.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.253+0000 7fa8f85d1740 1 --2- 172.21.15.67:0/4176962504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626780901b0 0x56267847ba10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa8e402f590 tx=0x7fa8e402f310 comp rx=0 tx=0).stop 2024-07-27T03:19:02.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.253+0000 7fa8f85d1740 1 -- 172.21.15.67:0/4176962504 shutdown_connections 2024-07-27T03:19:02.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.253+0000 7fa8f85d1740 1 --2- 172.21.15.67:0/4176962504 >> [v2:172.21.15.149:6836/2932951845,v1:172.21.15.149:6837/2932951845] conn(0x56267804cda0 0x5626784e5790 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.253+0000 7fa8f85d1740 1 --2- 172.21.15.67:0/4176962504 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x5626780b9070 0x5626784e18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.253+0000 7fa8f85d1740 1 --2- 172.21.15.67:0/4176962504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562678098390 0x56267847bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.253+0000 7fa8f85d1740 1 --2- 172.21.15.67:0/4176962504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5626780901b0 0x56267847ba10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.253+0000 7fa8caffd640 0 client.25606 ms_handle_reset on v2:172.21.15.149:6836/2932951845 2024-07-27T03:19:02.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.253+0000 7fa8f85d1740 1 -- 172.21.15.67:0/4176962504 >> 172.21.15.67:0/4176962504 conn(0x5626780ad460 msgr2=0x562678467bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:02.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.253+0000 7fa8f85d1740 1 -- 172.21.15.67:0/4176962504 shutdown_connections 2024-07-27T03:19:02.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.253+0000 7fa8f85d1740 1 -- 172.21.15.67:0/4176962504 wait complete. 2024-07-27T03:19:02.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.253+0000 7fa8f85d1740 20 client.25606 trim_cache size 0 max 16384 2024-07-27T03:19:02.262 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-27T03:19:02.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.254+0000 7fa8f204c640 1 -- 172.21.15.67:0/1303165450 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa8d40b3bd0 msgr2=0x7fa8d40b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:02.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.254+0000 7fa8f204c640 1 --2- 172.21.15.67:0/1303165450 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa8d40b3bd0 0x7fa8d40b60e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0aa360 tx=0x7fa8dc009f90 comp rx=0 tx=0).stop 2024-07-27T03:19:02.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.254+0000 7fa8f204c640 1 -- 172.21.15.67:0/1303165450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8ec1024f0 msgr2=0x7fa8ec10fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:02.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.254+0000 7fa8f204c640 1 --2- 172.21.15.67:0/1303165450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8ec1024f0 0x7fa8ec10fa40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc00d770 tx=0x7fa8dc00dc30 comp rx=0 tx=0).stop 2024-07-27T03:19:02.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.254+0000 7fa8f204c640 1 -- 172.21.15.67:0/1303165450 shutdown_connections 2024-07-27T03:19:02.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.254+0000 7fa8f204c640 1 --2- 172.21.15.67:0/1303165450 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa8d40b3bd0 0x7fa8d40b60e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.254+0000 7fa8f204c640 1 --2- 172.21.15.67:0/1303165450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8ec108780 0x7fa8ec1104c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.254+0000 7fa8f204c640 1 --2- 172.21.15.67:0/1303165450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8ec107e30 0x7fa8ec10ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.254+0000 7fa8f204c640 1 --2- 172.21.15.67:0/1303165450 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa8ec1024f0 0x7fa8ec10fa40 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:02.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.254+0000 7fa8f204c640 1 -- 172.21.15.67:0/1303165450 >> 172.21.15.67:0/1303165450 conn(0x7fa8ec0791b0 msgr2=0x7fa8ec103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:02.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.254+0000 7fa8f204c640 1 -- 172.21.15.67:0/1303165450 shutdown_connections 2024-07-27T03:19:02.263 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:02.254+0000 7fa8f204c640 1 -- 172.21.15.67:0/1303165450 wait complete. 2024-07-27T03:19:02.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:02 smithi067 ceph-mon[32814]: pgmap v504: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 692 KiB/s wr, 15 op/s 2024-07-27T03:19:02.905 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-07-27T03:19:03.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:02 smithi149 ceph-mon[37173]: pgmap v504: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 692 KiB/s wr, 15 op/s 2024-07-27T03:19:03.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:02 smithi175 ceph-mon[37125]: pgmap v504: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 692 KiB/s wr, 15 op/s 2024-07-27T03:19:03.183 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:19:04.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:04 smithi067 ceph-mon[32814]: pgmap v505: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 689 KiB/s wr, 11 op/s 2024-07-27T03:19:04.926 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:04 smithi175 ceph-mon[37125]: pgmap v505: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 689 KiB/s wr, 11 op/s 2024-07-27T03:19:04.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:04 smithi149 ceph-mon[37173]: pgmap v505: 129 pgs: 129 active+clean; 258 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 689 KiB/s wr, 11 op/s 2024-07-27T03:19:05.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.197+0000 7efe53e05640 1 -- 172.21.15.67:0/3100617447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe4c108f80 msgr2=0x7efe4c109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:05.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.197+0000 7efe53e05640 1 --2- 172.21.15.67:0/3100617447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe4c108f80 0x7efe4c109390 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009900 tx=0x7efe3c02a410 comp rx=0 tx=0).stop 2024-07-27T03:19:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.197+0000 7efe53e05640 1 -- 172.21.15.67:0/3100617447 shutdown_connections 2024-07-27T03:19:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.197+0000 7efe53e05640 1 --2- 172.21.15.67:0/3100617447 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe4c1098d0 0x7efe4c110190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.197+0000 7efe53e05640 1 --2- 172.21.15.67:0/3100617447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe4c108f80 0x7efe4c109390 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.197+0000 7efe53e05640 1 --2- 172.21.15.67:0/3100617447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe4c101ad0 0x7efe4c101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.197+0000 7efe53e05640 1 -- 172.21.15.67:0/3100617447 >> 172.21.15.67:0/3100617447 conn(0x7efe4c079a00 msgr2=0x7efe4c079e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.197+0000 7efe53e05640 1 -- 172.21.15.67:0/3100617447 shutdown_connections 2024-07-27T03:19:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.197+0000 7efe53e05640 1 -- 172.21.15.67:0/3100617447 wait complete. 2024-07-27T03:19:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.198+0000 7efe53e05640 1 Processor -- start 2024-07-27T03:19:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.198+0000 7efe53e05640 1 -- start start 2024-07-27T03:19:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.198+0000 7efe53e05640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe4c101ad0 0x7efe4c102a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.198+0000 7efe53e05640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe4c108f80 0x7efe4c102fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.198+0000 7efe53e05640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe4c1098d0 0x7efe4c1034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.198+0000 7efe53e05640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c113550 con 0x7efe4c101ad0 2024-07-27T03:19:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.198+0000 7efe53e05640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c1133d0 con 0x7efe4c1098d0 2024-07-27T03:19:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.198+0000 7efe53e05640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c1136d0 con 0x7efe4c108f80 2024-07-27T03:19:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.198+0000 7efe51ba1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe4c101ad0 0x7efe4c102a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.198+0000 7efe51ba1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe4c101ad0 0x7efe4c102a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42078/0 (socket says 172.21.15.67:42078) 2024-07-27T03:19:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.198+0000 7efe51ba1640 1 -- 172.21.15.67:0/2117176682 learned_addr learned my addr 172.21.15.67:0/2117176682 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:19:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.198+0000 7efe523a2640 1 --2- 172.21.15.67:0/2117176682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe4c1098d0 0x7efe4c1034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:05.203 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.198+0000 7efe513a0640 1 --2- 172.21.15.67:0/2117176682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe4c108f80 0x7efe4c102fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.199+0000 7efe51ba1640 1 -- 172.21.15.67:0/2117176682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe4c108f80 msgr2=0x7efe4c102fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.199+0000 7efe51ba1640 1 --2- 172.21.15.67:0/2117176682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe4c108f80 0x7efe4c102fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.199+0000 7efe51ba1640 1 -- 172.21.15.67:0/2117176682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe4c1098d0 msgr2=0x7efe4c1034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.199+0000 7efe51ba1640 1 --2- 172.21.15.67:0/2117176682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe4c1098d0 0x7efe4c1034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.199+0000 7efe51ba1640 1 -- 172.21.15.67:0/2117176682 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4c1a8170 con 0x7efe4c101ad0 2024-07-27T03:19:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.199+0000 7efe51ba1640 1 --2- 172.21.15.67:0/2117176682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe4c101ad0 0x7efe4c102a70 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7efe4800c920 tx=0x7efe4800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.199+0000 7efe42ffd640 1 -- 172.21.15.67:0/2117176682 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48007c70 con 0x7efe4c101ad0 2024-07-27T03:19:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.199+0000 7efe42ffd640 1 -- 172.21.15.67:0/2117176682 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe48007e10 con 0x7efe4c101ad0 2024-07-27T03:19:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.199+0000 7efe42ffd640 1 -- 172.21.15.67:0/2117176682 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48018820 con 0x7efe4c101ad0 2024-07-27T03:19:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.199+0000 7efe53e05640 1 -- 172.21.15.67:0/2117176682 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe4c1a8460 con 0x7efe4c101ad0 2024-07-27T03:19:05.204 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.199+0000 7efe53e05640 1 -- 172.21.15.67:0/2117176682 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe4c1a89c0 con 0x7efe4c101ad0 2024-07-27T03:19:05.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.202+0000 7efe42ffd640 1 -- 172.21.15.67:0/2117176682 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7efe4800f450 con 0x7efe4c101ad0 2024-07-27T03:19:05.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.202+0000 7efe5a38a740 1 Processor -- start 2024-07-27T03:19:05.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.202+0000 7efe5a38a740 1 -- start start 2024-07-27T03:19:05.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.203+0000 7efe5a38a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd7326c070 0x55fd73610e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:05.206 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.203+0000 7efe42ffd640 1 --2- 172.21.15.67:0/2117176682 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7efe380b3c50 0x7efe380b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:05.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.203+0000 7efe5a38a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd732431b0 0x55fd73611370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:05.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.203+0000 7efe5a38a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd7324b390 0x55fd736118b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:05.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.203+0000 7efe5a38a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fd734f9400 con 0x55fd7326c070 2024-07-27T03:19:05.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.203+0000 7efe5a38a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fd73587b20 con 0x55fd7324b390 2024-07-27T03:19:05.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.203+0000 7efe5a38a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fd73577020 con 0x55fd732431b0 2024-07-27T03:19:05.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.203+0000 7efe523a2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd7326c070 0x55fd73610e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:05.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.203+0000 7efe51ba1640 1 --2- 172.21.15.67:0/2117176682 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7efe380b3c50 0x7efe380b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:05.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.203+0000 7efe523a2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd7326c070 0x55fd73610e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42088/0 (socket says 172.21.15.67:42088) 2024-07-27T03:19:05.207 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.203+0000 7efe523a2640 1 -- 172.21.15.67:0/48053850 learned_addr learned my addr 172.21.15.67:0/48053850 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:19:05.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.203+0000 7efe42ffd640 1 -- 172.21.15.67:0/2117176682 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7efe480e4130 con 0x7efe4c101ad0 2024-07-27T03:19:05.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.203+0000 7efe513a0640 1 --2- 172.21.15.67:0/48053850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd732431b0 0x55fd73611370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:05.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.203+0000 7efe523a2640 1 --2- 172.21.15.67:0/48053850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd7324b390 0x55fd736118b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:05.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.203+0000 7efe523a2640 1 -- 172.21.15.67:0/48053850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd732431b0 msgr2=0x55fd73611370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:19:05.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.203+0000 7efe523a2640 1 --2- 172.21.15.67:0/48053850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd732431b0 0x55fd73611370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.203+0000 7efe523a2640 1 -- 172.21.15.67:0/48053850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd7324b390 msgr2=0x55fd736118b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:19:05.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.203+0000 7efe523a2640 1 --2- 172.21.15.67:0/48053850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd7324b390 0x55fd736118b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.203+0000 7efe523a2640 1 -- 172.21.15.67:0/48053850 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd7350c770 con 0x55fd7326c070 2024-07-27T03:19:05.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.204+0000 7efe51ba1640 1 --2- 172.21.15.67:0/2117176682 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7efe380b3c50 0x7efe380b6160 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efe480aa9d0 tx=0x7efe4806ffe0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:05.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.204+0000 7efe523a2640 1 --2- 172.21.15.67:0/48053850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd7326c070 0x55fd73610e30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7efe44009fd0 tx=0x7efe440098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613af0b24436556e server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:05.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.204+0000 7efe37fff640 1 -- 172.21.15.67:0/48053850 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4401a070 con 0x55fd7326c070 2024-07-27T03:19:05.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.204+0000 7efe37fff640 1 -- 172.21.15.67:0/48053850 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe44004d20 con 0x55fd7326c070 2024-07-27T03:19:05.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.204+0000 7efe37fff640 1 -- 172.21.15.67:0/48053850 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe440054c0 con 0x55fd7326c070 2024-07-27T03:19:05.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.205+0000 7efe5a38a740 1 -- 172.21.15.67:0/48053850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd7326c070 msgr2=0x55fd73610e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:05.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.205+0000 7efe5a38a740 1 --2- 172.21.15.67:0/48053850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd7326c070 0x55fd73610e30 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7efe44009fd0 tx=0x7efe440098f0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.205+0000 7efe5a38a740 1 -- 172.21.15.67:0/48053850 shutdown_connections 2024-07-27T03:19:05.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.205+0000 7efe5a38a740 1 --2- 172.21.15.67:0/48053850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd7324b390 0x55fd736118b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.205+0000 7efe5a38a740 1 --2- 172.21.15.67:0/48053850 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd732431b0 0x55fd73611370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.205+0000 7efe5a38a740 1 --2- 172.21.15.67:0/48053850 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd7326c070 0x55fd73610e30 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.205+0000 7efe5a38a740 1 -- 172.21.15.67:0/48053850 >> 172.21.15.67:0/48053850 conn(0x55fd73260460 msgr2=0x55fd7360ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:05.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.205+0000 7efe5a38a740 1 -- 172.21.15.67:0/48053850 shutdown_connections 2024-07-27T03:19:05.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.205+0000 7efe5a38a740 1 -- 172.21.15.67:0/48053850 wait complete. 2024-07-27T03:19:05.212 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.205+0000 7efe5a38a740 1 -- start start 2024-07-27T03:19:05.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.205+0000 7efe5a38a740 1 Processor -- start 2024-07-27T03:19:05.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.206+0000 7efe5a38a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd732431b0 0x55fd7361ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:05.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.206+0000 7efe5a38a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd7324b390 0x55fd7361cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:05.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.206+0000 7efe5a38a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd7326c070 0x55fd73628780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:05.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.206+0000 7efe5a38a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x55fd73519900 con 0x55fd7326c070 2024-07-27T03:19:05.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.206+0000 7efe5a38a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55fd7357b060 con 0x55fd732431b0 2024-07-27T03:19:05.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.206+0000 7efe5a38a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55fd733da700 con 0x55fd7324b390 2024-07-27T03:19:05.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.206+0000 7efe5a38a740 10 client.0 _ll_register_callbacks cb 0x55fd73521d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:19:05.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.206+0000 7efe523a2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd7326c070 0x55fd73628780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:05.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.206+0000 7efe523a2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd7326c070 0x55fd73628780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42100/0 (socket says 172.21.15.67:42100) 2024-07-27T03:19:05.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.206+0000 7efe523a2640 1 -- 172.21.15.67:0/121024615 learned_addr learned my addr 172.21.15.67:0/121024615 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:19:05.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.206+0000 7efe513a0640 1 --2- 172.21.15.67:0/121024615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd7324b390 0x55fd7361cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:05.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.206+0000 7efe523a2640 1 --2- 172.21.15.67:0/121024615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd732431b0 0x55fd7361ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:05.213 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.206+0000 7efe523a2640 1 -- 172.21.15.67:0/121024615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd7324b390 msgr2=0x55fd7361cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:19:05.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.206+0000 7efe523a2640 1 --2- 172.21.15.67:0/121024615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd7324b390 0x55fd7361cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.206+0000 7efe523a2640 1 -- 172.21.15.67:0/121024615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd732431b0 msgr2=0x55fd7361ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:19:05.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.206+0000 7efe523a2640 1 --2- 172.21.15.67:0/121024615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd732431b0 0x55fd7361ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.207+0000 7efe523a2640 1 -- 172.21.15.67:0/121024615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd7351dbf0 con 0x55fd7326c070 2024-07-27T03:19:05.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.207+0000 7efe523a2640 1 --2- 172.21.15.67:0/121024615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd7326c070 0x55fd73628780 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7efe44014fc0 tx=0x7efe44005160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1694054445aeb29 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:05.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.207+0000 7efe357fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:19:05.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.207+0000 7efe357fa640 1 -- 172.21.15.67:0/121024615 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44022070 con 0x55fd7326c070 2024-07-27T03:19:05.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.207+0000 7efe357fa640 1 -- 172.21.15.67:0/121024615 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7efe44012aa0 con 0x55fd7326c070 2024-07-27T03:19:05.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.207+0000 7efe5a38a740 1 -- 172.21.15.67:0/121024615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe44004d20 con 0x55fd7326c070 2024-07-27T03:19:05.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.207+0000 7efe357fa640 1 -- 172.21.15.67:0/121024615 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efe44034050 con 0x55fd7326c070 2024-07-27T03:19:05.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.207+0000 7efe5a38a740 10 client.16035 fetch_fsmap learned FSMap version 30 2024-07-27T03:19:05.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.207+0000 7efe5a38a740 1 -- 172.21.15.67:0/121024615 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fd732d7650 con 0x55fd7326c070 2024-07-27T03:19:05.214 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.207+0000 7efe357fa640 1 -- 172.21.15.67:0/121024615 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe44004d20 con 0x55fd7326c070 2024-07-27T03:19:05.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.207+0000 7efe357fa640 1 -- 172.21.15.67:0/121024615 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7efe4400f040 con 0x55fd7326c070 2024-07-27T03:19:05.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.208+0000 7efe5a38a740 10 client.16035 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:19:05.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.208+0000 7efe5a38a740 10 client.16035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:19:05.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.208+0000 7efe5a38a740 20 client.16035 populate_metadata read hostname 'smithi067' 2024-07-27T03:19:05.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.208+0000 7efe5a38a740 1 --2- 172.21.15.67:0/121024615 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55fd7322a9f0 0x55fd7362e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:05.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.208+0000 7efe5a38a740 4 client.16035 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-07-27T03:19:05.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.208+0000 7efe5a38a740 1 -- 172.21.15.67:0/121024615 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd73589a70 con 0x55fd7322a9f0 2024-07-27T03:19:05.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.208+0000 7efe513a0640 1 --2- 172.21.15.67:0/121024615 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55fd7322a9f0 0x55fd7362e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:05.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.208+0000 7efe357fa640 10 client.16035 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:19:05.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.208+0000 7efe513a0640 1 --2- 172.21.15.67:0/121024615 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55fd7322a9f0 0x55fd7362e6a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0618741c46b55ba server_cookie=4c10851170dfc02e in_seq=0 out_seq=0 2024-07-27T03:19:05.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.209+0000 7efe357fa640 1 -- 172.21.15.67:0/121024615 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd73589a70 con 0x55fd7322a9f0 2024-07-27T03:19:05.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.209+0000 7efe357fa640 10 client.16035 handle_command_reply: tid=0 2024-07-27T03:19:05.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.209+0000 7efe5a38a740 1 client.16035 shutdown 2024-07-27T03:19:05.215 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.209+0000 7efe5a38a740 1 -- 172.21.15.67:0/121024615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd7326c070 msgr2=0x55fd73628780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.209+0000 7efe5a38a740 1 --2- 172.21.15.67:0/121024615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd7326c070 0x55fd73628780 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7efe44014fc0 tx=0x7efe44005160 comp rx=0 tx=0).stop 2024-07-27T03:19:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.209+0000 7efe5a38a740 1 -- 172.21.15.67:0/121024615 shutdown_connections 2024-07-27T03:19:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.209+0000 7efe5a38a740 1 --2- 172.21.15.67:0/121024615 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55fd7322a9f0 0x55fd7362e6a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.209+0000 7efe5a38a740 1 --2- 172.21.15.67:0/121024615 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd7326c070 0x55fd73628780 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.209+0000 7efe5a38a740 1 --2- 172.21.15.67:0/121024615 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd7324b390 0x55fd7361cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.209+0000 7efe5a38a740 1 --2- 172.21.15.67:0/121024615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd732431b0 0x55fd7361ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.209+0000 7efe357fa640 0 client.16035 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:19:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.209+0000 7efe5a38a740 1 -- 172.21.15.67:0/121024615 >> 172.21.15.67:0/121024615 conn(0x55fd73260460 msgr2=0x55fd73614010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.209+0000 7efe5a38a740 1 -- 172.21.15.67:0/121024615 shutdown_connections 2024-07-27T03:19:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.210+0000 7efe5a38a740 1 -- 172.21.15.67:0/121024615 wait complete. 2024-07-27T03:19:05.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.210+0000 7efe5a38a740 20 client.16035 trim_cache size 0 max 16384 2024-07-27T03:19:05.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.223+0000 7efe5a38a740 1 Processor -- start 2024-07-27T03:19:05.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.223+0000 7efe5a38a740 1 -- start start 2024-07-27T03:19:05.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.223+0000 7efe5a38a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd732431b0 0x55fd7361e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:05.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.223+0000 7efe5a38a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd7324b390 0x55fd73628570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:05.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.223+0000 7efe5a38a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd7326c070 0x55fd73628ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:05.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.223+0000 7efe5a38a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c1b2860 con 0x55fd7326c070 2024-07-27T03:19:05.226 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.223+0000 7efe5a38a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c1b43a0 con 0x55fd732431b0 2024-07-27T03:19:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.223+0000 7efe5a38a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c1b2f30 con 0x55fd7324b390 2024-07-27T03:19:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.223+0000 7efe523a2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd7326c070 0x55fd73628ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.223+0000 7efe523a2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd7326c070 0x55fd73628ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42116/0 (socket says 172.21.15.67:42116) 2024-07-27T03:19:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.223+0000 7efe523a2640 1 -- 172.21.15.67:0/1453788286 learned_addr learned my addr 172.21.15.67:0/1453788286 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:19:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.223+0000 7efe513a0640 1 --2- 172.21.15.67:0/1453788286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd7324b390 0x55fd73628570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.223+0000 7efe523a2640 1 -- 172.21.15.67:0/1453788286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd7324b390 msgr2=0x55fd73628570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:19:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.223+0000 7efe523a2640 1 --2- 172.21.15.67:0/1453788286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd7324b390 0x55fd73628570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.223+0000 7efe523a2640 1 -- 172.21.15.67:0/1453788286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd732431b0 msgr2=0x55fd7361e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:19:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.223+0000 7efe523a2640 1 --2- 172.21.15.67:0/1453788286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd732431b0 0x55fd7361e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.223+0000 7efe523a2640 1 -- 172.21.15.67:0/1453788286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd73613120 con 0x55fd7326c070 2024-07-27T03:19:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.224+0000 7efe523a2640 1 --2- 172.21.15.67:0/1453788286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd7326c070 0x55fd73628ab0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7efe44002230 tx=0x7efe440096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6375711d58d637a5 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.224+0000 7efe377fe640 1 -- 172.21.15.67:0/1453788286 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44031020 con 0x55fd7326c070 2024-07-27T03:19:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.224+0000 7efe377fe640 1 -- 172.21.15.67:0/1453788286 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe44014620 con 0x55fd7326c070 2024-07-27T03:19:05.227 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.224+0000 7efe377fe640 1 -- 172.21.15.67:0/1453788286 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44022070 con 0x55fd7326c070 2024-07-27T03:19:05.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.224+0000 7efe5a38a740 1 -- 172.21.15.67:0/1453788286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd7326c070 msgr2=0x55fd73628ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:05.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.224+0000 7efe5a38a740 1 --2- 172.21.15.67:0/1453788286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd7326c070 0x55fd73628ab0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7efe44002230 tx=0x7efe440096e0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.225+0000 7efe5a38a740 1 -- 172.21.15.67:0/1453788286 shutdown_connections 2024-07-27T03:19:05.231 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.225+0000 7efe5a38a740 1 --2- 172.21.15.67:0/1453788286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd7326c070 0x55fd73628ab0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.225+0000 7efe5a38a740 1 --2- 172.21.15.67:0/1453788286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd7324b390 0x55fd73628570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.225+0000 7efe5a38a740 1 --2- 172.21.15.67:0/1453788286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd732431b0 0x55fd7361e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.225+0000 7efe5a38a740 1 -- 172.21.15.67:0/1453788286 >> 172.21.15.67:0/1453788286 conn(0x55fd73260460 msgr2=0x55fd7361dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:05.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.225+0000 7efe5a38a740 1 -- 172.21.15.67:0/1453788286 shutdown_connections 2024-07-27T03:19:05.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.225+0000 7efe5a38a740 1 -- 172.21.15.67:0/1453788286 wait complete. 2024-07-27T03:19:05.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.225+0000 7efe5a38a740 1 -- start start 2024-07-27T03:19:05.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.225+0000 7efe5a38a740 1 Processor -- start 2024-07-27T03:19:05.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.226+0000 7efe5a38a740 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd732431b0 0x55fd73620690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:05.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.226+0000 7efe5a38a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd7324b390 0x55fd73620bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:05.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.226+0000 7efe5a38a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd7326c070 0x55fd73621110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:05.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.226+0000 7efe5a38a740 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c1b3cd0 con 0x55fd732431b0 2024-07-27T03:19:05.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.226+0000 7efe5a38a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c1b3600 con 0x55fd7326c070 2024-07-27T03:19:05.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.226+0000 7efe5a38a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c1b0840 con 0x55fd7324b390 2024-07-27T03:19:05.232 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.226+0000 7efe5a38a740 10 client.0 _ll_register_callbacks cb 0x7efe3c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-07-27T03:19:05.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.226+0000 7efe523a2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd732431b0 0x55fd73620690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:05.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.226+0000 7efe523a2640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd732431b0 0x55fd73620690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42120/0 (socket says 172.21.15.67:42120) 2024-07-27T03:19:05.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.226+0000 7efe523a2640 1 -- 172.21.15.67:0/571910021 learned_addr learned my addr 172.21.15.67:0/571910021 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:19:05.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.226+0000 7efe513a0640 1 --2- 172.21.15.67:0/571910021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd7324b390 0x55fd73620bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:05.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.226+0000 7efe523a2640 1 -- 172.21.15.67:0/571910021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd7324b390 msgr2=0x55fd73620bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:19:05.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.226+0000 7efe523a2640 1 --2- 172.21.15.67:0/571910021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd7324b390 0x55fd73620bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.226+0000 7efe523a2640 1 -- 172.21.15.67:0/571910021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd7326c070 msgr2=0x55fd73621110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-07-27T03:19:05.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.226+0000 7efe523a2640 1 --2- 172.21.15.67:0/571910021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd7326c070 0x55fd73621110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.226+0000 7efe523a2640 1 -- 172.21.15.67:0/571910021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd73695dd0 con 0x55fd732431b0 2024-07-27T03:19:05.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.226+0000 7efe523a2640 1 --2- 172.21.15.67:0/571910021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd732431b0 0x55fd73620690 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7efe44004d60 tx=0x7efe4401e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49937d47bdf9d8c server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:05.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.226+0000 7efe357fa640 10 client.0 ms_handle_connect on v2:172.21.15.67:3300/0 2024-07-27T03:19:05.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.226+0000 7efe357fa640 1 -- 172.21.15.67:0/571910021 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44031030 con 0x55fd732431b0 2024-07-27T03:19:05.233 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.227+0000 7efe357fa640 1 -- 172.21.15.67:0/571910021 <== mon.0 v2:172.21.15.67:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7efe44012ca0 con 0x55fd732431b0 2024-07-27T03:19:05.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.227+0000 7efe5a38a740 1 -- 172.21.15.67:0/571910021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe44014620 con 0x55fd732431b0 2024-07-27T03:19:05.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.227+0000 7efe357fa640 1 -- 172.21.15.67:0/571910021 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efe4404a050 con 0x55fd732431b0 2024-07-27T03:19:05.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.227+0000 7efe5a38a740 10 client.16041 fetch_fsmap learned FSMap version 30 2024-07-27T03:19:05.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.227+0000 7efe5a38a740 1 -- 172.21.15.67:0/571910021 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fd73697030 con 0x55fd732431b0 2024-07-27T03:19:05.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.227+0000 7efe357fa640 1 -- 172.21.15.67:0/571910021 <== mon.0 v2:172.21.15.67:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe44014620 con 0x55fd732431b0 2024-07-27T03:19:05.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.227+0000 7efe357fa640 1 -- 172.21.15.67:0/571910021 <== mon.0 v2:172.21.15.67:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7efe440055f0 con 0x55fd732431b0 2024-07-27T03:19:05.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.227+0000 7efe5a38a740 10 client.16041 fetch_fsmap finished waiting for FSMap version 30 2024-07-27T03:19:05.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.227+0000 7efe5a38a740 10 client.16041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-07-27T03:19:05.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.227+0000 7efe5a38a740 20 client.16041 populate_metadata read hostname 'smithi067' 2024-07-27T03:19:05.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.227+0000 7efe5a38a740 1 --2- 172.21.15.67:0/571910021 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55fd731ffda0 0x55fd73699730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:05.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.227+0000 7efe5a38a740 4 client.16041 mds_command: new command op to 24620 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-07-27T03:19:05.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.227+0000 7efe5a38a740 1 -- 172.21.15.67:0/571910021 --> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd73585510 con 0x55fd731ffda0 2024-07-27T03:19:05.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.227+0000 7efe513a0640 1 --2- 172.21.15.67:0/571910021 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55fd731ffda0 0x55fd73699730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:05.234 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.228+0000 7efe357fa640 10 client.16041 ms_handle_connect on v2:172.21.15.149:6838/281746498 2024-07-27T03:19:05.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.228+0000 7efe513a0640 1 --2- 172.21.15.67:0/571910021 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55fd731ffda0 0x55fd73699730 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35bbe56eaeb8d99 server_cookie=bdd3f1a653eedd89 in_seq=0 out_seq=0 2024-07-27T03:19:05.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.228+0000 7efe357fa640 1 -- 172.21.15.67:0/571910021 <== mds.0 v2:172.21.15.149:6838/281746498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd73585510 con 0x55fd731ffda0 2024-07-27T03:19:05.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.228+0000 7efe357fa640 10 client.16041 handle_command_reply: tid=0 2024-07-27T03:19:05.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.228+0000 7efe5a38a740 1 client.16041 shutdown 2024-07-27T03:19:05.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.228+0000 7efe5a38a740 1 -- 172.21.15.67:0/571910021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd732431b0 msgr2=0x55fd73620690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:05.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.228+0000 7efe5a38a740 1 --2- 172.21.15.67:0/571910021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd732431b0 0x55fd73620690 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7efe44004d60 tx=0x7efe4401e950 comp rx=0 tx=0).stop 2024-07-27T03:19:05.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.228+0000 7efe5a38a740 1 -- 172.21.15.67:0/571910021 shutdown_connections 2024-07-27T03:19:05.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.228+0000 7efe5a38a740 1 --2- 172.21.15.67:0/571910021 >> [v2:172.21.15.149:6838/281746498,v1:172.21.15.149:6839/281746498] conn(0x55fd731ffda0 0x55fd73699730 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.228+0000 7efe5a38a740 1 --2- 172.21.15.67:0/571910021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd7326c070 0x55fd73621110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.228+0000 7efe357fa640 0 client.16041 ms_handle_reset on v2:172.21.15.149:6838/281746498 2024-07-27T03:19:05.235 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.228+0000 7efe5a38a740 1 --2- 172.21.15.67:0/571910021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55fd7324b390 0x55fd73620bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.228+0000 7efe5a38a740 1 --2- 172.21.15.67:0/571910021 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x55fd732431b0 0x55fd73620690 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.228+0000 7efe5a38a740 1 -- 172.21.15.67:0/571910021 >> 172.21.15.67:0/571910021 conn(0x55fd73260460 msgr2=0x55fd73611e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:05.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.229+0000 7efe5a38a740 1 -- 172.21.15.67:0/571910021 shutdown_connections 2024-07-27T03:19:05.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.229+0000 7efe5a38a740 1 -- 172.21.15.67:0/571910021 wait complete. 2024-07-27T03:19:05.236 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.229+0000 7efe5a38a740 20 client.16041 trim_cache size 0 max 16384 2024-07-27T03:19:05.236 INFO:teuthology.orchestra.run.smithi067.stdout:[] 2024-07-27T03:19:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.229+0000 7efe53e05640 1 -- 172.21.15.67:0/2117176682 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7efe380b3c50 msgr2=0x7efe380b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.229+0000 7efe53e05640 1 --2- 172.21.15.67:0/2117176682 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7efe380b3c50 0x7efe380b6160 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efe480aa9d0 tx=0x7efe4806ffe0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.229+0000 7efe53e05640 1 -- 172.21.15.67:0/2117176682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe4c101ad0 msgr2=0x7efe4c102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.229+0000 7efe53e05640 1 --2- 172.21.15.67:0/2117176682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe4c101ad0 0x7efe4c102a70 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7efe4800c920 tx=0x7efe4800cde0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.229+0000 7efe53e05640 1 -- 172.21.15.67:0/2117176682 shutdown_connections 2024-07-27T03:19:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.229+0000 7efe53e05640 1 --2- 172.21.15.67:0/2117176682 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7efe380b3c50 0x7efe380b6160 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.229+0000 7efe53e05640 1 --2- 172.21.15.67:0/2117176682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe4c1098d0 0x7efe4c1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.229+0000 7efe53e05640 1 --2- 172.21.15.67:0/2117176682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe4c108f80 0x7efe4c102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.229+0000 7efe53e05640 1 --2- 172.21.15.67:0/2117176682 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7efe4c101ad0 0x7efe4c102a70 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:05.237 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.229+0000 7efe53e05640 1 -- 172.21.15.67:0/2117176682 >> 172.21.15.67:0/2117176682 conn(0x7efe4c079a00 msgr2=0x7efe4c107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:05.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.229+0000 7efe53e05640 1 -- 172.21.15.67:0/2117176682 shutdown_connections 2024-07-27T03:19:05.238 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:05.229+0000 7efe53e05640 1 -- 172.21.15.67:0/2117176682 wait complete. 2024-07-27T03:19:06.017 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_af8fdbf079811fe158ce62927bd145c35b74d4b8/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_af8fdbf079811fe158ce62927bd145c35b74d4b8/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_af8fdbf079811fe158ce62927bd145c35b74d4b8/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-07-27T03:19:06.018 INFO:tasks.fwd_scrub:done joining 2024-07-27T03:19:06.018 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-07-27T03:19:06.031 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-07-27T03:19:06.156 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.152+0000 7fe75f2eb640 1 -- 172.21.15.67:0/3238859399 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe758103a00 msgr2=0x7fe75810a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:06.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.152+0000 7fe75f2eb640 1 --2- 172.21.15.67:0/3238859399 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe758103a00 0x7fe75810a3a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe750009900 tx=0x7fe75002a410 comp rx=0 tx=0).stop 2024-07-27T03:19:06.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.152+0000 7fe75f2eb640 1 -- 172.21.15.67:0/3238859399 shutdown_connections 2024-07-27T03:19:06.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.152+0000 7fe75f2eb640 1 --2- 172.21.15.67:0/3238859399 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe758103a00 0x7fe75810a3a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.152+0000 7fe75f2eb640 1 --2- 172.21.15.67:0/3238859399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe758103060 0x7fe758103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.152+0000 7fe75f2eb640 1 --2- 172.21.15.67:0/3238859399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe758101fe0 0x7fe758102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.152+0000 7fe75f2eb640 1 -- 172.21.15.67:0/3238859399 >> 172.21.15.67:0/3238859399 conn(0x7fe7580fd920 msgr2=0x7fe7580ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:06.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.152+0000 7fe75f2eb640 1 -- 172.21.15.67:0/3238859399 shutdown_connections 2024-07-27T03:19:06.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.152+0000 7fe75f2eb640 1 -- 172.21.15.67:0/3238859399 wait complete. 2024-07-27T03:19:06.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.152+0000 7fe75f2eb640 1 Processor -- start 2024-07-27T03:19:06.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.153+0000 7fe75f2eb640 1 -- start start 2024-07-27T03:19:06.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.153+0000 7fe75f2eb640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe758101fe0 0x7fe75806a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:06.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.153+0000 7fe75f2eb640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe758103060 0x7fe75819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:06.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.153+0000 7fe75f2eb640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe758103a00 0x7fe7581a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:06.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.153+0000 7fe75f2eb640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fe75810dcb0 con 0x7fe758101fe0 2024-07-27T03:19:06.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.153+0000 7fe75f2eb640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe75810db30 con 0x7fe758103060 2024-07-27T03:19:06.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.153+0000 7fe75f2eb640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe75810de30 con 0x7fe758103a00 2024-07-27T03:19:06.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.153+0000 7fe75d087640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe758101fe0 0x7fe75806a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:06.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.153+0000 7fe75d087640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe758101fe0 0x7fe75806a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42132/0 (socket says 172.21.15.67:42132) 2024-07-27T03:19:06.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.153+0000 7fe75d087640 1 -- 172.21.15.67:0/2988710646 learned_addr learned my addr 172.21.15.67:0/2988710646 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:19:06.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.153+0000 7fe75c886640 1 --2- 172.21.15.67:0/2988710646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe758103060 0x7fe75819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:06.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.153+0000 7fe75d888640 1 --2- 172.21.15.67:0/2988710646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe758103a00 0x7fe7581a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:06.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.153+0000 7fe75d087640 1 -- 172.21.15.67:0/2988710646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe758103a00 msgr2=0x7fe7581a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:06.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.153+0000 7fe75d087640 1 --2- 172.21.15.67:0/2988710646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe758103a00 0x7fe7581a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.153+0000 7fe75d087640 1 -- 172.21.15.67:0/2988710646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe758103060 msgr2=0x7fe75819e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:06.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.153+0000 7fe75d087640 1 --2- 172.21.15.67:0/2988710646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe758103060 0x7fe75819e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.153+0000 7fe75d087640 1 -- 172.21.15.67:0/2988710646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7581a1350 con 0x7fe758101fe0 2024-07-27T03:19:06.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.153+0000 7fe75d087640 1 --2- 172.21.15.67:0/2988710646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe758101fe0 0x7fe75806a9e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe74800ebc0 tx=0x7fe74800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:06.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.154+0000 7fe7567fc640 1 -- 172.21.15.67:0/2988710646 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe748010630 con 0x7fe758101fe0 2024-07-27T03:19:06.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.154+0000 7fe7567fc640 1 -- 172.21.15.67:0/2988710646 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe748010c80 con 0x7fe758101fe0 2024-07-27T03:19:06.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.154+0000 7fe7567fc640 1 -- 172.21.15.67:0/2988710646 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe748018b60 con 0x7fe758101fe0 2024-07-27T03:19:06.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.154+0000 7fe75f2eb640 1 -- 172.21.15.67:0/2988710646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7581a1640 con 0x7fe758101fe0 2024-07-27T03:19:06.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.154+0000 7fe75f2eb640 1 -- 172.21.15.67:0/2988710646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7581a1b90 con 0x7fe758101fe0 2024-07-27T03:19:06.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.155+0000 7fe75f2eb640 1 -- 172.21.15.67:0/2988710646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe758109190 con 0x7fe758101fe0 2024-07-27T03:19:06.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.156+0000 7fe7567fc640 1 -- 172.21.15.67:0/2988710646 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7fe748018d00 con 0x7fe758101fe0 2024-07-27T03:19:06.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.157+0000 7fe7567fc640 1 --2- 172.21.15.67:0/2988710646 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe7400b3bd0 0x7fe7400b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:06.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.157+0000 7fe75c886640 1 --2- 172.21.15.67:0/2988710646 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe7400b3bd0 0x7fe7400b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:06.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.157+0000 7fe7567fc640 1 -- 172.21.15.67:0/2988710646 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7fe7480e4b70 con 0x7fe758101fe0 2024-07-27T03:19:06.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.157+0000 7fe75c886640 1 --2- 172.21.15.67:0/2988710646 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe7400b3bd0 0x7fe7400b60e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe74c005e10 tx=0x7fe74c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:06.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.160+0000 7fe7567fc640 1 -- 172.21.15.67:0/2988710646 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fe7480abe10 con 0x7fe758101fe0 2024-07-27T03:19:06.357 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.355+0000 7fe75f2eb640 1 -- 172.21.15.67:0/2988710646 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe758063a30 con 0x7fe758101fe0 2024-07-27T03:19:06.358 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.356+0000 7fe7567fc640 1 -- 172.21.15.67:0/2988710646 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24829 (secure 0 0 0) 0x7fe7480b40e0 con 0x7fe758101fe0 2024-07-27T03:19:06.359 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:19:06.359 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":91,"fsid":"c0782a00-4bc4-11ef-bcc7-c7b262605968","created":"2024-07-27T03:04:08.096824+0000","modified":"2024-07-27T03:13:57.294556+0000","last_up_change":"2024-07-27T03:11:33.314800+0000","last_in_change":"2024-07-27T03:11:14.028095+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-27T03:08:01.042479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-27T03:12:27.565948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-27T03:12:31.353726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-27T03:12:40.599486+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9f614272-18e8-4562-812c-4f43e40efd2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6803","nonce":2648129674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6805","nonce":2648129674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6809","nonce":2648129674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6807","nonce":2648129674}]},"public_addr":"172.21.15.67:6803/2648129674","cluster_addr":"172.21.15.67:6805/2648129674","heartbeat_back_addr":"172.21.15.67:6809/2648129674","heartbeat_front_addr":"172.21.15.67:6807/2648129674","state":["exists","up"]},{"osd":1,"uuid":"f59f5879-163a-4269-bbe7-1200eea65341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6801","nonce":2561753152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6803","nonce":2561753152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6807","nonce":2561753152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6805","nonce":2561753152}]},"public_addr":"172.21.15.149:6801/2561753152","cluster_addr":"172.21.15.149:6803/2561753152","heartbeat_back_addr":"172.21.15.149:6807/2561753152","heartbeat_front_addr":"172.21.15.149:6805/2561753152","state":["exists","up"]},{"osd":2,"uuid":"444e0ebe-7ca3-4be9-8bd5-db859f824445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6801","nonce":3770940078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6803","nonce":3770940078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6807","nonce":3770940078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6805","nonce":3770940078}]},"public_addr":"172.21.15.175:6801/3770940078","cluster_addr":"172.21.15.175:6803/3770940078","heartbeat_back_addr":"172.21.15.175:6807/3770940078","heartbeat_front_addr":"172.21.15.175:6805/3770940078","state":["exists","up"]},{"osd":3,"uuid":"d554db3d-9604-427c-8736-b1bb379f8d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6811","nonce":3159144693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6813","nonce":3159144693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6817","nonce":3159144693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6815","nonce":3159144693}]},"public_addr":"172.21.15.67:6811/3159144693","cluster_addr":"172.21.15.67:6813/3159144693","heartbeat_back_addr":"172.21.15.67:6817/3159144693","heartbeat_front_addr":"172.21.15.67:6815/3159144693","state":["exists","up"]},{"osd":4,"uuid":"c317dbe5-3b49-47a6-8408-7042a3dac345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3393503532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3393503532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3393503532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3393503532}]},"public_addr":"172.21.15.149:6809/3393503532","cluster_addr":"172.21.15.149:6811/3393503532","heartbeat_back_addr":"172.21.15.149:6815/3393503532","heartbeat_front_addr":"172.21.15.149:6813/3393503532","state":["exists","up"]},{"osd":5,"uuid":"5c85abd6-2b42-4957-a9b7-59a9ee36b841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6809","nonce":2291204784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6811","nonce":2291204784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6815","nonce":2291204784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6813","nonce":2291204784}]},"public_addr":"172.21.15.175:6809/2291204784","cluster_addr":"172.21.15.175:6811/2291204784","heartbeat_back_addr":"172.21.15.175:6815/2291204784","heartbeat_front_addr":"172.21.15.175:6813/2291204784","state":["exists","up"]},{"osd":6,"uuid":"bf6c277d-ec24-4470-98de-cf6f28876ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6819","nonce":2660182879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6821","nonce":2660182879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6825","nonce":2660182879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6823","nonce":2660182879}]},"public_addr":"172.21.15.67:6819/2660182879","cluster_addr":"172.21.15.67:6821/2660182879","heartbeat_back_addr":"172.21.15.67:6825/2660182879","heartbeat_front_addr":"172.21.15.67:6823/2660182879","state":["exists","up"]},{"osd":7,"uuid":"fea2c7cf-d5cb-45ae-9b29-f6e56c903891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6817","nonce":2289624404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6819","nonce":2289624404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6823","nonce":2289624404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6821","nonce":2289624404}]},"public_addr":"172.21.15.149:6817/2289624404","cluster_addr":"172.21.15.149:6819/2289624404","heartbeat_back_addr":"172.21.15.149:6823/2289624404","heartbeat_front_addr":"172.21.15.149:6821/2289624404","state":["exists","up"]},{"osd":8,"uuid":"14d04d96-579a-4dc9-a4d4-50e4850b48e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6817","nonce":2286032466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6819","nonce":2286032466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6823","nonce":2286032466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6821","nonce":2286032466}]},"public_addr":"172.21.15.175:6817/2286032466","cluster_addr":"172.21.15.175:6819/2286032466","heartbeat_back_addr":"172.21.15.175:6823/2286032466","heartbeat_front_addr":"172.21.15.175:6821/2286032466","state":["exists","up"]},{"osd":9,"uuid":"b65dfb6b-1efe-4314-a8be-ce6a295e99fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6827","nonce":3411081513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6829","nonce":3411081513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6833","nonce":3411081513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6831","nonce":3411081513}]},"public_addr":"172.21.15.67:6827/3411081513","cluster_addr":"172.21.15.67:6829/3411081513","heartbeat_back_addr":"172.21.15.67:6833/3411081513","heartbeat_front_addr":"172.21.15.67:6831/3411081513","state":["exists","up"]},{"osd":10,"uuid":"e3dabae0-0d64-4222-984c-fc1cce883456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6825","nonce":2011495212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6827","nonce":2011495212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6831","nonce":2011495212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6829","nonce":2011495212}]},"public_addr":"172.21.15.149:6825/2011495212","cluster_addr":"172.21.15.149:6827/2011495212","heartbeat_back_addr":"172.21.15.149:6831/2011495212","heartbeat_front_addr":"172.21.15.149:6829/2011495212","state":["exists","up"]},{"osd":11,"uuid":"406c082c-9b5f-4781-8cbf-3c04fa49e410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6825","nonce":2565514213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6827","nonce":2565514213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6831","nonce":2565514213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6829","nonce":2565514213}]},"public_addr":"172.21.15.175:6825/2565514213","cluster_addr":"172.21.15.175:6827/2565514213","heartbeat_back_addr":"172.21.15.175:6831/2565514213","heartbeat_front_addr":"172.21.15.175:6829/2565514213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:11.399380+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:33.728518+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:56.411181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:19.871731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:42.994576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:06.712815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:31.321967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:55.070898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:18.686108+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:43.187795+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:07.318538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:31.374843+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/3477601235":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/4130679770":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6800/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/1029332005":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6801/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6801/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/284078926":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6800/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/2559121542":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/925406021":"2024-07-28T03:04:30.600273+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-27T03:19:06.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.358+0000 7fe75f2eb640 1 -- 172.21.15.67:0/2988710646 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe7400b3bd0 msgr2=0x7fe7400b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:06.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.358+0000 7fe75f2eb640 1 --2- 172.21.15.67:0/2988710646 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe7400b3bd0 0x7fe7400b60e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe74c005e10 tx=0x7fe74c005da0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.361 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.358+0000 7fe75f2eb640 1 -- 172.21.15.67:0/2988710646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe758101fe0 msgr2=0x7fe75806a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:06.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.358+0000 7fe75f2eb640 1 --2- 172.21.15.67:0/2988710646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe758101fe0 0x7fe75806a9e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe74800ebc0 tx=0x7fe74800c6a0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.358+0000 7fe75f2eb640 1 -- 172.21.15.67:0/2988710646 shutdown_connections 2024-07-27T03:19:06.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.358+0000 7fe75f2eb640 1 --2- 172.21.15.67:0/2988710646 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fe7400b3bd0 0x7fe7400b60e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.358+0000 7fe75f2eb640 1 --2- 172.21.15.67:0/2988710646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe758103a00 0x7fe7581a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.358+0000 7fe75f2eb640 1 --2- 172.21.15.67:0/2988710646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe758103060 0x7fe75819e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.358+0000 7fe75f2eb640 1 --2- 172.21.15.67:0/2988710646 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe758101fe0 0x7fe75806a9e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.358+0000 7fe75f2eb640 1 -- 172.21.15.67:0/2988710646 >> 172.21.15.67:0/2988710646 conn(0x7fe7580fd920 msgr2=0x7fe758106360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:06.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.359+0000 7fe75f2eb640 1 -- 172.21.15.67:0/2988710646 shutdown_connections 2024-07-27T03:19:06.362 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.359+0000 7fe75f2eb640 1 -- 172.21.15.67:0/2988710646 wait complete. 2024-07-27T03:19:06.373 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-07-27T03:08:01.042479+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-07-27T03:12:27.565948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-07-27T03:12:31.353726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-07-27T03:12:40.599486+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-07-27T03:19:06.374 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-07-27T03:19:06.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.490+0000 7f3923629640 1 -- 172.21.15.67:0/4267875273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f391c101ff0 msgr2=0x7f391c102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:06.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.490+0000 7f3923629640 1 --2- 172.21.15.67:0/4267875273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f391c101ff0 0x7f391c102460 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3914009900 tx=0x7f391402a410 comp rx=0 tx=0).stop 2024-07-27T03:19:06.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.490+0000 7f3923629640 1 -- 172.21.15.67:0/4267875273 shutdown_connections 2024-07-27T03:19:06.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.490+0000 7f3923629640 1 --2- 172.21.15.67:0/4267875273 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f391c103a10 0x7f391c10a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.490+0000 7f3923629640 1 --2- 172.21.15.67:0/4267875273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f391c103070 0x7f391c103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.490+0000 7f3923629640 1 --2- 172.21.15.67:0/4267875273 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f391c101ff0 0x7f391c102460 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.490+0000 7f3923629640 1 -- 172.21.15.67:0/4267875273 >> 172.21.15.67:0/4267875273 conn(0x7f391c0fd950 msgr2=0x7f391c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:06.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.490+0000 7f3923629640 1 -- 172.21.15.67:0/4267875273 shutdown_connections 2024-07-27T03:19:06.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.490+0000 7f3923629640 1 -- 172.21.15.67:0/4267875273 wait complete. 2024-07-27T03:19:06.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.491+0000 7f3923629640 1 Processor -- start 2024-07-27T03:19:06.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.491+0000 7f3923629640 1 -- start start 2024-07-27T03:19:06.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.491+0000 7f3923629640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f391c101ff0 0x7f391c06a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:06.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.491+0000 7f3923629640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f391c103070 0x7f391c19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:06.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.491+0000 7f3923629640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f391c103a10 0x7f391c1a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:06.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.491+0000 7f3923629640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f391c10dc70 con 0x7f391c103070 2024-07-27T03:19:06.495 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.491+0000 7f3923629640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f391c10daf0 con 0x7f391c101ff0 2024-07-27T03:19:06.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.491+0000 7f3923629640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f391c10ddf0 con 0x7f391c103a10 2024-07-27T03:19:06.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.491+0000 7f3920bc4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f391c103070 0x7f391c19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:06.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.491+0000 7f3920bc4640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f391c103070 0x7f391c19e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42152/0 (socket says 172.21.15.67:42152) 2024-07-27T03:19:06.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.491+0000 7f3920bc4640 1 -- 172.21.15.67:0/927115621 learned_addr learned my addr 172.21.15.67:0/927115621 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:19:06.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.491+0000 7f39213c5640 1 --2- 172.21.15.67:0/927115621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f391c101ff0 0x7f391c06a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:06.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.491+0000 7f3920bc4640 1 -- 172.21.15.67:0/927115621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f391c103a10 msgr2=0x7f391c1a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:06.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.491+0000 7f3921bc6640 1 --2- 172.21.15.67:0/927115621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f391c103a10 0x7f391c1a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:06.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.491+0000 7f3920bc4640 1 --2- 172.21.15.67:0/927115621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f391c103a10 0x7f391c1a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.491+0000 7f3920bc4640 1 -- 172.21.15.67:0/927115621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f391c101ff0 msgr2=0x7f391c06a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:06.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.491+0000 7f3920bc4640 1 --2- 172.21.15.67:0/927115621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f391c101ff0 0x7f391c06a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.491+0000 7f3920bc4640 1 -- 172.21.15.67:0/927115621 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391c1a1410 con 0x7f391c103070 2024-07-27T03:19:06.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.491+0000 7f3920bc4640 1 --2- 172.21.15.67:0/927115621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f391c103070 0x7f391c19e6c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f391000e9f0 tx=0x7f391000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:06.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.492+0000 7f391a7fc640 1 -- 172.21.15.67:0/927115621 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391000cd80 con 0x7f391c103070 2024-07-27T03:19:06.496 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.492+0000 7f3923629640 1 -- 172.21.15.67:0/927115621 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391c1a1700 con 0x7f391c103070 2024-07-27T03:19:06.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.492+0000 7f3923629640 1 -- 172.21.15.67:0/927115621 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f391c1a1ca0 con 0x7f391c103070 2024-07-27T03:19:06.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.492+0000 7f391a7fc640 1 -- 172.21.15.67:0/927115621 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39100108e0 con 0x7f391c103070 2024-07-27T03:19:06.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.492+0000 7f391a7fc640 1 -- 172.21.15.67:0/927115621 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910026060 con 0x7f391c103070 2024-07-27T03:19:06.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.493+0000 7f3923629640 1 -- 172.21.15.67:0/927115621 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39080052c0 con 0x7f391c103070 2024-07-27T03:19:06.497 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.495+0000 7f391a7fc640 1 -- 172.21.15.67:0/927115621 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f3910010430 con 0x7f391c103070 2024-07-27T03:19:06.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.496+0000 7f391a7fc640 1 --2- 172.21.15.67:0/927115621 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f39040b3bd0 0x7f39040b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:06.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.496+0000 7f39213c5640 1 --2- 172.21.15.67:0/927115621 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f39040b3bd0 0x7f39040b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:06.498 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.496+0000 7f391a7fc640 1 -- 172.21.15.67:0/927115621 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f39100e4270 con 0x7f391c103070 2024-07-27T03:19:06.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.497+0000 7f39213c5640 1 --2- 172.21.15.67:0/927115621 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f39040b3bd0 0x7f39040b60e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3914000c00 tx=0x7f3914031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:06.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.499+0000 7f391a7fc640 1 -- 172.21.15.67:0/927115621 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f39100ab510 con 0x7f391c103070 2024-07-27T03:19:06.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.698+0000 7f3923629640 1 -- 172.21.15.67:0/927115621 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3908005880 con 0x7f391c103070 2024-07-27T03:19:06.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.698+0000 7f391a7fc640 1 -- 172.21.15.67:0/927115621 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f39100aae10 con 0x7f391c103070 2024-07-27T03:19:06.701 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-07-27T03:19:06.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.700+0000 7f3923629640 1 -- 172.21.15.67:0/927115621 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f39040b3bd0 msgr2=0x7f39040b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:06.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.700+0000 7f3923629640 1 --2- 172.21.15.67:0/927115621 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f39040b3bd0 0x7f39040b60e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3914000c00 tx=0x7f3914031400 comp rx=0 tx=0).stop 2024-07-27T03:19:06.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.700+0000 7f3923629640 1 -- 172.21.15.67:0/927115621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f391c103070 msgr2=0x7f391c19e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:06.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.700+0000 7f3923629640 1 --2- 172.21.15.67:0/927115621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f391c103070 0x7f391c19e6c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f391000e9f0 tx=0x7f391000eeb0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.700+0000 7f3923629640 1 -- 172.21.15.67:0/927115621 shutdown_connections 2024-07-27T03:19:06.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.700+0000 7f3923629640 1 --2- 172.21.15.67:0/927115621 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f39040b3bd0 0x7f39040b60e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.700+0000 7f3923629640 1 --2- 172.21.15.67:0/927115621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f391c103a10 0x7f391c1a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.700+0000 7f3923629640 1 --2- 172.21.15.67:0/927115621 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f391c103070 0x7f391c19e6c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.703 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.700+0000 7f3923629640 1 --2- 172.21.15.67:0/927115621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f391c101ff0 0x7f391c06a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.700+0000 7f3923629640 1 -- 172.21.15.67:0/927115621 >> 172.21.15.67:0/927115621 conn(0x7f391c0fd950 msgr2=0x7f391c106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:06.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.700+0000 7f3923629640 1 -- 172.21.15.67:0/927115621 shutdown_connections 2024-07-27T03:19:06.704 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.700+0000 7f3923629640 1 -- 172.21.15.67:0/927115621 wait complete. 2024-07-27T03:19:06.714 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-07-27T03:19:06.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.831+0000 7f12459d9640 1 -- 172.21.15.67:0/4187739060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1240108e50 msgr2=0x7f124010b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:06.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.831+0000 7f12459d9640 1 --2- 172.21.15.67:0/4187739060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1240108e50 0x7f124010b360 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1238009c90 tx=0x7f123802a860 comp rx=0 tx=0).stop 2024-07-27T03:19:06.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.831+0000 7f12459d9640 1 -- 172.21.15.67:0/4187739060 shutdown_connections 2024-07-27T03:19:06.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.831+0000 7f12459d9640 1 --2- 172.21.15.67:0/4187739060 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1240108e50 0x7f124010b360 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.831+0000 7f12459d9640 1 --2- 172.21.15.67:0/4187739060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1240106290 0x7f1240108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.831+0000 7f12459d9640 1 --2- 172.21.15.67:0/4187739060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f124006be80 0x7f1240105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.831+0000 7f12459d9640 1 -- 172.21.15.67:0/4187739060 >> 172.21.15.67:0/4187739060 conn(0x7f12400fd950 msgr2=0x7f12400ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:06.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.831+0000 7f12459d9640 1 -- 172.21.15.67:0/4187739060 shutdown_connections 2024-07-27T03:19:06.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.831+0000 7f12459d9640 1 -- 172.21.15.67:0/4187739060 wait complete. 2024-07-27T03:19:06.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.832+0000 7f12459d9640 1 Processor -- start 2024-07-27T03:19:06.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.832+0000 7f12459d9640 1 -- start start 2024-07-27T03:19:06.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.832+0000 7f12459d9640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f124006be80 0x7f1240199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:06.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.832+0000 7f12459d9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1240106290 0x7f1240199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:06.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.832+0000 7f12459d9640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1240108e50 0x7f124019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:06.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.832+0000 7f12459d9640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f124010ed70 con 0x7f1240108e50 2024-07-27T03:19:06.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.832+0000 7f12459d9640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f124010ebf0 con 0x7f1240106290 2024-07-27T03:19:06.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.832+0000 7f12459d9640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f124010eef0 con 0x7f124006be80 2024-07-27T03:19:06.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.832+0000 7f123f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1240108e50 0x7f124019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:06.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.833+0000 7f123f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1240108e50 0x7f124019ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42170/0 (socket says 172.21.15.67:42170) 2024-07-27T03:19:06.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.833+0000 7f123f7fe640 1 -- 172.21.15.67:0/1406994793 learned_addr learned my addr 172.21.15.67:0/1406994793 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:19:06.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.833+0000 7f123e7fc640 1 --2- 172.21.15.67:0/1406994793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1240106290 0x7f1240199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:06.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.833+0000 7f123effd640 1 --2- 172.21.15.67:0/1406994793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f124006be80 0x7f1240199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:06.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.833+0000 7f123f7fe640 1 -- 172.21.15.67:0/1406994793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f124006be80 msgr2=0x7f1240199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:06.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.833+0000 7f123f7fe640 1 --2- 172.21.15.67:0/1406994793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f124006be80 0x7f1240199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.833+0000 7f123f7fe640 1 -- 172.21.15.67:0/1406994793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1240106290 msgr2=0x7f1240199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:06.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.833+0000 7f123f7fe640 1 --2- 172.21.15.67:0/1406994793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1240106290 0x7f1240199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:06.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.833+0000 7f123f7fe640 1 -- 172.21.15.67:0/1406994793 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124019b320 con 0x7f1240108e50 2024-07-27T03:19:06.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.833+0000 7f123f7fe640 1 --2- 172.21.15.67:0/1406994793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1240108e50 0x7f124019ab90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f12380011e0 tx=0x7f1238004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:06.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.833+0000 7f12449d7640 1 -- 172.21.15.67:0/1406994793 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238030560 con 0x7f1240108e50 2024-07-27T03:19:06.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.833+0000 7f12449d7640 1 -- 172.21.15.67:0/1406994793 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1238030bb0 con 0x7f1240108e50 2024-07-27T03:19:06.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.833+0000 7f12459d9640 1 -- 172.21.15.67:0/1406994793 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124019b5b0 con 0x7f1240108e50 2024-07-27T03:19:06.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.833+0000 7f12449d7640 1 -- 172.21.15.67:0/1406994793 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123803cb30 con 0x7f1240108e50 2024-07-27T03:19:06.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.833+0000 7f12459d9640 1 -- 172.21.15.67:0/1406994793 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12401a1bb0 con 0x7f1240108e50 2024-07-27T03:19:06.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.834+0000 7f12459d9640 1 -- 172.21.15.67:0/1406994793 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12400ff470 con 0x7f1240108e50 2024-07-27T03:19:06.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.836+0000 7f12449d7640 1 -- 172.21.15.67:0/1406994793 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f123803ccd0 con 0x7f1240108e50 2024-07-27T03:19:06.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.837+0000 7f12449d7640 1 --2- 172.21.15.67:0/1406994793 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f122c0b3bd0 0x7f122c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:06.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.837+0000 7f123effd640 1 --2- 172.21.15.67:0/1406994793 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f122c0b3bd0 0x7f122c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:06.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.837+0000 7f12449d7640 1 -- 172.21.15.67:0/1406994793 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f12381053a0 con 0x7f1240108e50 2024-07-27T03:19:06.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.837+0000 7f123effd640 1 --2- 172.21.15.67:0/1406994793 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f122c0b3bd0 0x7f122c0b60e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f123000abe0 tx=0x7f123000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:06.842 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:06.840+0000 7f12449d7640 1 -- 172.21.15.67:0/1406994793 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f12380cc6d0 con 0x7f1240108e50 2024-07-27T03:19:06.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:06 smithi067 ceph-mon[32814]: pgmap v506: 129 pgs: 129 active+clean; 257 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 MiB/s wr, 16 op/s 2024-07-27T03:19:06.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:06 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2988710646' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:19:07.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:06 smithi149 ceph-mon[37173]: pgmap v506: 129 pgs: 129 active+clean; 257 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 MiB/s wr, 16 op/s 2024-07-27T03:19:07.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:06 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2988710646' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:19:07.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.032+0000 7f12459d9640 1 -- 172.21.15.67:0/1406994793 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1240063a30 con 0x7f1240108e50 2024-07-27T03:19:07.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.033+0000 7f12449d7640 1 -- 172.21.15.67:0/1406994793 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f12380cbfd0 con 0x7f1240108e50 2024-07-27T03:19:07.035 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-07-27T03:19:07.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.035+0000 7f12459d9640 1 -- 172.21.15.67:0/1406994793 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f122c0b3bd0 msgr2=0x7f122c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:07.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.035+0000 7f12459d9640 1 --2- 172.21.15.67:0/1406994793 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f122c0b3bd0 0x7f122c0b60e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f123000abe0 tx=0x7f123000a2d0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.035+0000 7f12459d9640 1 -- 172.21.15.67:0/1406994793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1240108e50 msgr2=0x7f124019ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:07.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.035+0000 7f12459d9640 1 --2- 172.21.15.67:0/1406994793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1240108e50 0x7f124019ab90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f12380011e0 tx=0x7f1238004cc0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.035+0000 7f12459d9640 1 -- 172.21.15.67:0/1406994793 shutdown_connections 2024-07-27T03:19:07.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.035+0000 7f12459d9640 1 --2- 172.21.15.67:0/1406994793 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f122c0b3bd0 0x7f122c0b60e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.035+0000 7f12459d9640 1 --2- 172.21.15.67:0/1406994793 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1240108e50 0x7f124019ab90 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.035+0000 7f12459d9640 1 --2- 172.21.15.67:0/1406994793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1240106290 0x7f1240199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.035+0000 7f12459d9640 1 --2- 172.21.15.67:0/1406994793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f124006be80 0x7f1240199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.035+0000 7f12459d9640 1 -- 172.21.15.67:0/1406994793 >> 172.21.15.67:0/1406994793 conn(0x7f12400fd950 msgr2=0x7f12401047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:07.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.035+0000 7f12459d9640 1 -- 172.21.15.67:0/1406994793 shutdown_connections 2024-07-27T03:19:07.038 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.035+0000 7f12459d9640 1 -- 172.21.15.67:0/1406994793 wait complete. 2024-07-27T03:19:07.049 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-07-27T03:19:07.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:06 smithi175 ceph-mon[37125]: pgmap v506: 129 pgs: 129 active+clean; 257 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 MiB/s wr, 16 op/s 2024-07-27T03:19:07.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:06 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2988710646' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:19:07.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.164+0000 7f568e9ab640 1 -- 172.21.15.67:0/493883594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5688101ff0 msgr2=0x7f5688102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:07.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.164+0000 7f568e9ab640 1 --2- 172.21.15.67:0/493883594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5688101ff0 0x7f5688102460 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5678009900 tx=0x7f567802a410 comp rx=0 tx=0).stop 2024-07-27T03:19:07.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.164+0000 7f568e9ab640 1 -- 172.21.15.67:0/493883594 shutdown_connections 2024-07-27T03:19:07.167 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.164+0000 7f568e9ab640 1 --2- 172.21.15.67:0/493883594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5688103a10 0x7f568810a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.164+0000 7f568e9ab640 1 --2- 172.21.15.67:0/493883594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5688103070 0x7f5688103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.164+0000 7f568e9ab640 1 --2- 172.21.15.67:0/493883594 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5688101ff0 0x7f5688102460 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.164+0000 7f568e9ab640 1 -- 172.21.15.67:0/493883594 >> 172.21.15.67:0/493883594 conn(0x7f56880fd950 msgr2=0x7f56880ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:07.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.164+0000 7f568e9ab640 1 -- 172.21.15.67:0/493883594 shutdown_connections 2024-07-27T03:19:07.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.164+0000 7f568e9ab640 1 -- 172.21.15.67:0/493883594 wait complete. 2024-07-27T03:19:07.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.165+0000 7f568e9ab640 1 Processor -- start 2024-07-27T03:19:07.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.165+0000 7f568e9ab640 1 -- start start 2024-07-27T03:19:07.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.165+0000 7f568e9ab640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5688101ff0 0x7f5688199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:07.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.165+0000 7f568e9ab640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5688103070 0x7f5688199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:07.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.165+0000 7f568e9ab640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5688103a10 0x7f568819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:07.172 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.165+0000 7f568e9ab640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f568810dc70 con 0x7f5688103a10 2024-07-27T03:19:07.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.165+0000 7f568e9ab640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f568810daf0 con 0x7f5688101ff0 2024-07-27T03:19:07.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.165+0000 7f568e9ab640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f568810ddf0 con 0x7f5688103070 2024-07-27T03:19:07.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.165+0000 7f568cf48640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5688103a10 0x7f568819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:07.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.165+0000 7f5687fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5688101ff0 0x7f5688199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:07.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.166+0000 7f568cf48640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5688103a10 0x7f568819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42196/0 (socket says 172.21.15.67:42196) 2024-07-27T03:19:07.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.166+0000 7f568cf48640 1 -- 172.21.15.67:0/3516674236 learned_addr learned my addr 172.21.15.67:0/3516674236 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:19:07.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.166+0000 7f56877fe640 1 --2- 172.21.15.67:0/3516674236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5688103070 0x7f5688199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:07.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.166+0000 7f5687fff640 1 -- 172.21.15.67:0/3516674236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5688103070 msgr2=0x7f5688199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:07.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.166+0000 7f5687fff640 1 --2- 172.21.15.67:0/3516674236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5688103070 0x7f5688199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.166+0000 7f5687fff640 1 -- 172.21.15.67:0/3516674236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5688103a10 msgr2=0x7f568819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:07.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.166+0000 7f5687fff640 1 --2- 172.21.15.67:0/3516674236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5688103a10 0x7f568819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.166+0000 7f5687fff640 1 -- 172.21.15.67:0/3516674236 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568819b380 con 0x7f5688101ff0 2024-07-27T03:19:07.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.166+0000 7f568cf48640 1 --2- 172.21.15.67:0/3516674236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5688103a10 0x7f568819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-07-27T03:19:07.173 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.166+0000 7f5687fff640 1 --2- 172.21.15.67:0/3516674236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5688101ff0 0x7f5688199940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5678006010 tx=0x7f5678032f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:07.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.166+0000 7f56857fa640 1 -- 172.21.15.67:0/3516674236 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567803f070 con 0x7f5688101ff0 2024-07-27T03:19:07.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.166+0000 7f568e9ab640 1 -- 172.21.15.67:0/3516674236 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568819b610 con 0x7f5688101ff0 2024-07-27T03:19:07.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.166+0000 7f56857fa640 1 -- 172.21.15.67:0/3516674236 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5678003ca0 con 0x7f5688101ff0 2024-07-27T03:19:07.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.166+0000 7f56857fa640 1 -- 172.21.15.67:0/3516674236 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678035070 con 0x7f5688101ff0 2024-07-27T03:19:07.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.166+0000 7f568e9ab640 1 -- 172.21.15.67:0/3516674236 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56881a1d60 con 0x7f5688101ff0 2024-07-27T03:19:07.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.167+0000 7f568e9ab640 1 -- 172.21.15.67:0/3516674236 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5688102460 con 0x7f5688101ff0 2024-07-27T03:19:07.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.169+0000 7f56857fa640 1 -- 172.21.15.67:0/3516674236 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f567803b580 con 0x7f5688101ff0 2024-07-27T03:19:07.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.170+0000 7f56857fa640 1 --2- 172.21.15.67:0/3516674236 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f56700b3bd0 0x7f56700b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:07.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.170+0000 7f56877fe640 1 --2- 172.21.15.67:0/3516674236 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f56700b3bd0 0x7f56700b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:07.175 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.170+0000 7f56857fa640 1 -- 172.21.15.67:0/3516674236 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f5678104ba0 con 0x7f5688101ff0 2024-07-27T03:19:07.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.171+0000 7f56877fe640 1 --2- 172.21.15.67:0/3516674236 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f56700b3bd0 0x7f56700b60e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f567c00bbe0 tx=0x7f567c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:07.176 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.173+0000 7f56857fa640 1 -- 172.21.15.67:0/3516674236 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f5678104e20 con 0x7f5688101ff0 2024-07-27T03:19:07.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.365+0000 7f568e9ab640 1 -- 172.21.15.67:0/3516674236 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f56881a20f0 con 0x7f5688101ff0 2024-07-27T03:19:07.368 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.366+0000 7f56857fa640 1 -- 172.21.15.67:0/3516674236 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f567803be20 con 0x7f5688101ff0 2024-07-27T03:19:07.368 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 64 2024-07-27T03:19:07.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.368+0000 7f568e9ab640 1 -- 172.21.15.67:0/3516674236 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f56700b3bd0 msgr2=0x7f56700b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:07.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.368+0000 7f568e9ab640 1 --2- 172.21.15.67:0/3516674236 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f56700b3bd0 0x7f56700b60e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f567c00bbe0 tx=0x7f567c00b390 comp rx=0 tx=0).stop 2024-07-27T03:19:07.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.368+0000 7f568e9ab640 1 -- 172.21.15.67:0/3516674236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5688101ff0 msgr2=0x7f5688199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:07.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.368+0000 7f568e9ab640 1 --2- 172.21.15.67:0/3516674236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5688101ff0 0x7f5688199940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5678006010 tx=0x7f5678032f90 comp rx=0 tx=0).stop 2024-07-27T03:19:07.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.368+0000 7f568e9ab640 1 -- 172.21.15.67:0/3516674236 shutdown_connections 2024-07-27T03:19:07.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.368+0000 7f568e9ab640 1 --2- 172.21.15.67:0/3516674236 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f56700b3bd0 0x7f56700b60e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.368+0000 7f568e9ab640 1 --2- 172.21.15.67:0/3516674236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5688103a10 0x7f568819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.368+0000 7f568e9ab640 1 --2- 172.21.15.67:0/3516674236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5688103070 0x7f5688199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.368+0000 7f568e9ab640 1 --2- 172.21.15.67:0/3516674236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5688101ff0 0x7f5688199940 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.368+0000 7f568e9ab640 1 -- 172.21.15.67:0/3516674236 >> 172.21.15.67:0/3516674236 conn(0x7f56880fd950 msgr2=0x7f56880ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:07.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.368+0000 7f568e9ab640 1 -- 172.21.15.67:0/3516674236 shutdown_connections 2024-07-27T03:19:07.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.368+0000 7f568e9ab640 1 -- 172.21.15.67:0/3516674236 wait complete. 2024-07-27T03:19:07.382 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-07-27T03:19:07.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.498+0000 7faae3a9a640 1 -- 172.21.15.67:0/2720075653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faadc101ff0 msgr2=0x7faadc102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:07.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.498+0000 7faae3a9a640 1 --2- 172.21.15.67:0/2720075653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faadc101ff0 0x7faadc102460 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7faad8009900 tx=0x7faad802a410 comp rx=0 tx=0).stop 2024-07-27T03:19:07.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.498+0000 7faae3a9a640 1 -- 172.21.15.67:0/2720075653 shutdown_connections 2024-07-27T03:19:07.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.498+0000 7faae3a9a640 1 --2- 172.21.15.67:0/2720075653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faadc103a10 0x7faadc10a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.498+0000 7faae3a9a640 1 --2- 172.21.15.67:0/2720075653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faadc103070 0x7faadc103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.498+0000 7faae3a9a640 1 --2- 172.21.15.67:0/2720075653 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faadc101ff0 0x7faadc102460 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.498+0000 7faae3a9a640 1 -- 172.21.15.67:0/2720075653 >> 172.21.15.67:0/2720075653 conn(0x7faadc0fd950 msgr2=0x7faadc0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:07.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.498+0000 7faae3a9a640 1 -- 172.21.15.67:0/2720075653 shutdown_connections 2024-07-27T03:19:07.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.498+0000 7faae3a9a640 1 -- 172.21.15.67:0/2720075653 wait complete. 2024-07-27T03:19:07.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.499+0000 7faae3a9a640 1 Processor -- start 2024-07-27T03:19:07.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.499+0000 7faae3a9a640 1 -- start start 2024-07-27T03:19:07.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.499+0000 7faae3a9a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faadc101ff0 0x7faadc199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:07.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.499+0000 7faae3a9a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faadc103070 0x7faadc19a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:07.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.499+0000 7faae3a9a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faadc103a10 0x7faadc19acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:07.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.499+0000 7faae3a9a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7faadc10ee60 con 0x7faadc103a10 2024-07-27T03:19:07.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.499+0000 7faae3a9a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faadc10ece0 con 0x7faadc101ff0 2024-07-27T03:19:07.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.499+0000 7faae3a9a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faadc10efe0 con 0x7faadc103070 2024-07-27T03:19:07.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.499+0000 7faae2037640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faadc103a10 0x7faadc19acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:07.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.499+0000 7faae2037640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faadc103a10 0x7faadc19acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42214/0 (socket says 172.21.15.67:42214) 2024-07-27T03:19:07.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.499+0000 7faae2037640 1 -- 172.21.15.67:0/2437582541 learned_addr learned my addr 172.21.15.67:0/2437582541 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:19:07.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.499+0000 7faae1836640 1 --2- 172.21.15.67:0/2437582541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faadc101ff0 0x7faadc199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:07.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.499+0000 7faae1035640 1 --2- 172.21.15.67:0/2437582541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faadc103070 0x7faadc19a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:07.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.499+0000 7faae2037640 1 -- 172.21.15.67:0/2437582541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faadc103070 msgr2=0x7faadc19a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:07.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.499+0000 7faae2037640 1 --2- 172.21.15.67:0/2437582541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faadc103070 0x7faadc19a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.499+0000 7faae2037640 1 -- 172.21.15.67:0/2437582541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faadc101ff0 msgr2=0x7faadc199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:07.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.499+0000 7faae2037640 1 --2- 172.21.15.67:0/2437582541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faadc101ff0 0x7faadc199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.499+0000 7faae2037640 1 -- 172.21.15.67:0/2437582541 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faadc19b4f0 con 0x7faadc103a10 2024-07-27T03:19:07.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.499+0000 7faae1836640 1 --2- 172.21.15.67:0/2437582541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faadc101ff0 0x7faadc199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:19:07.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.499+0000 7faae1035640 1 --2- 172.21.15.67:0/2437582541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faadc103070 0x7faadc19a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:19:07.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.500+0000 7faae2037640 1 --2- 172.21.15.67:0/2437582541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faadc103a10 0x7faadc19acd0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7faad400b970 tx=0x7faad400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:07.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.500+0000 7faad2ffd640 1 -- 172.21.15.67:0/2437582541 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4019070 con 0x7faadc103a10 2024-07-27T03:19:07.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.500+0000 7faad2ffd640 1 -- 172.21.15.67:0/2437582541 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faad4014440 con 0x7faadc103a10 2024-07-27T03:19:07.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.500+0000 7faae3a9a640 1 -- 172.21.15.67:0/2437582541 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faadc19b7e0 con 0x7faadc103a10 2024-07-27T03:19:07.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.500+0000 7faad2ffd640 1 -- 172.21.15.67:0/2437582541 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad40075d0 con 0x7faadc103a10 2024-07-27T03:19:07.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.500+0000 7faae3a9a640 1 -- 172.21.15.67:0/2437582541 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faadc1a1ed0 con 0x7faadc103a10 2024-07-27T03:19:07.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.501+0000 7faae3a9a640 1 -- 172.21.15.67:0/2437582541 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faadc1091c0 con 0x7faadc103a10 2024-07-27T03:19:07.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.503+0000 7faad2ffd640 1 -- 172.21.15.67:0/2437582541 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7faad401c070 con 0x7faadc103a10 2024-07-27T03:19:07.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.503+0000 7faad2ffd640 1 --2- 172.21.15.67:0/2437582541 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7faac80b3bd0 0x7faac80b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:07.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.503+0000 7faae1836640 1 --2- 172.21.15.67:0/2437582541 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7faac80b3bd0 0x7faac80b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:07.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.503+0000 7faad2ffd640 1 -- 172.21.15.67:0/2437582541 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7faad40e3aa0 con 0x7faadc103a10 2024-07-27T03:19:07.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.504+0000 7faae1836640 1 --2- 172.21.15.67:0/2437582541 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7faac80b3bd0 0x7faac80b60e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faad8009900 tx=0x7faad8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:07.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.506+0000 7faad2ffd640 1 -- 172.21.15.67:0/2437582541 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7faad40aad40 con 0x7faadc103a10 2024-07-27T03:19:07.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.700+0000 7faae3a9a640 1 -- 172.21.15.67:0/2437582541 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7faadc063a30 con 0x7faadc103a10 2024-07-27T03:19:07.702 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.700+0000 7faad2ffd640 1 -- 172.21.15.67:0/2437582541 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7faad40aa640 con 0x7faadc103a10 2024-07-27T03:19:07.703 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-07-27T03:19:07.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.702+0000 7faae3a9a640 1 -- 172.21.15.67:0/2437582541 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7faac80b3bd0 msgr2=0x7faac80b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:07.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.702+0000 7faae3a9a640 1 --2- 172.21.15.67:0/2437582541 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7faac80b3bd0 0x7faac80b60e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faad8009900 tx=0x7faad8005fd0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.705 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.702+0000 7faae3a9a640 1 -- 172.21.15.67:0/2437582541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faadc103a10 msgr2=0x7faadc19acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:07.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.702+0000 7faae3a9a640 1 --2- 172.21.15.67:0/2437582541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faadc103a10 0x7faadc19acd0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7faad400b970 tx=0x7faad400be30 comp rx=0 tx=0).stop 2024-07-27T03:19:07.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.702+0000 7faae3a9a640 1 -- 172.21.15.67:0/2437582541 shutdown_connections 2024-07-27T03:19:07.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.702+0000 7faae3a9a640 1 --2- 172.21.15.67:0/2437582541 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7faac80b3bd0 0x7faac80b60e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.702+0000 7faae3a9a640 1 --2- 172.21.15.67:0/2437582541 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faadc103a10 0x7faadc19acd0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.702+0000 7faae3a9a640 1 --2- 172.21.15.67:0/2437582541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faadc103070 0x7faadc19a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.702+0000 7faae3a9a640 1 --2- 172.21.15.67:0/2437582541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faadc101ff0 0x7faadc199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.702+0000 7faae3a9a640 1 -- 172.21.15.67:0/2437582541 >> 172.21.15.67:0/2437582541 conn(0x7faadc0fd950 msgr2=0x7faadc106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:07.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.702+0000 7faae3a9a640 1 -- 172.21.15.67:0/2437582541 shutdown_connections 2024-07-27T03:19:07.706 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.702+0000 7faae3a9a640 1 -- 172.21.15.67:0/2437582541 wait complete. 2024-07-27T03:19:07.716 DEBUG:teuthology.orchestra.run.smithi067:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-07-27T03:19:07.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.830+0000 7f83c1b57640 1 -- 172.21.15.67:0/3311469135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83bc06acc0 msgr2=0x7f83bc105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:07.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.830+0000 7f83c1b57640 1 --2- 172.21.15.67:0/3311469135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83bc06acc0 0x7f83bc105d20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009900 tx=0x7f83ac02a410 comp rx=0 tx=0).stop 2024-07-27T03:19:07.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.830+0000 7f83c1b57640 1 -- 172.21.15.67:0/3311469135 shutdown_connections 2024-07-27T03:19:07.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.830+0000 7f83c1b57640 1 --2- 172.21.15.67:0/3311469135 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83bc108e20 0x7f83bc10b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.830+0000 7f83c1b57640 1 --2- 172.21.15.67:0/3311469135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83bc106260 0x7f83bc1086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.830+0000 7f83c1b57640 1 --2- 172.21.15.67:0/3311469135 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83bc06acc0 0x7f83bc105d20 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.830+0000 7f83c1b57640 1 -- 172.21.15.67:0/3311469135 >> 172.21.15.67:0/3311469135 conn(0x7f83bc0fd920 msgr2=0x7f83bc0ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:07.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.830+0000 7f83c1b57640 1 -- 172.21.15.67:0/3311469135 shutdown_connections 2024-07-27T03:19:07.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.830+0000 7f83c1b57640 1 -- 172.21.15.67:0/3311469135 wait complete. 2024-07-27T03:19:07.835 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.831+0000 7f83c1b57640 1 Processor -- start 2024-07-27T03:19:07.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.831+0000 7f83c1b57640 1 -- start start 2024-07-27T03:19:07.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.831+0000 7f83c1b57640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83bc06acc0 0x7f83bc1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:07.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.831+0000 7f83c1b57640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83bc106260 0x7f83bc19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:07.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.831+0000 7f83c1b57640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83bc108e20 0x7f83bc19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:07.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.831+0000 7f83c1b57640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc10ed40 con 0x7f83bc106260 2024-07-27T03:19:07.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.831+0000 7f83c1b57640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc10ebc0 con 0x7f83bc108e20 2024-07-27T03:19:07.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.831+0000 7f83c1b57640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc10eec0 con 0x7f83bc06acc0 2024-07-27T03:19:07.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.831+0000 7f83baffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83bc106260 0x7f83bc19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:07.836 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.831+0000 7f83baffd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83bc106260 0x7f83bc19a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42234/0 (socket says 172.21.15.67:42234) 2024-07-27T03:19:07.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.831+0000 7f83baffd640 1 -- 172.21.15.67:0/240751111 learned_addr learned my addr 172.21.15.67:0/240751111 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:19:07.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.831+0000 7f83bbfff640 1 --2- 172.21.15.67:0/240751111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83bc108e20 0x7f83bc19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:07.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.832+0000 7f83bb7fe640 1 --2- 172.21.15.67:0/240751111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83bc06acc0 0x7f83bc1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:07.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.832+0000 7f83baffd640 1 -- 172.21.15.67:0/240751111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83bc06acc0 msgr2=0x7f83bc1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:07.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.832+0000 7f83baffd640 1 --2- 172.21.15.67:0/240751111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83bc06acc0 0x7f83bc1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.832+0000 7f83baffd640 1 -- 172.21.15.67:0/240751111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83bc108e20 msgr2=0x7f83bc19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:07.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.832+0000 7f83baffd640 1 --2- 172.21.15.67:0/240751111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83bc108e20 0x7f83bc19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:07.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.832+0000 7f83baffd640 1 -- 172.21.15.67:0/240751111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83bc19b350 con 0x7f83bc106260 2024-07-27T03:19:07.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.832+0000 7f83baffd640 1 --2- 172.21.15.67:0/240751111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83bc106260 0x7f83bc19a010 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f83a400d770 tx=0x7f83a400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:07.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.832+0000 7f83b8ff9640 1 -- 172.21.15.67:0/240751111 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a4017070 con 0x7f83bc106260 2024-07-27T03:19:07.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.832+0000 7f83c1b57640 1 -- 172.21.15.67:0/240751111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83bc19b640 con 0x7f83bc106260 2024-07-27T03:19:07.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.832+0000 7f83b8ff9640 1 -- 172.21.15.67:0/240751111 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a4009c30 con 0x7f83bc106260 2024-07-27T03:19:07.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.832+0000 7f83b8ff9640 1 -- 172.21.15.67:0/240751111 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a400f620 con 0x7f83bc106260 2024-07-27T03:19:07.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.832+0000 7f83c1b57640 1 -- 172.21.15.67:0/240751111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83bc1a1dc0 con 0x7f83bc106260 2024-07-27T03:19:07.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.833+0000 7f83c1b57640 1 -- 172.21.15.67:0/240751111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83bc1066d0 con 0x7f83bc106260 2024-07-27T03:19:07.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.835+0000 7f83b8ff9640 1 -- 172.21.15.67:0/240751111 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f83a401a070 con 0x7f83bc106260 2024-07-27T03:19:07.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.836+0000 7f83b8ff9640 1 --2- 172.21.15.67:0/240751111 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f83a80b3bd0 0x7f83a80b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:07.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.837+0000 7f83bb7fe640 1 --2- 172.21.15.67:0/240751111 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f83a80b3bd0 0x7f83a80b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:07.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.837+0000 7f83b8ff9640 1 -- 172.21.15.67:0/240751111 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f83a40e3a90 con 0x7f83bc106260 2024-07-27T03:19:07.839 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.837+0000 7f83bb7fe640 1 --2- 172.21.15.67:0/240751111 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f83a80b3bd0 0x7f83a80b60e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f83ac031440 tx=0x7f83ac0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:07.841 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:07.839+0000 7f83b8ff9640 1 -- 172.21.15.67:0/240751111 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f83a4009de0 con 0x7f83bc106260 2024-07-27T03:19:07.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:07 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/927115621' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-27T03:19:07.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:07 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1406994793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-27T03:19:07.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:07 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3516674236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-27T03:19:08.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:07 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/927115621' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-27T03:19:08.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:07 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1406994793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-27T03:19:08.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:07 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3516674236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-27T03:19:08.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:08.084+0000 7f83c1b57640 1 -- 172.21.15.67:0/240751111 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f83bc063a30 con 0x7f83bc106260 2024-07-27T03:19:08.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:08.092+0000 7f83b8ff9640 1 -- 172.21.15.67:0/240751111 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+821377 (secure 0 0 0) 0x7f83a40aad30 con 0x7f83bc106260 2024-07-27T03:19:08.104 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:19:08.104 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:19:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: "epoch": 18, 2024-07-27T03:19:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: "active_gid": 14150, 2024-07-27T03:19:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: "active_name": "x", 2024-07-27T03:19:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: "active_addrs": { 2024-07-27T03:19:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-07-27T03:19:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-07-27T03:19:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6800", 2024-07-27T03:19:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 1534270904 2024-07-27T03:19:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.105 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v1", 2024-07-27T03:19:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:6801", 2024-07-27T03:19:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 1534270904 2024-07-27T03:19:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-07-27T03:19:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: "active_addr": "172.21.15.67:6801/1534270904", 2024-07-27T03:19:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: "active_change": "2024-07-27T03:04:54.948110+0000", 2024-07-27T03:19:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: "active_mgr_features": 4540701547738038271, 2024-07-27T03:19:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: "available": true, 2024-07-27T03:19:08.106 INFO:teuthology.orchestra.run.smithi067.stdout: "standbys": [ 2024-07-27T03:19:08.107 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.107 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 24109, 2024-07-27T03:19:08.107 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "z", 2024-07-27T03:19:08.107 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_features": 4540701547738038271, 2024-07-27T03:19:08.107 INFO:teuthology.orchestra.run.smithi067.stdout: "available_modules": [ 2024-07-27T03:19:08.107 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.107 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "alerts", 2024-07-27T03:19:08.107 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.107 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.107 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.107 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-27T03:19:08.107 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-27T03:19:08.107 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.108 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.108 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.108 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-27T03:19:08.108 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.108 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.108 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.108 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How frequently to reexamine health status", 2024-07-27T03:19:08.108 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.108 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.108 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.108 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.108 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.109 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.109 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.109 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.109 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.109 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.109 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.109 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.109 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.109 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.109 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.109 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.109 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.109 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.109 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.110 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.110 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.110 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.110 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.110 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.110 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.110 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.110 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.110 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.110 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.110 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.110 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.110 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.111 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.111 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.111 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.111 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.111 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.111 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.111 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.111 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.111 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.111 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.111 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.111 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.112 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.112 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.112 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.112 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.112 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.112 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.112 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.112 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.112 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.112 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.112 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.112 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.112 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.113 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.113 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.113 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.113 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.113 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.113 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.113 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.113 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.113 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.113 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.113 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.113 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.113 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.113 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.114 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_destination": { 2024-07-27T03:19:08.114 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_destination", 2024-07-27T03:19:08.114 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.114 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.114 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.114 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.114 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.114 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.114 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.114 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email address to send alerts to", 2024-07-27T03:19:08.114 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.114 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.115 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.115 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_from_name": { 2024-07-27T03:19:08.115 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_from_name", 2024-07-27T03:19:08.115 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.115 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.115 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.115 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "Ceph", 2024-07-27T03:19:08.115 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.115 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.115 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.115 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email From: name", 2024-07-27T03:19:08.115 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.115 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.116 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.116 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_host": { 2024-07-27T03:19:08.116 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_host", 2024-07-27T03:19:08.116 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.116 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.116 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.116 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.116 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.116 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.116 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.117 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP server", 2024-07-27T03:19:08.117 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.117 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.117 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.117 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_password": { 2024-07-27T03:19:08.117 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_password", 2024-07-27T03:19:08.117 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.117 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.117 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.117 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.117 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.117 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.118 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.118 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Password to authenticate with", 2024-07-27T03:19:08.118 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.118 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.118 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.118 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_port": { 2024-07-27T03:19:08.118 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_port", 2024-07-27T03:19:08.118 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.118 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.118 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.118 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "465", 2024-07-27T03:19:08.119 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.119 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.119 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.119 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP port", 2024-07-27T03:19:08.119 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.119 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.119 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.119 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_sender": { 2024-07-27T03:19:08.119 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_sender", 2024-07-27T03:19:08.119 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.119 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.119 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.120 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.120 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.120 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.120 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.120 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP envelope sender", 2024-07-27T03:19:08.120 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.120 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.120 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.120 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_ssl": { 2024-07-27T03:19:08.120 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_ssl", 2024-07-27T03:19:08.120 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.121 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.121 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.121 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.121 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.121 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.121 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.121 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use SSL to connect to SMTP server", 2024-07-27T03:19:08.121 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.121 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.121 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.121 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_user": { 2024-07-27T03:19:08.122 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_user", 2024-07-27T03:19:08.122 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.122 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.122 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.122 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.122 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.122 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.122 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.122 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "User to authenticate as", 2024-07-27T03:19:08.122 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.122 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.122 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.123 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.123 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.123 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.123 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.123 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.123 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.123 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.123 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.123 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.123 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.123 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.123 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.124 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.124 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.124 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.124 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.124 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "balancer", 2024-07-27T03:19:08.124 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.124 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.124 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.124 INFO:teuthology.orchestra.run.smithi067.stdout: "active": { 2024-07-27T03:19:08.124 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "active", 2024-07-27T03:19:08.124 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.124 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.125 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.125 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.125 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.125 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.125 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.125 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically balance PGs across cluster", 2024-07-27T03:19:08.125 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.125 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.125 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.125 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_time": { 2024-07-27T03:19:08.125 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_time", 2024-07-27T03:19:08.126 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.126 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.126 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.126 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0000", 2024-07-27T03:19:08.126 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.126 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.126 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.126 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "beginning time of day to automatically balance", 2024-07-27T03:19:08.126 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-27T03:19:08.126 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.126 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.126 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_weekday": { 2024-07-27T03:19:08.127 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_weekday", 2024-07-27T03:19:08.127 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-27T03:19:08.127 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.127 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.127 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.127 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-07-27T03:19:08.127 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-07-27T03:19:08.127 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.127 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-07-27T03:19:08.127 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-27T03:19:08.127 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.127 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.128 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_max_iterations": { 2024-07-27T03:19:08.128 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_max_iterations", 2024-07-27T03:19:08.128 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-27T03:19:08.128 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.128 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.128 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "25", 2024-07-27T03:19:08.128 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-27T03:19:08.128 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "250", 2024-07-27T03:19:08.128 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.128 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-07-27T03:19:08.128 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.128 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.128 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.129 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_metrics": { 2024-07-27T03:19:08.129 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_metrics", 2024-07-27T03:19:08.129 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.129 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.129 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.129 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "pgs,objects,bytes", 2024-07-27T03:19:08.129 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.129 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.129 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.129 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-07-27T03:19:08.129 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-07-27T03:19:08.129 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.130 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.130 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_step": { 2024-07-27T03:19:08.130 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_step", 2024-07-27T03:19:08.130 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-27T03:19:08.130 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.130 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.130 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.5", 2024-07-27T03:19:08.130 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0.001", 2024-07-27T03:19:08.130 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "0.999", 2024-07-27T03:19:08.130 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.130 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "aggressiveness of optimization", 2024-07-27T03:19:08.130 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-07-27T03:19:08.131 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.131 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.131 INFO:teuthology.orchestra.run.smithi067.stdout: "end_time": { 2024-07-27T03:19:08.131 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_time", 2024-07-27T03:19:08.131 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.131 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.131 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.131 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2359", 2024-07-27T03:19:08.131 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.131 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.131 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.131 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ending time of day to automatically balance", 2024-07-27T03:19:08.131 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-27T03:19:08.132 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.132 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.132 INFO:teuthology.orchestra.run.smithi067.stdout: "end_weekday": { 2024-07-27T03:19:08.132 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_weekday", 2024-07-27T03:19:08.132 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-27T03:19:08.132 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.132 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.132 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.132 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-07-27T03:19:08.132 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-07-27T03:19:08.132 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.132 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-07-27T03:19:08.133 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-27T03:19:08.133 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.133 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.133 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.133 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.133 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.133 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.133 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.133 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.133 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.133 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.133 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.133 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.133 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.134 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.134 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.134 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.134 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.134 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.134 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.134 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.134 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.134 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.135 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.135 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.135 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.135 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.135 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.135 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.135 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.135 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.135 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.135 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.135 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.135 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.136 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.136 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.136 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.136 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.136 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.136 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.136 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.136 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.136 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.136 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.136 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.136 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.137 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.137 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.137 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.137 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.137 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.137 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.137 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.137 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.137 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.137 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.137 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.137 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.137 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.138 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.138 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.138 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.138 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.138 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.138 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.138 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.138 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.138 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.138 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.138 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.139 INFO:teuthology.orchestra.run.smithi067.stdout: "min_score": { 2024-07-27T03:19:08.139 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_score", 2024-07-27T03:19:08.139 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-27T03:19:08.139 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.139 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.139 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.139 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.139 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.139 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.139 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-07-27T03:19:08.139 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.139 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.139 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.140 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.140 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-07-27T03:19:08.140 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-07-27T03:19:08.140 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.140 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.140 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.140 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "upmap", 2024-07-27T03:19:08.140 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.140 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.140 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.140 INFO:teuthology.orchestra.run.smithi067.stdout: "crush-compat", 2024-07-27T03:19:08.140 INFO:teuthology.orchestra.run.smithi067.stdout: "none", 2024-07-27T03:19:08.140 INFO:teuthology.orchestra.run.smithi067.stdout: "read", 2024-07-27T03:19:08.141 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap", 2024-07-27T03:19:08.141 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap-read" 2024-07-27T03:19:08.141 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.141 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Balancer mode", 2024-07-27T03:19:08.141 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.141 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.141 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.141 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_ids": { 2024-07-27T03:19:08.141 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_ids", 2024-07-27T03:19:08.141 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.142 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.142 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.142 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.142 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.142 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.142 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.142 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-07-27T03:19:08.142 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.142 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.142 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.142 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-27T03:19:08.142 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-27T03:19:08.143 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.143 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.143 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.143 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-27T03:19:08.143 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.143 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.143 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.143 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-07-27T03:19:08.143 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.143 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.143 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.143 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.144 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.144 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.144 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.144 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.144 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.144 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.144 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.144 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.144 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.144 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.144 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.144 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.144 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_deviation": { 2024-07-27T03:19:08.145 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_deviation", 2024-07-27T03:19:08.145 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.145 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.145 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.145 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-27T03:19:08.145 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-27T03:19:08.145 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.145 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.145 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "deviation below which no optimization is attempted", 2024-07-27T03:19:08.146 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-07-27T03:19:08.146 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.146 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.146 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_optimizations": { 2024-07-27T03:19:08.146 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_optimizations", 2024-07-27T03:19:08.146 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-27T03:19:08.146 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.146 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.146 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:08.146 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.146 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.146 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.147 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-07-27T03:19:08.147 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.147 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.147 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.147 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.147 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.147 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm", 2024-07-27T03:19:08.147 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.147 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.147 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.147 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_down_multiplier": { 2024-07-27T03:19:08.148 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_down_multiplier", 2024-07-27T03:19:08.148 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-27T03:19:08.148 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.148 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.148 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-07-27T03:19:08.148 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.148 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.148 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.148 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-07-27T03:19:08.148 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.148 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.148 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.148 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_refresh_rate": { 2024-07-27T03:19:08.149 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_refresh_rate", 2024-07-27T03:19:08.149 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.149 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.149 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.149 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "20", 2024-07-27T03:19:08.149 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.149 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.149 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.149 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-07-27T03:19:08.149 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.149 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.149 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.150 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_starting_port": { 2024-07-27T03:19:08.150 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_starting_port", 2024-07-27T03:19:08.150 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.150 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.150 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.150 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4721", 2024-07-27T03:19:08.150 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.150 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.150 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.150 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-07-27T03:19:08.150 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.150 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.151 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.151 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_ptrace": { 2024-07-27T03:19:08.151 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_ptrace", 2024-07-27T03:19:08.151 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.151 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.151 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.151 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.151 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.151 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.151 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.151 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-07-27T03:19:08.151 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-07-27T03:19:08.151 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.152 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.152 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_interval": { 2024-07-27T03:19:08.152 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_interval", 2024-07-27T03:19:08.152 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.152 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.152 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.152 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-27T03:19:08.152 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.152 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.152 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.152 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to autotune daemon memory", 2024-07-27T03:19:08.152 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.153 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.153 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.153 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_memory_target_ratio": { 2024-07-27T03:19:08.153 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_memory_target_ratio", 2024-07-27T03:19:08.153 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-27T03:19:08.153 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.153 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.153 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.7", 2024-07-27T03:19:08.153 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.153 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.153 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.153 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-07-27T03:19:08.154 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.154 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.154 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.154 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm_log_destination": { 2024-07-27T03:19:08.154 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm_log_destination", 2024-07-27T03:19:08.154 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.154 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.154 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.155 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.155 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.155 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.155 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.155 INFO:teuthology.orchestra.run.smithi067.stdout: "file", 2024-07-27T03:19:08.155 INFO:teuthology.orchestra.run.smithi067.stdout: "file,syslog", 2024-07-27T03:19:08.155 INFO:teuthology.orchestra.run.smithi067.stdout: "syslog" 2024-07-27T03:19:08.155 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.155 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-07-27T03:19:08.155 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.155 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.155 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.156 INFO:teuthology.orchestra.run.smithi067.stdout: "cgroups_split": { 2024-07-27T03:19:08.156 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cgroups_split", 2024-07-27T03:19:08.156 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.156 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.156 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.156 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.156 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.156 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.156 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.156 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-07-27T03:19:08.156 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.156 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.156 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.157 INFO:teuthology.orchestra.run.smithi067.stdout: "config_checks_enabled": { 2024-07-27T03:19:08.157 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_checks_enabled", 2024-07-27T03:19:08.157 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.157 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.157 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.157 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.157 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.157 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.157 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.157 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-07-27T03:19:08.157 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.157 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.157 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.158 INFO:teuthology.orchestra.run.smithi067.stdout: "config_dashboard": { 2024-07-27T03:19:08.158 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_dashboard", 2024-07-27T03:19:08.158 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.158 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.158 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.158 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.158 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.158 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.158 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.158 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-07-27T03:19:08.158 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.158 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.158 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.159 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.159 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_alertmanager": { 2024-07-27T03:19:08.159 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_alertmanager", 2024-07-27T03:19:08.159 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.159 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.159 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.159 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-07-27T03:19:08.159 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.159 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.159 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.159 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-27T03:19:08.159 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.159 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.159 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.160 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_base": { 2024-07-27T03:19:08.160 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_base", 2024-07-27T03:19:08.160 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.160 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.160 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.160 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/ceph", 2024-07-27T03:19:08.160 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.160 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.160 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.160 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Container image name, without the tag", 2024-07-27T03:19:08.160 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.160 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.160 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.161 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.161 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_elasticsearch": { 2024-07-27T03:19:08.161 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_elasticsearch", 2024-07-27T03:19:08.161 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.161 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.161 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.161 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-07-27T03:19:08.161 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.161 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.161 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.161 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "elasticsearch container image", 2024-07-27T03:19:08.161 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.162 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.162 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.162 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_grafana": { 2024-07-27T03:19:08.162 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_grafana", 2024-07-27T03:19:08.162 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.162 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.162 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.162 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-07-27T03:19:08.162 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.162 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.162 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.163 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-27T03:19:08.163 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.163 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.163 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.163 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_haproxy": { 2024-07-27T03:19:08.163 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_haproxy", 2024-07-27T03:19:08.163 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.163 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.163 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.163 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-07-27T03:19:08.163 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.163 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.163 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.164 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "HAproxy container image", 2024-07-27T03:19:08.164 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.164 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.164 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.164 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_agent": { 2024-07-27T03:19:08.164 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_agent", 2024-07-27T03:19:08.164 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.164 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.164 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.164 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-07-27T03:19:08.164 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.164 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.164 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.165 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger agent container image", 2024-07-27T03:19:08.165 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.165 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.165 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.165 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_collector": { 2024-07-27T03:19:08.165 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_collector", 2024-07-27T03:19:08.165 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.165 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.165 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.165 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-07-27T03:19:08.166 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.166 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.166 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.166 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger collector container image", 2024-07-27T03:19:08.166 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.166 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.166 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.166 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_query": { 2024-07-27T03:19:08.166 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_query", 2024-07-27T03:19:08.166 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.167 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.167 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.167 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-07-27T03:19:08.167 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.167 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.167 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.167 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger query container image", 2024-07-27T03:19:08.167 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.167 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.167 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.167 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_keepalived": { 2024-07-27T03:19:08.167 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_keepalived", 2024-07-27T03:19:08.167 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.167 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.168 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.168 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-07-27T03:19:08.168 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.168 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.168 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.168 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Keepalived container image", 2024-07-27T03:19:08.168 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.168 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.168 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.168 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.168 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_loki": { 2024-07-27T03:19:08.168 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_loki", 2024-07-27T03:19:08.168 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.169 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.169 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.169 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-07-27T03:19:08.169 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.169 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.169 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.169 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Loki container image", 2024-07-27T03:19:08.169 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.169 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.169 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.169 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.169 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nginx": { 2024-07-27T03:19:08.169 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nginx", 2024-07-27T03:19:08.169 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.170 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.170 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.170 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-07-27T03:19:08.170 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.170 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.170 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.170 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nginx container image", 2024-07-27T03:19:08.170 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.170 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.170 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.170 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.170 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_node_exporter": { 2024-07-27T03:19:08.170 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_node_exporter", 2024-07-27T03:19:08.171 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.171 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.171 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.171 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-07-27T03:19:08.171 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.171 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.171 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.171 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-27T03:19:08.171 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.171 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.171 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.171 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.171 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nvmeof": { 2024-07-27T03:19:08.171 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nvmeof", 2024-07-27T03:19:08.172 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.172 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.172 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.172 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-07-27T03:19:08.172 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.172 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.172 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.172 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nvme-of container image", 2024-07-27T03:19:08.172 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.172 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.172 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.172 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.172 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_prometheus": { 2024-07-27T03:19:08.172 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_prometheus", 2024-07-27T03:19:08.173 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.173 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.173 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.173 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-07-27T03:19:08.173 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.173 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.173 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.173 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-27T03:19:08.173 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.173 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.173 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.173 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.173 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_promtail": { 2024-07-27T03:19:08.173 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_promtail", 2024-07-27T03:19:08.173 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.174 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.174 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.174 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-07-27T03:19:08.174 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.174 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.174 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.174 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Promtail container image", 2024-07-27T03:19:08.174 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.174 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.174 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.174 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.174 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_samba": { 2024-07-27T03:19:08.174 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_samba", 2024-07-27T03:19:08.175 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.175 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.175 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.175 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-07-27T03:19:08.175 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.175 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.175 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.175 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Samba/SMB container image", 2024-07-27T03:19:08.175 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.175 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.175 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.175 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.175 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_snmp_gateway": { 2024-07-27T03:19:08.176 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_snmp_gateway", 2024-07-27T03:19:08.176 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.176 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.176 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.176 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-07-27T03:19:08.176 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.176 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.176 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.176 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SNMP Gateway container image", 2024-07-27T03:19:08.176 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.176 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.176 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.176 INFO:teuthology.orchestra.run.smithi067.stdout: "container_init": { 2024-07-27T03:19:08.177 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_init", 2024-07-27T03:19:08.177 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.177 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.177 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.177 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.177 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.177 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.177 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.177 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Run podman/docker with `--init`", 2024-07-27T03:19:08.177 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.177 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.177 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.177 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.177 INFO:teuthology.orchestra.run.smithi067.stdout: "daemon_cache_timeout": { 2024-07-27T03:19:08.178 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "daemon_cache_timeout", 2024-07-27T03:19:08.178 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.178 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.178 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.178 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-27T03:19:08.178 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.178 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.178 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.178 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-07-27T03:19:08.178 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.178 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.178 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.179 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.179 INFO:teuthology.orchestra.run.smithi067.stdout: "default_cephadm_command_timeout": { 2024-07-27T03:19:08.179 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_cephadm_command_timeout", 2024-07-27T03:19:08.179 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.179 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.179 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.179 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "900", 2024-07-27T03:19:08.179 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.179 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.179 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.179 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-07-27T03:19:08.179 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.179 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.180 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.180 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.180 INFO:teuthology.orchestra.run.smithi067.stdout: "default_registry": { 2024-07-27T03:19:08.180 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_registry", 2024-07-27T03:19:08.180 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.180 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.180 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.180 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io", 2024-07-27T03:19:08.180 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.180 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.180 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.180 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-07-27T03:19:08.180 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.180 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.181 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.181 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.181 INFO:teuthology.orchestra.run.smithi067.stdout: "device_cache_timeout": { 2024-07-27T03:19:08.181 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_cache_timeout", 2024-07-27T03:19:08.181 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.181 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.181 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.181 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1800", 2024-07-27T03:19:08.181 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.181 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.181 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.181 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache device inventory", 2024-07-27T03:19:08.181 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.181 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.182 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.182 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.182 INFO:teuthology.orchestra.run.smithi067.stdout: "device_enhanced_scan": { 2024-07-27T03:19:08.182 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_enhanced_scan", 2024-07-27T03:19:08.182 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.182 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.182 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.182 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.182 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.182 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.182 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.182 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use libstoragemgmt during device scans", 2024-07-27T03:19:08.182 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.183 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.183 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.183 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.183 INFO:teuthology.orchestra.run.smithi067.stdout: "facts_cache_timeout": { 2024-07-27T03:19:08.183 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "facts_cache_timeout", 2024-07-27T03:19:08.183 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.183 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.183 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.183 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-27T03:19:08.183 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.183 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.183 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.183 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache host facts data", 2024-07-27T03:19:08.184 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.184 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.184 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.184 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.184 INFO:teuthology.orchestra.run.smithi067.stdout: "grafana_dashboards_path": { 2024-07-27T03:19:08.184 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "grafana_dashboards_path", 2024-07-27T03:19:08.184 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.184 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.184 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.184 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-07-27T03:19:08.184 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.184 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.184 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.184 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-07-27T03:19:08.185 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.185 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.185 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.185 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.185 INFO:teuthology.orchestra.run.smithi067.stdout: "host_check_interval": { 2024-07-27T03:19:08.185 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "host_check_interval", 2024-07-27T03:19:08.185 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.185 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.185 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.185 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-27T03:19:08.185 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.185 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.185 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.185 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to perform a host check", 2024-07-27T03:19:08.186 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.186 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.186 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.186 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.186 INFO:teuthology.orchestra.run.smithi067.stdout: "hw_monitoring": { 2024-07-27T03:19:08.186 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hw_monitoring", 2024-07-27T03:19:08.186 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.186 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.186 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.186 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.186 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.186 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.186 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.186 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-07-27T03:19:08.187 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.187 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.187 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.187 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.187 INFO:teuthology.orchestra.run.smithi067.stdout: "inventory_list_all": { 2024-07-27T03:19:08.187 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "inventory_list_all", 2024-07-27T03:19:08.187 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.187 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.187 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.187 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.187 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.187 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.188 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.188 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-07-27T03:19:08.188 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.188 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.188 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.188 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.188 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.188 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.188 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.188 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.188 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.188 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.188 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.188 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.189 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.189 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.189 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.189 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.189 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.189 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.189 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.189 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.189 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.189 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.189 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.189 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.189 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.190 INFO:teuthology.orchestra.run.smithi067.stdout: "log_refresh_metadata": { 2024-07-27T03:19:08.190 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_refresh_metadata", 2024-07-27T03:19:08.190 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.190 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.190 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.190 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.190 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.190 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.190 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.190 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-07-27T03:19:08.190 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.190 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.191 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.191 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.191 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.191 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.191 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.191 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.191 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.191 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.191 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.191 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.191 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.191 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-07-27T03:19:08.191 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.192 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.192 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.192 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.192 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.192 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.192 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.194 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.194 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.194 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.194 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.194 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.194 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.194 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.194 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.194 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.194 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.194 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.194 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.194 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.195 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.195 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.195 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.195 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.195 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.195 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.195 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.195 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.195 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.195 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.195 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.195 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.195 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.196 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.196 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.196 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.196 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.196 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.196 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.196 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf": { 2024-07-27T03:19:08.196 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-07-27T03:19:08.196 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.196 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.196 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.196 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.196 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.196 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.197 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.197 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-07-27T03:19:08.197 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.197 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.197 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.197 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.197 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-07-27T03:19:08.197 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-07-27T03:19:08.197 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.197 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.197 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.197 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "*", 2024-07-27T03:19:08.197 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.197 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.198 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.198 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-07-27T03:19:08.198 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.198 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.198 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.198 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.198 INFO:teuthology.orchestra.run.smithi067.stdout: "max_count_per_host": { 2024-07-27T03:19:08.198 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_count_per_host", 2024-07-27T03:19:08.198 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.198 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.198 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.198 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:08.198 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.198 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.199 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.199 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of daemons per service per host", 2024-07-27T03:19:08.199 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.199 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.199 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.199 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.199 INFO:teuthology.orchestra.run.smithi067.stdout: "max_osd_draining_count": { 2024-07-27T03:19:08.199 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_osd_draining_count", 2024-07-27T03:19:08.199 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.199 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.199 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.199 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:08.199 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.200 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.200 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.200 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-07-27T03:19:08.200 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.200 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.200 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.200 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.200 INFO:teuthology.orchestra.run.smithi067.stdout: "migration_current": { 2024-07-27T03:19:08.200 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "migration_current", 2024-07-27T03:19:08.200 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.200 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.200 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.200 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.200 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.201 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.201 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.201 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "internal - do not modify", 2024-07-27T03:19:08.201 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.201 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.201 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.201 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.201 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-07-27T03:19:08.201 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-07-27T03:19:08.201 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.201 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.201 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.201 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "root", 2024-07-27T03:19:08.201 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.202 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.202 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.202 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm-package", 2024-07-27T03:19:08.202 INFO:teuthology.orchestra.run.smithi067.stdout: "root" 2024-07-27T03:19:08.202 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.202 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "mode for remote execution of cephadm", 2024-07-27T03:19:08.202 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.202 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.202 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.202 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.202 INFO:teuthology.orchestra.run.smithi067.stdout: "oob_default_addr": { 2024-07-27T03:19:08.202 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "oob_default_addr", 2024-07-27T03:19:08.203 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.203 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.203 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.203 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "169.254.1.1", 2024-07-27T03:19:08.203 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.203 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.203 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.203 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default address for RedFish API (oob management).", 2024-07-27T03:19:08.203 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.203 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.203 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.203 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.203 INFO:teuthology.orchestra.run.smithi067.stdout: "prometheus_alerts_path": { 2024-07-27T03:19:08.203 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus_alerts_path", 2024-07-27T03:19:08.204 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.204 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.204 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.204 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-07-27T03:19:08.204 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.204 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.204 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.204 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-07-27T03:19:08.204 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.204 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.204 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.204 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.204 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_insecure": { 2024-07-27T03:19:08.205 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_insecure", 2024-07-27T03:19:08.205 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.205 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.205 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.205 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.205 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.205 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.205 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.205 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-07-27T03:19:08.205 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.205 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.205 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.205 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.205 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_password": { 2024-07-27T03:19:08.206 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_password", 2024-07-27T03:19:08.206 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.206 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.206 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.206 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.206 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.206 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.206 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.206 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-07-27T03:19:08.206 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.206 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.206 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.206 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.207 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_url": { 2024-07-27T03:19:08.207 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_url", 2024-07-27T03:19:08.207 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.207 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.207 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.207 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.207 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.207 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.207 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.207 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-07-27T03:19:08.207 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.207 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.207 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.207 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.208 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_username": { 2024-07-27T03:19:08.208 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_username", 2024-07-27T03:19:08.208 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.208 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.208 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.208 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.208 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.208 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.208 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.208 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-07-27T03:19:08.208 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.208 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.208 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.208 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.209 INFO:teuthology.orchestra.run.smithi067.stdout: "secure_monitoring_stack": { 2024-07-27T03:19:08.209 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secure_monitoring_stack", 2024-07-27T03:19:08.209 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.209 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.209 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.209 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.209 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.209 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.209 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.209 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-07-27T03:19:08.209 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.209 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.209 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.210 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.210 INFO:teuthology.orchestra.run.smithi067.stdout: "service_discovery_port": { 2024-07-27T03:19:08.210 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "service_discovery_port", 2024-07-27T03:19:08.210 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.210 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.210 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.210 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8765", 2024-07-27T03:19:08.210 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.210 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.210 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.210 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "cephadm service discovery port", 2024-07-27T03:19:08.210 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.210 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.211 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.211 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.211 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.211 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.211 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.211 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.211 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.211 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.211 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.211 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.211 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.211 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.212 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.212 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.212 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.212 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.212 INFO:teuthology.orchestra.run.smithi067.stdout: "ssh_config_file": { 2024-07-27T03:19:08.212 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssh_config_file", 2024-07-27T03:19:08.212 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.212 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.212 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.212 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.212 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.212 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.212 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.213 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-07-27T03:19:08.213 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.213 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.213 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.213 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.213 INFO:teuthology.orchestra.run.smithi067.stdout: "use_agent": { 2024-07-27T03:19:08.213 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_agent", 2024-07-27T03:19:08.213 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.213 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.213 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.213 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.213 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.213 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.213 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.214 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-07-27T03:19:08.214 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.214 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.214 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.214 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.214 INFO:teuthology.orchestra.run.smithi067.stdout: "use_repo_digest": { 2024-07-27T03:19:08.214 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_repo_digest", 2024-07-27T03:19:08.214 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.214 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.214 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.214 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.214 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.215 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.215 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.215 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-07-27T03:19:08.215 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.215 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.215 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.215 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.215 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_failed_host_check": { 2024-07-27T03:19:08.215 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_failed_host_check", 2024-07-27T03:19:08.215 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.215 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.215 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.215 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if the host check fails", 2024-07-27T03:19:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_daemons": { 2024-07-27T03:19:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_daemons", 2024-07-27T03:19:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-07-27T03:19:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_hosts": { 2024-07-27T03:19:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_hosts", 2024-07-27T03:19:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-07-27T03:19:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crash", 2024-07-27T03:19:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: "retain_interval": { 2024-07-27T03:19:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retain_interval", 2024-07-27T03:19:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "31536000", 2024-07-27T03:19:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain crashes before pruning them", 2024-07-27T03:19:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_recent_interval": { 2024-07-27T03:19:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_recent_interval", 2024-07-27T03:19:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1209600", 2024-07-27T03:19:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "time interval in which to warn about recent crashes", 2024-07-27T03:19:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dashboard", 2024-07-27T03:19:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-07-27T03:19:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-07-27T03:19:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_HOST": { 2024-07-27T03:19:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_HOST", 2024-07-27T03:19:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-07-27T03:19:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-07-27T03:19:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_ENABLED": { 2024-07-27T03:19:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_ENABLED", 2024-07-27T03:19:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-07-27T03:19:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-07-27T03:19:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "ENABLE_BROWSABLE_API": { 2024-07-27T03:19:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ENABLE_BROWSABLE_API", 2024-07-27T03:19:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-07-27T03:19:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-07-27T03:19:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-07-27T03:19:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-07-27T03:19:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-07-27T03:19:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-07-27T03:19:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-07-27T03:19:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-07-27T03:19:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_NFS": { 2024-07-27T03:19:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-07-27T03:19:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RBD": { 2024-07-27T03:19:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-07-27T03:19:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RGW": { 2024-07-27T03:19:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-07-27T03:19:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-07-27T03:19:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-07-27T03:19:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_PASSWORD": { 2024-07-27T03:19:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_PASSWORD", 2024-07-27T03:19:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-07-27T03:19:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-07-27T03:19:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-07-27T03:19:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_URL": { 2024-07-27T03:19:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_URL", 2024-07-27T03:19:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_USERNAME": { 2024-07-27T03:19:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_USERNAME", 2024-07-27T03:19:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-07-27T03:19:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-07-27T03:19:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-07-27T03:19:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-07-27T03:19:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-07-27T03:19:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-07-27T03:19:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-07-27T03:19:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: "ISSUE_TRACKER_API_KEY": { 2024-07-27T03:19:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-07-27T03:19:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: "MANAGED_BY_CLUSTERS": { 2024-07-27T03:19:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-07-27T03:19:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "[]", 2024-07-27T03:19:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "MULTICLUSTER_CONFIG": { 2024-07-27T03:19:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MULTICLUSTER_CONFIG", 2024-07-27T03:19:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "{}", 2024-07-27T03:19:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_HOST": { 2024-07-27T03:19:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_HOST", 2024-07-27T03:19:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-07-27T03:19:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-07-27T03:19:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-07-27T03:19:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-07-27T03:19:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-07-27T03:19:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-07-27T03:19:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-07-27T03:19:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-07-27T03:19:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-07-27T03:19:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-07-27T03:19:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-07-27T03:19:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-07-27T03:19:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-07-27T03:19:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-07-27T03:19:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-07-27T03:19:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-07-27T03:19:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_ENABLED": { 2024-07-27T03:19:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_ENABLED", 2024-07-27T03:19:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-07-27T03:19:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-07-27T03:19:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-07-27T03:19:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-07-27T03:19:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-07-27T03:19:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-07-27T03:19:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-07-27T03:19:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8", 2024-07-27T03:19:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: "REST_REQUESTS_TIMEOUT": { 2024-07-27T03:19:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-07-27T03:19:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "45", 2024-07-27T03:19:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ACCESS_KEY": { 2024-07-27T03:19:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ACCESS_KEY", 2024-07-27T03:19:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-07-27T03:19:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-07-27T03:19:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-07-27T03:19:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SECRET_KEY": { 2024-07-27T03:19:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SECRET_KEY", 2024-07-27T03:19:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SSL_VERIFY": { 2024-07-27T03:19:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SSL_VERIFY", 2024-07-27T03:19:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: "UNSAFE_TLS_v1_2": { 2024-07-27T03:19:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "UNSAFE_TLS_v1_2", 2024-07-27T03:19:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-07-27T03:19:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-07-27T03:19:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-07-27T03:19:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-07-27T03:19:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-07-27T03:19:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-07-27T03:19:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-27T03:19:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: "cross_origin_url": { 2024-07-27T03:19:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cross_origin_url", 2024-07-27T03:19:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: "crt_file": { 2024-07-27T03:19:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crt_file", 2024-07-27T03:19:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: "debug": { 2024-07-27T03:19:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "debug", 2024-07-27T03:19:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable/disable debug options", 2024-07-27T03:19:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: "jwt_token_ttl": { 2024-07-27T03:19:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "jwt_token_ttl", 2024-07-27T03:19:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "28800", 2024-07-27T03:19:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-07-27T03:19:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-07-27T03:19:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.283 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.283 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.283 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.283 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.283 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.283 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.283 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.283 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.283 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.283 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.283 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.283 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.283 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.284 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.284 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.284 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.284 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.284 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.284 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.284 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.284 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.284 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.284 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.284 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.284 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.284 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: "motd": { 2024-07-27T03:19:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "motd", 2024-07-27T03:19:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "The message of the day", 2024-07-27T03:19:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect_resolve_ip_addr": { 2024-07-27T03:19:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "redirect_resolve_ip_addr", 2024-07-27T03:19:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-07-27T03:19:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-07-27T03:19:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-07-27T03:19:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-07-27T03:19:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-07-27T03:19:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8080", 2024-07-27T03:19:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-07-27T03:19:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-07-27T03:19:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl_server_port": { 2024-07-27T03:19:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl_server_port", 2024-07-27T03:19:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8443", 2024-07-27T03:19:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-07-27T03:19:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-07-27T03:19:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "redirect", 2024-07-27T03:19:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect" 2024-07-27T03:19:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-07-27T03:19:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-07-27T03:19:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-07-27T03:19:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-07-27T03:19:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-07-27T03:19:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: "url_prefix": { 2024-07-27T03:19:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url_prefix", 2024-07-27T03:19:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "devicehealth", 2024-07-27T03:19:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_monitoring": { 2024-07-27T03:19:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_monitoring", 2024-07-27T03:19:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "monitor device health metrics", 2024-07-27T03:19:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: "mark_out_threshold": { 2024-07-27T03:19:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mark_out_threshold", 2024-07-27T03:19:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2419200", 2024-07-27T03:19:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-07-27T03:19:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_name": { 2024-07-27T03:19:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_name", 2024-07-27T03:19:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "device_health_metrics", 2024-07-27T03:19:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name of pool in which to store device health metrics", 2024-07-27T03:19:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: "retention_period": { 2024-07-27T03:19:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retention_period", 2024-07-27T03:19:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15552000", 2024-07-27T03:19:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain device health metrics", 2024-07-27T03:19:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_frequency": { 2024-07-27T03:19:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_frequency", 2024-07-27T03:19:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-07-27T03:19:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to scrape device health metrics", 2024-07-27T03:19:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: "self_heal": { 2024-07-27T03:19:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "self_heal", 2024-07-27T03:19:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-07-27T03:19:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-27T03:19:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-27T03:19:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-27T03:19:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and check device health", 2024-07-27T03:19:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_threshold": { 2024-07-27T03:19:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_threshold", 2024-07-27T03:19:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7257600", 2024-07-27T03:19:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-07-27T03:19:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "diskprediction_local", 2024-07-27T03:19:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: "predict_interval": { 2024-07-27T03:19:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predict_interval", 2024-07-27T03:19:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-07-27T03:19:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: "predictor_model": { 2024-07-27T03:19:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predictor_model", 2024-07-27T03:19:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "prophetstor", 2024-07-27T03:19:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-27T03:19:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-27T03:19:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-27T03:19:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "influx", 2024-07-27T03:19:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": false, 2024-07-27T03:19:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "influxdb python module not found", 2024-07-27T03:19:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: "batch_size": { 2024-07-27T03:19:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "batch_size", 2024-07-27T03:19:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5000", 2024-07-27T03:19:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-07-27T03:19:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: "database": { 2024-07-27T03:19:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "database", 2024-07-27T03:19:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "ceph", 2024-07-27T03:19:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-07-27T03:19:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": { 2024-07-27T03:19:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hostname", 2024-07-27T03:19:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server hostname", 2024-07-27T03:19:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-27T03:19:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-27T03:19:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "30", 2024-07-27T03:19:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "5", 2024-07-27T03:19:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-07-27T03:19:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.332 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.332 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.332 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.332 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.332 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.332 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.332 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.332 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.332 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.332 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.332 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.332 INFO:teuthology.orchestra.run.smithi067.stdout: "password": { 2024-07-27T03:19:08.332 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "password", 2024-07-27T03:19:08.333 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.333 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.333 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.333 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.333 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.333 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.333 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.333 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "password of InfluxDB server user", 2024-07-27T03:19:08.333 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.333 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.333 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.333 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.333 INFO:teuthology.orchestra.run.smithi067.stdout: "port": { 2024-07-27T03:19:08.333 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "port", 2024-07-27T03:19:08.334 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.334 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.334 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.334 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8086", 2024-07-27T03:19:08.334 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.334 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.334 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.334 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server port", 2024-07-27T03:19:08.334 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.334 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.334 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.334 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.335 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.335 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.335 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.335 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.335 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.335 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.335 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.335 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.335 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.335 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.335 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.335 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.335 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.336 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-07-27T03:19:08.336 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-07-27T03:19:08.336 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.336 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.336 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.336 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "false", 2024-07-27T03:19:08.336 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.336 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.336 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.336 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-07-27T03:19:08.336 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.336 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.336 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.337 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.337 INFO:teuthology.orchestra.run.smithi067.stdout: "threads": { 2024-07-27T03:19:08.337 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threads", 2024-07-27T03:19:08.337 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.337 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.337 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.337 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-27T03:19:08.337 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-27T03:19:08.337 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "32", 2024-07-27T03:19:08.337 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.337 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-07-27T03:19:08.337 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.337 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.337 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.338 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.338 INFO:teuthology.orchestra.run.smithi067.stdout: "username": { 2024-07-27T03:19:08.338 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "username", 2024-07-27T03:19:08.338 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.338 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.338 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.338 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.338 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.338 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.338 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.338 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "username of InfluxDB server user", 2024-07-27T03:19:08.338 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.338 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.339 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.339 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.339 INFO:teuthology.orchestra.run.smithi067.stdout: "verify_ssl": { 2024-07-27T03:19:08.339 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "verify_ssl", 2024-07-27T03:19:08.339 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.339 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.339 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.339 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "true", 2024-07-27T03:19:08.339 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.339 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.339 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.339 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-07-27T03:19:08.339 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.340 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.340 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.340 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.340 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.340 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.340 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.340 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "insights", 2024-07-27T03:19:08.340 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.340 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.340 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.340 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.340 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.341 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.341 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.341 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.341 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.341 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.341 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.341 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.341 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.341 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.341 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.341 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.341 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.341 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.341 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.342 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.342 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.342 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.342 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.342 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.342 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.342 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.342 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.342 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.342 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.342 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.343 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.343 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.343 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.343 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.343 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.343 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.343 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.343 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.343 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.343 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.343 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.343 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.344 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.344 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.344 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.344 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.344 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.344 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.344 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.344 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.344 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.344 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.344 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.344 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.344 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.345 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.345 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.345 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.345 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.345 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.345 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.345 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.345 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.345 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.345 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.345 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.346 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.346 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.346 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.346 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.346 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.346 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.346 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.346 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.346 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.346 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.346 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.346 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.347 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.347 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.347 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.347 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.347 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.347 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.347 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.347 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.347 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "iostat", 2024-07-27T03:19:08.347 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.347 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.347 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.347 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.348 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.348 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.348 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.348 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.348 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.348 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.348 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.348 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.348 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.348 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.348 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.348 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.348 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.349 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.349 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.349 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.349 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.349 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.349 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.349 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.349 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.349 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.349 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.349 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.349 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.349 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.350 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.350 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.350 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.350 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.350 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.350 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.350 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.350 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.350 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.350 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.350 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.350 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.351 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.351 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.351 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.351 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.351 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.351 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.351 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.351 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.351 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.351 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.351 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.351 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.352 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.352 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.352 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.352 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.352 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.352 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.352 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.352 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.352 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.352 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.352 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.352 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.352 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.352 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.353 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.353 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.353 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.353 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.353 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.353 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.353 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.353 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.353 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.353 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.353 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.353 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.353 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.353 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.354 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.354 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.354 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.354 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.354 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.354 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "k8sevents", 2024-07-27T03:19:08.354 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.354 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.354 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.354 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_event_retention_days": { 2024-07-27T03:19:08.354 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ceph_event_retention_days", 2024-07-27T03:19:08.354 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.354 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.355 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.355 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7", 2024-07-27T03:19:08.355 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.355 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.355 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.355 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Days to hold ceph event information within local cache", 2024-07-27T03:19:08.355 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.355 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.355 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.355 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.355 INFO:teuthology.orchestra.run.smithi067.stdout: "config_check_secs": { 2024-07-27T03:19:08.355 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_check_secs", 2024-07-27T03:19:08.355 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.356 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.356 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.356 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:08.356 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "10", 2024-07-27T03:19:08.356 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.356 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.356 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-07-27T03:19:08.356 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.356 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.356 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.356 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.356 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.356 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.357 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.357 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.357 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.357 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.357 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.357 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.357 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.357 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.357 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.357 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.357 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.357 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.357 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.357 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.357 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.358 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.358 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.358 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.358 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.358 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.358 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.358 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.358 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.358 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.358 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.358 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.358 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.358 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.358 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.359 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.359 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.359 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.359 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.359 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.359 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.359 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.359 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.359 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.359 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.359 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.359 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.359 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.359 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.359 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.360 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.360 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.360 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.360 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.360 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.360 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.360 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.360 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.360 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.360 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.360 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.360 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.360 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.361 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.361 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.361 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.361 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.361 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.361 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.361 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.361 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.361 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.361 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.361 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.361 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.361 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.362 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.362 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.362 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.362 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.362 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.362 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.362 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.362 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.362 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.362 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.362 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.362 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.363 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.363 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "localpool", 2024-07-27T03:19:08.363 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.363 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.363 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.363 INFO:teuthology.orchestra.run.smithi067.stdout: "failure_domain": { 2024-07-27T03:19:08.363 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "failure_domain", 2024-07-27T03:19:08.363 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.363 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.363 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.363 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "host", 2024-07-27T03:19:08.363 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.364 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.364 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.364 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "failure domain for any created local pool", 2024-07-27T03:19:08.364 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-07-27T03:19:08.364 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.364 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.364 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.364 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.364 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.364 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.364 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.364 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.364 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.365 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.365 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.365 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.365 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.365 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.365 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.365 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.365 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.365 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.365 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.365 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.365 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.365 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.365 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.366 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.366 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.366 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.366 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.366 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.366 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.366 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.366 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.366 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.366 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.366 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.366 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.366 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.367 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.367 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.367 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.367 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.367 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.367 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.367 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.367 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.367 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.367 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.367 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.367 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.367 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.367 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.368 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.368 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.368 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.368 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.368 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.368 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.368 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.368 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.368 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.368 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.368 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.368 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.368 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.368 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.369 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.369 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.369 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.369 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.369 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.369 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.369 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.369 INFO:teuthology.orchestra.run.smithi067.stdout: "min_size": { 2024-07-27T03:19:08.369 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_size", 2024-07-27T03:19:08.369 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.369 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.369 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.370 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.370 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.370 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.370 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.370 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default min_size for any created local pool", 2024-07-27T03:19:08.370 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-07-27T03:19:08.370 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.370 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.370 INFO:teuthology.orchestra.run.smithi067.stdout: "num_rep": { 2024-07-27T03:19:08.370 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "num_rep", 2024-07-27T03:19:08.370 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.371 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.371 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.371 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3", 2024-07-27T03:19:08.371 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.371 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.371 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.371 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default replica count for any created local pool", 2024-07-27T03:19:08.371 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.371 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.371 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.371 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.371 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_num": { 2024-07-27T03:19:08.372 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_num", 2024-07-27T03:19:08.372 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.372 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.372 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.372 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "128", 2024-07-27T03:19:08.372 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.372 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.372 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.372 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default pg_num for any created local pool", 2024-07-27T03:19:08.372 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.372 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.372 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.373 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.373 INFO:teuthology.orchestra.run.smithi067.stdout: "prefix": { 2024-07-27T03:19:08.373 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prefix", 2024-07-27T03:19:08.373 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.373 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.373 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.373 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.373 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.373 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.373 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.373 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name prefix for any created local pool", 2024-07-27T03:19:08.373 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.373 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.374 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.374 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.374 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.374 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.374 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.374 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.374 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.374 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.374 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.374 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.374 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.374 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.374 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.375 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.375 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.375 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.375 INFO:teuthology.orchestra.run.smithi067.stdout: "subtree": { 2024-07-27T03:19:08.375 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subtree", 2024-07-27T03:19:08.375 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.375 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.375 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.375 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "rack", 2024-07-27T03:19:08.375 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.375 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.375 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.375 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "CRUSH level for which to create a local pool", 2024-07-27T03:19:08.375 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-07-27T03:19:08.376 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.376 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.376 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.376 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.376 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.376 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mds_autoscaler", 2024-07-27T03:19:08.376 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.376 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.376 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.376 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.376 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.376 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.377 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.377 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.377 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.377 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.377 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.377 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.377 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.377 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.377 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.377 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.377 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.377 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.377 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.378 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.378 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.378 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.378 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.378 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.378 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.378 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.378 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.378 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.378 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.378 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.378 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.378 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.379 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.379 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.379 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.379 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.379 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.379 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.379 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.379 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.379 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.379 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.379 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.379 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.380 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.380 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.380 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.380 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.380 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.380 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.380 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.380 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.380 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.380 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.380 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.380 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.380 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.380 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.381 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.381 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.381 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.381 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.381 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.381 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.381 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.381 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.381 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.381 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.381 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.381 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.382 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.382 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.382 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.382 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.382 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.382 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.382 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.382 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.382 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.382 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.382 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.382 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.382 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.382 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.383 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.383 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.383 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirroring", 2024-07-27T03:19:08.383 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.383 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.383 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.383 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.383 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.383 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.383 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.383 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.383 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.383 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.384 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.384 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.384 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.384 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.384 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.384 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.384 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.384 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.384 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.384 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.384 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.384 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.385 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.385 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.385 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.385 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.385 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.385 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.385 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.385 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.385 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.385 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.385 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.385 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.385 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.385 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.386 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.386 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.386 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.386 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.386 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.386 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.386 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.386 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.386 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.386 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.386 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.386 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.386 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.386 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.387 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.387 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.387 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.387 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.387 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.387 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.387 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.387 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.387 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.387 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.387 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.387 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.387 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.388 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.388 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.388 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.388 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.388 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.388 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.388 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.388 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.388 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.388 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.388 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.388 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.388 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.388 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.389 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.389 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.389 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.389 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.389 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.389 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.389 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.389 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.389 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.389 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.389 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.389 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.389 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "nfs", 2024-07-27T03:19:08.389 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.390 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.390 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.390 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.390 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.390 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.390 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.390 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.390 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.390 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.390 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.390 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.390 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.390 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.391 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.391 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.391 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.391 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.391 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.391 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.391 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.391 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.391 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.391 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.391 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.391 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.391 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.392 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.392 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.392 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.392 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.392 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.392 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.392 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.392 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.392 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.392 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.392 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.392 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.392 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.392 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.393 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.393 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.393 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.393 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.393 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.393 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.393 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.393 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.393 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.393 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.393 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.393 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.393 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.394 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.394 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.394 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.394 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.394 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.394 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.394 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.394 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.394 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.394 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.394 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.394 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.394 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.394 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.395 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.395 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.395 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.395 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.395 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.395 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.395 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.395 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.395 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.395 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.395 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.395 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.395 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.396 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.396 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.396 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.396 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.396 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.396 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.396 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.396 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.396 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.396 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-07-27T03:19:08.396 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.396 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.396 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.397 INFO:teuthology.orchestra.run.smithi067.stdout: "fail_fs": { 2024-07-27T03:19:08.397 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "fail_fs", 2024-07-27T03:19:08.397 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.397 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.397 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.397 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.397 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.397 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.397 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.397 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-07-27T03:19:08.397 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.397 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.397 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.397 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.398 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.398 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.398 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.398 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.398 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.398 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.398 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.398 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.398 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.398 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.398 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.398 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.399 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.399 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.399 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.399 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.399 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.399 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.399 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.399 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.399 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.399 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.399 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.399 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.399 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.400 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.400 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.400 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.400 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.400 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.400 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.400 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.400 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.400 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.400 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.400 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.400 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.401 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.401 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.401 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.401 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.401 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.401 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.401 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.401 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.401 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.401 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.401 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.401 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.401 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.402 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.402 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.402 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.402 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.402 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.402 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.402 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.402 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.402 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.402 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.402 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.402 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.402 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.403 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.403 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.403 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.403 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.403 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.403 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.403 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator": { 2024-07-27T03:19:08.403 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-07-27T03:19:08.403 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.403 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.403 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.403 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.403 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.403 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.404 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.404 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm", 2024-07-27T03:19:08.404 INFO:teuthology.orchestra.run.smithi067.stdout: "rook", 2024-07-27T03:19:08.404 INFO:teuthology.orchestra.run.smithi067.stdout: "test_orchestrator" 2024-07-27T03:19:08.404 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.404 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Orchestrator backend", 2024-07-27T03:19:08.404 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.404 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.404 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.404 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.404 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.404 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.404 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.405 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.405 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.405 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.405 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.405 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.405 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.405 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.405 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.405 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.405 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.405 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.405 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.405 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.405 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.406 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_perf_query", 2024-07-27T03:19:08.406 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.406 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.406 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.406 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.406 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.406 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.406 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.406 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.406 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.406 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.406 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.406 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.407 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.407 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.407 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.407 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.407 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.407 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.407 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.407 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.407 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.407 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.407 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.407 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.407 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.408 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.408 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.408 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.408 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.408 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.408 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.408 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.408 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.408 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.408 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.408 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.408 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.408 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.409 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.409 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.409 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.409 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.409 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.409 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.409 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.409 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.409 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.409 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.409 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.409 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.410 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.410 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.410 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.410 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.410 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.410 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.410 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.410 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.410 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.410 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.410 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.410 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.410 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.411 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.411 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.411 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.411 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.411 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.411 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.411 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.411 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.411 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.411 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.411 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.411 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.411 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.411 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.412 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.412 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.412 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.412 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.412 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.412 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.412 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.412 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.412 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.412 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.412 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.412 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_support", 2024-07-27T03:19:08.413 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.413 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.413 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.413 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.413 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.413 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.413 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.413 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.413 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.413 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.413 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.413 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.413 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.414 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.414 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.414 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.414 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.414 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.414 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.414 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.414 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.414 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.414 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.414 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.414 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.414 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.415 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.415 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.415 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.415 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.415 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.415 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.415 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.415 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.415 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.415 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.415 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.415 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.415 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.415 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.416 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.416 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.416 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.416 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.416 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.416 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.416 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.416 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.416 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.416 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.416 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.416 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.416 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.417 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.417 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.417 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.417 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.417 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.417 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.417 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.417 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.417 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.417 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.417 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.417 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.417 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.417 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.418 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.418 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.418 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.418 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.418 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.418 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.418 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.418 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.418 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.418 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.418 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.419 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.419 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.419 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.419 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.419 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.419 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.419 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.419 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.419 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.419 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.419 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_autoscaler", 2024-07-27T03:19:08.419 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.419 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.420 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.420 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.420 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.420 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.420 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.420 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.420 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.420 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.420 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.420 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.420 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.420 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.420 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.420 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.421 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.421 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.421 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.421 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.421 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.421 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.421 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.421 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.421 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.421 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.421 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.421 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.421 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.422 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.422 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.422 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.422 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.422 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.422 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.422 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.422 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.422 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.422 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.422 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.422 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.423 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.423 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.423 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.423 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.423 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.423 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.423 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.423 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.423 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.423 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.423 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.423 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.423 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.423 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.424 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.424 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.424 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.424 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.424 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.424 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.424 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.424 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.424 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.424 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.424 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.424 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.424 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.424 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.425 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-27T03:19:08.425 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-27T03:19:08.425 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.425 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.425 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.425 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-27T03:19:08.425 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.425 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.425 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.425 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.425 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.425 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.425 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.426 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.426 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.426 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.426 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.426 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.426 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.426 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.426 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.426 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.426 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.426 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.426 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.426 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.427 INFO:teuthology.orchestra.run.smithi067.stdout: "threshold": { 2024-07-27T03:19:08.427 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threshold", 2024-07-27T03:19:08.427 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-27T03:19:08.427 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.427 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.427 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-07-27T03:19:08.427 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1.0", 2024-07-27T03:19:08.427 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.427 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.427 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "scaling threshold", 2024-07-27T03:19:08.427 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-07-27T03:19:08.428 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.428 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.428 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.428 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.428 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.428 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "progress", 2024-07-27T03:19:08.428 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.428 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.428 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.428 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_pg_recovery_event": { 2024-07-27T03:19:08.428 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_pg_recovery_event", 2024-07-27T03:19:08.428 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.428 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.429 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.429 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.429 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.429 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.429 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.429 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow the module to show pg recovery progress", 2024-07-27T03:19:08.429 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.429 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.429 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.429 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.429 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-07-27T03:19:08.429 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-07-27T03:19:08.429 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.429 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.430 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.430 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.430 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.430 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.430 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.430 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.430 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.430 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.430 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.430 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.430 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.430 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.430 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.431 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.431 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.431 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.431 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.431 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.431 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.431 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.431 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.431 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.431 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.431 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.431 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.431 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.432 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.432 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.432 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.432 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.432 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.432 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.432 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.432 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.432 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.432 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.432 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.432 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.433 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.433 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.433 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.433 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.433 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.433 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.433 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.433 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.433 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.433 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.433 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.433 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.433 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.434 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.434 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.434 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.434 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.434 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.434 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.434 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.434 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.434 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.434 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.434 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.434 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.434 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.434 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.435 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.435 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.435 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.435 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.435 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.435 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.435 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.435 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.435 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.435 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.435 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.435 INFO:teuthology.orchestra.run.smithi067.stdout: "max_completed_events": { 2024-07-27T03:19:08.435 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_completed_events", 2024-07-27T03:19:08.436 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.436 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.436 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.436 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "50", 2024-07-27T03:19:08.436 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.436 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.436 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.436 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "number of past completed events to remember", 2024-07-27T03:19:08.436 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.436 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.436 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.436 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-27T03:19:08.437 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-27T03:19:08.437 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.437 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.437 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.437 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-27T03:19:08.437 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.437 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.437 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.437 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long the module is going to sleep", 2024-07-27T03:19:08.437 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.437 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.437 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.437 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.438 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.438 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.438 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.438 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.438 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.438 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.438 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.438 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.438 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.438 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.438 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.438 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.439 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.439 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.439 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.439 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.439 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus", 2024-07-27T03:19:08.439 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.439 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.439 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.439 INFO:teuthology.orchestra.run.smithi067.stdout: "cache": { 2024-07-27T03:19:08.439 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cache", 2024-07-27T03:19:08.439 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.439 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.439 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.439 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.440 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.440 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.440 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.440 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.440 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.440 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.440 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.440 INFO:teuthology.orchestra.run.smithi067.stdout: "exclude_perf_counters": { 2024-07-27T03:19:08.440 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "exclude_perf_counters", 2024-07-27T03:19:08.440 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.440 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.440 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.441 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.441 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.441 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.441 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.441 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-07-27T03:19:08.441 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-07-27T03:19:08.441 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.441 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.442 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.442 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.442 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.442 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.442 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.442 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.442 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.442 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.442 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.442 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.442 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.442 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.442 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.443 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.444 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.445 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.446 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools": { 2024-07-27T03:19:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools", 2024-07-27T03:19:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.447 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools_refresh_interval": { 2024-07-27T03:19:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-07-27T03:19:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "300", 2024-07-27T03:19:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.448 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_interval": { 2024-07-27T03:19:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_interval", 2024-07-27T03:19:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-27T03:19:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15.0", 2024-07-27T03:19:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.449 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-07-27T03:19:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-07-27T03:19:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-07-27T03:19:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.450 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-07-27T03:19:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-07-27T03:19:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-07-27T03:19:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "9283", 2024-07-27T03:19:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.451 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-07-27T03:19:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.452 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: "stale_cache_strategy": { 2024-07-27T03:19:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stale_cache_strategy", 2024-07-27T03:19:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "log", 2024-07-27T03:19:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.453 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.454 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.454 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.454 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.454 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.454 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-07-27T03:19:08.454 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-07-27T03:19:08.454 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.454 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.454 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.454 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "default", 2024-07-27T03:19:08.454 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.454 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: "default", 2024-07-27T03:19:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: "error" 2024-07-27T03:19:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-07-27T03:19:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-07-27T03:19:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.455 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-07-27T03:19:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-07-27T03:19:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-07-27T03:19:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_support", 2024-07-27T03:19:08.456 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.457 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.457 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.457 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.457 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.457 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.457 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.457 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.457 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.457 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.457 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.457 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.457 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.457 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.457 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.458 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.458 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.458 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.458 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.458 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.458 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.458 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.458 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.458 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.458 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.458 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.458 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.459 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.459 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.459 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.459 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.459 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.459 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.459 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.459 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.459 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.459 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.459 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.459 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.459 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.460 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.460 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.460 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.460 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.460 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.460 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.460 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.460 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.460 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.460 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.460 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.460 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.461 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.461 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.461 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.461 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.461 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.461 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.461 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.461 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.461 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.461 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.461 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.461 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.462 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.462 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.462 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.462 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.462 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.462 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_snap_create": { 2024-07-27T03:19:08.462 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_snap_create", 2024-07-27T03:19:08.462 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.462 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.462 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.462 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:08.462 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.463 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.463 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.463 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.463 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.463 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.463 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.463 INFO:teuthology.orchestra.run.smithi067.stdout: "mirror_snapshot_schedule": { 2024-07-27T03:19:08.463 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirror_snapshot_schedule", 2024-07-27T03:19:08.463 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.463 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.463 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.463 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.463 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.464 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.464 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.464 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.464 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.464 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.464 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.464 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.464 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.464 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.464 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.464 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.464 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.464 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.464 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.465 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.465 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.465 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.465 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.465 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.465 INFO:teuthology.orchestra.run.smithi067.stdout: "trash_purge_schedule": { 2024-07-27T03:19:08.465 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "trash_purge_schedule", 2024-07-27T03:19:08.465 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.465 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.465 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.465 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.465 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.465 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.466 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.466 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.466 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.466 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.466 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.466 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.466 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.466 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "restful", 2024-07-27T03:19:08.466 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.466 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.466 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.467 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_auth": { 2024-07-27T03:19:08.467 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_auth", 2024-07-27T03:19:08.467 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.467 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.467 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.467 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.467 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.467 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.467 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.467 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.467 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.467 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.467 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.467 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.467 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-07-27T03:19:08.468 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-07-27T03:19:08.468 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.468 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.468 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.468 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.468 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.468 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.468 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.468 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.468 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.468 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.468 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.468 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.468 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.469 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.469 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.469 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.469 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.469 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.469 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.469 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.469 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.469 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.469 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.469 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.469 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.469 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.469 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.469 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.470 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.470 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.470 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.470 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.470 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.470 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.470 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.470 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.470 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.470 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.470 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.471 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.471 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.471 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.471 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.471 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.471 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.471 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.471 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.471 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.471 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.471 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.471 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.472 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.472 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.472 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.472 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.472 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.472 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.472 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.472 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.472 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.472 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.472 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.472 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.472 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.473 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.473 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.473 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.473 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.473 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.473 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.473 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.473 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.473 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.473 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.473 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.473 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.473 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-07-27T03:19:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-07-27T03:19:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.474 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-07-27T03:19:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-07-27T03:19:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.475 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.476 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rgw", 2024-07-27T03:19:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.477 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.478 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.479 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.480 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.481 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.482 INFO:teuthology.orchestra.run.smithi067.stdout: "secondary_zone_period_retry_limit": { 2024-07-27T03:19:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secondary_zone_period_retry_limit", 2024-07-27T03:19:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-27T03:19:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-07-27T03:19:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.483 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.484 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rook", 2024-07-27T03:19:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.485 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.486 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.487 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.487 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.487 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.487 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.487 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.487 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.487 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.487 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.487 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.487 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.487 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.487 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.487 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.488 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.488 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.488 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.488 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.488 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.488 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.488 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.488 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.488 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.488 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.488 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.488 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.488 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.489 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.489 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.489 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.489 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.489 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.489 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.489 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.489 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.489 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.489 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.489 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.489 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.489 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.490 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.490 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.490 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.490 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.490 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.490 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.490 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.490 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.490 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.490 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.490 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.490 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.491 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.491 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.491 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.491 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.491 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.491 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.491 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.491 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.491 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.491 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.491 INFO:teuthology.orchestra.run.smithi067.stdout: "storage_class": { 2024-07-27T03:19:08.491 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "storage_class", 2024-07-27T03:19:08.491 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.492 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.492 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.492 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "local", 2024-07-27T03:19:08.492 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.492 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.492 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.492 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-07-27T03:19:08.492 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.492 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.492 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.492 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.492 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.493 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.493 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.493 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "selftest", 2024-07-27T03:19:08.493 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.493 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.493 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.493 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.493 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.493 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.493 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.493 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.493 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.493 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.493 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.494 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.494 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.494 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.494 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.494 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.494 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.494 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.494 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.494 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.494 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.494 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.494 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.495 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.495 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.495 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.495 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.495 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.495 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.495 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.495 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.495 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.495 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.495 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.495 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.495 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.496 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.496 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.496 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.496 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.496 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.496 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.496 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.496 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.496 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.496 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.496 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.496 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.497 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.497 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.497 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.497 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.497 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.497 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.497 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.497 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.497 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.497 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.497 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.497 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.497 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.498 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.498 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.498 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.498 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.498 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.498 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.498 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.498 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.498 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.498 INFO:teuthology.orchestra.run.smithi067.stdout: "roption1": { 2024-07-27T03:19:08.498 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption1", 2024-07-27T03:19:08.498 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.499 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.499 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.499 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.499 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.499 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.499 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.499 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.499 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.499 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.499 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.499 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.499 INFO:teuthology.orchestra.run.smithi067.stdout: "roption2": { 2024-07-27T03:19:08.499 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption2", 2024-07-27T03:19:08.500 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.500 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.500 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.500 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "xyz", 2024-07-27T03:19:08.500 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.500 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.500 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.500 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.500 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.500 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.500 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.500 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption1": { 2024-07-27T03:19:08.501 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption1", 2024-07-27T03:19:08.501 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.501 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.501 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.501 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.501 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.501 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.501 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.501 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.501 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.501 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.501 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.501 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption2": { 2024-07-27T03:19:08.502 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption2", 2024-07-27T03:19:08.502 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.502 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.502 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.502 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.502 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.502 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.502 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.502 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.502 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.502 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.502 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.503 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption3": { 2024-07-27T03:19:08.503 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption3", 2024-07-27T03:19:08.503 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-27T03:19:08.503 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.503 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.503 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.503 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.503 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.503 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.503 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.503 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.503 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.503 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.504 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.504 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption4": { 2024-07-27T03:19:08.504 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption4", 2024-07-27T03:19:08.504 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.504 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.504 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.504 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.504 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.504 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.504 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.504 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.504 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.504 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.504 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.505 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.505 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption5": { 2024-07-27T03:19:08.505 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption5", 2024-07-27T03:19:08.505 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.505 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.505 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.505 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.505 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.505 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.505 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.505 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.505 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.505 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.506 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.506 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.506 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption6": { 2024-07-27T03:19:08.506 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption6", 2024-07-27T03:19:08.506 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.506 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.506 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.506 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.506 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.506 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.506 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.506 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.507 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.507 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.507 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.507 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.507 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption7": { 2024-07-27T03:19:08.507 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption7", 2024-07-27T03:19:08.507 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.507 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.507 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.507 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.507 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-27T03:19:08.507 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "42", 2024-07-27T03:19:08.507 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.508 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.508 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.508 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.508 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.508 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.508 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.508 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.508 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.508 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.508 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.508 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.509 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.509 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.509 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.509 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.509 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.509 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.509 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.509 INFO:teuthology.orchestra.run.smithi067.stdout: "testkey": { 2024-07-27T03:19:08.509 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testkey", 2024-07-27T03:19:08.509 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.509 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.509 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.509 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.509 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.510 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.510 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.510 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.510 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.510 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.510 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.510 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.510 INFO:teuthology.orchestra.run.smithi067.stdout: "testlkey": { 2024-07-27T03:19:08.510 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testlkey", 2024-07-27T03:19:08.510 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.510 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.510 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.510 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.510 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.511 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.511 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.511 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.511 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.511 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.511 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.511 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.511 INFO:teuthology.orchestra.run.smithi067.stdout: "testnewline": { 2024-07-27T03:19:08.511 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testnewline", 2024-07-27T03:19:08.511 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.511 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.511 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.511 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.511 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.512 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.512 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.512 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.512 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.512 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.512 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.512 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.512 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.512 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.512 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.512 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smb", 2024-07-27T03:19:08.512 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.512 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.512 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.513 INFO:teuthology.orchestra.run.smithi067.stdout: "internal_store_backend": { 2024-07-27T03:19:08.513 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "internal_store_backend", 2024-07-27T03:19:08.513 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.513 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.513 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.513 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.513 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.513 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.513 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.513 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-07-27T03:19:08.513 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.513 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.513 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.514 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.514 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.514 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.514 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.514 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.514 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.514 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.514 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.514 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.514 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.514 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.514 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.514 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.515 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.515 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.515 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.515 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.515 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.515 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.515 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.515 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.515 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.515 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.516 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.516 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.516 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.516 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.516 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.516 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.516 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.516 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.516 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.516 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.516 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.516 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.516 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.516 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.517 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.517 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.517 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.517 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.517 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.517 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.517 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.517 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.517 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.517 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.517 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.517 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.517 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.518 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.518 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.518 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.518 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.518 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.518 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.518 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.518 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.518 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.518 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.518 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.518 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.518 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.519 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.519 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.519 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.519 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.519 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.519 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.519 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.519 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.519 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.519 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.519 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.519 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.519 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.519 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.520 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.520 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.520 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.520 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.520 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.520 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.520 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.520 INFO:teuthology.orchestra.run.smithi067.stdout: "update_orchestration": { 2024-07-27T03:19:08.520 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "update_orchestration", 2024-07-27T03:19:08.520 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.520 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.520 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.520 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.521 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.521 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.521 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.521 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-07-27T03:19:08.521 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.521 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.521 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.521 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.521 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.521 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.521 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snap_schedule", 2024-07-27T03:19:08.521 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.521 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.522 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.522 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_m_granularity": { 2024-07-27T03:19:08.522 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_m_granularity", 2024-07-27T03:19:08.522 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.522 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.522 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.522 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.522 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.522 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.522 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.522 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow minute scheduled snapshots", 2024-07-27T03:19:08.522 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.522 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.523 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.523 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.523 INFO:teuthology.orchestra.run.smithi067.stdout: "dump_on_update": { 2024-07-27T03:19:08.523 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dump_on_update", 2024-07-27T03:19:08.523 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.523 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.523 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.523 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.523 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.523 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.523 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.523 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "dump database to debug log on update", 2024-07-27T03:19:08.523 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.524 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.524 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.524 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.524 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.524 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.524 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.524 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.524 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.524 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.524 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.524 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.524 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.524 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.525 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.525 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.525 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.525 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.525 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.525 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.525 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.525 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.525 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.525 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.525 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.525 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.525 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.526 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.526 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.526 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.526 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.526 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.526 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.526 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.526 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.526 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.526 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.526 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.526 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.526 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.527 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.527 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.527 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.527 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.527 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.527 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.527 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.527 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.527 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.527 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.527 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.527 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.528 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.528 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.528 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.528 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.528 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.528 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.528 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.528 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.528 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.528 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.528 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.529 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.529 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.529 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.529 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.529 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.529 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.529 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.529 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.529 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.529 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.529 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.529 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.529 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.530 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.530 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.530 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.530 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.530 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.530 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.530 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.530 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.530 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.530 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.530 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.530 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.531 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stats", 2024-07-27T03:19:08.531 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.531 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.531 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.531 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.531 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.531 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.531 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.531 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.531 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.531 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.531 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.531 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.531 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.532 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.532 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.532 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.532 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.532 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.532 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.532 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.532 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.532 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.532 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.532 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.532 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.532 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.533 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.533 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.533 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.533 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.533 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.533 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.533 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.533 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.533 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.533 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.533 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.533 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.533 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.533 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.534 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.534 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.534 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.534 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.534 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.534 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.534 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.534 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.534 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.534 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.534 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.534 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.534 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.535 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.535 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.535 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.535 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.535 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.535 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.535 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.535 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.535 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.535 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.535 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.535 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.535 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.536 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.536 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.536 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.536 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.536 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.536 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.536 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.536 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.536 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.536 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.536 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.536 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.537 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.537 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.537 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.537 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.537 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.537 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.537 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.537 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.537 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.537 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "status", 2024-07-27T03:19:08.537 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.537 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.537 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.537 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.538 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.538 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.538 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.538 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.538 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.538 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.538 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.538 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.538 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.538 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.538 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.538 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.539 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.539 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.539 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.539 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.539 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.539 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.539 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.539 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.539 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.539 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.539 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.539 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.539 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.539 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.540 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.540 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.540 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.540 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.540 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.540 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.540 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.540 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.540 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.540 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.540 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.540 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.540 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.540 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.541 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.541 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.541 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.541 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.541 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.541 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.541 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.541 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.541 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.541 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.541 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.541 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.541 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.541 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.542 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.542 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.542 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.542 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.542 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.542 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.542 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.542 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.542 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.542 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.542 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.542 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.543 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.543 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.543 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.543 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.543 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.543 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.543 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.543 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.543 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.543 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.543 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.543 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.543 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.544 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.544 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.544 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.544 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.544 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.544 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telegraf", 2024-07-27T03:19:08.544 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.544 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.544 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.544 INFO:teuthology.orchestra.run.smithi067.stdout: "address": { 2024-07-27T03:19:08.544 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "address", 2024-07-27T03:19:08.544 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.544 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.545 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.545 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-07-27T03:19:08.545 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.545 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.545 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.545 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.545 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.545 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.545 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.545 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-27T03:19:08.545 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-27T03:19:08.545 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.545 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.546 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.546 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15", 2024-07-27T03:19:08.546 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.546 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.546 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.546 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.546 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.546 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.546 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.546 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.546 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.546 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.546 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.547 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.547 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.547 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.549 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.550 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.550 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.550 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.550 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.550 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.550 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.550 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.550 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.550 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.550 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.550 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.551 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.551 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.551 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.551 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.551 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.551 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.551 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.551 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.551 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.551 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.551 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.551 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.551 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.552 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.552 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.552 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.552 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.552 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.552 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.552 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.552 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.552 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.552 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.552 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.552 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.552 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.553 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.553 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.553 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.553 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.553 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.553 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.553 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.553 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.553 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.553 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.553 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.553 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.553 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.553 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.554 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.554 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.554 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.554 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.554 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.554 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.554 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.554 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.554 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.554 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.554 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.554 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.555 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.555 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.555 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.555 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.555 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.555 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.555 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.555 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.555 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.555 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.555 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.555 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.555 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.556 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.556 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telemetry", 2024-07-27T03:19:08.556 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.556 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.556 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.556 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_basic": { 2024-07-27T03:19:08.556 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_basic", 2024-07-27T03:19:08.556 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.556 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.556 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.556 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.556 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.556 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.557 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.557 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share basic cluster information (size, version)", 2024-07-27T03:19:08.557 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.557 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.557 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.557 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.557 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_crash": { 2024-07-27T03:19:08.557 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_crash", 2024-07-27T03:19:08.557 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.557 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.557 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.557 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.557 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.558 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.558 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.558 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-07-27T03:19:08.558 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.558 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.558 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.558 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_device": { 2024-07-27T03:19:08.558 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_device", 2024-07-27T03:19:08.558 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.558 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.558 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.558 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.559 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.559 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.559 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.559 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-07-27T03:19:08.559 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.559 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.559 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.559 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.559 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_ident": { 2024-07-27T03:19:08.559 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_ident", 2024-07-27T03:19:08.559 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.559 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.559 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.560 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.560 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.560 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.560 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.560 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-07-27T03:19:08.560 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.560 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.560 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.560 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_perf": { 2024-07-27T03:19:08.560 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_perf", 2024-07-27T03:19:08.560 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.560 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.561 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.561 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.561 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.561 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.561 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.561 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share various performance metrics of a cluster", 2024-07-27T03:19:08.561 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.561 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.561 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.561 INFO:teuthology.orchestra.run.smithi067.stdout: "contact": { 2024-07-27T03:19:08.561 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "contact", 2024-07-27T03:19:08.561 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.562 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.562 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.562 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.562 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.562 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.562 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.562 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.562 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.562 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.562 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.562 INFO:teuthology.orchestra.run.smithi067.stdout: "description": { 2024-07-27T03:19:08.562 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "description", 2024-07-27T03:19:08.563 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.563 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.563 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.563 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.563 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.563 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.563 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.563 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.563 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.563 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.563 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.564 INFO:teuthology.orchestra.run.smithi067.stdout: "device_url": { 2024-07-27T03:19:08.564 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_url", 2024-07-27T03:19:08.564 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.564 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.564 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.564 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-07-27T03:19:08.564 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.564 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.564 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.564 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.564 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.564 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.564 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.565 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-07-27T03:19:08.565 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-07-27T03:19:08.565 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.565 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.565 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.565 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.565 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.565 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.565 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.565 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.565 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.565 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.565 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.566 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-27T03:19:08.566 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-27T03:19:08.566 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.566 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.566 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.566 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "24", 2024-07-27T03:19:08.566 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "8", 2024-07-27T03:19:08.566 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.566 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.566 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.566 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.567 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.567 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.567 INFO:teuthology.orchestra.run.smithi067.stdout: "last_opt_revision": { 2024-07-27T03:19:08.567 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "last_opt_revision", 2024-07-27T03:19:08.567 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.567 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.567 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.567 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1", 2024-07-27T03:19:08.567 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.567 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.567 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.567 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.567 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.567 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.568 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.568 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard": { 2024-07-27T03:19:08.568 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard", 2024-07-27T03:19:08.568 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.568 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.568 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.568 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.568 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.568 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.568 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.568 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.568 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.568 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.568 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.569 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard_description": { 2024-07-27T03:19:08.569 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard_description", 2024-07-27T03:19:08.569 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.569 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.569 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.569 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.569 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.569 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.569 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.569 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.569 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.569 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.569 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.570 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.570 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.570 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.570 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.570 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.570 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.570 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.570 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.570 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.570 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.570 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.570 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.570 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.571 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.571 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.571 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.571 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.571 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.571 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.571 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.571 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.571 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.571 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.571 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.571 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.571 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.572 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.572 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.572 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.572 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.572 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.572 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.572 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.572 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.572 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.572 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.572 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.572 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.572 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.573 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.573 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.573 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.573 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.573 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.573 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.573 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.573 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.573 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.573 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.573 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.573 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.573 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.574 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.574 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.574 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.574 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.574 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.574 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.574 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.574 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.574 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.574 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.574 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.575 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.575 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.575 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.575 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.575 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.575 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.575 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.575 INFO:teuthology.orchestra.run.smithi067.stdout: "organization": { 2024-07-27T03:19:08.575 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "organization", 2024-07-27T03:19:08.575 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.575 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.575 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.575 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.576 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.576 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.576 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.576 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.576 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.576 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.576 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.576 INFO:teuthology.orchestra.run.smithi067.stdout: "proxy": { 2024-07-27T03:19:08.576 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "proxy", 2024-07-27T03:19:08.576 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.576 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.576 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.577 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.577 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.577 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.577 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.577 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.577 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.577 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.577 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.577 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.577 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.577 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.577 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.577 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.578 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.578 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.578 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.578 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.578 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.578 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.578 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.578 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.578 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.578 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.578 INFO:teuthology.orchestra.run.smithi067.stdout: "url": { 2024-07-27T03:19:08.578 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url", 2024-07-27T03:19:08.578 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.579 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.579 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.579 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-07-27T03:19:08.579 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.579 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.579 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.579 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.579 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.579 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.579 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.579 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.579 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.580 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.580 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "test_orchestrator", 2024-07-27T03:19:08.580 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.580 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.580 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.580 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.580 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.580 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.580 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.580 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.580 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.580 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.581 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.581 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.581 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.581 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.581 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.581 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.581 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.581 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.581 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.581 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.581 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.581 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.581 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.581 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.582 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.582 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.582 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.582 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.582 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.582 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.582 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.582 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.582 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.582 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.582 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.582 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.582 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.583 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.583 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.583 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.583 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.583 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.583 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.583 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.583 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.583 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.583 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.583 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.583 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.584 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.584 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.584 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.584 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.584 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.584 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.584 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.584 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.584 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.584 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.584 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.584 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.584 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.584 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.585 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.585 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.585 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.585 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.585 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.585 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.585 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.585 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.585 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.585 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.585 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.585 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.585 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.585 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.586 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.586 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.586 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.586 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.586 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.586 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.586 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.586 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.586 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.586 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.586 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.586 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "volumes", 2024-07-27T03:19:08.586 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.586 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.587 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.587 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.587 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.587 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.587 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.587 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.587 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.587 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.587 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.587 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.587 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.587 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.587 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.587 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.588 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.588 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.588 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.588 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.588 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.588 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.588 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.588 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.588 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.588 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.588 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.588 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.588 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.589 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.589 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.589 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.589 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.589 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.589 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.589 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.589 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.589 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.589 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.589 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.589 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.590 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.590 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.590 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.590 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.590 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.590 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.590 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.590 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.590 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.590 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.590 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.590 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.590 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.591 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.591 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.591 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.591 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.591 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.591 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.591 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.591 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.591 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.591 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.591 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.591 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.591 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.592 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.592 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.592 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.592 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.592 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_clones": { 2024-07-27T03:19:08.592 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_clones", 2024-07-27T03:19:08.592 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.592 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.592 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.592 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4", 2024-07-27T03:19:08.592 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.592 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.592 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.592 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Number of asynchronous cloner threads", 2024-07-27T03:19:08.593 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.593 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.593 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.593 INFO:teuthology.orchestra.run.smithi067.stdout: "periodic_async_work": { 2024-07-27T03:19:08.593 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "periodic_async_work", 2024-07-27T03:19:08.593 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.593 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.593 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.593 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.593 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.593 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.593 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.594 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Periodically check for async work", 2024-07-27T03:19:08.594 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.594 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.594 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.594 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.594 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_delay": { 2024-07-27T03:19:08.594 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_delay", 2024-07-27T03:19:08.594 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.594 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.594 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.594 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.594 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.594 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.595 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.595 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-07-27T03:19:08.595 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.595 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.595 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.595 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.595 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_no_wait": { 2024-07-27T03:19:08.595 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_no_wait", 2024-07-27T03:19:08.595 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.595 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.595 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.595 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.595 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.595 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.596 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.596 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-07-27T03:19:08.596 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.596 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.596 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.596 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.596 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.596 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.596 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.596 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.596 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.596 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.596 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.597 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.597 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.597 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.597 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.597 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.597 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.597 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.597 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.597 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix", 2024-07-27T03:19:08.597 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.597 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.597 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.598 INFO:teuthology.orchestra.run.smithi067.stdout: "discovery_interval": { 2024-07-27T03:19:08.598 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "discovery_interval", 2024-07-27T03:19:08.598 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-27T03:19:08.598 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.598 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.598 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "100", 2024-07-27T03:19:08.598 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.598 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.598 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.598 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.598 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.598 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.598 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.599 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.599 INFO:teuthology.orchestra.run.smithi067.stdout: "identifier": { 2024-07-27T03:19:08.599 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "identifier", 2024-07-27T03:19:08.599 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.599 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.599 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.599 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.599 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.599 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.599 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.599 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.599 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.599 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.599 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.600 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.600 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-27T03:19:08.600 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-27T03:19:08.600 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.600 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.600 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.600 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-27T03:19:08.600 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.600 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.600 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.600 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.600 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.600 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.601 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.601 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.601 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.601 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.601 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.601 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.601 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.601 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.601 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.601 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.601 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.601 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.601 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.602 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.602 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.602 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.602 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.602 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.602 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.602 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.602 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.602 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.602 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.602 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.602 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.602 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.603 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.603 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.603 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.603 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.603 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.603 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.603 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.603 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.603 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.603 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.603 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.603 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.604 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.604 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.604 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.604 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.604 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.604 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.604 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.604 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.604 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.604 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.604 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.604 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.605 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.605 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.605 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.605 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.605 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.605 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.605 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.605 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.605 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.605 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.605 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.605 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.605 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.606 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.606 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.606 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.606 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.606 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.606 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.606 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.606 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.606 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.606 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.606 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.606 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.606 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.607 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.607 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.607 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.607 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.607 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.607 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.607 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.607 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.607 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.607 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.607 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_host": { 2024-07-27T03:19:08.607 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_host", 2024-07-27T03:19:08.607 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.607 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.608 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.608 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.608 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.608 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.608 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.608 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.608 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.608 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.608 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.608 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_port": { 2024-07-27T03:19:08.608 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_port", 2024-07-27T03:19:08.608 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.608 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.609 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.609 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10051", 2024-07-27T03:19:08.609 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.609 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.609 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.609 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.609 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.609 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.609 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.609 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.609 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_sender": { 2024-07-27T03:19:08.609 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_sender", 2024-07-27T03:19:08.610 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.610 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.610 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.610 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-07-27T03:19:08.610 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.610 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.610 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.610 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.610 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.610 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.610 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.610 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.610 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.611 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.611 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-07-27T03:19:08.611 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.611 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.611 INFO:teuthology.orchestra.run.smithi067.stdout: "gid": 24122, 2024-07-27T03:19:08.611 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "y", 2024-07-27T03:19:08.611 INFO:teuthology.orchestra.run.smithi067.stdout: "mgr_features": 4540701547738038271, 2024-07-27T03:19:08.611 INFO:teuthology.orchestra.run.smithi067.stdout: "available_modules": [ 2024-07-27T03:19:08.611 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.611 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "alerts", 2024-07-27T03:19:08.611 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.611 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.611 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.612 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-27T03:19:08.612 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-27T03:19:08.612 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.612 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.612 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.612 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-27T03:19:08.612 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.612 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.612 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.612 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How frequently to reexamine health status", 2024-07-27T03:19:08.612 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.612 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.612 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.612 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.613 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.613 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.613 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.613 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.613 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.613 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.613 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.613 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.613 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.613 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.613 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.613 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.613 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.613 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.613 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.614 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.614 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.614 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.614 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.614 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.614 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.614 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.614 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.614 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.614 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.614 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.614 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.614 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.615 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.615 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.615 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.615 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.615 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.615 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.615 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.615 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.615 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.615 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.615 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.615 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.615 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.615 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.616 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.616 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.616 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.616 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.616 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.616 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.616 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.616 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.616 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.616 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.616 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.616 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.616 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.616 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.616 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.617 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.617 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.617 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.617 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.617 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.617 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.617 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.617 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.617 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.617 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.617 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.617 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.617 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.618 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_destination": { 2024-07-27T03:19:08.618 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_destination", 2024-07-27T03:19:08.618 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.618 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.618 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.618 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.618 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.618 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.618 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.618 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email address to send alerts to", 2024-07-27T03:19:08.618 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.618 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.618 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.618 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.619 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_from_name": { 2024-07-27T03:19:08.619 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_from_name", 2024-07-27T03:19:08.619 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.619 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.619 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.619 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "Ceph", 2024-07-27T03:19:08.619 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.619 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.619 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.619 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email From: name", 2024-07-27T03:19:08.619 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.619 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.619 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.620 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_host": { 2024-07-27T03:19:08.620 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_host", 2024-07-27T03:19:08.620 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.620 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.620 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.620 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.620 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.620 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.620 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.620 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP server", 2024-07-27T03:19:08.620 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.620 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.620 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.621 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.621 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_password": { 2024-07-27T03:19:08.621 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_password", 2024-07-27T03:19:08.621 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.621 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.621 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.621 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.621 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.621 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.621 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.621 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Password to authenticate with", 2024-07-27T03:19:08.621 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.621 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.622 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.622 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.622 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_port": { 2024-07-27T03:19:08.622 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_port", 2024-07-27T03:19:08.622 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.622 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.622 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.622 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "465", 2024-07-27T03:19:08.622 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.622 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.622 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.622 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP port", 2024-07-27T03:19:08.622 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.623 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.623 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.623 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.623 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_sender": { 2024-07-27T03:19:08.623 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_sender", 2024-07-27T03:19:08.623 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.623 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.623 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.623 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.623 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.623 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.623 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.623 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP envelope sender", 2024-07-27T03:19:08.623 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.624 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.624 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.624 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.624 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_ssl": { 2024-07-27T03:19:08.624 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_ssl", 2024-07-27T03:19:08.624 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.624 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.624 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.624 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.624 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.624 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.624 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.624 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use SSL to connect to SMTP server", 2024-07-27T03:19:08.625 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.625 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.625 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.625 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.625 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_user": { 2024-07-27T03:19:08.625 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_user", 2024-07-27T03:19:08.625 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.625 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.625 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.625 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.625 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.625 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.625 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.626 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "User to authenticate as", 2024-07-27T03:19:08.626 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.626 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.626 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.626 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.626 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.626 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.626 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.626 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.626 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.626 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.626 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "balancer", 2024-07-27T03:19:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.627 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: "active": { 2024-07-27T03:19:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "active", 2024-07-27T03:19:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically balance PGs across cluster", 2024-07-27T03:19:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.628 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.629 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.629 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_time": { 2024-07-27T03:19:08.629 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_time", 2024-07-27T03:19:08.629 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.629 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.629 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.629 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0000", 2024-07-27T03:19:08.629 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.629 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.629 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.629 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "beginning time of day to automatically balance", 2024-07-27T03:19:08.629 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-27T03:19:08.629 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.629 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.630 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.630 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_weekday": { 2024-07-27T03:19:08.630 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_weekday", 2024-07-27T03:19:08.630 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-27T03:19:08.630 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.630 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.630 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.630 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-07-27T03:19:08.630 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-07-27T03:19:08.630 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.630 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-07-27T03:19:08.630 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-27T03:19:08.630 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.630 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.631 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.631 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_max_iterations": { 2024-07-27T03:19:08.631 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_max_iterations", 2024-07-27T03:19:08.631 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-27T03:19:08.631 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.631 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.631 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "25", 2024-07-27T03:19:08.631 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-27T03:19:08.631 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "250", 2024-07-27T03:19:08.631 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.631 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-07-27T03:19:08.631 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.632 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.632 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.632 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.632 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_metrics": { 2024-07-27T03:19:08.632 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_metrics", 2024-07-27T03:19:08.632 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.632 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.632 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.632 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "pgs,objects,bytes", 2024-07-27T03:19:08.632 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.632 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.632 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.632 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-07-27T03:19:08.632 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-07-27T03:19:08.633 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.633 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.633 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.633 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_step": { 2024-07-27T03:19:08.633 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_step", 2024-07-27T03:19:08.633 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-27T03:19:08.633 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.633 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.633 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.5", 2024-07-27T03:19:08.633 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0.001", 2024-07-27T03:19:08.633 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "0.999", 2024-07-27T03:19:08.633 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.633 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "aggressiveness of optimization", 2024-07-27T03:19:08.633 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-07-27T03:19:08.634 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.634 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.634 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.634 INFO:teuthology.orchestra.run.smithi067.stdout: "end_time": { 2024-07-27T03:19:08.634 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_time", 2024-07-27T03:19:08.634 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.634 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.634 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.634 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2359", 2024-07-27T03:19:08.634 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.634 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.634 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.634 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ending time of day to automatically balance", 2024-07-27T03:19:08.635 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-27T03:19:08.635 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.635 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.635 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.635 INFO:teuthology.orchestra.run.smithi067.stdout: "end_weekday": { 2024-07-27T03:19:08.635 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_weekday", 2024-07-27T03:19:08.635 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-27T03:19:08.635 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.635 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.635 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.635 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-07-27T03:19:08.635 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-07-27T03:19:08.635 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.635 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-07-27T03:19:08.636 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-27T03:19:08.636 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.636 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.636 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.636 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.636 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.636 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.636 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.636 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.636 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.636 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.636 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.636 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.636 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.637 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.637 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.637 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.637 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.637 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.637 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.637 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.637 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.637 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.637 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.637 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.637 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.637 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.637 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.638 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.638 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.638 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.638 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.638 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.638 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.638 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.638 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.638 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.638 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.638 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.638 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.638 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.638 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.639 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.639 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.639 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.639 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.639 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.639 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.639 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.639 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.639 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.639 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.639 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.639 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.639 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.639 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.640 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.640 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.640 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.640 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.640 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.640 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.640 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.640 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.640 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.640 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.640 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.640 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.640 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.640 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.641 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.641 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.641 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.641 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.641 INFO:teuthology.orchestra.run.smithi067.stdout: "min_score": { 2024-07-27T03:19:08.641 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_score", 2024-07-27T03:19:08.641 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-27T03:19:08.641 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.641 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.641 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.641 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.641 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.641 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.641 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-07-27T03:19:08.642 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.642 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.642 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.642 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.642 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-07-27T03:19:08.642 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-07-27T03:19:08.642 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.642 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.642 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.642 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "upmap", 2024-07-27T03:19:08.642 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.642 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.642 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.643 INFO:teuthology.orchestra.run.smithi067.stdout: "crush-compat", 2024-07-27T03:19:08.643 INFO:teuthology.orchestra.run.smithi067.stdout: "none", 2024-07-27T03:19:08.643 INFO:teuthology.orchestra.run.smithi067.stdout: "read", 2024-07-27T03:19:08.643 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap", 2024-07-27T03:19:08.643 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap-read" 2024-07-27T03:19:08.643 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.643 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Balancer mode", 2024-07-27T03:19:08.643 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.643 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.643 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.643 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.643 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_ids": { 2024-07-27T03:19:08.643 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_ids", 2024-07-27T03:19:08.644 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.644 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.644 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.644 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.644 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.644 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.644 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.644 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-07-27T03:19:08.644 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.644 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.644 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.644 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.644 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-27T03:19:08.644 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-27T03:19:08.645 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.645 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.645 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.645 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-27T03:19:08.645 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.645 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.645 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.645 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-07-27T03:19:08.645 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.645 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.645 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.645 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.646 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.646 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.646 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.646 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.646 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.646 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.646 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.646 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.646 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.646 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.646 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.646 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.647 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.647 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.647 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_deviation": { 2024-07-27T03:19:08.647 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_deviation", 2024-07-27T03:19:08.647 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.647 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.647 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.647 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-27T03:19:08.647 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-27T03:19:08.647 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.647 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.647 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "deviation below which no optimization is attempted", 2024-07-27T03:19:08.647 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-07-27T03:19:08.648 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.648 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.648 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.648 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_optimizations": { 2024-07-27T03:19:08.648 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_optimizations", 2024-07-27T03:19:08.648 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-27T03:19:08.648 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.648 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.648 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:08.648 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.648 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.648 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.648 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-07-27T03:19:08.649 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.649 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.649 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.649 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.649 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.649 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.649 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.649 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm", 2024-07-27T03:19:08.649 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.649 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.649 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.650 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_down_multiplier": { 2024-07-27T03:19:08.650 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_down_multiplier", 2024-07-27T03:19:08.650 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-27T03:19:08.650 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.650 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.650 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-07-27T03:19:08.650 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.650 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.650 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.650 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-07-27T03:19:08.650 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.651 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.651 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.651 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.651 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_refresh_rate": { 2024-07-27T03:19:08.651 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_refresh_rate", 2024-07-27T03:19:08.651 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.651 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.651 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.651 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "20", 2024-07-27T03:19:08.651 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.651 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.651 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.651 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-07-27T03:19:08.652 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.652 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.652 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.652 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.652 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_starting_port": { 2024-07-27T03:19:08.652 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_starting_port", 2024-07-27T03:19:08.652 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.652 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.652 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.652 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4721", 2024-07-27T03:19:08.652 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.652 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.652 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.652 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-07-27T03:19:08.653 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.653 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.653 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.653 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.653 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_ptrace": { 2024-07-27T03:19:08.653 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_ptrace", 2024-07-27T03:19:08.653 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.653 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.653 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.653 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.653 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.653 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.653 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.654 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-07-27T03:19:08.654 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-07-27T03:19:08.654 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.654 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.654 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.654 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_interval": { 2024-07-27T03:19:08.654 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_interval", 2024-07-27T03:19:08.654 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.654 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.654 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.654 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-27T03:19:08.655 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.655 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.655 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.655 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to autotune daemon memory", 2024-07-27T03:19:08.655 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.655 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.655 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.655 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.655 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_memory_target_ratio": { 2024-07-27T03:19:08.655 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_memory_target_ratio", 2024-07-27T03:19:08.655 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-27T03:19:08.655 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.655 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.656 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.7", 2024-07-27T03:19:08.656 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.656 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.656 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.656 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-07-27T03:19:08.656 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.656 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.656 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.656 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.656 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm_log_destination": { 2024-07-27T03:19:08.656 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm_log_destination", 2024-07-27T03:19:08.656 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.656 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.657 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.657 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.657 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.657 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.657 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.657 INFO:teuthology.orchestra.run.smithi067.stdout: "file", 2024-07-27T03:19:08.657 INFO:teuthology.orchestra.run.smithi067.stdout: "file,syslog", 2024-07-27T03:19:08.657 INFO:teuthology.orchestra.run.smithi067.stdout: "syslog" 2024-07-27T03:19:08.657 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.657 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-07-27T03:19:08.657 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.657 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.657 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.657 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.658 INFO:teuthology.orchestra.run.smithi067.stdout: "cgroups_split": { 2024-07-27T03:19:08.658 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cgroups_split", 2024-07-27T03:19:08.658 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.658 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.658 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.658 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.658 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.658 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.658 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.658 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-07-27T03:19:08.658 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.658 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.659 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.659 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.659 INFO:teuthology.orchestra.run.smithi067.stdout: "config_checks_enabled": { 2024-07-27T03:19:08.659 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_checks_enabled", 2024-07-27T03:19:08.659 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.659 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.659 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.659 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.659 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.659 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.659 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.659 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-07-27T03:19:08.659 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.660 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.660 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.660 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.660 INFO:teuthology.orchestra.run.smithi067.stdout: "config_dashboard": { 2024-07-27T03:19:08.660 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_dashboard", 2024-07-27T03:19:08.660 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.660 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.660 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.660 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.660 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.660 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.660 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.660 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-07-27T03:19:08.661 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.661 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.661 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.661 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.661 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_alertmanager": { 2024-07-27T03:19:08.661 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_alertmanager", 2024-07-27T03:19:08.661 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.661 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.661 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.661 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-07-27T03:19:08.661 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.661 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.661 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.662 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-27T03:19:08.662 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.662 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.662 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.662 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.662 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_base": { 2024-07-27T03:19:08.662 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_base", 2024-07-27T03:19:08.662 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.662 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.662 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.662 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/ceph", 2024-07-27T03:19:08.662 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.662 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.663 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.663 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Container image name, without the tag", 2024-07-27T03:19:08.663 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.663 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.663 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.663 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.663 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_elasticsearch": { 2024-07-27T03:19:08.663 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_elasticsearch", 2024-07-27T03:19:08.663 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.663 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.663 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.663 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-07-27T03:19:08.663 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.663 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.664 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.664 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "elasticsearch container image", 2024-07-27T03:19:08.664 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.664 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.664 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.664 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.664 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_grafana": { 2024-07-27T03:19:08.664 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_grafana", 2024-07-27T03:19:08.664 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.664 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.664 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.664 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-07-27T03:19:08.664 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.664 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.665 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.665 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-27T03:19:08.665 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.665 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.665 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.665 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.665 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_haproxy": { 2024-07-27T03:19:08.665 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_haproxy", 2024-07-27T03:19:08.665 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.665 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.665 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.665 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-07-27T03:19:08.665 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.665 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.666 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.666 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "HAproxy container image", 2024-07-27T03:19:08.666 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.666 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.666 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.666 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.666 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_agent": { 2024-07-27T03:19:08.666 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_agent", 2024-07-27T03:19:08.666 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.666 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.666 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.666 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-07-27T03:19:08.666 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.666 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.667 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.667 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger agent container image", 2024-07-27T03:19:08.667 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.667 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.667 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.667 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.667 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_collector": { 2024-07-27T03:19:08.667 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_collector", 2024-07-27T03:19:08.667 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.667 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.667 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.667 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-07-27T03:19:08.667 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.667 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.668 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.668 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger collector container image", 2024-07-27T03:19:08.668 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.668 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.668 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.668 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.668 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_query": { 2024-07-27T03:19:08.668 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_query", 2024-07-27T03:19:08.668 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.668 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.668 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.668 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-07-27T03:19:08.668 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.668 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.669 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.669 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger query container image", 2024-07-27T03:19:08.669 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.669 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.669 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.669 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.669 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_keepalived": { 2024-07-27T03:19:08.669 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_keepalived", 2024-07-27T03:19:08.669 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.669 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.669 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.669 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-07-27T03:19:08.669 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.669 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.670 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.670 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Keepalived container image", 2024-07-27T03:19:08.670 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.670 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.670 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.670 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.670 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_loki": { 2024-07-27T03:19:08.670 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_loki", 2024-07-27T03:19:08.670 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.670 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.670 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.670 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-07-27T03:19:08.671 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.671 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.671 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.671 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Loki container image", 2024-07-27T03:19:08.671 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.671 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.671 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.671 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.671 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nginx": { 2024-07-27T03:19:08.671 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nginx", 2024-07-27T03:19:08.671 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.671 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.671 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.672 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-07-27T03:19:08.672 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.672 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.672 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.672 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nginx container image", 2024-07-27T03:19:08.672 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.672 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.672 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.672 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.672 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_node_exporter": { 2024-07-27T03:19:08.672 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_node_exporter", 2024-07-27T03:19:08.672 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.672 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.673 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.673 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-07-27T03:19:08.673 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.673 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.673 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.673 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-27T03:19:08.673 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.673 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.673 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.673 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.673 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nvmeof": { 2024-07-27T03:19:08.673 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nvmeof", 2024-07-27T03:19:08.673 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.673 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.674 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.674 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-07-27T03:19:08.674 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.674 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.674 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.674 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nvme-of container image", 2024-07-27T03:19:08.674 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.674 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.674 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.674 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.674 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_prometheus": { 2024-07-27T03:19:08.674 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_prometheus", 2024-07-27T03:19:08.674 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.675 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.675 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.675 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-07-27T03:19:08.675 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.675 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.675 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.675 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-27T03:19:08.675 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.675 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.675 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.675 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.675 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_promtail": { 2024-07-27T03:19:08.675 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_promtail", 2024-07-27T03:19:08.675 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.676 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.676 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.676 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-07-27T03:19:08.676 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.676 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.676 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.676 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Promtail container image", 2024-07-27T03:19:08.676 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.676 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.676 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.676 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.676 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_samba": { 2024-07-27T03:19:08.676 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_samba", 2024-07-27T03:19:08.676 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.676 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.677 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.677 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-07-27T03:19:08.677 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.677 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.677 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.677 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Samba/SMB container image", 2024-07-27T03:19:08.677 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.677 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.677 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.677 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.677 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_snmp_gateway": { 2024-07-27T03:19:08.677 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_snmp_gateway", 2024-07-27T03:19:08.678 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.678 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.678 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.678 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-07-27T03:19:08.678 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.678 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.678 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.678 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SNMP Gateway container image", 2024-07-27T03:19:08.678 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.678 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.678 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.678 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.678 INFO:teuthology.orchestra.run.smithi067.stdout: "container_init": { 2024-07-27T03:19:08.679 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_init", 2024-07-27T03:19:08.679 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.679 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.679 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.679 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.679 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.679 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.679 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.679 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Run podman/docker with `--init`", 2024-07-27T03:19:08.679 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.679 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.679 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.679 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.679 INFO:teuthology.orchestra.run.smithi067.stdout: "daemon_cache_timeout": { 2024-07-27T03:19:08.680 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "daemon_cache_timeout", 2024-07-27T03:19:08.680 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.680 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.680 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.680 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-27T03:19:08.680 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.680 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.680 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.680 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-07-27T03:19:08.680 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.680 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.680 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.680 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.681 INFO:teuthology.orchestra.run.smithi067.stdout: "default_cephadm_command_timeout": { 2024-07-27T03:19:08.681 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_cephadm_command_timeout", 2024-07-27T03:19:08.681 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.681 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.681 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.681 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "900", 2024-07-27T03:19:08.681 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.681 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.681 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.681 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-07-27T03:19:08.681 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.681 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.681 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.681 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.682 INFO:teuthology.orchestra.run.smithi067.stdout: "default_registry": { 2024-07-27T03:19:08.682 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_registry", 2024-07-27T03:19:08.682 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.682 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.682 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.682 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io", 2024-07-27T03:19:08.682 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.682 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.682 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.682 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-07-27T03:19:08.682 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.682 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.683 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.683 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.683 INFO:teuthology.orchestra.run.smithi067.stdout: "device_cache_timeout": { 2024-07-27T03:19:08.683 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_cache_timeout", 2024-07-27T03:19:08.683 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.683 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.683 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.683 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1800", 2024-07-27T03:19:08.683 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.683 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.683 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.683 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache device inventory", 2024-07-27T03:19:08.684 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.684 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.684 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.684 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.684 INFO:teuthology.orchestra.run.smithi067.stdout: "device_enhanced_scan": { 2024-07-27T03:19:08.684 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_enhanced_scan", 2024-07-27T03:19:08.684 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.684 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.684 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.684 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.684 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.684 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.684 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.685 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use libstoragemgmt during device scans", 2024-07-27T03:19:08.685 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.685 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.685 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.685 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.685 INFO:teuthology.orchestra.run.smithi067.stdout: "facts_cache_timeout": { 2024-07-27T03:19:08.685 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "facts_cache_timeout", 2024-07-27T03:19:08.685 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.685 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.685 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.685 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-27T03:19:08.685 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.685 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.685 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.686 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache host facts data", 2024-07-27T03:19:08.686 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.686 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.686 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.686 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.686 INFO:teuthology.orchestra.run.smithi067.stdout: "grafana_dashboards_path": { 2024-07-27T03:19:08.686 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "grafana_dashboards_path", 2024-07-27T03:19:08.686 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.686 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.686 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.686 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-07-27T03:19:08.686 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.687 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.687 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.687 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-07-27T03:19:08.687 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.687 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.687 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.687 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.687 INFO:teuthology.orchestra.run.smithi067.stdout: "host_check_interval": { 2024-07-27T03:19:08.687 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "host_check_interval", 2024-07-27T03:19:08.687 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.687 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.687 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.687 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-27T03:19:08.688 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.688 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.688 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.688 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to perform a host check", 2024-07-27T03:19:08.688 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.688 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.688 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.688 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.688 INFO:teuthology.orchestra.run.smithi067.stdout: "hw_monitoring": { 2024-07-27T03:19:08.688 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hw_monitoring", 2024-07-27T03:19:08.688 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.688 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.688 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.689 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.689 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.689 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.689 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.689 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-07-27T03:19:08.689 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.689 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.689 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.689 INFO:teuthology.orchestra.run.smithi067.stdout: "inventory_list_all": { 2024-07-27T03:19:08.689 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "inventory_list_all", 2024-07-27T03:19:08.689 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.689 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.689 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.690 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.690 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.690 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.690 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.690 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-07-27T03:19:08.690 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.690 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.690 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.690 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.690 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.690 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.690 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.690 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.691 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.691 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.691 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.691 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.691 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.691 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.691 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.691 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.691 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.691 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.691 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.691 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.691 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.692 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.692 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.692 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.692 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.692 INFO:teuthology.orchestra.run.smithi067.stdout: "log_refresh_metadata": { 2024-07-27T03:19:08.692 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_refresh_metadata", 2024-07-27T03:19:08.692 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.692 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.692 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.692 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.692 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.692 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.692 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.692 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-07-27T03:19:08.693 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.693 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.693 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.693 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.693 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.693 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.693 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.693 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.693 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.693 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.693 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.693 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.693 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.694 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-07-27T03:19:08.694 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.694 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.694 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.694 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.694 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.694 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.694 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.694 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.694 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.694 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.694 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.695 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.695 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.695 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.695 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.695 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.695 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.695 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.695 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.695 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.695 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.695 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.695 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.695 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.695 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.696 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.696 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.696 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.696 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.696 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.696 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.696 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.696 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.696 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.696 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.696 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.696 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.696 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.697 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.697 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf": { 2024-07-27T03:19:08.697 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-07-27T03:19:08.697 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.697 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.697 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.697 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.697 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.697 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.697 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.697 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-07-27T03:19:08.697 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.698 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.698 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.698 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.698 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-07-27T03:19:08.698 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-07-27T03:19:08.698 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.698 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.698 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.698 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "*", 2024-07-27T03:19:08.698 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.698 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.698 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.698 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-07-27T03:19:08.699 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.699 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.699 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.699 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.699 INFO:teuthology.orchestra.run.smithi067.stdout: "max_count_per_host": { 2024-07-27T03:19:08.699 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_count_per_host", 2024-07-27T03:19:08.699 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.699 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.699 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.699 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:08.699 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.699 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.699 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.699 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of daemons per service per host", 2024-07-27T03:19:08.700 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.700 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.700 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.700 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.700 INFO:teuthology.orchestra.run.smithi067.stdout: "max_osd_draining_count": { 2024-07-27T03:19:08.700 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_osd_draining_count", 2024-07-27T03:19:08.700 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.700 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.700 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.700 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:08.700 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.700 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.701 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.701 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-07-27T03:19:08.701 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.701 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.701 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.701 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.701 INFO:teuthology.orchestra.run.smithi067.stdout: "migration_current": { 2024-07-27T03:19:08.701 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "migration_current", 2024-07-27T03:19:08.701 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.701 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.701 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.701 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.701 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.701 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.702 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.702 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "internal - do not modify", 2024-07-27T03:19:08.702 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.702 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.702 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.702 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.702 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-07-27T03:19:08.702 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-07-27T03:19:08.702 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.702 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.702 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.702 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "root", 2024-07-27T03:19:08.702 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.703 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.703 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.703 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm-package", 2024-07-27T03:19:08.703 INFO:teuthology.orchestra.run.smithi067.stdout: "root" 2024-07-27T03:19:08.703 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.703 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "mode for remote execution of cephadm", 2024-07-27T03:19:08.703 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.703 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.703 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.703 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.703 INFO:teuthology.orchestra.run.smithi067.stdout: "oob_default_addr": { 2024-07-27T03:19:08.703 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "oob_default_addr", 2024-07-27T03:19:08.703 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.703 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "169.254.1.1", 2024-07-27T03:19:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default address for RedFish API (oob management).", 2024-07-27T03:19:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: "prometheus_alerts_path": { 2024-07-27T03:19:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus_alerts_path", 2024-07-27T03:19:08.704 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-07-27T03:19:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-07-27T03:19:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_insecure": { 2024-07-27T03:19:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_insecure", 2024-07-27T03:19:08.705 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-07-27T03:19:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_password": { 2024-07-27T03:19:08.706 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_password", 2024-07-27T03:19:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-07-27T03:19:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_url": { 2024-07-27T03:19:08.707 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_url", 2024-07-27T03:19:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-07-27T03:19:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_username": { 2024-07-27T03:19:08.708 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_username", 2024-07-27T03:19:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-07-27T03:19:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.709 INFO:teuthology.orchestra.run.smithi067.stdout: "secure_monitoring_stack": { 2024-07-27T03:19:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secure_monitoring_stack", 2024-07-27T03:19:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-07-27T03:19:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.710 INFO:teuthology.orchestra.run.smithi067.stdout: "service_discovery_port": { 2024-07-27T03:19:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "service_discovery_port", 2024-07-27T03:19:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8765", 2024-07-27T03:19:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "cephadm service discovery port", 2024-07-27T03:19:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.711 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.712 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: "ssh_config_file": { 2024-07-27T03:19:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssh_config_file", 2024-07-27T03:19:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-07-27T03:19:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.713 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.714 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.714 INFO:teuthology.orchestra.run.smithi067.stdout: "use_agent": { 2024-07-27T03:19:08.714 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_agent", 2024-07-27T03:19:08.714 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.714 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.714 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.714 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.714 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.714 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.714 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.714 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-07-27T03:19:08.714 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: "use_repo_digest": { 2024-07-27T03:19:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_repo_digest", 2024-07-27T03:19:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.715 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-07-27T03:19:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_failed_host_check": { 2024-07-27T03:19:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_failed_host_check", 2024-07-27T03:19:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.716 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if the host check fails", 2024-07-27T03:19:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_daemons": { 2024-07-27T03:19:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_daemons", 2024-07-27T03:19:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.717 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.718 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.718 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.718 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-07-27T03:19:08.718 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.718 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.718 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.718 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.718 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_hosts": { 2024-07-27T03:19:08.718 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_hosts", 2024-07-27T03:19:08.718 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.718 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.718 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.719 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.719 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.719 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.719 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.719 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-07-27T03:19:08.719 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.719 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.719 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.719 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.719 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.719 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.719 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.719 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crash", 2024-07-27T03:19:08.719 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.720 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.720 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.720 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.720 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.720 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.720 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.720 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.720 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.720 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.720 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.720 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.720 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.720 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.720 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.721 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.721 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.721 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.721 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.721 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.721 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.721 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.721 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.721 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.721 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.721 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.721 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.721 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.721 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.722 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.722 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.722 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.722 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.722 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.722 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.722 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.722 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.722 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.722 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.722 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.722 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.722 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.723 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.723 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.723 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.723 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.723 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.723 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.723 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.723 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.723 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.723 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.723 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.723 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.723 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.724 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.724 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.724 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.724 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.724 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.724 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.724 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.724 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.724 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.724 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.724 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.724 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.724 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.725 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.725 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.725 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.725 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.725 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.725 INFO:teuthology.orchestra.run.smithi067.stdout: "retain_interval": { 2024-07-27T03:19:08.725 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retain_interval", 2024-07-27T03:19:08.725 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.725 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.725 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.725 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "31536000", 2024-07-27T03:19:08.725 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.725 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.725 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.725 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain crashes before pruning them", 2024-07-27T03:19:08.726 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.726 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.726 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.726 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.726 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.726 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.726 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.726 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.726 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.726 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.726 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.726 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.726 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.727 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.727 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.727 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.727 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.727 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.727 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_recent_interval": { 2024-07-27T03:19:08.727 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_recent_interval", 2024-07-27T03:19:08.727 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.727 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.727 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.727 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1209600", 2024-07-27T03:19:08.727 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.727 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.727 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.728 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "time interval in which to warn about recent crashes", 2024-07-27T03:19:08.728 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.728 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.728 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.728 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.728 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.728 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.728 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.728 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dashboard", 2024-07-27T03:19:08.728 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.728 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.728 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.728 INFO:teuthology.orchestra.run.smithi067.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-07-27T03:19:08.729 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-07-27T03:19:08.729 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.729 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.729 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.729 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:08.729 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.729 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.729 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.729 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.729 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.729 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.729 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.729 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.729 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_HOST": { 2024-07-27T03:19:08.730 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_HOST", 2024-07-27T03:19:08.730 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.730 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.730 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.730 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.730 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.730 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.730 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.730 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.730 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.730 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.730 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.730 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.730 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-07-27T03:19:08.731 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-07-27T03:19:08.731 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.731 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.731 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.731 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.731 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.731 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.731 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.731 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.731 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.731 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.731 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.731 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.731 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_ENABLED": { 2024-07-27T03:19:08.732 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_ENABLED", 2024-07-27T03:19:08.732 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.732 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.732 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.732 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.732 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.732 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.732 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.732 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.732 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.732 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.732 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.732 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.733 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-07-27T03:19:08.733 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-07-27T03:19:08.733 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.733 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.733 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.733 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.733 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.733 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.733 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.733 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.733 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.733 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.733 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.733 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.733 INFO:teuthology.orchestra.run.smithi067.stdout: "ENABLE_BROWSABLE_API": { 2024-07-27T03:19:08.734 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ENABLE_BROWSABLE_API", 2024-07-27T03:19:08.734 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.734 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.734 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.734 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.734 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.734 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.734 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.734 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.734 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.734 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.734 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.734 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.734 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-07-27T03:19:08.735 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-07-27T03:19:08.735 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.735 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.735 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.735 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.735 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.735 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.735 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.735 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.735 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.735 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.735 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.735 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.735 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-07-27T03:19:08.736 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-07-27T03:19:08.736 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.736 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.736 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.736 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.736 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.736 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.736 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.736 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.736 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.736 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.736 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.736 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.736 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-07-27T03:19:08.736 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-07-27T03:19:08.737 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.737 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.737 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.737 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.737 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.737 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.737 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.737 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.737 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.737 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.737 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.737 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.737 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-07-27T03:19:08.737 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-07-27T03:19:08.738 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.738 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.738 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.738 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.738 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.738 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.738 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.738 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.738 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.738 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.738 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.738 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.738 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_NFS": { 2024-07-27T03:19:08.739 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-07-27T03:19:08.739 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.739 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.739 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.739 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.739 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.739 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.739 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.739 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.739 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.739 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.739 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.739 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.740 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RBD": { 2024-07-27T03:19:08.740 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-07-27T03:19:08.740 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.740 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.740 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.740 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.740 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.740 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.740 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.740 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.740 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.740 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.740 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.740 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.741 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RGW": { 2024-07-27T03:19:08.741 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-07-27T03:19:08.741 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.741 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.741 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.741 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.741 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.741 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.741 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.741 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.741 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.741 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.741 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.742 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.742 INFO:teuthology.orchestra.run.smithi067.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-07-27T03:19:08.742 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-07-27T03:19:08.742 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.742 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.742 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.742 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.742 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.742 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.742 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.742 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.742 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.742 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.743 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.743 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.743 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_PASSWORD": { 2024-07-27T03:19:08.743 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_PASSWORD", 2024-07-27T03:19:08.743 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.743 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.743 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.743 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-07-27T03:19:08.743 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.743 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.743 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.743 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.743 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.743 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.743 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.744 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.744 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-07-27T03:19:08.744 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-07-27T03:19:08.744 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.744 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.744 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.744 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.744 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.744 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.744 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.744 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.744 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.744 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.745 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.745 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.745 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_URL": { 2024-07-27T03:19:08.745 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_URL", 2024-07-27T03:19:08.745 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.745 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.745 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.745 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.745 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.745 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.745 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.745 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.745 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.745 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.745 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.746 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.746 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_USERNAME": { 2024-07-27T03:19:08.746 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_USERNAME", 2024-07-27T03:19:08.746 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.746 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.746 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.746 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-07-27T03:19:08.746 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.746 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.746 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.746 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.746 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.746 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.747 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.747 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.747 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-07-27T03:19:08.747 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-07-27T03:19:08.747 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.747 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.747 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.747 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.747 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.747 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.747 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.747 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.747 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.747 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.748 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.748 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-07-27T03:19:08.748 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-07-27T03:19:08.748 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.748 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.748 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.748 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.748 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.748 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.748 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.748 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.748 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.748 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.749 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.749 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.749 INFO:teuthology.orchestra.run.smithi067.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-07-27T03:19:08.749 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-07-27T03:19:08.749 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.749 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.749 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.749 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.749 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.749 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.749 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.749 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.749 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.749 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.750 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.750 INFO:teuthology.orchestra.run.smithi067.stdout: "ISSUE_TRACKER_API_KEY": { 2024-07-27T03:19:08.750 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-07-27T03:19:08.750 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.750 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.750 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.750 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.750 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.750 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.750 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.750 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.750 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.751 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.751 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.751 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.751 INFO:teuthology.orchestra.run.smithi067.stdout: "MANAGED_BY_CLUSTERS": { 2024-07-27T03:19:08.751 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-07-27T03:19:08.751 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.751 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.751 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.751 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "[]", 2024-07-27T03:19:08.751 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.751 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.751 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.751 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.752 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.752 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.752 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.752 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.752 INFO:teuthology.orchestra.run.smithi067.stdout: "MULTICLUSTER_CONFIG": { 2024-07-27T03:19:08.752 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MULTICLUSTER_CONFIG", 2024-07-27T03:19:08.752 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.752 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.752 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.752 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "{}", 2024-07-27T03:19:08.752 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.752 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.752 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.753 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.753 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.753 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.753 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.753 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.753 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_HOST": { 2024-07-27T03:19:08.753 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_HOST", 2024-07-27T03:19:08.753 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.753 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.753 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.753 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.753 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.753 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.754 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.754 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.754 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.754 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.754 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.754 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.754 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-07-27T03:19:08.754 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-07-27T03:19:08.754 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.754 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.754 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.754 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.755 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.755 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.755 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.755 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.755 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.755 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.755 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.755 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.755 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-07-27T03:19:08.755 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-07-27T03:19:08.755 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.755 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.755 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.756 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.756 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.756 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.756 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.756 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.756 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.756 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.756 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.756 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.756 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-07-27T03:19:08.756 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-07-27T03:19:08.756 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.756 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.756 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.757 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.757 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.757 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.757 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.757 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.757 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.757 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.757 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.757 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.757 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-07-27T03:19:08.757 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-07-27T03:19:08.757 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.757 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.758 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.758 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.758 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.758 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.758 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.758 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.758 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.758 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.758 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.758 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.758 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-07-27T03:19:08.758 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-07-27T03:19:08.758 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.759 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.759 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.759 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.759 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.759 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.759 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.759 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.759 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.759 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.759 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.759 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.759 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-07-27T03:19:08.759 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-07-27T03:19:08.759 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.760 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.760 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.760 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.760 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.760 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.760 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.760 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.760 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.760 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.760 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.760 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.760 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-07-27T03:19:08.760 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-07-27T03:19:08.760 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.761 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.761 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.761 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.761 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.761 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.761 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.761 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.761 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.761 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.761 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.761 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.761 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-07-27T03:19:08.761 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-07-27T03:19:08.762 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.762 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.762 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.762 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.762 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.762 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.762 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.762 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.762 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.762 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.762 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.762 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.762 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_ENABLED": { 2024-07-27T03:19:08.762 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_ENABLED", 2024-07-27T03:19:08.763 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.763 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.763 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.763 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.763 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.763 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.763 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.763 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.763 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.763 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.763 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.763 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.763 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-07-27T03:19:08.764 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-07-27T03:19:08.764 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.764 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.764 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.764 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-07-27T03:19:08.764 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.764 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.764 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.764 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.764 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.764 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.764 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.764 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.765 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-07-27T03:19:08.765 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-07-27T03:19:08.765 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.765 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.765 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.765 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:08.765 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.765 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.765 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.765 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.765 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.765 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.765 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.766 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.766 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-07-27T03:19:08.766 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-07-27T03:19:08.766 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.766 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.766 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.766 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8", 2024-07-27T03:19:08.766 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.766 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.766 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.766 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.766 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.766 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.767 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.767 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.767 INFO:teuthology.orchestra.run.smithi067.stdout: "REST_REQUESTS_TIMEOUT": { 2024-07-27T03:19:08.767 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-07-27T03:19:08.767 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.767 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.767 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.767 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "45", 2024-07-27T03:19:08.767 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.767 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.767 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.767 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.767 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.767 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ACCESS_KEY": { 2024-07-27T03:19:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ACCESS_KEY", 2024-07-27T03:19:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.768 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-07-27T03:19:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-07-27T03:19:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-07-27T03:19:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.769 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SECRET_KEY": { 2024-07-27T03:19:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SECRET_KEY", 2024-07-27T03:19:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.770 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SSL_VERIFY": { 2024-07-27T03:19:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SSL_VERIFY", 2024-07-27T03:19:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.771 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "UNSAFE_TLS_v1_2": { 2024-07-27T03:19:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "UNSAFE_TLS_v1_2", 2024-07-27T03:19:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.772 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-07-27T03:19:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-07-27T03:19:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.773 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.774 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.774 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.774 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.774 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.774 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.774 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.774 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.774 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-07-27T03:19:08.774 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-07-27T03:19:08.774 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.774 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.774 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.775 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:08.775 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.775 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.775 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.775 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.775 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.775 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.775 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.775 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.775 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-07-27T03:19:08.775 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-07-27T03:19:08.775 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.775 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.776 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.776 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-27T03:19:08.776 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.776 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.776 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.776 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.776 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.776 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.776 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.776 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.776 INFO:teuthology.orchestra.run.smithi067.stdout: "cross_origin_url": { 2024-07-27T03:19:08.776 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cross_origin_url", 2024-07-27T03:19:08.776 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.777 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.777 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.777 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.777 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.777 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.777 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.777 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.777 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.777 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.777 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.777 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.777 INFO:teuthology.orchestra.run.smithi067.stdout: "crt_file": { 2024-07-27T03:19:08.777 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crt_file", 2024-07-27T03:19:08.778 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.778 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.778 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.778 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.778 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.778 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.778 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.778 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.778 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.778 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.778 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.778 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.779 INFO:teuthology.orchestra.run.smithi067.stdout: "debug": { 2024-07-27T03:19:08.779 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "debug", 2024-07-27T03:19:08.779 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.779 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.779 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.779 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.779 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.779 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.779 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.779 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable/disable debug options", 2024-07-27T03:19:08.779 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.779 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.779 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.780 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.780 INFO:teuthology.orchestra.run.smithi067.stdout: "jwt_token_ttl": { 2024-07-27T03:19:08.780 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "jwt_token_ttl", 2024-07-27T03:19:08.780 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.780 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.780 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.780 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "28800", 2024-07-27T03:19:08.780 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.780 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.780 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.780 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.780 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.780 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.781 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.781 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.781 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-07-27T03:19:08.781 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-07-27T03:19:08.781 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.781 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.781 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.781 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.781 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.781 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.781 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.781 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.781 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.781 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.782 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.782 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.782 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.782 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.782 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.782 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.782 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.782 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.782 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.782 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.782 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.782 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.782 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.783 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.783 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.783 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.783 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.783 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.783 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.783 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.783 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.783 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.783 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.783 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.783 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.783 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.784 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.784 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.784 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.784 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.784 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.784 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.784 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.784 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.784 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.784 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.784 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.784 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.784 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.785 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.785 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.785 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.785 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.785 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.785 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.785 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.785 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.785 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.785 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.785 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.785 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.785 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.785 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.785 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.786 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.786 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.786 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.786 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.786 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.786 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.786 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.786 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.786 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.786 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.786 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.786 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.786 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.787 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.787 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.787 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.787 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.787 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.787 INFO:teuthology.orchestra.run.smithi067.stdout: "motd": { 2024-07-27T03:19:08.787 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "motd", 2024-07-27T03:19:08.787 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.787 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.787 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.787 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.787 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.787 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.787 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.788 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "The message of the day", 2024-07-27T03:19:08.788 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.788 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.788 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.788 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.788 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect_resolve_ip_addr": { 2024-07-27T03:19:08.788 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "redirect_resolve_ip_addr", 2024-07-27T03:19:08.788 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.788 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.788 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.788 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.788 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.788 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.788 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.789 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.789 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.789 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.789 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.789 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.789 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-07-27T03:19:08.789 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-07-27T03:19:08.789 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.789 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.789 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.789 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-07-27T03:19:08.789 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.789 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-07-27T03:19:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-07-27T03:19:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8080", 2024-07-27T03:19:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.790 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.791 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-07-27T03:19:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-07-27T03:19:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.792 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl_server_port": { 2024-07-27T03:19:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl_server_port", 2024-07-27T03:19:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8443", 2024-07-27T03:19:08.793 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-07-27T03:19:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-07-27T03:19:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.794 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "redirect", 2024-07-27T03:19:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect" 2024-07-27T03:19:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-07-27T03:19:08.795 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-07-27T03:19:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-07-27T03:19:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-07-27T03:19:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-07-27T03:19:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.796 INFO:teuthology.orchestra.run.smithi067.stdout: "url_prefix": { 2024-07-27T03:19:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url_prefix", 2024-07-27T03:19:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.797 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "devicehealth", 2024-07-27T03:19:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_monitoring": { 2024-07-27T03:19:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_monitoring", 2024-07-27T03:19:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.798 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.799 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.799 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.799 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.799 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "monitor device health metrics", 2024-07-27T03:19:08.799 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.799 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.799 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.799 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.799 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.799 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.799 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.799 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.799 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.800 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.801 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.802 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.803 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.804 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.804 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.804 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.804 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.804 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.804 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.804 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.804 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.804 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.804 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.804 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.804 INFO:teuthology.orchestra.run.smithi067.stdout: "mark_out_threshold": { 2024-07-27T03:19:08.804 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mark_out_threshold", 2024-07-27T03:19:08.804 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.804 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.805 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.805 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2419200", 2024-07-27T03:19:08.805 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.805 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.805 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.805 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-07-27T03:19:08.805 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.805 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.805 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.805 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.805 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_name": { 2024-07-27T03:19:08.805 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_name", 2024-07-27T03:19:08.805 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.806 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.806 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.806 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "device_health_metrics", 2024-07-27T03:19:08.806 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.806 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.806 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.806 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name of pool in which to store device health metrics", 2024-07-27T03:19:08.806 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.806 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.806 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.806 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.809 INFO:teuthology.orchestra.run.smithi067.stdout: "retention_period": { 2024-07-27T03:19:08.809 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retention_period", 2024-07-27T03:19:08.810 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.810 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.810 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.810 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15552000", 2024-07-27T03:19:08.810 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.810 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.810 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.810 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain device health metrics", 2024-07-27T03:19:08.810 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.810 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.810 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.810 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.811 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_frequency": { 2024-07-27T03:19:08.811 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_frequency", 2024-07-27T03:19:08.811 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.811 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.811 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.811 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-07-27T03:19:08.811 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.811 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.811 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.811 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to scrape device health metrics", 2024-07-27T03:19:08.811 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.811 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.811 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.812 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.812 INFO:teuthology.orchestra.run.smithi067.stdout: "self_heal": { 2024-07-27T03:19:08.812 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "self_heal", 2024-07-27T03:19:08.812 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.812 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.812 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.812 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.812 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.812 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.812 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.812 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-07-27T03:19:08.812 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.812 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.813 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.813 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.813 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-27T03:19:08.813 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-27T03:19:08.813 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.813 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.813 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.813 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-27T03:19:08.813 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.813 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.813 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.813 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and check device health", 2024-07-27T03:19:08.813 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.813 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.814 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.814 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.814 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.814 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.814 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.814 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.814 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.814 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.814 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.814 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.814 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.814 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.815 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.815 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.815 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.815 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.815 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_threshold": { 2024-07-27T03:19:08.815 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_threshold", 2024-07-27T03:19:08.815 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.815 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.815 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.815 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7257600", 2024-07-27T03:19:08.815 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.815 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.815 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.816 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-07-27T03:19:08.816 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.816 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.816 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.816 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.816 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.816 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "diskprediction_local", 2024-07-27T03:19:08.816 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.816 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.816 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.816 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.817 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.817 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.817 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.817 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.817 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.817 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.817 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.817 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.817 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.817 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.817 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.817 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.817 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.817 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.818 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.818 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.818 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.818 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.818 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.818 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.818 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.818 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.818 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.818 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.818 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.818 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.818 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.819 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.819 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.819 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.819 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.819 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.819 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.819 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.819 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.819 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.819 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.819 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.819 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.820 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.820 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.820 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.820 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.820 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.820 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.820 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.820 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.820 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.820 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.820 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.820 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.820 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.821 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.821 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.821 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.821 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.821 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.821 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.821 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.821 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.821 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.821 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.821 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.821 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.821 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.822 INFO:teuthology.orchestra.run.smithi067.stdout: "predict_interval": { 2024-07-27T03:19:08.822 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predict_interval", 2024-07-27T03:19:08.822 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.822 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.822 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.822 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-07-27T03:19:08.822 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.822 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.822 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.822 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.822 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.822 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.822 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.823 INFO:teuthology.orchestra.run.smithi067.stdout: "predictor_model": { 2024-07-27T03:19:08.823 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predictor_model", 2024-07-27T03:19:08.823 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.823 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.823 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.823 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "prophetstor", 2024-07-27T03:19:08.823 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.823 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.823 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.823 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.823 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.823 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.824 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.824 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-27T03:19:08.824 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-27T03:19:08.824 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.824 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.824 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.824 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-27T03:19:08.824 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.824 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.824 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.824 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.824 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.824 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.825 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.825 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.825 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.825 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.825 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.825 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.825 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.825 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.825 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.825 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.825 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.825 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.825 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.826 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.826 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.826 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.826 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.826 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.826 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "influx", 2024-07-27T03:19:08.826 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": false, 2024-07-27T03:19:08.826 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "influxdb python module not found", 2024-07-27T03:19:08.826 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.826 INFO:teuthology.orchestra.run.smithi067.stdout: "batch_size": { 2024-07-27T03:19:08.826 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "batch_size", 2024-07-27T03:19:08.826 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.826 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.827 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.827 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5000", 2024-07-27T03:19:08.827 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.827 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.827 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.827 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-07-27T03:19:08.827 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.827 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.827 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.827 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.827 INFO:teuthology.orchestra.run.smithi067.stdout: "database": { 2024-07-27T03:19:08.827 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "database", 2024-07-27T03:19:08.827 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.827 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.828 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.828 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "ceph", 2024-07-27T03:19:08.828 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.828 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.828 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.828 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-07-27T03:19:08.828 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.828 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.828 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.828 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.828 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": { 2024-07-27T03:19:08.828 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hostname", 2024-07-27T03:19:08.828 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.829 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.829 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.829 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.829 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.829 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.829 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.829 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server hostname", 2024-07-27T03:19:08.829 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.829 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.829 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.829 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.829 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-27T03:19:08.830 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-27T03:19:08.830 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.830 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.830 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.830 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "30", 2024-07-27T03:19:08.830 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "5", 2024-07-27T03:19:08.830 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.830 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.830 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-07-27T03:19:08.830 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.830 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.830 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.831 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.831 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.831 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.831 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.831 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.831 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.831 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.831 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.831 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.831 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.831 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.831 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.831 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.831 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.832 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.832 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.832 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.832 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.832 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.832 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.832 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.832 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.832 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.832 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.832 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.832 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.832 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.832 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.833 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.833 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.833 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.833 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.833 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.833 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.833 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.833 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.833 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.833 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.833 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.833 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.833 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.833 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.834 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.834 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.834 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.834 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.834 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.834 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.834 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.834 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.834 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.834 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.834 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.834 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.835 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.835 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.835 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.835 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.835 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.835 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.835 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.835 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.835 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.835 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.835 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.835 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.835 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.836 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.836 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.836 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.836 INFO:teuthology.orchestra.run.smithi067.stdout: "password": { 2024-07-27T03:19:08.836 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "password", 2024-07-27T03:19:08.836 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.836 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.836 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.836 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.836 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.836 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.836 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.836 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "password of InfluxDB server user", 2024-07-27T03:19:08.836 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.836 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.837 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.837 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.837 INFO:teuthology.orchestra.run.smithi067.stdout: "port": { 2024-07-27T03:19:08.837 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "port", 2024-07-27T03:19:08.837 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.837 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.837 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.837 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8086", 2024-07-27T03:19:08.837 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.837 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.837 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.837 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server port", 2024-07-27T03:19:08.837 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.838 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.838 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.838 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.838 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.838 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.838 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.838 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.838 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.838 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.838 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.838 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.838 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.839 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.839 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.839 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.839 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.839 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.839 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-07-27T03:19:08.839 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-07-27T03:19:08.839 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.839 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.839 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.839 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "false", 2024-07-27T03:19:08.839 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.839 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.840 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.840 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-07-27T03:19:08.840 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.840 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.840 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.840 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.840 INFO:teuthology.orchestra.run.smithi067.stdout: "threads": { 2024-07-27T03:19:08.840 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threads", 2024-07-27T03:19:08.840 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.840 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.840 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.840 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-27T03:19:08.840 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-27T03:19:08.840 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "32", 2024-07-27T03:19:08.841 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.841 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-07-27T03:19:08.841 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.841 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.841 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.841 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.841 INFO:teuthology.orchestra.run.smithi067.stdout: "username": { 2024-07-27T03:19:08.841 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "username", 2024-07-27T03:19:08.841 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.841 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.841 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.841 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.841 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.841 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.842 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.842 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "username of InfluxDB server user", 2024-07-27T03:19:08.842 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.842 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.842 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.842 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.842 INFO:teuthology.orchestra.run.smithi067.stdout: "verify_ssl": { 2024-07-27T03:19:08.842 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "verify_ssl", 2024-07-27T03:19:08.842 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.842 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.842 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.842 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "true", 2024-07-27T03:19:08.842 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.843 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.843 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.843 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-07-27T03:19:08.843 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.843 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.843 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.843 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.843 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.843 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.843 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.843 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "insights", 2024-07-27T03:19:08.843 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.843 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.843 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.844 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.844 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.844 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.844 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.844 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.844 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.844 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.844 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.844 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.844 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.844 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.844 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.844 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.844 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.845 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.845 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.845 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.845 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.845 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.845 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.845 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.845 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.845 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.845 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.845 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.845 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.845 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.846 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.846 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.846 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.846 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.846 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.846 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.846 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.846 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.846 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.846 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.846 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.846 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.846 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.847 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.847 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.847 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.847 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.847 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.847 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.847 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.847 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.847 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.847 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.847 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.847 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.847 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.847 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.848 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.848 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.848 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.848 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.848 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.848 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.848 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.848 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.848 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.848 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.848 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.848 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.848 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.848 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.849 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.849 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.849 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.849 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.849 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.849 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.849 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.849 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.849 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.849 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.849 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.849 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.849 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.850 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.850 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.850 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.850 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.850 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.850 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.850 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "iostat", 2024-07-27T03:19:08.850 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.850 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.850 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.850 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.850 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.850 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.851 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.851 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.851 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.851 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.851 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.851 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.851 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.851 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.851 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.851 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.851 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.851 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.851 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.851 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.852 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.852 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.852 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.852 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.852 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.852 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.852 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.852 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.852 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.852 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.852 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.853 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.853 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.853 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.853 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.853 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.853 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.853 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.853 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.853 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.853 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.853 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.853 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.854 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.854 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.854 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.854 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.854 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.854 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.854 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.854 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.854 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.854 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.854 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.854 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.855 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.855 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.855 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.855 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.855 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.855 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.855 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.855 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.855 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.855 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.855 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.855 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.855 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.856 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.856 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.856 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.856 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.856 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.856 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.856 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.856 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.856 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.856 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.856 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.856 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.856 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.857 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.857 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.857 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.857 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.857 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.857 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.857 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.857 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.857 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.857 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "k8sevents", 2024-07-27T03:19:08.857 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.857 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.857 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.857 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_event_retention_days": { 2024-07-27T03:19:08.858 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ceph_event_retention_days", 2024-07-27T03:19:08.858 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.858 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.858 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.858 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7", 2024-07-27T03:19:08.858 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.858 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.858 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.858 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Days to hold ceph event information within local cache", 2024-07-27T03:19:08.858 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.858 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.858 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.859 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.859 INFO:teuthology.orchestra.run.smithi067.stdout: "config_check_secs": { 2024-07-27T03:19:08.859 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_check_secs", 2024-07-27T03:19:08.859 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.859 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.859 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.859 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:08.859 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "10", 2024-07-27T03:19:08.859 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.859 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.859 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-07-27T03:19:08.859 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.859 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.860 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.860 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.860 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.860 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.860 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.860 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.860 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.860 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.860 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.860 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.860 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.860 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.860 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.860 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.861 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.861 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.861 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.861 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.861 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.861 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.861 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.861 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.861 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.861 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.861 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.861 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.861 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.861 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.862 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.862 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.862 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.862 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.862 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.862 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.862 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.862 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.862 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.862 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.862 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.862 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.862 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.862 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.863 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.863 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.863 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.863 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.863 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.863 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.863 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.863 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.863 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.863 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.863 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.863 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.863 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.864 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.864 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.864 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.864 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.864 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.864 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.864 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.864 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.864 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.864 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.864 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.864 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.864 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.865 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.865 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.865 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.865 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.865 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.865 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.865 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.865 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.865 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.865 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.865 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.865 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.866 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.866 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.866 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.866 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.866 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.866 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.866 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.866 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.866 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.866 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "localpool", 2024-07-27T03:19:08.866 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.866 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.866 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.866 INFO:teuthology.orchestra.run.smithi067.stdout: "failure_domain": { 2024-07-27T03:19:08.867 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "failure_domain", 2024-07-27T03:19:08.867 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.867 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.867 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.867 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "host", 2024-07-27T03:19:08.867 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.867 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.867 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.867 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "failure domain for any created local pool", 2024-07-27T03:19:08.867 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-07-27T03:19:08.867 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.867 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.867 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.867 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.868 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.868 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.868 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.868 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.868 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.868 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.868 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.868 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.868 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.868 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.868 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.868 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.868 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.869 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.869 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.869 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.869 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.869 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.869 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.869 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.869 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.869 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.869 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.869 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.869 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.869 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.869 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.870 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.870 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.870 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.870 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.870 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.870 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.870 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.870 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.870 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.870 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.870 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.870 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.870 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.871 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.871 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.871 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.871 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.871 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.871 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.871 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.871 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.871 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.871 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.871 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.871 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.871 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.871 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.872 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.872 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.872 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.872 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.872 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.872 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.872 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.872 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.872 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.872 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.872 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.872 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.872 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.872 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.873 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.873 INFO:teuthology.orchestra.run.smithi067.stdout: "min_size": { 2024-07-27T03:19:08.873 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_size", 2024-07-27T03:19:08.873 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.873 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.873 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.873 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.873 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.873 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.873 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.873 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default min_size for any created local pool", 2024-07-27T03:19:08.873 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-07-27T03:19:08.873 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.874 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.874 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.874 INFO:teuthology.orchestra.run.smithi067.stdout: "num_rep": { 2024-07-27T03:19:08.874 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "num_rep", 2024-07-27T03:19:08.874 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.874 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.874 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.874 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3", 2024-07-27T03:19:08.874 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.874 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.874 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.874 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default replica count for any created local pool", 2024-07-27T03:19:08.875 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.875 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.875 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.875 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.875 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_num": { 2024-07-27T03:19:08.875 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_num", 2024-07-27T03:19:08.875 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.875 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.875 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.875 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "128", 2024-07-27T03:19:08.875 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.875 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.875 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.876 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default pg_num for any created local pool", 2024-07-27T03:19:08.876 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.876 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.876 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.876 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.876 INFO:teuthology.orchestra.run.smithi067.stdout: "prefix": { 2024-07-27T03:19:08.876 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prefix", 2024-07-27T03:19:08.876 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.876 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.876 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.876 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.876 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.876 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.876 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.877 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name prefix for any created local pool", 2024-07-27T03:19:08.877 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.877 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.877 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.877 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.877 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.877 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.877 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.877 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.877 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.877 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.877 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.877 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.877 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.878 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.878 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.878 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.878 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.878 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.878 INFO:teuthology.orchestra.run.smithi067.stdout: "subtree": { 2024-07-27T03:19:08.878 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subtree", 2024-07-27T03:19:08.878 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.878 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.878 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.878 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "rack", 2024-07-27T03:19:08.878 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.878 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.879 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.879 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "CRUSH level for which to create a local pool", 2024-07-27T03:19:08.879 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-07-27T03:19:08.879 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.879 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.879 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.879 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.879 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.879 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.879 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mds_autoscaler", 2024-07-27T03:19:08.879 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.879 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.879 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.880 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.880 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.880 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.880 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.880 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.880 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.880 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.880 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.880 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.880 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.880 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.880 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.880 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.880 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.881 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.881 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.881 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.881 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.881 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.881 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.881 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.881 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.881 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.881 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.881 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.881 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.881 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.881 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.882 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.882 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.882 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.882 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.882 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.882 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.882 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.882 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.882 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.882 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.882 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.882 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.882 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.883 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.883 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.883 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.883 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.883 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.883 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.883 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.883 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.883 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.883 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.883 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.883 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.883 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.883 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.884 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.884 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.884 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.884 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.884 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.884 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.884 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.884 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.884 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.884 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.884 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.884 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.884 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.884 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.885 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.885 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.885 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.885 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.885 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.885 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.885 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.885 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.885 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.885 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.885 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.885 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.885 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.886 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.886 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.886 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.886 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.886 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.886 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirroring", 2024-07-27T03:19:08.886 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.886 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.886 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.886 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.886 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.886 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.886 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.886 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.887 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.887 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.887 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.887 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.887 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.887 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.887 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.887 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.887 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.887 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.887 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.887 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.887 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.887 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.888 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.888 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.888 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.888 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.888 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.888 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.888 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.888 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.888 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.888 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.888 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.888 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.888 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.888 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.889 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.889 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.889 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.889 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.889 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.889 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.889 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.889 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.889 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.889 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.889 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.889 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.889 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.889 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.890 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.890 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.890 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.890 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.890 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.890 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.890 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.890 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.890 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.890 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.890 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.890 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.890 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.891 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.891 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.891 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.891 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.891 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.891 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.891 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.891 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.891 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.891 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.891 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.891 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.891 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.891 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.892 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.892 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.892 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.892 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.892 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.892 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.892 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.892 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.892 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.892 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.892 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.892 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.892 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.892 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "nfs", 2024-07-27T03:19:08.893 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.893 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.893 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.893 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.893 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.893 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.893 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.893 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.893 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.893 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.893 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.893 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.893 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.894 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.894 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.894 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.894 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.894 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.894 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.894 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.894 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.894 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.894 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.894 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.894 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.894 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.895 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.895 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.895 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.895 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.895 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.895 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.895 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.895 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.895 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.895 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.895 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.895 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.895 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.895 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.896 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.896 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.896 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.896 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.896 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.896 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.896 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.896 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.896 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.896 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.896 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.896 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.897 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.897 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.897 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.897 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.897 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.897 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.897 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.897 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.897 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.897 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.897 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.897 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.897 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.897 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.898 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.898 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.898 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.898 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.898 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.898 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.898 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.898 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.898 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.898 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.898 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.898 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.898 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.899 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.899 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.899 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.899 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.899 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.899 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.899 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.899 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.899 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.899 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.899 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.899 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-07-27T03:19:08.899 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.900 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.900 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.900 INFO:teuthology.orchestra.run.smithi067.stdout: "fail_fs": { 2024-07-27T03:19:08.900 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "fail_fs", 2024-07-27T03:19:08.900 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.900 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.900 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.900 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.900 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.900 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.900 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.900 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-07-27T03:19:08.900 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.901 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.901 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.901 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.901 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.901 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.901 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.901 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.901 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.901 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.901 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.901 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.901 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.901 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.901 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.902 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.902 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.902 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.902 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.902 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.902 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.902 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.902 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.902 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.902 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.902 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.902 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.902 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.903 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.903 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.903 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.903 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.903 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.903 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.903 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.903 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.903 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.903 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.903 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.903 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.903 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.903 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.904 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.904 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.904 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.904 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.904 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.904 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.904 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.904 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.904 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.904 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.904 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.904 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.904 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.904 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.905 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.905 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.905 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.905 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.905 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.905 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.905 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.905 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.905 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.905 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.905 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.905 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.905 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.906 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.906 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.906 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.906 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.906 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.906 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator": { 2024-07-27T03:19:08.906 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-07-27T03:19:08.906 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.906 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.906 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.906 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.906 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.906 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.906 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.907 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm", 2024-07-27T03:19:08.907 INFO:teuthology.orchestra.run.smithi067.stdout: "rook", 2024-07-27T03:19:08.907 INFO:teuthology.orchestra.run.smithi067.stdout: "test_orchestrator" 2024-07-27T03:19:08.907 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.907 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Orchestrator backend", 2024-07-27T03:19:08.907 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.907 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.907 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.907 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.907 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.907 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.907 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.907 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.908 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.908 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.908 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.908 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.908 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.908 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.908 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.908 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.908 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.908 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.908 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.908 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.908 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.908 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_perf_query", 2024-07-27T03:19:08.909 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.909 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.909 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.909 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.909 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.909 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.909 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.909 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.909 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.909 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.909 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.909 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.910 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.910 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.910 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.910 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.910 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.910 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.910 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.910 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.910 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.910 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.910 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.910 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.910 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.911 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.911 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.911 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.911 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.911 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.911 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.911 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.911 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.911 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.911 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.911 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.911 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.911 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.912 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.912 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.912 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.912 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.912 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.912 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.912 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.912 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.912 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.912 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.912 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.912 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.912 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.912 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.913 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.913 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.913 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.913 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.913 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.913 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.913 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.913 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.913 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.913 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.913 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.913 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.913 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.914 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.914 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.914 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.914 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.914 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.914 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.914 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.914 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.914 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.914 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.914 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.914 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.914 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.915 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.915 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.915 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.915 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.915 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.915 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.915 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.915 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.915 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.915 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.915 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.915 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.916 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_support", 2024-07-27T03:19:08.916 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.916 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.916 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.916 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.916 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.916 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.916 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.916 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.916 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.916 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.916 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.916 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.916 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.917 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.917 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.917 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.917 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.917 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.917 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.917 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.917 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.917 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.917 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.917 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.917 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.917 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.918 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.918 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.918 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.918 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.918 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.918 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.918 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.918 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.918 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.918 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.918 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.918 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.918 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.919 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.919 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.919 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.919 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.919 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.919 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.919 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.919 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.919 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.919 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.919 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.919 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.919 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.919 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.920 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.920 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.920 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.920 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.920 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.920 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.920 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.920 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.920 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.920 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.920 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.920 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.920 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.921 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.921 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.921 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.921 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.921 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.921 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.921 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.921 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.921 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.921 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.921 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.921 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.921 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.922 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.922 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.922 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.922 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.922 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.922 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.922 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.922 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.922 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.922 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.922 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.922 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_autoscaler", 2024-07-27T03:19:08.922 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.922 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.923 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.923 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.923 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.923 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.923 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.923 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.923 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.923 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.923 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.923 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.923 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.923 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.923 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.924 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.924 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.924 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.924 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.924 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.924 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.924 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.924 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.924 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.924 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.924 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.924 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.924 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.925 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.925 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.925 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.925 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.925 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.925 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.925 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.925 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.925 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.925 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.925 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.925 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.925 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.926 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.926 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.926 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.926 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.926 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.926 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.926 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.926 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.926 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.926 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.926 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.926 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.926 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.927 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.927 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.927 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.927 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.927 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.927 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.927 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.927 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.927 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.927 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.927 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.927 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.927 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.927 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.928 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.928 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.928 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.928 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.928 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-27T03:19:08.928 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-27T03:19:08.928 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.928 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.928 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.928 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-27T03:19:08.928 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.928 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.928 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.929 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.929 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.929 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.929 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.929 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.929 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.929 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.929 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.929 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.929 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.929 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.929 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.929 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.930 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.930 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.930 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.930 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.930 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.930 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.930 INFO:teuthology.orchestra.run.smithi067.stdout: "threshold": { 2024-07-27T03:19:08.930 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threshold", 2024-07-27T03:19:08.930 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-27T03:19:08.930 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.930 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.930 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-07-27T03:19:08.931 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1.0", 2024-07-27T03:19:08.931 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.931 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.931 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "scaling threshold", 2024-07-27T03:19:08.931 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-07-27T03:19:08.931 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.931 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.931 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.931 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.931 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.931 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.931 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "progress", 2024-07-27T03:19:08.931 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.932 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.932 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.932 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_pg_recovery_event": { 2024-07-27T03:19:08.932 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_pg_recovery_event", 2024-07-27T03:19:08.932 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.932 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.932 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.932 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.932 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.932 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.932 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.932 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow the module to show pg recovery progress", 2024-07-27T03:19:08.932 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.933 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.933 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.933 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.933 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-07-27T03:19:08.933 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-07-27T03:19:08.933 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.933 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.933 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.933 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.933 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.933 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.933 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.933 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.933 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.933 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.934 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.934 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.934 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.934 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.934 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.934 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.934 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.934 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.934 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.934 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.934 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.934 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.934 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.935 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.935 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.935 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.935 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.935 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.935 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.935 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.935 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.935 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.935 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.935 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.935 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.935 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.936 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.936 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.936 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.936 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.936 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.936 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.936 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.936 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.936 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.936 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.936 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.936 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.936 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.937 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.937 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.937 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.937 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.937 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.937 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.937 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.937 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.937 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.937 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.937 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.937 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.937 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.937 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.938 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.938 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.938 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.938 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.938 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.938 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.938 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.938 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.938 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.938 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.938 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.938 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.938 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.938 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.939 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.939 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.939 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.939 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.939 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.939 INFO:teuthology.orchestra.run.smithi067.stdout: "max_completed_events": { 2024-07-27T03:19:08.939 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_completed_events", 2024-07-27T03:19:08.939 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.939 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.939 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.939 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "50", 2024-07-27T03:19:08.939 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.939 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.940 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.940 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "number of past completed events to remember", 2024-07-27T03:19:08.940 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.940 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.940 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.940 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.940 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-27T03:19:08.940 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-27T03:19:08.940 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:08.940 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.940 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.940 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-27T03:19:08.940 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.941 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.941 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.941 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long the module is going to sleep", 2024-07-27T03:19:08.941 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.941 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.941 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.941 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.941 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.941 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.941 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.941 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.941 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.941 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.941 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.942 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.942 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.942 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.942 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.942 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.942 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.942 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.942 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.942 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.942 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.942 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus", 2024-07-27T03:19:08.942 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.942 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.943 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.943 INFO:teuthology.orchestra.run.smithi067.stdout: "cache": { 2024-07-27T03:19:08.943 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cache", 2024-07-27T03:19:08.943 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.943 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.943 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.943 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.943 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.943 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.943 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.943 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.943 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.943 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.944 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.944 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.944 INFO:teuthology.orchestra.run.smithi067.stdout: "exclude_perf_counters": { 2024-07-27T03:19:08.944 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "exclude_perf_counters", 2024-07-27T03:19:08.944 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.944 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.944 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.944 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.944 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.944 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.944 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.944 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-07-27T03:19:08.944 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-07-27T03:19:08.944 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.945 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.945 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.945 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.945 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.945 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.945 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.945 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.945 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.945 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.945 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.945 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.945 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.945 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.945 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.946 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.946 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.946 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.946 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.946 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.946 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.946 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.946 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.946 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.946 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.946 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.946 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.947 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.947 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.947 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.947 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.947 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.947 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.947 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.947 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.947 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.947 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.947 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.947 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.947 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.947 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.948 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.948 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.948 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.948 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.948 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.948 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.948 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.948 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.948 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.948 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.948 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.948 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.948 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.948 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.949 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.949 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.949 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.949 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.949 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.949 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.949 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.949 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.949 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.949 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.949 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.949 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.949 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.950 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.950 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.950 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.950 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.950 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.950 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools": { 2024-07-27T03:19:08.950 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools", 2024-07-27T03:19:08.950 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.950 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.950 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.950 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.950 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.950 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.950 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.951 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.951 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.951 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.951 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.951 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.951 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools_refresh_interval": { 2024-07-27T03:19:08.951 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-07-27T03:19:08.951 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.951 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.951 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.951 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "300", 2024-07-27T03:19:08.951 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.951 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.951 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.952 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.952 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.952 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.952 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.952 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.952 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_interval": { 2024-07-27T03:19:08.952 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_interval", 2024-07-27T03:19:08.952 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-27T03:19:08.952 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.952 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.952 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15.0", 2024-07-27T03:19:08.952 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.952 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.953 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.953 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.953 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.953 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.953 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.953 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.953 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-07-27T03:19:08.953 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-07-27T03:19:08.953 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.953 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.953 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.953 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-07-27T03:19:08.953 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.953 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.954 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.954 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-07-27T03:19:08.954 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.954 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.954 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.954 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.954 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-07-27T03:19:08.954 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-07-27T03:19:08.954 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.954 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.954 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.954 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "9283", 2024-07-27T03:19:08.954 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.955 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.955 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.955 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-07-27T03:19:08.955 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.955 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.955 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.955 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.955 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.955 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.955 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.955 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.955 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.955 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.955 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.956 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.956 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.956 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.956 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.956 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.956 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.956 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.956 INFO:teuthology.orchestra.run.smithi067.stdout: "stale_cache_strategy": { 2024-07-27T03:19:08.956 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stale_cache_strategy", 2024-07-27T03:19:08.956 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.956 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.956 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.956 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "log", 2024-07-27T03:19:08.957 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.957 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.957 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.957 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.957 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.957 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.957 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.957 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.957 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-07-27T03:19:08.957 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-07-27T03:19:08.957 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.957 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.957 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.958 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "default", 2024-07-27T03:19:08.958 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.958 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.958 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.958 INFO:teuthology.orchestra.run.smithi067.stdout: "default", 2024-07-27T03:19:08.958 INFO:teuthology.orchestra.run.smithi067.stdout: "error" 2024-07-27T03:19:08.958 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.958 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.958 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.958 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.958 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.958 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.958 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-07-27T03:19:08.959 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-07-27T03:19:08.959 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.959 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.959 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.959 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-07-27T03:19:08.959 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-07-27T03:19:08.959 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-07-27T03:19:08.959 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.959 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.959 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.959 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.959 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.959 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.960 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.960 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.960 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.960 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_support", 2024-07-27T03:19:08.960 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.960 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.960 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.960 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.960 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.960 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.960 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.960 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.960 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.961 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.961 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.961 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.961 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.961 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.961 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.961 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.961 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.961 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.961 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.961 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.961 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.961 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.962 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.962 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.962 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.962 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.962 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.962 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.962 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.962 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.962 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.962 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.962 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.962 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.962 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.963 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.963 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.963 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.963 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.963 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.963 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.963 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.963 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.963 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.963 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.963 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.963 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.963 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.964 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.964 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.964 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.964 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.964 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.964 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.964 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.964 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.964 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.964 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.964 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.964 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.964 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.965 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.965 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.965 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.965 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.965 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.965 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.965 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.965 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.965 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.965 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.965 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.965 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.965 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_snap_create": { 2024-07-27T03:19:08.966 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_snap_create", 2024-07-27T03:19:08.966 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.966 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.966 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.966 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:08.966 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.966 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.966 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.966 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.966 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.966 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.966 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.966 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.966 INFO:teuthology.orchestra.run.smithi067.stdout: "mirror_snapshot_schedule": { 2024-07-27T03:19:08.967 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirror_snapshot_schedule", 2024-07-27T03:19:08.967 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.967 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.967 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.967 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.967 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.967 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.967 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.967 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.967 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.967 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.967 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.968 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.968 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.968 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.968 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.968 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.968 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.968 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.968 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.968 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.968 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.968 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.968 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.968 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.969 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.969 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.969 INFO:teuthology.orchestra.run.smithi067.stdout: "trash_purge_schedule": { 2024-07-27T03:19:08.969 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "trash_purge_schedule", 2024-07-27T03:19:08.969 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.969 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.969 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.969 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.969 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.969 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.969 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.969 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.969 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.970 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.970 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.970 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.970 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.970 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.970 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.970 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "restful", 2024-07-27T03:19:08.970 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.970 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.970 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.970 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_auth": { 2024-07-27T03:19:08.970 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_auth", 2024-07-27T03:19:08.970 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.970 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.971 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.971 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:08.971 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.971 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.971 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.971 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.971 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.971 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.971 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.971 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.971 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-07-27T03:19:08.971 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-07-27T03:19:08.971 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.972 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.972 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.972 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.972 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.972 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.972 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.972 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.972 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.972 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.972 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.972 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.972 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.972 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.973 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.973 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.973 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.973 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.973 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.973 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.973 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.973 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.973 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.973 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.973 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.973 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.973 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.974 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.974 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.974 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.974 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.974 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.974 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.974 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.974 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.974 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.974 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.974 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.974 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.974 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.975 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.975 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.975 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.975 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.975 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.975 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.975 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.975 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.975 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.975 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.975 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.975 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.976 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.976 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.976 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.976 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.976 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.976 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.976 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.976 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.976 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.976 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.976 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.976 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.976 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.976 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.977 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.977 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.977 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.977 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.977 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.977 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.977 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.977 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.977 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.977 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.977 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.977 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.977 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.977 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.978 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.978 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.978 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-07-27T03:19:08.978 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-07-27T03:19:08.978 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.978 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.978 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.978 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.978 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.978 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.978 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.978 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.978 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.979 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.979 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.979 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.979 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-07-27T03:19:08.979 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-07-27T03:19:08.979 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.979 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.979 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.979 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.979 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.979 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.979 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.979 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.979 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.980 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.980 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.980 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.980 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.980 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.980 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.980 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.980 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.980 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.980 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.980 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.980 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.980 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.980 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.981 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.981 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.981 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.981 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.981 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.981 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.981 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rgw", 2024-07-27T03:19:08.981 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.981 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.981 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.981 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.981 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.981 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.982 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.982 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.982 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.982 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.982 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.982 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.982 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.982 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.982 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.982 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.982 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.982 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.982 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.983 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.983 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.983 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.983 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.983 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.983 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.983 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.983 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.983 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.983 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.983 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.983 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.983 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.984 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.984 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.984 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.984 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.984 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.984 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.984 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.984 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.984 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.984 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.984 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.984 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.984 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.984 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.985 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.985 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.985 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.985 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.985 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.985 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.985 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.985 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.985 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.985 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.985 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.985 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.985 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.985 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.986 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.986 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.986 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.986 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.986 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.986 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.986 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.986 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.986 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.986 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.986 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.986 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.986 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.987 INFO:teuthology.orchestra.run.smithi067.stdout: "secondary_zone_period_retry_limit": { 2024-07-27T03:19:08.987 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secondary_zone_period_retry_limit", 2024-07-27T03:19:08.987 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.987 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.987 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.987 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-27T03:19:08.987 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.987 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.987 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.987 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-07-27T03:19:08.987 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.987 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.987 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.987 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.988 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.988 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.988 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.988 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.988 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.988 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.988 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.988 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.988 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.988 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.988 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.988 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.988 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.989 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.989 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.989 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.989 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rook", 2024-07-27T03:19:08.989 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.989 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.989 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.989 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.989 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.989 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.989 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.989 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.989 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.990 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.990 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.990 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.990 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.990 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.990 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.990 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.990 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.990 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.990 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.990 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.990 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.990 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.991 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.991 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.991 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.991 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.991 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.991 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.991 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.991 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.991 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.991 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.991 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.991 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.991 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.991 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.992 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.992 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.992 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:08.992 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:08.992 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.992 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.992 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.992 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:08.992 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.992 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.992 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.992 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.992 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.992 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.993 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.993 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.993 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.993 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.993 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.993 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.993 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.993 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.993 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.993 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:08.993 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:08.993 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.993 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.993 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.994 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.994 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.994 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.994 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.994 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.994 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.994 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.994 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.994 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.994 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:08.994 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:08.995 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:08.995 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:08.995 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.995 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:08.995 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.995 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.995 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.995 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.995 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.995 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.995 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.995 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.995 INFO:teuthology.orchestra.run.smithi067.stdout: "storage_class": { 2024-07-27T03:19:08.995 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "storage_class", 2024-07-27T03:19:08.996 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.996 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.996 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:08.996 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "local", 2024-07-27T03:19:08.996 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.996 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.996 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.996 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-07-27T03:19:08.996 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.996 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.996 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.996 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.996 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:08.997 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.997 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:08.997 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "selftest", 2024-07-27T03:19:08.997 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:08.997 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:08.997 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:08.997 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:08.997 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:08.997 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:08.997 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.997 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.997 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:08.998 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.998 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.998 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:08.998 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:08.998 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:08.998 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:08.998 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:08.998 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:08.998 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:08.998 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:08.998 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.998 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:08.998 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:08.999 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:08.999 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:08.999 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:08.999 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:08.999 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:08.999 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:08.999 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:08.999 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:08.999 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:08.999 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:08.999 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:08.999 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:08.999 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.000 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.000 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.000 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.000 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.000 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.000 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.000 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.000 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.000 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.000 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.000 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.000 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.000 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.001 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.001 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.001 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.001 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.001 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.001 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.001 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.001 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.001 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.001 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.001 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.001 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.001 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.001 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.002 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.002 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.002 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.002 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.002 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.002 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.002 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.002 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.002 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.002 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.002 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.002 INFO:teuthology.orchestra.run.smithi067.stdout: "roption1": { 2024-07-27T03:19:09.003 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption1", 2024-07-27T03:19:09.003 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.003 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.003 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.003 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.003 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.003 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.003 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.003 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.003 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.003 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.003 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.003 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.003 INFO:teuthology.orchestra.run.smithi067.stdout: "roption2": { 2024-07-27T03:19:09.004 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption2", 2024-07-27T03:19:09.004 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.004 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.004 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.004 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "xyz", 2024-07-27T03:19:09.004 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.004 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.004 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.004 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.004 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.004 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.004 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.004 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.005 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption1": { 2024-07-27T03:19:09.005 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption1", 2024-07-27T03:19:09.005 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.005 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.005 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.005 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.005 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.005 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.005 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.005 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.005 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.005 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.005 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.005 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.005 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption2": { 2024-07-27T03:19:09.006 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption2", 2024-07-27T03:19:09.006 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.006 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.006 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.006 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.006 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.006 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.006 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.006 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.006 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.006 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.006 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.006 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.007 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption3": { 2024-07-27T03:19:09.007 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption3", 2024-07-27T03:19:09.007 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-27T03:19:09.007 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.007 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.007 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.007 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.007 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.007 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.007 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.007 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.007 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.007 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.008 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.008 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption4": { 2024-07-27T03:19:09.008 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption4", 2024-07-27T03:19:09.008 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.008 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.008 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.008 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.008 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.008 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.008 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.008 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.008 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.008 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.008 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.009 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.009 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption5": { 2024-07-27T03:19:09.009 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption5", 2024-07-27T03:19:09.009 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.009 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.009 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.009 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.009 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.009 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.009 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.009 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.009 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.009 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.009 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.010 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.010 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption6": { 2024-07-27T03:19:09.010 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption6", 2024-07-27T03:19:09.010 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.010 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.010 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.010 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.010 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.010 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.010 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.010 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.010 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.011 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.013 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.014 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.014 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption7": { 2024-07-27T03:19:09.014 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption7", 2024-07-27T03:19:09.014 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.014 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.014 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.014 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.014 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-27T03:19:09.014 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "42", 2024-07-27T03:19:09.014 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.014 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.014 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.014 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.015 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.015 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.015 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.015 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.015 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.015 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.015 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.015 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.015 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.015 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.015 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.015 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.016 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.016 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.016 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.016 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.016 INFO:teuthology.orchestra.run.smithi067.stdout: "testkey": { 2024-07-27T03:19:09.016 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testkey", 2024-07-27T03:19:09.016 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.016 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.016 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.016 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.016 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.016 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.016 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.017 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.017 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.017 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.017 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.017 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.017 INFO:teuthology.orchestra.run.smithi067.stdout: "testlkey": { 2024-07-27T03:19:09.017 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testlkey", 2024-07-27T03:19:09.017 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.017 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.017 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.017 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.017 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.017 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: "testnewline": { 2024-07-27T03:19:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testnewline", 2024-07-27T03:19:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.018 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smb", 2024-07-27T03:19:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.019 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: "internal_store_backend": { 2024-07-27T03:19:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "internal_store_backend", 2024-07-27T03:19:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-07-27T03:19:09.020 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.021 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.022 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.023 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.024 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.024 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.024 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.024 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.024 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.024 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.024 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.024 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.024 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.024 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.024 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.024 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.024 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.025 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.025 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.025 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.025 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.025 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.025 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.025 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.025 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.025 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.025 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.025 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.025 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.025 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.025 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.026 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.026 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.026 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.026 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.026 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.026 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.026 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.026 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.026 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.026 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.026 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.026 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.026 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.027 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.027 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.027 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.027 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.027 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.027 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.027 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.027 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.027 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.027 INFO:teuthology.orchestra.run.smithi067.stdout: "update_orchestration": { 2024-07-27T03:19:09.027 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "update_orchestration", 2024-07-27T03:19:09.027 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.027 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.027 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.028 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.028 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.028 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.028 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.028 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-07-27T03:19:09.028 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.028 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.028 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.028 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.028 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.028 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.028 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.029 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snap_schedule", 2024-07-27T03:19:09.029 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.029 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.029 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.029 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_m_granularity": { 2024-07-27T03:19:09.029 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_m_granularity", 2024-07-27T03:19:09.029 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.029 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.029 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.029 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.029 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.029 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.029 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.030 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow minute scheduled snapshots", 2024-07-27T03:19:09.030 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.030 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.030 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.030 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.030 INFO:teuthology.orchestra.run.smithi067.stdout: "dump_on_update": { 2024-07-27T03:19:09.030 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dump_on_update", 2024-07-27T03:19:09.030 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.030 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.030 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.030 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.030 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.030 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.031 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.031 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "dump database to debug log on update", 2024-07-27T03:19:09.031 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.031 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.031 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.031 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.031 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.031 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.031 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.031 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.031 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.031 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.031 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.031 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.032 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.032 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.032 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.032 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.032 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.032 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.032 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.032 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.032 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.032 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.032 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.032 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.032 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.032 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.033 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.033 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.033 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.033 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.033 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.033 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.033 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.033 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.033 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.033 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.033 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.033 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.033 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.034 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.034 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.034 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.034 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.034 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.034 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.034 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.034 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.034 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.034 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.034 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.034 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.034 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.035 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.035 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.035 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.035 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.035 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.035 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.035 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.035 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.035 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.035 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.035 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.035 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.035 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.036 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.036 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.036 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.036 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.036 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.036 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.036 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.036 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.036 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.036 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.036 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.036 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.036 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.037 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.037 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.037 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.037 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.037 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.037 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.037 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.037 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.037 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.037 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.037 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.037 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.037 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.037 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stats", 2024-07-27T03:19:09.038 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.038 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.038 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.038 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.038 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.038 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.038 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.038 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.038 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.038 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.038 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.038 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.038 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.039 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.039 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.039 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.039 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.039 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.039 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.039 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.039 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.039 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.039 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.039 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.039 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.039 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.040 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.040 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.040 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.040 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.040 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.040 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.040 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.040 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.040 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.040 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.040 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.040 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.040 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.040 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.041 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.041 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.041 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.041 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.041 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.041 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.041 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.041 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.041 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.041 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.041 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.041 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.041 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.041 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.042 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.042 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.042 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.042 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.042 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.042 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.042 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.042 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.042 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.042 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.042 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.042 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.042 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.043 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.043 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.043 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.043 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.043 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.043 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.043 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.043 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.043 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.043 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.043 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.043 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.043 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.043 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.044 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.044 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.044 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.044 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.044 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.044 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.044 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.044 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.044 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.044 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "status", 2024-07-27T03:19:09.044 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.044 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.044 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.045 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.045 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.045 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.045 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.045 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.045 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.045 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.045 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.045 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.045 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.045 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.045 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.045 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.045 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.046 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.046 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.046 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.046 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.046 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.046 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.046 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.046 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.046 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.046 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.046 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.046 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.046 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.046 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.047 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.047 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.047 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.047 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.047 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.047 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.047 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.047 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.047 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.047 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.047 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.047 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.047 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.047 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.048 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.048 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.048 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.048 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.048 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.048 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.048 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.048 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.048 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.048 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.048 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.048 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.049 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.049 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.049 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.049 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.049 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.049 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.049 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.049 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.049 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.049 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.049 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.049 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.049 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.050 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.050 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.050 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.050 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.050 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.050 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.050 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.050 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.050 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.050 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.050 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.050 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.050 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.050 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.051 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.051 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.051 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.051 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.051 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.051 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.051 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telegraf", 2024-07-27T03:19:09.051 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.051 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.051 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.051 INFO:teuthology.orchestra.run.smithi067.stdout: "address": { 2024-07-27T03:19:09.051 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "address", 2024-07-27T03:19:09.051 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.051 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.052 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.052 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-07-27T03:19:09.052 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.052 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.052 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.052 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.052 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.052 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.052 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.052 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-27T03:19:09.052 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-27T03:19:09.052 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:09.052 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.053 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.053 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15", 2024-07-27T03:19:09.053 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.053 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.053 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.053 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.053 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.053 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.053 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.053 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.053 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.053 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.053 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.054 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.054 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.054 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.054 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.054 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.054 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.054 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.054 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.054 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.054 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.054 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.054 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.054 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.054 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.055 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.055 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.055 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.055 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.055 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.055 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.055 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.055 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.055 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.055 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.055 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.055 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.055 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.055 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.056 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.056 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.056 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.056 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.056 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.056 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.056 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.056 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.056 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.056 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.056 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.056 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.056 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.056 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.057 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.057 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.057 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.057 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.057 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.057 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.057 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.057 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.057 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.057 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.057 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.057 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.058 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.058 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.058 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.058 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.058 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.058 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.058 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.058 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.058 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.058 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.058 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.058 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.059 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.059 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.059 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.059 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.059 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.059 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.059 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.059 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.059 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.059 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.059 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.059 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.059 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.059 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.060 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.060 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telemetry", 2024-07-27T03:19:09.060 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.060 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.060 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.060 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_basic": { 2024-07-27T03:19:09.060 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_basic", 2024-07-27T03:19:09.060 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.060 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.060 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.060 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.060 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.061 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.061 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.061 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share basic cluster information (size, version)", 2024-07-27T03:19:09.061 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.061 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.061 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.061 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_crash": { 2024-07-27T03:19:09.061 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_crash", 2024-07-27T03:19:09.061 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.061 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.061 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.061 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.061 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.062 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.062 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.062 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-07-27T03:19:09.062 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.062 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.062 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.062 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_device": { 2024-07-27T03:19:09.062 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_device", 2024-07-27T03:19:09.062 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.062 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.062 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.063 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.063 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.063 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.063 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.063 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-07-27T03:19:09.063 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.063 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.063 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.063 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.063 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_ident": { 2024-07-27T03:19:09.063 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_ident", 2024-07-27T03:19:09.063 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.063 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.063 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.064 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.064 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.064 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.064 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.064 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-07-27T03:19:09.064 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.064 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.064 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.064 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_perf": { 2024-07-27T03:19:09.064 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_perf", 2024-07-27T03:19:09.064 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.064 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.065 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.065 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.065 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.065 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.065 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.065 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share various performance metrics of a cluster", 2024-07-27T03:19:09.065 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.065 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.065 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.065 INFO:teuthology.orchestra.run.smithi067.stdout: "contact": { 2024-07-27T03:19:09.065 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "contact", 2024-07-27T03:19:09.065 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.065 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.066 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.066 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.066 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.066 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.066 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.066 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.066 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.066 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.066 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.066 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.066 INFO:teuthology.orchestra.run.smithi067.stdout: "description": { 2024-07-27T03:19:09.066 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "description", 2024-07-27T03:19:09.066 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.066 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.067 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.067 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.067 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.067 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.067 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.067 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.067 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.067 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.067 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.067 INFO:teuthology.orchestra.run.smithi067.stdout: "device_url": { 2024-07-27T03:19:09.067 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_url", 2024-07-27T03:19:09.067 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.067 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.068 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.068 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-07-27T03:19:09.068 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.068 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.068 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.068 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.068 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.068 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.068 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.068 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.068 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-07-27T03:19:09.068 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-07-27T03:19:09.068 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-27T03:19:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-27T03:19:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.069 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "24", 2024-07-27T03:19:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "8", 2024-07-27T03:19:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "last_opt_revision": { 2024-07-27T03:19:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "last_opt_revision", 2024-07-27T03:19:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.070 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1", 2024-07-27T03:19:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard": { 2024-07-27T03:19:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard", 2024-07-27T03:19:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.071 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard_description": { 2024-07-27T03:19:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard_description", 2024-07-27T03:19:09.072 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.073 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.074 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.075 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.076 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.077 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: "organization": { 2024-07-27T03:19:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "organization", 2024-07-27T03:19:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: "proxy": { 2024-07-27T03:19:09.079 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "proxy", 2024-07-27T03:19:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.080 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: "url": { 2024-07-27T03:19:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url", 2024-07-27T03:19:09.081 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-07-27T03:19:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.082 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.083 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "test_orchestrator", 2024-07-27T03:19:09.083 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.083 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.083 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.083 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.083 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.083 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.083 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.083 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.083 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.083 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.083 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.083 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.083 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.083 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.084 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.084 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.084 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.084 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.084 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.084 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.084 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.084 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.084 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.084 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.084 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.084 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.084 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.084 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.085 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.085 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.085 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.085 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.085 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.085 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.085 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.085 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.085 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.085 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.085 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.085 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.085 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.086 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.086 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.086 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.086 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.086 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.086 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.086 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.086 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.086 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.086 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.086 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.086 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.086 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.086 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.086 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.087 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.087 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.087 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.087 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.087 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.087 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.087 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.087 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.087 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.087 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.087 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.087 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.087 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.087 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.088 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.088 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.088 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.088 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.088 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.088 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.088 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.088 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.088 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.088 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.088 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.088 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.088 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.089 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.089 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.089 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.089 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.089 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.089 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "volumes", 2024-07-27T03:19:09.089 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.089 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.089 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.089 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.089 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.089 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.089 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.089 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.089 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.090 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.090 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.090 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.090 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.090 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.090 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.090 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.090 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.090 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.090 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.090 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.090 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.090 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.090 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.091 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.091 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.091 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.091 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.091 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.091 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.091 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.091 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.091 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.091 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.091 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.091 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.091 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.091 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.092 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.092 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.092 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.092 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.092 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.092 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.092 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.092 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.092 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.092 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.092 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.092 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.092 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.092 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.093 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.093 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.093 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.093 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.093 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.093 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.093 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.093 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.093 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.093 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.093 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.093 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.093 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.093 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.093 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.094 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.094 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.094 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.094 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.094 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.094 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.094 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.094 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_clones": { 2024-07-27T03:19:09.094 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_clones", 2024-07-27T03:19:09.094 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.094 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.094 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.094 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4", 2024-07-27T03:19:09.094 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.095 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.095 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.095 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Number of asynchronous cloner threads", 2024-07-27T03:19:09.095 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.095 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.095 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.095 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.095 INFO:teuthology.orchestra.run.smithi067.stdout: "periodic_async_work": { 2024-07-27T03:19:09.095 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "periodic_async_work", 2024-07-27T03:19:09.095 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.095 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.095 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.095 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.095 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.096 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.096 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.096 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Periodically check for async work", 2024-07-27T03:19:09.096 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.096 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.096 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.096 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.096 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_delay": { 2024-07-27T03:19:09.096 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_delay", 2024-07-27T03:19:09.096 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.096 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.096 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.096 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.096 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.096 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.097 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.097 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-07-27T03:19:09.097 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.097 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.097 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.097 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.097 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_no_wait": { 2024-07-27T03:19:09.097 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_no_wait", 2024-07-27T03:19:09.097 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.097 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.097 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.097 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.097 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.097 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.098 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.098 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-07-27T03:19:09.098 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.098 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.098 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.098 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.098 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.098 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.098 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.098 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.098 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.098 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.098 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.098 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.099 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.099 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.099 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.099 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.099 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.099 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.099 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.099 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.099 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.099 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix", 2024-07-27T03:19:09.099 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.099 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.099 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.099 INFO:teuthology.orchestra.run.smithi067.stdout: "discovery_interval": { 2024-07-27T03:19:09.100 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "discovery_interval", 2024-07-27T03:19:09.100 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-27T03:19:09.100 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.100 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.100 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "100", 2024-07-27T03:19:09.100 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.100 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.100 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.100 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.100 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.100 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.100 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.100 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.100 INFO:teuthology.orchestra.run.smithi067.stdout: "identifier": { 2024-07-27T03:19:09.101 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "identifier", 2024-07-27T03:19:09.101 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.101 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.101 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.101 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.101 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.101 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.101 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.101 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.101 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.101 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.101 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.101 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.101 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-27T03:19:09.102 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-27T03:19:09.102 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:09.102 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.102 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.102 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-27T03:19:09.102 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.102 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.102 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.102 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.102 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.102 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.102 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.102 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.103 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.103 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.103 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.103 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.103 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.103 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.103 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.103 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.103 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.103 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.103 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.103 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.103 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.103 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.104 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.104 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.104 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.104 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.104 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.104 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.104 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.104 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.104 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.104 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.104 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.104 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.104 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.105 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.105 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.105 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.105 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.105 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.105 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.105 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.105 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.105 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.105 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.105 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.105 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.105 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.105 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.106 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.106 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.106 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.106 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.106 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.106 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.106 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.106 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.106 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.106 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.106 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.106 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.107 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.107 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.107 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.107 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.107 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.107 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.107 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.107 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.107 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.107 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.107 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.107 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.107 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.108 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.108 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.108 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.108 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.108 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.108 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.108 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.108 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.108 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.108 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.108 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.108 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.108 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.109 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_host": { 2024-07-27T03:19:09.109 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_host", 2024-07-27T03:19:09.109 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.109 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.109 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.109 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.109 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.109 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.109 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.109 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.109 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.109 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.109 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.110 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_port": { 2024-07-27T03:19:09.110 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_port", 2024-07-27T03:19:09.110 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.110 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.110 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.110 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10051", 2024-07-27T03:19:09.110 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.110 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.110 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.110 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.110 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.110 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.110 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.110 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.111 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_sender": { 2024-07-27T03:19:09.111 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_sender", 2024-07-27T03:19:09.111 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.111 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.111 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.111 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-07-27T03:19:09.111 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.111 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.111 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.111 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.111 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.111 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.111 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.112 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.112 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.112 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.112 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-07-27T03:19:09.112 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.112 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.112 INFO:teuthology.orchestra.run.smithi067.stdout: "modules": [ 2024-07-27T03:19:09.112 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm", 2024-07-27T03:19:09.112 INFO:teuthology.orchestra.run.smithi067.stdout: "dashboard", 2024-07-27T03:19:09.112 INFO:teuthology.orchestra.run.smithi067.stdout: "iostat", 2024-07-27T03:19:09.112 INFO:teuthology.orchestra.run.smithi067.stdout: "nfs", 2024-07-27T03:19:09.112 INFO:teuthology.orchestra.run.smithi067.stdout: "restful" 2024-07-27T03:19:09.112 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.112 INFO:teuthology.orchestra.run.smithi067.stdout: "available_modules": [ 2024-07-27T03:19:09.113 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.113 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "alerts", 2024-07-27T03:19:09.113 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.113 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.113 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.113 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-27T03:19:09.113 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-27T03:19:09.113 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:09.113 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.113 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.113 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-27T03:19:09.113 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.113 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.113 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.114 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How frequently to reexamine health status", 2024-07-27T03:19:09.114 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.114 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.114 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.114 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.114 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.114 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.114 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.114 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.114 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.114 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.114 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.114 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.115 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.115 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.115 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.115 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.115 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.115 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.115 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.115 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.115 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.115 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.115 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.115 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.115 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.115 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.116 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.116 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.116 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.116 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.116 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.116 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.116 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.116 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.116 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.116 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.116 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.116 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.116 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.117 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.117 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.117 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.117 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.117 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.117 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.117 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.117 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.117 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.117 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.117 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.117 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.117 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.118 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.118 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.118 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.118 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.118 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.118 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.118 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.118 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.118 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.118 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.118 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.118 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.118 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.119 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.119 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.119 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.119 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.119 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_destination": { 2024-07-27T03:19:09.119 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_destination", 2024-07-27T03:19:09.119 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.119 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.119 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.119 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.119 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.119 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.119 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.120 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email address to send alerts to", 2024-07-27T03:19:09.120 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.120 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.120 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.120 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_from_name": { 2024-07-27T03:19:09.120 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_from_name", 2024-07-27T03:19:09.120 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.120 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.120 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.120 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "Ceph", 2024-07-27T03:19:09.120 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.120 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.120 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.121 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Email From: name", 2024-07-27T03:19:09.121 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.121 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.121 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.121 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_host": { 2024-07-27T03:19:09.121 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_host", 2024-07-27T03:19:09.121 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.121 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.121 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.121 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.121 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.121 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.121 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.122 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP server", 2024-07-27T03:19:09.122 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.122 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.122 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.122 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_password": { 2024-07-27T03:19:09.122 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_password", 2024-07-27T03:19:09.122 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.122 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.122 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.122 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.122 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.122 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.123 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.123 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Password to authenticate with", 2024-07-27T03:19:09.123 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.123 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.123 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.123 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_port": { 2024-07-27T03:19:09.123 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_port", 2024-07-27T03:19:09.123 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.123 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.123 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.123 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "465", 2024-07-27T03:19:09.123 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.123 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.123 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.124 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP port", 2024-07-27T03:19:09.124 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.124 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.124 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.124 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_sender": { 2024-07-27T03:19:09.124 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_sender", 2024-07-27T03:19:09.124 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.124 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.124 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.124 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.124 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.124 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.124 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.125 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SMTP envelope sender", 2024-07-27T03:19:09.125 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.125 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.125 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.125 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_ssl": { 2024-07-27T03:19:09.125 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_ssl", 2024-07-27T03:19:09.125 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.125 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.125 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.125 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.125 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.125 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.125 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.126 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use SSL to connect to SMTP server", 2024-07-27T03:19:09.126 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.126 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.126 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.126 INFO:teuthology.orchestra.run.smithi067.stdout: "smtp_user": { 2024-07-27T03:19:09.126 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smtp_user", 2024-07-27T03:19:09.126 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.126 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.126 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.126 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.126 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.126 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.127 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.127 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "User to authenticate as", 2024-07-27T03:19:09.127 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.127 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.127 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.127 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.127 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.127 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.127 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.127 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.127 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.127 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.127 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.128 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.128 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.128 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.128 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.128 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.128 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.128 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.128 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.128 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "balancer", 2024-07-27T03:19:09.128 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.128 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.128 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.128 INFO:teuthology.orchestra.run.smithi067.stdout: "active": { 2024-07-27T03:19:09.129 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "active", 2024-07-27T03:19:09.129 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.129 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.129 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.129 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.129 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.129 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.129 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.129 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically balance PGs across cluster", 2024-07-27T03:19:09.129 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.129 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.129 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.130 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_time": { 2024-07-27T03:19:09.130 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_time", 2024-07-27T03:19:09.130 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.130 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.130 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.130 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0000", 2024-07-27T03:19:09.130 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.130 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.130 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.130 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "beginning time of day to automatically balance", 2024-07-27T03:19:09.130 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-27T03:19:09.130 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.130 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.131 INFO:teuthology.orchestra.run.smithi067.stdout: "begin_weekday": { 2024-07-27T03:19:09.131 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "begin_weekday", 2024-07-27T03:19:09.131 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-27T03:19:09.131 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.131 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.131 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.131 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-07-27T03:19:09.131 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-07-27T03:19:09.131 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.131 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-07-27T03:19:09.131 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-27T03:19:09.131 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.131 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.132 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_max_iterations": { 2024-07-27T03:19:09.132 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_max_iterations", 2024-07-27T03:19:09.132 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-27T03:19:09.132 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.132 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.132 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "25", 2024-07-27T03:19:09.132 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-27T03:19:09.132 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "250", 2024-07-27T03:19:09.132 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.132 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-07-27T03:19:09.132 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.132 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.132 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.133 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.133 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_metrics": { 2024-07-27T03:19:09.133 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_metrics", 2024-07-27T03:19:09.133 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.133 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.133 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.133 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "pgs,objects,bytes", 2024-07-27T03:19:09.133 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.133 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.133 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.133 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-07-27T03:19:09.133 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-07-27T03:19:09.133 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.133 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.134 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.134 INFO:teuthology.orchestra.run.smithi067.stdout: "crush_compat_step": { 2024-07-27T03:19:09.134 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crush_compat_step", 2024-07-27T03:19:09.134 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-27T03:19:09.134 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.134 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.134 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.5", 2024-07-27T03:19:09.134 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0.001", 2024-07-27T03:19:09.134 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "0.999", 2024-07-27T03:19:09.134 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.134 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "aggressiveness of optimization", 2024-07-27T03:19:09.134 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-07-27T03:19:09.134 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.135 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.135 INFO:teuthology.orchestra.run.smithi067.stdout: "end_time": { 2024-07-27T03:19:09.135 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_time", 2024-07-27T03:19:09.135 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.135 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.135 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.135 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2359", 2024-07-27T03:19:09.135 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.135 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.135 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.135 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ending time of day to automatically balance", 2024-07-27T03:19:09.135 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-07-27T03:19:09.135 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.135 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.136 INFO:teuthology.orchestra.run.smithi067.stdout: "end_weekday": { 2024-07-27T03:19:09.136 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "end_weekday", 2024-07-27T03:19:09.136 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-27T03:19:09.136 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.136 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.136 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.136 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "0", 2024-07-27T03:19:09.136 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "6", 2024-07-27T03:19:09.136 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.136 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-07-27T03:19:09.136 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-07-27T03:19:09.136 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.136 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.137 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.137 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.137 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.137 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.137 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.137 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.137 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.137 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.137 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.137 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.137 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.137 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.137 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.138 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.138 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.138 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.138 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.138 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.138 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.138 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.138 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.138 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.138 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.138 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.138 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.138 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.139 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.139 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.139 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.139 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.139 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.139 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.139 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.139 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.139 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.139 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.139 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.139 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.139 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.140 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.140 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.140 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.140 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.140 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.140 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.140 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.140 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.140 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.140 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.140 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.140 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.140 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.140 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.140 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.141 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.141 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.141 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.141 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.141 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.141 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.141 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.141 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.141 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.141 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.141 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.141 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.141 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.142 INFO:teuthology.orchestra.run.smithi067.stdout: "min_score": { 2024-07-27T03:19:09.142 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_score", 2024-07-27T03:19:09.142 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-27T03:19:09.142 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.142 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.142 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.142 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.142 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.142 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.142 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-07-27T03:19:09.142 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.142 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.143 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.143 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-07-27T03:19:09.143 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-07-27T03:19:09.143 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.143 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.143 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.143 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "upmap", 2024-07-27T03:19:09.143 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.143 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.143 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.143 INFO:teuthology.orchestra.run.smithi067.stdout: "crush-compat", 2024-07-27T03:19:09.143 INFO:teuthology.orchestra.run.smithi067.stdout: "none", 2024-07-27T03:19:09.143 INFO:teuthology.orchestra.run.smithi067.stdout: "read", 2024-07-27T03:19:09.144 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap", 2024-07-27T03:19:09.144 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap-read" 2024-07-27T03:19:09.144 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.144 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Balancer mode", 2024-07-27T03:19:09.144 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.144 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.144 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.144 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_ids": { 2024-07-27T03:19:09.144 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_ids", 2024-07-27T03:19:09.144 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.144 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.144 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.144 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.145 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.145 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.145 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.145 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-07-27T03:19:09.145 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.145 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.145 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.145 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-27T03:19:09.145 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-27T03:19:09.145 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:09.145 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.145 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.146 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-27T03:19:09.146 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.146 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.146 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.146 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-07-27T03:19:09.146 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.146 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.146 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.146 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.146 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.146 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.146 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.147 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.147 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.147 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.147 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.147 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.147 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.147 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.147 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.147 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.147 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_deviation": { 2024-07-27T03:19:09.147 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_deviation", 2024-07-27T03:19:09.147 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.147 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.147 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.148 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-27T03:19:09.148 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-27T03:19:09.148 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.148 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.148 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "deviation below which no optimization is attempted", 2024-07-27T03:19:09.148 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-07-27T03:19:09.148 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.148 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.148 INFO:teuthology.orchestra.run.smithi067.stdout: "upmap_max_optimizations": { 2024-07-27T03:19:09.148 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "upmap_max_optimizations", 2024-07-27T03:19:09.148 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-27T03:19:09.148 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.148 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.149 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:09.149 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.149 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.149 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.149 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-07-27T03:19:09.149 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.149 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.149 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.149 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.149 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.149 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm", 2024-07-27T03:19:09.149 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.150 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.150 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.150 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_down_multiplier": { 2024-07-27T03:19:09.150 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_down_multiplier", 2024-07-27T03:19:09.150 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-27T03:19:09.150 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.150 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.150 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-07-27T03:19:09.150 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.150 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.150 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.150 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-07-27T03:19:09.150 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.151 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.151 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.151 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_refresh_rate": { 2024-07-27T03:19:09.151 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_refresh_rate", 2024-07-27T03:19:09.151 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:09.151 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.151 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.151 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "20", 2024-07-27T03:19:09.151 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.151 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.151 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.151 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-07-27T03:19:09.151 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.152 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.152 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.152 INFO:teuthology.orchestra.run.smithi067.stdout: "agent_starting_port": { 2024-07-27T03:19:09.152 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "agent_starting_port", 2024-07-27T03:19:09.152 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.152 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.152 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.152 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4721", 2024-07-27T03:19:09.152 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.152 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.152 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.152 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-07-27T03:19:09.152 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.153 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.153 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.153 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_ptrace": { 2024-07-27T03:19:09.153 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_ptrace", 2024-07-27T03:19:09.153 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.153 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.153 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.153 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.153 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.153 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.153 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.153 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-07-27T03:19:09.153 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-07-27T03:19:09.154 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.154 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.154 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_interval": { 2024-07-27T03:19:09.154 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_interval", 2024-07-27T03:19:09.154 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:09.154 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.154 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.154 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-27T03:19:09.154 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.154 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.154 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.154 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to autotune daemon memory", 2024-07-27T03:19:09.154 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.155 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.155 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.155 INFO:teuthology.orchestra.run.smithi067.stdout: "autotune_memory_target_ratio": { 2024-07-27T03:19:09.155 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "autotune_memory_target_ratio", 2024-07-27T03:19:09.155 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-27T03:19:09.155 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.155 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.155 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0.7", 2024-07-27T03:19:09.155 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.155 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.155 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.155 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-07-27T03:19:09.156 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.156 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.156 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.156 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm_log_destination": { 2024-07-27T03:19:09.156 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cephadm_log_destination", 2024-07-27T03:19:09.156 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.156 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.156 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.156 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.156 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.156 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.156 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.156 INFO:teuthology.orchestra.run.smithi067.stdout: "file", 2024-07-27T03:19:09.157 INFO:teuthology.orchestra.run.smithi067.stdout: "file,syslog", 2024-07-27T03:19:09.157 INFO:teuthology.orchestra.run.smithi067.stdout: "syslog" 2024-07-27T03:19:09.157 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.157 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-07-27T03:19:09.157 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.157 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.157 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.157 INFO:teuthology.orchestra.run.smithi067.stdout: "cgroups_split": { 2024-07-27T03:19:09.157 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cgroups_split", 2024-07-27T03:19:09.157 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.157 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.157 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.157 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.157 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.158 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.158 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.158 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-07-27T03:19:09.158 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.158 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.158 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.158 INFO:teuthology.orchestra.run.smithi067.stdout: "config_checks_enabled": { 2024-07-27T03:19:09.158 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_checks_enabled", 2024-07-27T03:19:09.158 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.158 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.158 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.159 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.159 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.159 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.159 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.159 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-07-27T03:19:09.159 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.159 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.159 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.159 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.159 INFO:teuthology.orchestra.run.smithi067.stdout: "config_dashboard": { 2024-07-27T03:19:09.159 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_dashboard", 2024-07-27T03:19:09.159 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.159 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.159 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.159 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.160 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.160 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.160 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.160 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-07-27T03:19:09.160 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.160 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.160 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.160 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_alertmanager": { 2024-07-27T03:19:09.160 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_alertmanager", 2024-07-27T03:19:09.160 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.160 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.160 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.161 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-07-27T03:19:09.161 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.161 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.161 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.161 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-27T03:19:09.161 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.161 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.161 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.161 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.161 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_base": { 2024-07-27T03:19:09.161 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_base", 2024-07-27T03:19:09.161 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.161 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.161 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.162 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/ceph", 2024-07-27T03:19:09.162 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.162 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.162 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.162 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Container image name, without the tag", 2024-07-27T03:19:09.162 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.162 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.162 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.162 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_elasticsearch": { 2024-07-27T03:19:09.162 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_elasticsearch", 2024-07-27T03:19:09.162 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.162 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.163 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.163 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-07-27T03:19:09.163 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.163 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.163 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.163 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "elasticsearch container image", 2024-07-27T03:19:09.163 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.163 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.163 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.163 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_grafana": { 2024-07-27T03:19:09.163 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_grafana", 2024-07-27T03:19:09.163 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.163 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.163 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.164 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-07-27T03:19:09.164 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.164 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.164 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.164 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-27T03:19:09.164 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.164 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.164 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.164 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_haproxy": { 2024-07-27T03:19:09.164 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_haproxy", 2024-07-27T03:19:09.164 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.164 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.164 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.164 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-07-27T03:19:09.165 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.165 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.165 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.165 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "HAproxy container image", 2024-07-27T03:19:09.165 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.165 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.165 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.165 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_agent": { 2024-07-27T03:19:09.165 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_agent", 2024-07-27T03:19:09.165 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.165 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.165 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.165 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-07-27T03:19:09.165 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.166 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.166 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.166 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger agent container image", 2024-07-27T03:19:09.166 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.166 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.166 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.166 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_collector": { 2024-07-27T03:19:09.166 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_collector", 2024-07-27T03:19:09.166 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.166 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.166 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.166 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-07-27T03:19:09.167 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.167 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.167 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.167 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger collector container image", 2024-07-27T03:19:09.167 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.167 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.167 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.167 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_jaeger_query": { 2024-07-27T03:19:09.167 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_jaeger_query", 2024-07-27T03:19:09.167 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.167 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.167 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.167 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-07-27T03:19:09.168 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.168 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.168 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.168 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Jaeger query container image", 2024-07-27T03:19:09.168 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.168 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.168 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.168 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.168 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_keepalived": { 2024-07-27T03:19:09.168 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_keepalived", 2024-07-27T03:19:09.168 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.168 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.168 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.168 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-07-27T03:19:09.168 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.169 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.169 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.169 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Keepalived container image", 2024-07-27T03:19:09.169 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.169 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.169 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.169 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.169 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_loki": { 2024-07-27T03:19:09.169 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_loki", 2024-07-27T03:19:09.169 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.169 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.169 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.169 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-07-27T03:19:09.169 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.169 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.170 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.170 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Loki container image", 2024-07-27T03:19:09.170 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.170 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.170 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.170 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.170 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nginx": { 2024-07-27T03:19:09.170 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nginx", 2024-07-27T03:19:09.170 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.170 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.170 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.170 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-07-27T03:19:09.170 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.170 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.171 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.171 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nginx container image", 2024-07-27T03:19:09.171 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.171 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.171 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.171 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.171 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_node_exporter": { 2024-07-27T03:19:09.171 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_node_exporter", 2024-07-27T03:19:09.171 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.171 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.171 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.171 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-07-27T03:19:09.171 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.172 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.172 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.172 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-27T03:19:09.172 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.172 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.172 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.172 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.172 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_nvmeof": { 2024-07-27T03:19:09.172 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_nvmeof", 2024-07-27T03:19:09.172 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.172 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.172 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.172 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-07-27T03:19:09.172 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.172 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.173 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.173 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Nvme-of container image", 2024-07-27T03:19:09.173 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.173 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.173 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.173 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.173 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_prometheus": { 2024-07-27T03:19:09.173 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_prometheus", 2024-07-27T03:19:09.173 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.173 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.173 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.173 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-07-27T03:19:09.173 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.173 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.173 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.174 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Prometheus container image", 2024-07-27T03:19:09.174 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.174 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.174 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.174 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.174 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_promtail": { 2024-07-27T03:19:09.174 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_promtail", 2024-07-27T03:19:09.174 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.174 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.174 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.174 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-07-27T03:19:09.174 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.174 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.174 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.175 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Promtail container image", 2024-07-27T03:19:09.175 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.175 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.175 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.175 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.175 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_samba": { 2024-07-27T03:19:09.175 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_samba", 2024-07-27T03:19:09.175 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.175 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.175 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.175 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-07-27T03:19:09.175 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.175 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.175 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.175 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Samba/SMB container image", 2024-07-27T03:19:09.176 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.176 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.176 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.176 INFO:teuthology.orchestra.run.smithi067.stdout: "container_image_snmp_gateway": { 2024-07-27T03:19:09.176 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_image_snmp_gateway", 2024-07-27T03:19:09.176 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.176 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.176 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.176 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-07-27T03:19:09.176 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.176 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.176 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.176 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "SNMP Gateway container image", 2024-07-27T03:19:09.177 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.177 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.177 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.177 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.177 INFO:teuthology.orchestra.run.smithi067.stdout: "container_init": { 2024-07-27T03:19:09.177 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "container_init", 2024-07-27T03:19:09.177 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.177 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.177 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.177 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.177 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.177 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.177 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.177 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Run podman/docker with `--init`", 2024-07-27T03:19:09.177 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.178 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.178 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.178 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.178 INFO:teuthology.orchestra.run.smithi067.stdout: "daemon_cache_timeout": { 2024-07-27T03:19:09.178 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "daemon_cache_timeout", 2024-07-27T03:19:09.178 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:09.178 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.178 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.178 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-27T03:19:09.178 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.178 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.178 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.178 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-07-27T03:19:09.179 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.179 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.179 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.179 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.179 INFO:teuthology.orchestra.run.smithi067.stdout: "default_cephadm_command_timeout": { 2024-07-27T03:19:09.179 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_cephadm_command_timeout", 2024-07-27T03:19:09.179 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.179 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.179 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.179 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "900", 2024-07-27T03:19:09.179 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.179 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.179 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.180 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-07-27T03:19:09.180 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.180 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.180 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.180 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.180 INFO:teuthology.orchestra.run.smithi067.stdout: "default_registry": { 2024-07-27T03:19:09.180 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "default_registry", 2024-07-27T03:19:09.180 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.180 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.180 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.180 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "docker.io", 2024-07-27T03:19:09.180 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.180 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.180 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.180 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-07-27T03:19:09.180 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.181 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.181 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.181 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.181 INFO:teuthology.orchestra.run.smithi067.stdout: "device_cache_timeout": { 2024-07-27T03:19:09.181 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_cache_timeout", 2024-07-27T03:19:09.181 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:09.181 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.181 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.181 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1800", 2024-07-27T03:19:09.181 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.181 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.181 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.181 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache device inventory", 2024-07-27T03:19:09.181 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.181 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.182 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.182 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.182 INFO:teuthology.orchestra.run.smithi067.stdout: "device_enhanced_scan": { 2024-07-27T03:19:09.182 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_enhanced_scan", 2024-07-27T03:19:09.182 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.182 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.182 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.182 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.182 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.182 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.182 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.182 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use libstoragemgmt during device scans", 2024-07-27T03:19:09.182 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.183 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.183 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.183 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.183 INFO:teuthology.orchestra.run.smithi067.stdout: "facts_cache_timeout": { 2024-07-27T03:19:09.183 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "facts_cache_timeout", 2024-07-27T03:19:09.183 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:09.183 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.183 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.183 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-27T03:19:09.183 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.183 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.183 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.183 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "seconds to cache host facts data", 2024-07-27T03:19:09.183 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.184 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.184 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.184 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.184 INFO:teuthology.orchestra.run.smithi067.stdout: "grafana_dashboards_path": { 2024-07-27T03:19:09.184 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "grafana_dashboards_path", 2024-07-27T03:19:09.184 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.184 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.184 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.184 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-07-27T03:19:09.184 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.184 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.184 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.184 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-07-27T03:19:09.184 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.185 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.185 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.185 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.185 INFO:teuthology.orchestra.run.smithi067.stdout: "host_check_interval": { 2024-07-27T03:19:09.185 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "host_check_interval", 2024-07-27T03:19:09.185 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:09.185 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.185 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.185 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-27T03:19:09.185 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.185 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.185 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.185 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to perform a host check", 2024-07-27T03:19:09.185 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.186 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.186 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.186 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.186 INFO:teuthology.orchestra.run.smithi067.stdout: "hw_monitoring": { 2024-07-27T03:19:09.186 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hw_monitoring", 2024-07-27T03:19:09.186 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.186 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.186 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.186 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.186 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.186 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.186 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.186 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-07-27T03:19:09.187 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.187 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.187 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.187 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.187 INFO:teuthology.orchestra.run.smithi067.stdout: "inventory_list_all": { 2024-07-27T03:19:09.187 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "inventory_list_all", 2024-07-27T03:19:09.187 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.187 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.187 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.187 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.187 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.187 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.187 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.187 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-07-27T03:19:09.187 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.188 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.188 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.188 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.188 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.188 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.188 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.188 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.188 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.188 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.188 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.188 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.188 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.189 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.189 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.189 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.189 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.189 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.189 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.189 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.189 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.189 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.189 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.189 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.189 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.189 INFO:teuthology.orchestra.run.smithi067.stdout: "log_refresh_metadata": { 2024-07-27T03:19:09.190 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_refresh_metadata", 2024-07-27T03:19:09.190 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.190 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.190 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.190 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.190 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.190 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.190 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.190 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-07-27T03:19:09.190 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.190 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.190 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.190 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.191 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.191 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.191 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.191 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.191 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.191 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.191 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.191 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.191 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.191 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-07-27T03:19:09.191 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.191 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.191 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.191 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.191 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.192 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.192 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.192 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.192 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.192 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.192 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.192 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.192 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.192 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.192 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.192 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.192 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.192 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.193 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.193 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.193 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.193 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.193 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.193 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.193 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.193 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.193 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.193 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.193 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.193 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.193 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.193 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.193 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.194 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.194 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.194 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.194 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.194 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.194 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.194 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf": { 2024-07-27T03:19:09.194 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-07-27T03:19:09.194 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.194 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.194 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.194 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.194 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.194 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.195 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.195 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-07-27T03:19:09.195 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.195 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.195 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.195 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.195 INFO:teuthology.orchestra.run.smithi067.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-07-27T03:19:09.195 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-07-27T03:19:09.195 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.195 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.195 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.195 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "*", 2024-07-27T03:19:09.195 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.195 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.196 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.196 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-07-27T03:19:09.196 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.196 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.196 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.196 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.196 INFO:teuthology.orchestra.run.smithi067.stdout: "max_count_per_host": { 2024-07-27T03:19:09.196 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_count_per_host", 2024-07-27T03:19:09.196 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.196 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.196 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.196 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:09.196 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.196 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.197 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.197 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of daemons per service per host", 2024-07-27T03:19:09.197 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.197 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.197 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.197 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.197 INFO:teuthology.orchestra.run.smithi067.stdout: "max_osd_draining_count": { 2024-07-27T03:19:09.197 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_osd_draining_count", 2024-07-27T03:19:09.197 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.197 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.197 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.197 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:09.197 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.197 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.198 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.198 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-07-27T03:19:09.198 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.198 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.198 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.198 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.198 INFO:teuthology.orchestra.run.smithi067.stdout: "migration_current": { 2024-07-27T03:19:09.198 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "migration_current", 2024-07-27T03:19:09.198 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.198 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.198 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.198 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.198 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.198 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.199 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.199 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "internal - do not modify", 2024-07-27T03:19:09.199 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.199 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.199 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.199 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.199 INFO:teuthology.orchestra.run.smithi067.stdout: "mode": { 2024-07-27T03:19:09.199 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mode", 2024-07-27T03:19:09.199 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.199 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.199 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.199 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "root", 2024-07-27T03:19:09.199 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.199 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.200 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.200 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm-package", 2024-07-27T03:19:09.200 INFO:teuthology.orchestra.run.smithi067.stdout: "root" 2024-07-27T03:19:09.200 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.200 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "mode for remote execution of cephadm", 2024-07-27T03:19:09.200 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.200 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.200 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.200 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.200 INFO:teuthology.orchestra.run.smithi067.stdout: "oob_default_addr": { 2024-07-27T03:19:09.200 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "oob_default_addr", 2024-07-27T03:19:09.200 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.200 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.200 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.201 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "169.254.1.1", 2024-07-27T03:19:09.201 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.201 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.201 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.201 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Default address for RedFish API (oob management).", 2024-07-27T03:19:09.201 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.201 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.201 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.201 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.201 INFO:teuthology.orchestra.run.smithi067.stdout: "prometheus_alerts_path": { 2024-07-27T03:19:09.201 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus_alerts_path", 2024-07-27T03:19:09.201 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.201 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.201 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.201 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-07-27T03:19:09.202 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.202 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.202 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.202 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-07-27T03:19:09.202 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.202 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.202 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.202 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.202 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_insecure": { 2024-07-27T03:19:09.202 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_insecure", 2024-07-27T03:19:09.202 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.202 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.202 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.203 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.203 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.203 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.203 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.203 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-07-27T03:19:09.203 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.203 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.203 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.203 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.203 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_password": { 2024-07-27T03:19:09.203 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_password", 2024-07-27T03:19:09.203 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.203 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.203 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.204 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.204 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.204 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.204 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.204 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-07-27T03:19:09.204 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.204 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.204 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.204 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.204 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_url": { 2024-07-27T03:19:09.204 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_url", 2024-07-27T03:19:09.204 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.204 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.204 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.205 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.205 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.205 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.205 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.205 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-07-27T03:19:09.205 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.205 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.205 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.205 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.205 INFO:teuthology.orchestra.run.smithi067.stdout: "registry_username": { 2024-07-27T03:19:09.205 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "registry_username", 2024-07-27T03:19:09.205 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.205 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.205 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.205 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.206 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.206 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.206 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.206 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-07-27T03:19:09.206 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.206 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.206 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.206 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.206 INFO:teuthology.orchestra.run.smithi067.stdout: "secure_monitoring_stack": { 2024-07-27T03:19:09.206 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secure_monitoring_stack", 2024-07-27T03:19:09.206 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.206 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.206 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.207 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.207 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.207 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.207 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.207 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-07-27T03:19:09.207 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.207 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.207 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.207 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.207 INFO:teuthology.orchestra.run.smithi067.stdout: "service_discovery_port": { 2024-07-27T03:19:09.207 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "service_discovery_port", 2024-07-27T03:19:09.207 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.207 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.208 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.208 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8765", 2024-07-27T03:19:09.208 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.208 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.208 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.208 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "cephadm service discovery port", 2024-07-27T03:19:09.208 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.208 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.208 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.208 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.208 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.208 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.208 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.208 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.209 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.209 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.209 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.209 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.209 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.209 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.209 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.209 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.209 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.209 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.209 INFO:teuthology.orchestra.run.smithi067.stdout: "ssh_config_file": { 2024-07-27T03:19:09.209 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssh_config_file", 2024-07-27T03:19:09.209 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.209 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.209 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.210 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.210 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.210 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.210 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.210 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-07-27T03:19:09.210 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.210 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.210 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.210 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.210 INFO:teuthology.orchestra.run.smithi067.stdout: "use_agent": { 2024-07-27T03:19:09.210 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_agent", 2024-07-27T03:19:09.210 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.210 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.211 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.211 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.211 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.211 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.211 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.211 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-07-27T03:19:09.211 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.211 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.211 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.211 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.211 INFO:teuthology.orchestra.run.smithi067.stdout: "use_repo_digest": { 2024-07-27T03:19:09.211 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "use_repo_digest", 2024-07-27T03:19:09.211 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.211 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.212 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.212 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.212 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.212 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.212 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.212 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-07-27T03:19:09.212 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.212 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.212 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.212 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.212 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_failed_host_check": { 2024-07-27T03:19:09.212 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_failed_host_check", 2024-07-27T03:19:09.212 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.212 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.213 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.213 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.213 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.213 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.213 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.213 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if the host check fails", 2024-07-27T03:19:09.213 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.213 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.213 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.213 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.213 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_daemons": { 2024-07-27T03:19:09.213 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_daemons", 2024-07-27T03:19:09.213 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.214 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.214 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.214 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.214 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.214 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.214 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.214 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-07-27T03:19:09.214 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.214 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.214 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.214 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.214 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_on_stray_hosts": { 2024-07-27T03:19:09.214 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_on_stray_hosts", 2024-07-27T03:19:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-07-27T03:19:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.215 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.216 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.216 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crash", 2024-07-27T03:19:09.217 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.217 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.217 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.218 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.219 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.221 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.222 INFO:teuthology.orchestra.run.smithi067.stdout: "retain_interval": { 2024-07-27T03:19:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retain_interval", 2024-07-27T03:19:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "31536000", 2024-07-27T03:19:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain crashes before pruning them", 2024-07-27T03:19:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.223 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.224 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_recent_interval": { 2024-07-27T03:19:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_recent_interval", 2024-07-27T03:19:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1209600", 2024-07-27T03:19:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "time interval in which to warn about recent crashes", 2024-07-27T03:19:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.225 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.226 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.226 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.226 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.226 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dashboard", 2024-07-27T03:19:09.226 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.226 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.226 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.226 INFO:teuthology.orchestra.run.smithi067.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-07-27T03:19:09.226 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-07-27T03:19:09.226 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.226 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.226 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.227 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:09.227 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.227 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.227 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.227 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.227 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.227 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.227 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.227 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.227 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_HOST": { 2024-07-27T03:19:09.227 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_HOST", 2024-07-27T03:19:09.227 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.227 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.227 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.228 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.228 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.228 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.228 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.228 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.228 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.228 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.228 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.228 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.228 INFO:teuthology.orchestra.run.smithi067.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-07-27T03:19:09.228 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-07-27T03:19:09.228 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.228 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.228 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.229 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.229 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.229 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.229 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.229 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.229 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.229 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.229 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.229 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.229 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_ENABLED": { 2024-07-27T03:19:09.229 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_ENABLED", 2024-07-27T03:19:09.229 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.229 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.229 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.230 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.230 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.230 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.230 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.230 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.230 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.230 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.230 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.230 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.230 INFO:teuthology.orchestra.run.smithi067.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-07-27T03:19:09.230 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-07-27T03:19:09.230 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.230 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.231 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.231 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.231 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.231 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.231 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.231 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.231 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.231 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.231 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.231 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.231 INFO:teuthology.orchestra.run.smithi067.stdout: "ENABLE_BROWSABLE_API": { 2024-07-27T03:19:09.231 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ENABLE_BROWSABLE_API", 2024-07-27T03:19:09.231 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.231 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.232 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.232 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.232 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.232 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.232 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.232 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.232 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.232 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.232 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.232 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.232 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-07-27T03:19:09.232 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-07-27T03:19:09.232 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.232 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.233 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.233 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.233 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.233 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.233 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.233 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.233 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.233 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.233 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.233 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.233 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-07-27T03:19:09.233 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-07-27T03:19:09.233 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.233 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.234 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.234 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.234 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.234 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.234 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.234 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.234 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.234 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.234 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.234 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.234 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-07-27T03:19:09.234 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-07-27T03:19:09.234 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.235 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.235 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.235 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.235 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.235 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.235 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.235 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.235 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.235 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.235 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.235 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.235 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-07-27T03:19:09.235 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-07-27T03:19:09.235 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.236 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.236 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.236 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.236 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.236 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.236 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.236 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.236 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.236 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.236 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.236 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.236 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_NFS": { 2024-07-27T03:19:09.236 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-07-27T03:19:09.236 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.236 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.237 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.237 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.237 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.237 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.237 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.237 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.237 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.237 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.237 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.237 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.237 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RBD": { 2024-07-27T03:19:09.237 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-07-27T03:19:09.237 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.237 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.238 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.238 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.238 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.238 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.238 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.238 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.238 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.238 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.238 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.238 INFO:teuthology.orchestra.run.smithi067.stdout: "FEATURE_TOGGLE_RGW": { 2024-07-27T03:19:09.238 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-07-27T03:19:09.238 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.238 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.239 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.239 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.239 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.239 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.239 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.239 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.239 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.239 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.239 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.239 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.239 INFO:teuthology.orchestra.run.smithi067.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-07-27T03:19:09.239 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-07-27T03:19:09.239 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.239 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.240 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.240 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.240 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.240 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.240 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.240 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.240 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.240 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.240 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.240 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.240 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_PASSWORD": { 2024-07-27T03:19:09.240 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_PASSWORD", 2024-07-27T03:19:09.240 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.241 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.241 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.241 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-07-27T03:19:09.241 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.241 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.241 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.241 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.241 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.241 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.241 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.241 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.241 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-07-27T03:19:09.241 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-07-27T03:19:09.241 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.242 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.242 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.242 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.242 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.242 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.242 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.242 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.242 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.242 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.242 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.242 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.242 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_URL": { 2024-07-27T03:19:09.242 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_URL", 2024-07-27T03:19:09.243 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.243 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.243 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.243 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.243 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.243 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.243 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.243 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.243 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.243 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.243 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.243 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.243 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_API_USERNAME": { 2024-07-27T03:19:09.244 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_API_USERNAME", 2024-07-27T03:19:09.244 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.244 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.244 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.244 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-07-27T03:19:09.244 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.244 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.244 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.244 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.244 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.244 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.244 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.244 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.244 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-07-27T03:19:09.244 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-07-27T03:19:09.245 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.245 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.245 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.245 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.245 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.245 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.245 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.245 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.245 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.245 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.245 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.245 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.245 INFO:teuthology.orchestra.run.smithi067.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-07-27T03:19:09.246 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-07-27T03:19:09.246 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.246 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.246 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.246 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.246 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.246 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.246 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.246 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.246 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.246 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.246 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.246 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.247 INFO:teuthology.orchestra.run.smithi067.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-07-27T03:19:09.247 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-07-27T03:19:09.247 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.247 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.247 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.247 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.247 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.247 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.247 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.247 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.247 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.247 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.247 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.247 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.248 INFO:teuthology.orchestra.run.smithi067.stdout: "ISSUE_TRACKER_API_KEY": { 2024-07-27T03:19:09.248 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-07-27T03:19:09.248 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.248 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.248 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.248 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.248 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.248 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.248 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.248 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.248 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.248 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.248 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.248 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.249 INFO:teuthology.orchestra.run.smithi067.stdout: "MANAGED_BY_CLUSTERS": { 2024-07-27T03:19:09.249 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-07-27T03:19:09.249 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.249 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.249 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.249 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "[]", 2024-07-27T03:19:09.249 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.249 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.249 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.249 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.249 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.249 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.249 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.250 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.250 INFO:teuthology.orchestra.run.smithi067.stdout: "MULTICLUSTER_CONFIG": { 2024-07-27T03:19:09.250 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "MULTICLUSTER_CONFIG", 2024-07-27T03:19:09.250 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.250 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.250 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.250 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "{}", 2024-07-27T03:19:09.250 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.250 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.250 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.250 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.250 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.250 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.251 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.251 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.251 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_HOST": { 2024-07-27T03:19:09.251 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_HOST", 2024-07-27T03:19:09.251 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.251 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.251 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.251 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.251 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.251 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.251 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.251 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.251 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.251 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.252 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.252 INFO:teuthology.orchestra.run.smithi067.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-07-27T03:19:09.252 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-07-27T03:19:09.252 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.252 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.252 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.252 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.252 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.252 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.252 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.252 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.252 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.252 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.253 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.253 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-07-27T03:19:09.253 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-07-27T03:19:09.253 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.253 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.253 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.253 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.253 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.253 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.253 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.253 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.253 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.253 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.254 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.254 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.254 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-07-27T03:19:09.254 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-07-27T03:19:09.254 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.254 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.254 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.254 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.254 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.254 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.254 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.254 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.254 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.255 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.255 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.255 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-07-27T03:19:09.255 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-07-27T03:19:09.255 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.255 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.255 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.255 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.255 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.255 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.255 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.255 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.255 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.255 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.256 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.256 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-07-27T03:19:09.256 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-07-27T03:19:09.256 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.256 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.256 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.256 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.256 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.256 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.256 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.256 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.256 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.256 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.257 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.257 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-07-27T03:19:09.257 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-07-27T03:19:09.257 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.257 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.257 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.257 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.257 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.257 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.257 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.257 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.257 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.257 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.258 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.258 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.258 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-07-27T03:19:09.258 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-07-27T03:19:09.258 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.258 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.258 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.258 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.258 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.258 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.258 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.258 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.258 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.258 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.259 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.259 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.259 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-07-27T03:19:09.259 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-07-27T03:19:09.259 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.259 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.259 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.259 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.259 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.259 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.259 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.259 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.259 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.259 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.260 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.260 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.260 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_ENABLED": { 2024-07-27T03:19:09.260 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_ENABLED", 2024-07-27T03:19:09.260 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.260 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.260 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.260 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.260 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.260 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.260 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.260 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.260 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.260 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.261 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.261 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.261 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-07-27T03:19:09.261 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-07-27T03:19:09.261 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.261 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.261 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.261 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-07-27T03:19:09.261 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.261 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.261 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.261 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.261 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.262 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.262 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.262 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.262 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-07-27T03:19:09.262 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-07-27T03:19:09.262 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.262 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.262 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.262 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:09.262 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.262 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.262 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.262 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.263 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.263 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.263 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.263 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.263 INFO:teuthology.orchestra.run.smithi067.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-07-27T03:19:09.263 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-07-27T03:19:09.263 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.263 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.263 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.263 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8", 2024-07-27T03:19:09.263 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.263 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.263 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.264 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.264 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.264 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.264 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.264 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.264 INFO:teuthology.orchestra.run.smithi067.stdout: "REST_REQUESTS_TIMEOUT": { 2024-07-27T03:19:09.264 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-07-27T03:19:09.264 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.264 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.264 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.264 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "45", 2024-07-27T03:19:09.264 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.264 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.264 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.265 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.265 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.265 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.265 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.265 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.265 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ACCESS_KEY": { 2024-07-27T03:19:09.265 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ACCESS_KEY", 2024-07-27T03:19:09.265 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.265 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.265 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.265 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.265 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.265 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.265 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.265 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.266 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.266 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.266 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.266 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.266 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-07-27T03:19:09.266 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-07-27T03:19:09.266 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.266 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.266 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.266 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "admin", 2024-07-27T03:19:09.266 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.266 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.266 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.266 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.267 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.267 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.267 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.267 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.267 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SECRET_KEY": { 2024-07-27T03:19:09.267 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SECRET_KEY", 2024-07-27T03:19:09.267 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.267 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.267 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.267 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.267 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.267 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.267 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.267 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.268 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.268 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.268 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.268 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.268 INFO:teuthology.orchestra.run.smithi067.stdout: "RGW_API_SSL_VERIFY": { 2024-07-27T03:19:09.268 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "RGW_API_SSL_VERIFY", 2024-07-27T03:19:09.268 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.268 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.268 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.268 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.268 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.268 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.268 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.268 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.268 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.269 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.269 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.269 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.269 INFO:teuthology.orchestra.run.smithi067.stdout: "UNSAFE_TLS_v1_2": { 2024-07-27T03:19:09.269 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "UNSAFE_TLS_v1_2", 2024-07-27T03:19:09.269 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.269 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.269 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.269 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.269 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.269 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.269 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.269 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.269 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.269 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.270 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.270 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.270 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-07-27T03:19:09.270 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-07-27T03:19:09.270 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.270 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.270 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.270 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.270 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.270 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.270 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.270 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.270 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.271 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.271 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.271 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.271 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-07-27T03:19:09.271 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-07-27T03:19:09.271 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.271 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.271 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.271 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:09.271 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.271 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.271 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.271 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.271 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.272 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.272 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.272 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.272 INFO:teuthology.orchestra.run.smithi067.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-07-27T03:19:09.272 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-07-27T03:19:09.272 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.272 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.272 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.272 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-27T03:19:09.272 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.272 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.272 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.272 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.272 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.273 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.273 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.273 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.273 INFO:teuthology.orchestra.run.smithi067.stdout: "cross_origin_url": { 2024-07-27T03:19:09.273 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cross_origin_url", 2024-07-27T03:19:09.273 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.273 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.273 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.273 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.273 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.273 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.273 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.273 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.273 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.274 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.274 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.274 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.274 INFO:teuthology.orchestra.run.smithi067.stdout: "crt_file": { 2024-07-27T03:19:09.274 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "crt_file", 2024-07-27T03:19:09.274 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.274 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.274 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.274 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.274 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.274 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.274 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.274 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.274 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.275 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.275 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.275 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.275 INFO:teuthology.orchestra.run.smithi067.stdout: "debug": { 2024-07-27T03:19:09.275 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "debug", 2024-07-27T03:19:09.275 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.275 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.275 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.275 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.275 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.275 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.275 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.275 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Enable/disable debug options", 2024-07-27T03:19:09.275 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.275 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.276 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.276 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.276 INFO:teuthology.orchestra.run.smithi067.stdout: "jwt_token_ttl": { 2024-07-27T03:19:09.276 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "jwt_token_ttl", 2024-07-27T03:19:09.276 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.276 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.276 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.276 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "28800", 2024-07-27T03:19:09.276 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.276 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.276 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.276 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.276 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.277 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.277 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.277 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-07-27T03:19:09.277 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-07-27T03:19:09.277 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.277 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.277 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.277 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.277 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.277 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.277 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.277 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.277 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.278 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.278 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.278 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.278 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.278 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.278 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.278 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.278 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.278 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.278 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.278 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.278 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.279 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.279 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.279 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.279 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.279 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.279 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.279 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.279 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.279 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.279 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.279 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.279 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.279 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.279 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.279 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.280 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.280 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.280 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.280 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.280 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.280 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.280 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.280 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.280 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.280 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.280 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.280 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.280 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.280 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.281 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.281 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.281 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.281 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.281 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.281 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.281 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.281 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.281 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.281 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.281 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.281 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.281 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.281 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.282 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.282 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.282 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.282 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.282 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.282 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.282 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.282 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.282 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.282 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.282 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.282 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.282 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.283 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.283 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.283 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.283 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.283 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.283 INFO:teuthology.orchestra.run.smithi067.stdout: "motd": { 2024-07-27T03:19:09.283 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "motd", 2024-07-27T03:19:09.283 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.283 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.283 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.283 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.283 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.283 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.283 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.284 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "The message of the day", 2024-07-27T03:19:09.284 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.284 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.284 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.284 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.284 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect_resolve_ip_addr": { 2024-07-27T03:19:09.284 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "redirect_resolve_ip_addr", 2024-07-27T03:19:09.284 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.284 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.284 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.284 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.284 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.284 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.284 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.285 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.285 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.285 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.285 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.285 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.285 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-07-27T03:19:09.285 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-07-27T03:19:09.285 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.285 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.285 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.285 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-07-27T03:19:09.285 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.285 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.285 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.286 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.286 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.286 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.286 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.286 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.286 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-07-27T03:19:09.286 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-07-27T03:19:09.286 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.286 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.286 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.286 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8080", 2024-07-27T03:19:09.286 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.286 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.286 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.287 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.287 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.287 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.287 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.287 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.287 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.287 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.287 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.287 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.287 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.287 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.287 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.287 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.288 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.288 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.288 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.288 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.288 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.288 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.288 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-07-27T03:19:09.288 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-07-27T03:19:09.288 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.288 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.288 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.288 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.288 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.288 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.289 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.289 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.289 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.289 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.289 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.289 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.289 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl_server_port": { 2024-07-27T03:19:09.289 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl_server_port", 2024-07-27T03:19:09.289 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.289 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.289 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.289 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8443", 2024-07-27T03:19:09.289 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.289 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.290 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.290 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.290 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.290 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.290 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.290 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.290 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-07-27T03:19:09.290 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-07-27T03:19:09.290 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.290 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.290 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.290 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "redirect", 2024-07-27T03:19:09.290 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.291 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.291 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.291 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.291 INFO:teuthology.orchestra.run.smithi067.stdout: "redirect" 2024-07-27T03:19:09.291 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.291 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.291 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.291 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.291 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.291 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.291 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-07-27T03:19:09.291 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-07-27T03:19:09.291 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.291 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.292 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.292 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-07-27T03:19:09.292 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-07-27T03:19:09.292 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-07-27T03:19:09.292 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.292 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.292 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.292 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.292 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.292 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.292 INFO:teuthology.orchestra.run.smithi067.stdout: "url_prefix": { 2024-07-27T03:19:09.292 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url_prefix", 2024-07-27T03:19:09.292 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.293 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.293 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.293 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.293 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.293 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.293 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.293 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.293 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.293 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.293 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.293 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.293 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.293 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.293 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.294 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "devicehealth", 2024-07-27T03:19:09.294 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.294 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.294 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.294 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_monitoring": { 2024-07-27T03:19:09.294 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_monitoring", 2024-07-27T03:19:09.294 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.294 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.294 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.294 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.294 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.294 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.294 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.295 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "monitor device health metrics", 2024-07-27T03:19:09.295 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.295 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.295 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.295 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.295 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.295 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.295 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.295 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.295 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.295 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.295 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.295 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.295 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.296 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.296 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.296 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.296 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.296 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.296 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.296 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.296 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.296 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.296 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.296 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.296 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.296 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.296 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.297 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.297 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.297 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.297 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.297 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.297 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.297 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.297 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.297 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.297 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.297 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.297 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.297 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.298 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.298 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.298 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.298 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.298 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.298 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.298 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.298 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.298 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.298 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.298 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.298 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.298 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.299 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.299 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.299 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.299 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.299 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.299 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.299 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.299 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.299 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.299 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.299 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.299 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.299 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.299 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.299 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.300 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.300 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.300 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.300 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.300 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.300 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.300 INFO:teuthology.orchestra.run.smithi067.stdout: "mark_out_threshold": { 2024-07-27T03:19:09.300 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mark_out_threshold", 2024-07-27T03:19:09.300 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:09.300 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.300 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.300 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "2419200", 2024-07-27T03:19:09.300 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.300 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.301 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.301 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-07-27T03:19:09.301 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.301 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.301 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.301 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.301 INFO:teuthology.orchestra.run.smithi067.stdout: "pool_name": { 2024-07-27T03:19:09.301 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pool_name", 2024-07-27T03:19:09.301 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.301 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.301 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.301 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "device_health_metrics", 2024-07-27T03:19:09.301 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.301 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.302 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.302 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name of pool in which to store device health metrics", 2024-07-27T03:19:09.302 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.302 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.302 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.302 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.302 INFO:teuthology.orchestra.run.smithi067.stdout: "retention_period": { 2024-07-27T03:19:09.302 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "retention_period", 2024-07-27T03:19:09.302 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:09.302 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.302 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.302 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15552000", 2024-07-27T03:19:09.302 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.302 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long to retain device health metrics", 2024-07-27T03:19:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_frequency": { 2024-07-27T03:19:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_frequency", 2024-07-27T03:19:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-07-27T03:19:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.303 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.304 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.304 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to scrape device health metrics", 2024-07-27T03:19:09.304 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.304 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.304 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.304 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.304 INFO:teuthology.orchestra.run.smithi067.stdout: "self_heal": { 2024-07-27T03:19:09.304 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "self_heal", 2024-07-27T03:19:09.304 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.304 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.304 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.304 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.304 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.305 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.305 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.305 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-07-27T03:19:09.305 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.305 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.305 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.305 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.305 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-27T03:19:09.305 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-27T03:19:09.305 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:09.305 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.305 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.305 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-27T03:19:09.306 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.306 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.306 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.306 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how frequently to wake up and check device health", 2024-07-27T03:19:09.306 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.306 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.306 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.306 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.306 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.306 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.306 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.306 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.306 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.306 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.307 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.307 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.307 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.307 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.307 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.307 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.307 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.307 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.307 INFO:teuthology.orchestra.run.smithi067.stdout: "warn_threshold": { 2024-07-27T03:19:09.307 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "warn_threshold", 2024-07-27T03:19:09.307 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:09.307 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.307 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.307 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7257600", 2024-07-27T03:19:09.308 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.308 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.308 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.308 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-07-27T03:19:09.308 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.308 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.308 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.308 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.308 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.308 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.308 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.308 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "diskprediction_local", 2024-07-27T03:19:09.308 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.309 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.309 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.309 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.309 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.309 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.309 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.309 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.309 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.309 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.309 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.309 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.309 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.309 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.310 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.310 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.310 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.310 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.310 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.310 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.310 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.310 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.310 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.310 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.310 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.310 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.310 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.311 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.311 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.311 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.311 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.311 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.311 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.311 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.311 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.311 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.311 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.311 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.311 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.311 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.311 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.312 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.312 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.312 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.312 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.312 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.312 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.312 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.312 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.312 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.312 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.312 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.312 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.312 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.312 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.313 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.313 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.313 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.313 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.313 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.313 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.313 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.313 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.313 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.313 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.313 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.313 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.313 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.313 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.314 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.314 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.314 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.314 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.314 INFO:teuthology.orchestra.run.smithi067.stdout: "predict_interval": { 2024-07-27T03:19:09.314 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predict_interval", 2024-07-27T03:19:09.314 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.314 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.314 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.314 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "86400", 2024-07-27T03:19:09.314 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.314 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.314 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.314 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.315 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.315 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.315 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.315 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.315 INFO:teuthology.orchestra.run.smithi067.stdout: "predictor_model": { 2024-07-27T03:19:09.315 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "predictor_model", 2024-07-27T03:19:09.315 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.315 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.315 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.315 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "prophetstor", 2024-07-27T03:19:09.315 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.315 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.315 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.315 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.316 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.316 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.316 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.316 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.316 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-27T03:19:09.316 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-27T03:19:09.316 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.316 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.316 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.316 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "600", 2024-07-27T03:19:09.316 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.316 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.316 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.317 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.317 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.317 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.317 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.317 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.317 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.317 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.317 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.317 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.317 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.317 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.317 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.317 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.317 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.317 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.318 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.318 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.318 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.318 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.318 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.318 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.318 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.318 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "influx", 2024-07-27T03:19:09.318 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": false, 2024-07-27T03:19:09.318 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "influxdb python module not found", 2024-07-27T03:19:09.318 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.318 INFO:teuthology.orchestra.run.smithi067.stdout: "batch_size": { 2024-07-27T03:19:09.318 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "batch_size", 2024-07-27T03:19:09.318 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5000", 2024-07-27T03:19:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-07-27T03:19:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: "database": { 2024-07-27T03:19:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "database", 2024-07-27T03:19:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.319 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "ceph", 2024-07-27T03:19:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-07-27T03:19:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: "hostname": { 2024-07-27T03:19:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "hostname", 2024-07-27T03:19:09.320 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server hostname", 2024-07-27T03:19:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-27T03:19:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-27T03:19:09.321 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "30", 2024-07-27T03:19:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "5", 2024-07-27T03:19:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-07-27T03:19:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.322 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.323 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.324 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.325 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.326 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: "password": { 2024-07-27T03:19:09.327 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "password", 2024-07-27T03:19:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "password of InfluxDB server user", 2024-07-27T03:19:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: "port": { 2024-07-27T03:19:09.328 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "port", 2024-07-27T03:19:09.329 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.329 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.329 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.329 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "8086", 2024-07-27T03:19:09.329 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.329 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.329 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.329 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "InfluxDB server port", 2024-07-27T03:19:09.329 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.329 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.329 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.329 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.329 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.330 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.330 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.330 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.330 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.330 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.330 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.330 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.330 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.330 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.330 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.330 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.330 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.330 INFO:teuthology.orchestra.run.smithi067.stdout: "ssl": { 2024-07-27T03:19:09.330 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ssl", 2024-07-27T03:19:09.331 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.331 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.331 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.331 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "false", 2024-07-27T03:19:09.331 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.331 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.331 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.331 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-07-27T03:19:09.331 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.331 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.331 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.331 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.331 INFO:teuthology.orchestra.run.smithi067.stdout: "threads": { 2024-07-27T03:19:09.332 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threads", 2024-07-27T03:19:09.332 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.332 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.332 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.332 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-27T03:19:09.332 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-27T03:19:09.332 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "32", 2024-07-27T03:19:09.332 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.332 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-07-27T03:19:09.332 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.332 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.332 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.332 INFO:teuthology.orchestra.run.smithi067.stdout: "username": { 2024-07-27T03:19:09.333 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "username", 2024-07-27T03:19:09.333 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.333 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.333 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.333 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.333 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.333 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.333 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.333 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "username of InfluxDB server user", 2024-07-27T03:19:09.333 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.333 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.333 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.333 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.333 INFO:teuthology.orchestra.run.smithi067.stdout: "verify_ssl": { 2024-07-27T03:19:09.333 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "verify_ssl", 2024-07-27T03:19:09.334 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.334 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.334 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.334 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "true", 2024-07-27T03:19:09.334 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.334 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.334 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.334 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-07-27T03:19:09.334 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.334 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.334 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.334 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.334 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.335 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.335 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.335 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "insights", 2024-07-27T03:19:09.335 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.335 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.335 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.335 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.335 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.335 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.335 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.335 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.335 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.335 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.335 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.336 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.336 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.336 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.336 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.336 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.336 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.336 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.336 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.336 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.336 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.336 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.336 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.336 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.336 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.336 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.337 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.337 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.337 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.337 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.337 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.337 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.337 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.337 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.337 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.337 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.337 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.337 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.337 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.337 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.338 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.338 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.338 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.338 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.338 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.338 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.338 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.338 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.338 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.338 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.338 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.338 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.338 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.339 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.339 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.339 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.339 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.339 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.339 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.339 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.339 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.339 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.339 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.339 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.339 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.339 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.339 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.340 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.340 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.340 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.340 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.340 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.340 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.340 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.340 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.340 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.340 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.340 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.340 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.341 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.341 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.341 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.341 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.341 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.341 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.341 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.341 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.341 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.341 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.341 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.341 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "iostat", 2024-07-27T03:19:09.341 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.341 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.342 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.342 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.342 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.342 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.342 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.342 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.342 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.342 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.342 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.342 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.342 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.342 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.342 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.343 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.343 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.343 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.343 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.343 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.343 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.343 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.343 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.343 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.343 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.343 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.343 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.344 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.344 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.344 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.344 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.344 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.344 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.344 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.344 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.344 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.344 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.344 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.344 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.344 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.344 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.345 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.345 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.345 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.345 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.345 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.345 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.345 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.345 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.345 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.345 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.345 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.345 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.345 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.345 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.346 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.346 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.346 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.346 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.346 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.346 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.346 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.346 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.346 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.346 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.346 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.346 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.347 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.347 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.347 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.347 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.347 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.347 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.347 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.347 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.347 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.347 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.347 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.347 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.348 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.348 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.348 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.348 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.348 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.348 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.348 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.348 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "k8sevents", 2024-07-27T03:19:09.348 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.348 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.348 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.348 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_event_retention_days": { 2024-07-27T03:19:09.348 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "ceph_event_retention_days", 2024-07-27T03:19:09.349 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.349 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.349 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.349 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "7", 2024-07-27T03:19:09.349 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.349 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.349 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.349 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Days to hold ceph event information within local cache", 2024-07-27T03:19:09.349 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.349 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.349 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.349 INFO:teuthology.orchestra.run.smithi067.stdout: "config_check_secs": { 2024-07-27T03:19:09.350 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "config_check_secs", 2024-07-27T03:19:09.350 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.350 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.350 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.350 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:09.350 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "10", 2024-07-27T03:19:09.350 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.350 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.350 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-07-27T03:19:09.350 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.350 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.350 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.351 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.351 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.351 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.351 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.351 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.351 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.351 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.351 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.351 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.351 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.351 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.351 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.351 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.352 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.352 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.352 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.352 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.352 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.352 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.352 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.352 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.352 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.352 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.352 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.352 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.352 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.352 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.352 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.353 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.353 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.353 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.353 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.353 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.353 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.353 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.353 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.353 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.353 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.353 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.353 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.353 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.354 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.354 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.354 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.354 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.354 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.354 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.354 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.354 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.354 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.354 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.354 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.354 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.354 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.355 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.355 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.355 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.355 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.355 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.355 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.355 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.355 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.355 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.355 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.355 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.355 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.355 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.356 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.356 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.356 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.356 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.356 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.356 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.356 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.356 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.356 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.356 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.356 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.356 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.356 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.357 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.357 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.357 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.357 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.357 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.357 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.357 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "localpool", 2024-07-27T03:19:09.357 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.357 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.357 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.357 INFO:teuthology.orchestra.run.smithi067.stdout: "failure_domain": { 2024-07-27T03:19:09.357 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "failure_domain", 2024-07-27T03:19:09.357 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.358 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.358 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.358 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "host", 2024-07-27T03:19:09.358 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.358 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.358 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.358 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "failure domain for any created local pool", 2024-07-27T03:19:09.358 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-07-27T03:19:09.358 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.358 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.358 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.358 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.358 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.359 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.359 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.359 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.359 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.359 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.359 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.359 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.359 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.359 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.359 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.359 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.359 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.359 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.359 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.359 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.360 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.360 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.360 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.360 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.360 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.360 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.360 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.360 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.360 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.360 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.360 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.360 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.360 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.361 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.361 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.361 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.361 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.361 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.361 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.361 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.361 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.361 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.361 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.361 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.361 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.361 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.362 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.362 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.362 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.362 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.362 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.362 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.362 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.362 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.362 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.362 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.362 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.362 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.363 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.363 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.363 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.363 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.363 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.363 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.363 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.363 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.363 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.363 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.363 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.363 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.363 INFO:teuthology.orchestra.run.smithi067.stdout: "min_size": { 2024-07-27T03:19:09.364 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "min_size", 2024-07-27T03:19:09.364 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.364 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.364 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.364 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.364 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.364 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.364 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.364 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default min_size for any created local pool", 2024-07-27T03:19:09.364 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-07-27T03:19:09.364 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.364 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.364 INFO:teuthology.orchestra.run.smithi067.stdout: "num_rep": { 2024-07-27T03:19:09.365 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "num_rep", 2024-07-27T03:19:09.365 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.365 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.365 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.365 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3", 2024-07-27T03:19:09.365 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.365 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.365 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.365 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default replica count for any created local pool", 2024-07-27T03:19:09.365 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.365 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.365 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.365 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.365 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_num": { 2024-07-27T03:19:09.366 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_num", 2024-07-27T03:19:09.366 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.366 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.366 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.366 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "128", 2024-07-27T03:19:09.366 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.366 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.366 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.366 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "default pg_num for any created local pool", 2024-07-27T03:19:09.366 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.366 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.366 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.366 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.367 INFO:teuthology.orchestra.run.smithi067.stdout: "prefix": { 2024-07-27T03:19:09.367 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prefix", 2024-07-27T03:19:09.367 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.367 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.367 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.367 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.367 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.367 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.367 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.367 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "name prefix for any created local pool", 2024-07-27T03:19:09.367 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.367 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.367 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.368 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.368 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.368 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.368 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.368 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.368 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.368 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.368 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.368 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.368 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.368 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.368 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.368 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.368 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.369 INFO:teuthology.orchestra.run.smithi067.stdout: "subtree": { 2024-07-27T03:19:09.369 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "subtree", 2024-07-27T03:19:09.369 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.369 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.369 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.369 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "rack", 2024-07-27T03:19:09.369 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.369 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.369 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.369 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "CRUSH level for which to create a local pool", 2024-07-27T03:19:09.369 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-07-27T03:19:09.369 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.369 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.369 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.369 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.370 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.370 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mds_autoscaler", 2024-07-27T03:19:09.370 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.370 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.370 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.370 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.370 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.370 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.370 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.370 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.370 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.370 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.371 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.371 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.371 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.371 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.371 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.371 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.371 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.371 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.371 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.371 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.371 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.371 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.371 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.372 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.372 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.372 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.372 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.372 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.372 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.372 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.372 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.372 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.372 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.372 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.372 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.372 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.372 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.373 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.373 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.373 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.373 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.373 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.373 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.373 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.373 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.373 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.373 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.373 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.373 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.373 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.373 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.374 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.374 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.374 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.374 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.374 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.374 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.374 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.374 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.374 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.374 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.374 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.374 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.374 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.374 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.375 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.375 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.375 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.375 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.375 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.375 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.375 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.375 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.375 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.375 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.375 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.375 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.375 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.375 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.376 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.376 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.376 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.376 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.376 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.376 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.376 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.376 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.376 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.376 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.376 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirroring", 2024-07-27T03:19:09.376 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.377 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.377 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.377 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.377 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.377 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.377 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.377 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.377 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.377 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.377 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.377 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.377 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.377 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.378 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.378 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.378 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.378 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.378 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.378 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.378 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.378 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.378 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.378 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.378 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.378 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.378 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.379 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.379 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.379 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.379 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.379 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.379 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.379 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.379 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.379 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.379 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.379 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.379 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.379 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.379 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.380 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.380 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.380 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.380 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.380 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.380 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.380 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.380 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.380 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.380 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.380 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.380 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.380 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.380 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.380 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.381 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.381 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.381 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.381 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.381 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.381 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.381 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.381 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.381 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.381 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.381 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.381 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.381 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.381 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.381 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.382 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.382 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.382 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.382 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.382 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.382 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.382 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.382 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.382 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.382 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.382 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.382 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.382 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.382 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.382 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.383 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "nfs", 2024-07-27T03:19:09.383 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.383 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.383 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.383 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.383 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.383 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.383 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.383 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.383 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.383 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.383 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.383 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.383 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.383 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.384 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.384 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.384 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.384 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.384 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.384 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.384 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.384 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.384 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.384 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.384 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.384 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.384 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.384 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.384 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.385 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.385 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.385 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.385 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.385 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.385 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.385 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.385 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.385 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.385 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.385 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.385 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.385 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.385 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.386 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.386 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.386 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.386 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.386 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.386 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.386 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.386 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.386 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.386 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.386 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.386 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.386 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.386 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.387 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.387 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.387 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.387 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.387 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.387 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.387 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.387 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.387 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.387 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.387 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.387 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.387 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.387 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.388 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.388 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.388 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.388 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.388 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.388 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.388 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.388 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.388 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.388 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.388 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.388 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.388 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.389 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.389 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.389 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.389 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.389 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.389 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-07-27T03:19:09.389 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.389 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.389 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.389 INFO:teuthology.orchestra.run.smithi067.stdout: "fail_fs": { 2024-07-27T03:19:09.389 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "fail_fs", 2024-07-27T03:19:09.389 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.389 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.389 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.389 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.389 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.390 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.390 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.390 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-07-27T03:19:09.390 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.390 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.390 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.390 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.390 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.390 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.390 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.390 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.390 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.390 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.390 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.391 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.391 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.391 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.391 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.391 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.391 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.391 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.391 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.391 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.391 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.391 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.391 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.391 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.391 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.391 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.392 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.392 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.392 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.392 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.392 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.392 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.392 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.392 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.392 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.392 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.392 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.392 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.392 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.392 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.392 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.392 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.393 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.393 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.393 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.393 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.393 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.393 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.393 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.393 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.393 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.393 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.393 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.393 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.393 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.393 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.393 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.394 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.394 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.394 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.394 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.394 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.394 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.394 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.394 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.394 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.394 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.394 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.394 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.394 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.394 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.395 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.395 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.395 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.395 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator": { 2024-07-27T03:19:09.395 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "orchestrator", 2024-07-27T03:19:09.395 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.395 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.395 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.395 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.395 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.395 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.395 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.395 INFO:teuthology.orchestra.run.smithi067.stdout: "cephadm", 2024-07-27T03:19:09.395 INFO:teuthology.orchestra.run.smithi067.stdout: "rook", 2024-07-27T03:19:09.395 INFO:teuthology.orchestra.run.smithi067.stdout: "test_orchestrator" 2024-07-27T03:19:09.396 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.396 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Orchestrator backend", 2024-07-27T03:19:09.396 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.396 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.396 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.396 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.396 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.396 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.396 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.396 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.396 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.396 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.396 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.396 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.397 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.397 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.397 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.397 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.397 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.397 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.397 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.397 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.397 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.397 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_perf_query", 2024-07-27T03:19:09.397 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.397 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.397 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.398 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.398 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.398 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.398 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.398 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.398 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.398 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.398 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.398 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.398 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.398 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.398 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.398 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.399 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.399 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.399 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.399 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.399 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.399 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.399 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.399 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.399 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.399 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.399 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.399 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.399 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.399 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.400 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.400 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.400 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.400 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.400 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.400 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.400 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.400 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.400 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.400 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.400 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.400 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.400 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.401 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.401 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.401 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.401 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.401 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.401 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.401 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.401 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.401 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.401 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.401 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.401 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.401 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.401 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.402 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.402 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.402 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.402 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.402 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.402 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.402 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.402 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.402 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.402 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.402 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.402 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.402 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.402 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.402 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.403 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.403 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.403 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.403 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.403 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.403 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.403 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.403 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.403 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.403 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.403 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.403 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.403 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.404 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.404 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.404 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.404 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "osd_support", 2024-07-27T03:19:09.404 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.404 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.404 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.404 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.404 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.404 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.404 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.404 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.404 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.404 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.404 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.405 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.405 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.405 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.405 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.405 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.405 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.405 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.405 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.405 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.405 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.405 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.405 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.405 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.406 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.406 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.406 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.406 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.406 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.406 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.406 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.406 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.406 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.406 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.406 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.406 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.406 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.407 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.407 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.407 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.407 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.407 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.407 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.407 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.407 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.407 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.407 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.407 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.407 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.407 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.407 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.408 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.408 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.408 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.408 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.408 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.408 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.408 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.408 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.408 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.408 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.408 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.408 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.408 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.409 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.409 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.409 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.409 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.409 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.409 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.409 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.409 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.409 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.409 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.409 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.409 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.410 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.410 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.410 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.410 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.410 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.410 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.410 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.410 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.410 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.410 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.410 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.410 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.410 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "pg_autoscaler", 2024-07-27T03:19:09.411 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.411 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.411 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.411 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.411 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.411 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.411 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.411 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.411 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.411 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.411 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.411 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.411 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.411 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.412 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.412 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.412 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.412 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.412 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.412 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.412 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.412 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.412 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.412 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.412 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.412 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.412 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.413 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.413 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.413 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.413 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.413 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.413 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.413 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.413 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.413 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.413 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.413 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.413 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.414 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.414 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.414 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.414 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.414 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.414 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.414 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.414 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.414 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.414 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.414 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.414 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.414 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.415 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.415 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.415 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.415 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.415 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.415 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.415 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.415 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.415 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.415 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.415 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.415 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.415 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.415 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.416 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.416 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.416 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.416 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.416 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-27T03:19:09.416 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-27T03:19:09.416 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:09.416 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.416 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.416 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-27T03:19:09.416 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.416 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.416 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.417 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.417 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.417 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.417 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.417 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.417 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.417 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.417 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.417 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.417 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.417 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.417 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.417 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.418 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.418 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.418 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.418 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.418 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.418 INFO:teuthology.orchestra.run.smithi067.stdout: "threshold": { 2024-07-27T03:19:09.418 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "threshold", 2024-07-27T03:19:09.418 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-27T03:19:09.418 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.418 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.418 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "3.0", 2024-07-27T03:19:09.418 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1.0", 2024-07-27T03:19:09.419 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.419 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.419 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "scaling threshold", 2024-07-27T03:19:09.419 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-07-27T03:19:09.419 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.419 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.419 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.419 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.419 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.419 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "progress", 2024-07-27T03:19:09.419 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.419 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.419 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.419 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_pg_recovery_event": { 2024-07-27T03:19:09.420 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_pg_recovery_event", 2024-07-27T03:19:09.420 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.420 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.420 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.420 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.420 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.420 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.420 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.420 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow the module to show pg recovery progress", 2024-07-27T03:19:09.420 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.420 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.420 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.420 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-07-27T03:19:09.421 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-07-27T03:19:09.421 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.421 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.421 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.421 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.421 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.421 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.421 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.421 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.421 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.421 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.421 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.421 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.422 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.422 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.422 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.422 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.422 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.422 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.422 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.422 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.422 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.422 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.422 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.422 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.422 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.423 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.423 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.423 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.423 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.423 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.423 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.423 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.423 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.423 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.423 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.423 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.423 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.423 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.423 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.423 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.424 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.424 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.424 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.424 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.424 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.424 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.425 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.426 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.426 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.426 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.426 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.426 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.426 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.426 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.426 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.426 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.426 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.426 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.426 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.426 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.426 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.427 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.427 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.427 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.427 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.427 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.427 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.427 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.427 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.427 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.427 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.427 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.427 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.427 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.428 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.428 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.428 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.428 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.428 INFO:teuthology.orchestra.run.smithi067.stdout: "max_completed_events": { 2024-07-27T03:19:09.428 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_completed_events", 2024-07-27T03:19:09.428 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.428 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.428 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.428 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "50", 2024-07-27T03:19:09.428 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.428 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.428 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.429 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "number of past completed events to remember", 2024-07-27T03:19:09.429 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.429 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.429 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.429 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.429 INFO:teuthology.orchestra.run.smithi067.stdout: "sleep_interval": { 2024-07-27T03:19:09.429 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sleep_interval", 2024-07-27T03:19:09.429 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:09.429 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.429 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.429 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-27T03:19:09.429 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.429 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.429 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.430 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "how long the module is going to sleep", 2024-07-27T03:19:09.430 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.430 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.430 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.430 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.430 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.430 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.430 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.430 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.430 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.430 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.430 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.430 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.431 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.431 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.431 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.431 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.431 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.431 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.431 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.431 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.431 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "prometheus", 2024-07-27T03:19:09.431 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.431 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.431 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.431 INFO:teuthology.orchestra.run.smithi067.stdout: "cache": { 2024-07-27T03:19:09.431 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "cache", 2024-07-27T03:19:09.432 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.432 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.432 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.432 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.432 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.432 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.432 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.432 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.432 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.432 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.432 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.432 INFO:teuthology.orchestra.run.smithi067.stdout: "exclude_perf_counters": { 2024-07-27T03:19:09.432 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "exclude_perf_counters", 2024-07-27T03:19:09.432 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.433 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.433 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.433 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.433 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.433 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.433 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.433 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-07-27T03:19:09.433 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-07-27T03:19:09.433 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.433 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.433 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.433 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.433 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.434 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.434 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.434 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.434 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.434 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.434 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.434 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.434 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.434 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.434 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.434 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.434 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.434 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.435 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.435 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.435 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.435 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.435 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.435 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.435 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.435 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.435 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.435 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.435 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.435 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.435 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.435 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.436 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.436 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.436 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.436 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.436 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.436 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.436 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.436 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.436 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.436 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.436 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.436 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.437 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.437 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.437 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.437 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.437 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.437 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.437 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.437 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.437 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.437 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.437 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.437 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.437 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.438 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.438 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.438 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.438 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.438 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.438 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.438 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.438 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.438 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.438 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.438 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.438 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools": { 2024-07-27T03:19:09.439 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools", 2024-07-27T03:19:09.439 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.439 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.439 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.439 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.439 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.439 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.439 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.439 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.439 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.439 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.439 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.439 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.439 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_stats_pools_refresh_interval": { 2024-07-27T03:19:09.439 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-07-27T03:19:09.440 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.440 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.440 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.440 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "300", 2024-07-27T03:19:09.440 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.440 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.440 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.440 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.440 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.440 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.440 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.440 INFO:teuthology.orchestra.run.smithi067.stdout: "scrape_interval": { 2024-07-27T03:19:09.440 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "scrape_interval", 2024-07-27T03:19:09.441 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-27T03:19:09.441 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.441 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.441 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15.0", 2024-07-27T03:19:09.441 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.441 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.441 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.441 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.441 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.441 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.441 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.441 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-07-27T03:19:09.442 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-07-27T03:19:09.442 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.442 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.442 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.442 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "::", 2024-07-27T03:19:09.442 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.442 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.442 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.442 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-07-27T03:19:09.442 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.442 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.442 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.442 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-07-27T03:19:09.443 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-07-27T03:19:09.443 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.443 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.443 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.443 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "9283", 2024-07-27T03:19:09.443 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.443 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.443 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.443 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-07-27T03:19:09.443 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.443 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.443 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.443 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.444 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.444 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.444 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.444 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.444 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.444 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.444 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.444 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.444 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.444 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.444 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.444 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.444 INFO:teuthology.orchestra.run.smithi067.stdout: "stale_cache_strategy": { 2024-07-27T03:19:09.445 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stale_cache_strategy", 2024-07-27T03:19:09.445 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.445 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.445 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.445 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "log", 2024-07-27T03:19:09.445 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.445 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.445 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.445 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.445 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.445 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.445 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.445 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_behaviour": { 2024-07-27T03:19:09.446 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_behaviour", 2024-07-27T03:19:09.446 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.446 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.446 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.446 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "default", 2024-07-27T03:19:09.446 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.446 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.446 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.446 INFO:teuthology.orchestra.run.smithi067.stdout: "default", 2024-07-27T03:19:09.446 INFO:teuthology.orchestra.run.smithi067.stdout: "error" 2024-07-27T03:19:09.446 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.446 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.446 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.447 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.447 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.447 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.447 INFO:teuthology.orchestra.run.smithi067.stdout: "standby_error_status_code": { 2024-07-27T03:19:09.447 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "standby_error_status_code", 2024-07-27T03:19:09.447 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.447 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.447 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.447 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "500", 2024-07-27T03:19:09.447 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "400", 2024-07-27T03:19:09.447 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "599", 2024-07-27T03:19:09.447 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.447 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.447 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.448 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.448 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.448 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.448 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.448 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_support", 2024-07-27T03:19:09.448 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.448 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.448 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.448 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.448 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.448 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.448 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.449 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.449 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.449 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.449 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.449 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.449 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.449 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.449 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.449 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.449 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.449 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.449 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.449 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.449 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.450 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.450 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.450 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.450 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.450 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.450 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.450 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.450 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.450 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.450 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.450 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.450 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.451 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.451 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.451 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.451 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.451 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.451 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.451 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.451 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.451 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.451 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.451 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.451 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.452 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.452 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.452 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.452 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.452 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.452 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.452 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.452 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.452 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.452 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.452 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.452 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.452 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.453 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.453 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.453 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.453 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.453 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.453 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.453 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.453 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.453 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.453 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.453 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.453 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_snap_create": { 2024-07-27T03:19:09.454 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_snap_create", 2024-07-27T03:19:09.454 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.454 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.454 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.454 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10", 2024-07-27T03:19:09.454 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.454 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.454 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.454 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.454 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.454 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.454 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.455 INFO:teuthology.orchestra.run.smithi067.stdout: "mirror_snapshot_schedule": { 2024-07-27T03:19:09.455 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "mirror_snapshot_schedule", 2024-07-27T03:19:09.455 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.455 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.455 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.455 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.455 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.455 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.455 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.455 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.455 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.455 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.455 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.455 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.455 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.456 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.456 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.456 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.456 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.456 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.456 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.456 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.456 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.456 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.456 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.456 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.456 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.456 INFO:teuthology.orchestra.run.smithi067.stdout: "trash_purge_schedule": { 2024-07-27T03:19:09.457 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "trash_purge_schedule", 2024-07-27T03:19:09.457 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.457 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.457 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.457 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.457 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.457 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.457 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.457 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.457 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.457 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.457 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.457 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.457 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.458 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.458 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "restful", 2024-07-27T03:19:09.458 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.458 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.458 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.458 INFO:teuthology.orchestra.run.smithi067.stdout: "enable_auth": { 2024-07-27T03:19:09.458 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enable_auth", 2024-07-27T03:19:09.458 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.458 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.458 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.458 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.458 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.459 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.459 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.459 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.459 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.459 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.459 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.459 INFO:teuthology.orchestra.run.smithi067.stdout: "key_file": { 2024-07-27T03:19:09.459 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "key_file", 2024-07-27T03:19:09.459 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.459 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.459 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.459 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.459 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.460 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.460 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.460 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.460 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.460 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.460 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.460 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.460 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.460 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.460 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.460 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.460 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.460 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.461 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.461 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.461 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.461 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.461 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.461 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.461 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.461 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.461 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.461 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.461 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.461 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.461 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.462 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.462 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.462 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.462 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.462 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.462 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.462 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.462 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.462 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.462 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.462 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.462 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.462 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.463 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.463 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.463 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.463 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.463 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.463 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.463 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.463 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.463 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.463 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.463 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.463 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.464 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.464 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.464 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.464 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.464 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.464 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.464 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.464 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.464 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.464 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.464 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.464 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.464 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.465 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.465 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.465 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.465 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.465 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.465 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.465 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.465 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.465 INFO:teuthology.orchestra.run.smithi067.stdout: "server_addr": { 2024-07-27T03:19:09.465 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_addr", 2024-07-27T03:19:09.465 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.465 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.465 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.466 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.466 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.466 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.466 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.466 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.466 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.466 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.466 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.466 INFO:teuthology.orchestra.run.smithi067.stdout: "server_port": { 2024-07-27T03:19:09.466 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "server_port", 2024-07-27T03:19:09.466 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.466 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.467 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.467 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.467 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.467 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.467 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.467 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.467 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.467 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.467 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.467 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.467 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.467 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.467 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.467 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.468 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.468 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.468 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.468 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.468 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.468 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.468 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.468 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.468 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.468 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.468 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.468 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.468 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rgw", 2024-07-27T03:19:09.469 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.469 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.469 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.469 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.469 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.469 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.469 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.469 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.469 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.469 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.469 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.469 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.469 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.469 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.469 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.470 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.470 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.470 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.470 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.470 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.470 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.470 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.470 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.470 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.470 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.470 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.470 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.470 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.471 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.471 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.471 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.471 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.471 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.471 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.471 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.471 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.471 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.471 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.471 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.471 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.471 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.472 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.472 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.472 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.472 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.472 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.472 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.472 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.472 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.472 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.472 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.472 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.472 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.472 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.472 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.473 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.473 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.473 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.473 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.473 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.473 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.473 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.473 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.473 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.473 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.473 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.473 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.473 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.474 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.474 INFO:teuthology.orchestra.run.smithi067.stdout: "secondary_zone_period_retry_limit": { 2024-07-27T03:19:09.474 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "secondary_zone_period_retry_limit", 2024-07-27T03:19:09.474 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.474 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.474 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.474 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "5", 2024-07-27T03:19:09.474 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.474 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.474 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.474 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-07-27T03:19:09.474 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.475 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.475 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.475 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.475 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.475 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.475 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.475 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.475 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.475 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.475 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.475 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.475 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.475 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.476 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.476 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.476 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.476 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.476 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.476 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rook", 2024-07-27T03:19:09.476 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.476 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.476 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.476 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.476 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.476 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.476 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.477 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.477 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.477 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.477 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.477 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.477 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.477 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.477 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.477 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.477 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.477 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.477 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.477 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.477 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.478 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.478 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.478 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.478 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.478 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.478 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.478 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.478 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.478 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.478 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.478 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.478 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.478 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.479 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.479 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.479 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.479 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.479 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.479 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.479 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.479 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.479 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.479 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.479 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.479 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.480 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.480 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.480 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.480 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.480 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.480 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.480 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.480 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.480 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.480 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.480 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.480 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.480 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.481 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.481 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.481 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.481 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.481 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.481 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.481 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.481 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.481 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.481 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.481 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.481 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.481 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.482 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.482 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.482 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.482 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.482 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.482 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.482 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.482 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.482 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.482 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.482 INFO:teuthology.orchestra.run.smithi067.stdout: "storage_class": { 2024-07-27T03:19:09.482 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "storage_class", 2024-07-27T03:19:09.483 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.483 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.483 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.483 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "local", 2024-07-27T03:19:09.483 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.483 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.483 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.483 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-07-27T03:19:09.483 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.483 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.483 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.483 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.483 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.484 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "selftest", 2024-07-27T03:19:09.484 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.484 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.484 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.484 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.484 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.484 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.484 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.484 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.484 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.484 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.484 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.485 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.485 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.485 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.485 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.485 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.485 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.485 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.485 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.485 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.485 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.485 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.485 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.485 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.486 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.486 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.486 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.486 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.486 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.486 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.486 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.486 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.486 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.486 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.486 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.486 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.487 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.487 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.487 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.487 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.487 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.487 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.487 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.487 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.487 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.487 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.487 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.487 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.487 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.487 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.488 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.488 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.488 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.488 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.488 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.488 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.488 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.488 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.488 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.488 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.488 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.488 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.488 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.488 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.488 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.489 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.489 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.489 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.489 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.489 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.489 INFO:teuthology.orchestra.run.smithi067.stdout: "roption1": { 2024-07-27T03:19:09.489 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption1", 2024-07-27T03:19:09.489 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.489 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.489 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.489 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.489 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.489 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.490 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.490 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.490 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.490 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.490 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.490 INFO:teuthology.orchestra.run.smithi067.stdout: "roption2": { 2024-07-27T03:19:09.490 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "roption2", 2024-07-27T03:19:09.490 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.490 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.490 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.490 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "xyz", 2024-07-27T03:19:09.490 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.490 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.491 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.491 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.491 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.491 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.491 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.491 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption1": { 2024-07-27T03:19:09.491 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption1", 2024-07-27T03:19:09.491 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.491 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.491 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.491 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.491 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.491 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.491 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.492 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.492 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.492 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.492 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.492 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption2": { 2024-07-27T03:19:09.492 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption2", 2024-07-27T03:19:09.492 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.492 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.492 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.492 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.492 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.492 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.493 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.493 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.493 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.493 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.493 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.493 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.493 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption3": { 2024-07-27T03:19:09.493 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption3", 2024-07-27T03:19:09.493 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "float", 2024-07-27T03:19:09.493 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.493 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.493 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.493 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.493 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.493 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.494 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.494 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.494 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.494 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.494 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.494 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption4": { 2024-07-27T03:19:09.494 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption4", 2024-07-27T03:19:09.494 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.494 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.494 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.494 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.494 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.494 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.495 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.495 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.495 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.495 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.495 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.495 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption5": { 2024-07-27T03:19:09.495 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption5", 2024-07-27T03:19:09.495 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.495 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.495 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.495 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.495 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.495 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.495 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.496 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.496 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.496 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.496 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.496 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption6": { 2024-07-27T03:19:09.496 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption6", 2024-07-27T03:19:09.496 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.496 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.496 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.496 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.496 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.496 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.496 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.497 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.497 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.497 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.497 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.497 INFO:teuthology.orchestra.run.smithi067.stdout: "rwoption7": { 2024-07-27T03:19:09.497 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rwoption7", 2024-07-27T03:19:09.497 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.497 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.497 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.497 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.497 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "1", 2024-07-27T03:19:09.497 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "42", 2024-07-27T03:19:09.498 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.498 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.498 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.498 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.498 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.498 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.498 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.498 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.498 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.498 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.498 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.498 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.498 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.499 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.499 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.499 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.499 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.499 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.499 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.499 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.499 INFO:teuthology.orchestra.run.smithi067.stdout: "testkey": { 2024-07-27T03:19:09.499 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testkey", 2024-07-27T03:19:09.499 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.499 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.499 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.499 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.500 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.500 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.500 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.500 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.500 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.500 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.500 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.500 INFO:teuthology.orchestra.run.smithi067.stdout: "testlkey": { 2024-07-27T03:19:09.500 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testlkey", 2024-07-27T03:19:09.500 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.500 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.500 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.500 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.500 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.501 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.501 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.501 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.501 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.501 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.501 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.501 INFO:teuthology.orchestra.run.smithi067.stdout: "testnewline": { 2024-07-27T03:19:09.501 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "testnewline", 2024-07-27T03:19:09.501 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.501 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.501 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.501 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.502 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.502 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.502 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.502 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.502 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.502 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.502 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.502 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.502 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.502 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.502 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "smb", 2024-07-27T03:19:09.502 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.503 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.503 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.503 INFO:teuthology.orchestra.run.smithi067.stdout: "internal_store_backend": { 2024-07-27T03:19:09.503 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "internal_store_backend", 2024-07-27T03:19:09.503 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.503 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.503 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.503 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.503 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.503 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.503 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.503 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-07-27T03:19:09.503 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.503 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.504 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.504 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.504 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.504 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.504 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.504 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.504 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.504 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.504 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.504 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.504 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.504 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.504 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.504 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.504 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.505 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.505 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.505 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.505 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.505 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.505 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.505 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.505 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.505 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.505 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.505 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.505 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.505 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.505 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.506 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.506 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.506 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.506 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.506 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.506 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.506 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.506 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.506 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.506 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.506 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.506 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.506 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.507 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.507 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.507 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.507 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.507 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.507 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.507 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.507 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.507 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.507 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.507 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.507 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.507 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.507 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.507 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.508 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.508 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.508 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.508 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.508 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.508 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.508 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.508 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.508 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.508 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.508 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.508 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.508 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.509 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.509 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.509 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.509 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.509 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.509 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.509 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.509 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.509 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.509 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.509 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.509 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.509 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.509 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.510 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.510 INFO:teuthology.orchestra.run.smithi067.stdout: "update_orchestration": { 2024-07-27T03:19:09.510 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "update_orchestration", 2024-07-27T03:19:09.510 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.510 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.510 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.510 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.510 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.510 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.510 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.510 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-07-27T03:19:09.510 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.510 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.510 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.511 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.511 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.511 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.511 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.511 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snap_schedule", 2024-07-27T03:19:09.511 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.511 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.511 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.511 INFO:teuthology.orchestra.run.smithi067.stdout: "allow_m_granularity": { 2024-07-27T03:19:09.511 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "allow_m_granularity", 2024-07-27T03:19:09.511 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.511 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.511 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.511 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.512 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.512 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.512 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.512 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "allow minute scheduled snapshots", 2024-07-27T03:19:09.512 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.512 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.512 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.512 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.512 INFO:teuthology.orchestra.run.smithi067.stdout: "dump_on_update": { 2024-07-27T03:19:09.512 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "dump_on_update", 2024-07-27T03:19:09.512 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.512 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.512 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.512 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.512 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.513 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.513 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.513 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "dump database to debug log on update", 2024-07-27T03:19:09.513 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.513 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.513 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.513 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.513 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.513 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.513 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.513 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.513 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.513 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.514 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.514 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.514 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.514 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.514 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.514 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.514 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.514 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.514 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.514 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.514 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.514 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.514 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.515 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.515 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.515 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.515 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.515 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.515 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.515 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.515 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.515 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.515 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.515 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.515 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.516 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.516 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.516 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.516 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.516 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.516 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.516 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.516 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.516 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.516 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.516 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.516 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.516 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.516 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.517 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.517 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.517 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.517 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.517 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.517 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.517 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.517 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.517 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.517 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.517 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.517 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.517 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.517 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.518 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.518 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.518 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.518 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.518 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.518 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.518 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.518 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.518 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.518 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.518 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.518 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.518 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.519 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.519 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.519 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.519 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.519 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.519 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.519 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.519 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.519 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.519 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.519 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.519 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.519 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.520 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.520 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "stats", 2024-07-27T03:19:09.520 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.520 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.520 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.520 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.520 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.520 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.520 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.520 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.520 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.520 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.520 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.520 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.521 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.521 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.521 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.521 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.521 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.521 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.521 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.521 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.521 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.521 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.521 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.521 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.521 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.522 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.522 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.522 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.522 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.522 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.522 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.522 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.522 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.522 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.522 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.522 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.522 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.522 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.522 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.523 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.523 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.523 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.523 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.523 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.523 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.523 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.523 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.523 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.523 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.523 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.523 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.523 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.523 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.523 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.524 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.524 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.524 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.524 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.524 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.524 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.524 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.524 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.524 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.524 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.524 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.524 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.525 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.525 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.525 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.525 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.525 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.525 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.525 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.525 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.525 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.525 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.525 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.525 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.525 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.525 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.526 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.526 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.526 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.526 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.526 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.526 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.526 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.526 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.526 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "status", 2024-07-27T03:19:09.526 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.526 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.526 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.526 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.526 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.527 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.527 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.527 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.527 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.527 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.527 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.527 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.527 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.527 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.527 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.527 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.527 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.527 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.527 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.528 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.528 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.528 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.528 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.528 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.528 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.528 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.528 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.528 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.528 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.528 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.528 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.529 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.529 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.529 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.529 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.529 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.529 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.529 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.529 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.529 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.529 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.529 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.529 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.529 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.530 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.530 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.530 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.530 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.530 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.530 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.530 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.530 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.530 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.530 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.530 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.530 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.530 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.530 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.531 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.531 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.531 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.531 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.531 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.531 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.531 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.531 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.531 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.531 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.531 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.531 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.531 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.531 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.531 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.532 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.532 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.532 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.532 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.532 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.532 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.532 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.532 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.532 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.532 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.532 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.532 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.532 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.532 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.533 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.533 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telegraf", 2024-07-27T03:19:09.533 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.533 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.533 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.533 INFO:teuthology.orchestra.run.smithi067.stdout: "address": { 2024-07-27T03:19:09.533 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "address", 2024-07-27T03:19:09.533 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.533 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.533 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.533 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-07-27T03:19:09.533 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.533 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.533 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.533 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.534 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.534 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.534 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.534 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.534 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-27T03:19:09.534 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-27T03:19:09.534 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:09.534 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.534 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.534 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "15", 2024-07-27T03:19:09.534 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.534 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.534 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.535 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.535 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.535 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.535 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.535 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.535 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.535 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.535 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.535 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.535 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.535 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.535 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.535 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.535 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.536 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.536 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.536 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.536 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.536 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.536 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.536 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.536 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.536 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.536 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.536 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.536 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.537 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.537 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.537 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.537 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.537 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.537 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.537 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.537 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.537 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.537 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.537 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.537 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.537 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.538 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.538 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.538 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.538 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.538 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.538 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.538 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.538 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.538 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.538 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.538 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.538 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.538 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.538 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.539 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.539 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.539 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.539 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.539 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.539 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.539 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.539 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.539 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.539 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.539 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.539 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.539 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.539 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.539 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.539 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.540 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.540 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.540 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.540 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.540 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.540 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.540 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.540 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.540 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.540 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.540 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.540 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.540 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.540 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.540 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.541 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.541 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.541 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.541 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.541 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "telemetry", 2024-07-27T03:19:09.541 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.541 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.541 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.541 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_basic": { 2024-07-27T03:19:09.541 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_basic", 2024-07-27T03:19:09.541 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.541 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.541 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.541 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.541 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.541 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.542 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.542 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share basic cluster information (size, version)", 2024-07-27T03:19:09.542 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.542 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.542 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.542 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_crash": { 2024-07-27T03:19:09.542 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_crash", 2024-07-27T03:19:09.542 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.542 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.542 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.542 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.542 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.542 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.543 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.543 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-07-27T03:19:09.543 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.543 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.543 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.543 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_device": { 2024-07-27T03:19:09.543 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_device", 2024-07-27T03:19:09.543 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.543 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.543 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.543 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.543 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.543 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.543 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.544 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-07-27T03:19:09.544 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.544 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.544 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.544 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.544 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_ident": { 2024-07-27T03:19:09.544 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_ident", 2024-07-27T03:19:09.544 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.544 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.544 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.544 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.544 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.544 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.544 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.545 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-07-27T03:19:09.545 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.545 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.545 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.545 INFO:teuthology.orchestra.run.smithi067.stdout: "channel_perf": { 2024-07-27T03:19:09.545 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "channel_perf", 2024-07-27T03:19:09.545 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.545 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.545 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.545 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.545 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.545 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.545 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.545 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Share various performance metrics of a cluster", 2024-07-27T03:19:09.546 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.546 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.546 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.546 INFO:teuthology.orchestra.run.smithi067.stdout: "contact": { 2024-07-27T03:19:09.546 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "contact", 2024-07-27T03:19:09.546 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.546 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.546 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.546 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.546 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.546 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.546 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.547 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.547 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.547 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.547 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.547 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.547 INFO:teuthology.orchestra.run.smithi067.stdout: "description": { 2024-07-27T03:19:09.547 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "description", 2024-07-27T03:19:09.547 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.547 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.547 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.547 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.547 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.547 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.547 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.548 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.548 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.548 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.548 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.548 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.548 INFO:teuthology.orchestra.run.smithi067.stdout: "device_url": { 2024-07-27T03:19:09.548 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "device_url", 2024-07-27T03:19:09.548 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.548 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.548 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.548 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-07-27T03:19:09.548 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.548 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.548 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.549 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.549 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.549 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.549 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.549 INFO:teuthology.orchestra.run.smithi067.stdout: "enabled": { 2024-07-27T03:19:09.549 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "enabled", 2024-07-27T03:19:09.549 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.549 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.549 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.549 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.549 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.549 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.549 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.550 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.550 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.550 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.550 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.550 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.550 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-27T03:19:09.550 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-27T03:19:09.550 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.550 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.550 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.550 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "24", 2024-07-27T03:19:09.550 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "8", 2024-07-27T03:19:09.550 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.550 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.551 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.551 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.551 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.551 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.551 INFO:teuthology.orchestra.run.smithi067.stdout: "last_opt_revision": { 2024-07-27T03:19:09.551 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "last_opt_revision", 2024-07-27T03:19:09.551 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.551 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.551 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.551 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "1", 2024-07-27T03:19:09.551 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.551 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.552 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.552 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.552 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.552 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.552 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.552 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.552 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard": { 2024-07-27T03:19:09.552 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard", 2024-07-27T03:19:09.552 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.552 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.552 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.552 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.552 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.552 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.552 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.553 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.553 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.553 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.553 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.553 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.553 INFO:teuthology.orchestra.run.smithi067.stdout: "leaderboard_description": { 2024-07-27T03:19:09.553 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "leaderboard_description", 2024-07-27T03:19:09.553 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.553 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.553 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.553 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.553 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.553 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.553 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.554 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.554 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.554 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.554 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.554 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.554 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.554 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.554 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.554 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.554 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.554 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.554 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.554 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.555 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.555 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.555 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.555 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.555 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.555 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.555 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.555 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.555 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.555 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.555 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.555 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.555 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.555 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.556 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.556 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.556 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.556 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.556 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.556 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.556 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.556 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.556 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.556 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.556 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.556 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.556 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.557 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.557 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.557 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.557 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.557 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.557 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.557 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.557 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.557 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.557 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.557 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.557 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.557 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.557 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.558 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.558 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.558 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.558 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.558 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.558 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.558 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.558 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.558 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.558 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.558 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.558 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.558 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.559 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.559 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.559 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.559 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.559 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.559 INFO:teuthology.orchestra.run.smithi067.stdout: "organization": { 2024-07-27T03:19:09.559 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "organization", 2024-07-27T03:19:09.559 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.559 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.559 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.559 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.559 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.559 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.559 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.560 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.560 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.560 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.560 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.560 INFO:teuthology.orchestra.run.smithi067.stdout: "proxy": { 2024-07-27T03:19:09.560 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "proxy", 2024-07-27T03:19:09.560 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.560 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.560 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.560 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.560 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.560 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.560 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.561 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.561 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.561 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.561 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.561 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.561 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.561 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.561 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.561 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.561 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.561 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.561 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.561 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.561 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.562 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.562 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.562 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.562 INFO:teuthology.orchestra.run.smithi067.stdout: "url": { 2024-07-27T03:19:09.562 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "url", 2024-07-27T03:19:09.562 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.562 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.562 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.562 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-07-27T03:19:09.562 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.562 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.562 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.563 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.563 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.563 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.563 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.563 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.563 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.563 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.563 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "test_orchestrator", 2024-07-27T03:19:09.563 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.563 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.563 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.563 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.563 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.564 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.564 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.564 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.564 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.564 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.564 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.564 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.564 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.564 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.564 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.564 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.564 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.564 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.564 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.565 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.565 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.565 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.565 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.565 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.565 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.565 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.565 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.565 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.565 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.565 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.565 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.565 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.565 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.566 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.566 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.566 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.566 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.566 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.566 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.566 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.566 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.566 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.566 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.566 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.566 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.567 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.567 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.567 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.567 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.567 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.567 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.567 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.567 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.567 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.567 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.567 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.567 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.567 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.568 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.568 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.568 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.568 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.568 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.568 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.568 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.568 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.568 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.568 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.568 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.568 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.568 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.569 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.569 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.569 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.569 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.569 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.569 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.569 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.569 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.569 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.569 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.569 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.569 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.569 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.569 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.570 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "volumes", 2024-07-27T03:19:09.570 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.570 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.570 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.570 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.570 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.570 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.570 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.570 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.570 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.570 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.570 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.570 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.570 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.571 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.571 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.571 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.571 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.571 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.571 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.571 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.571 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.571 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.571 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.571 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.571 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.571 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.571 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.571 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.572 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.572 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.572 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.572 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.572 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.572 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.572 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.572 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.572 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.572 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.572 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.572 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.573 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.573 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.573 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.573 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.573 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.573 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.573 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.573 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.573 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.573 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.573 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.573 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.573 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.573 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.573 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.574 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.574 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.574 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.574 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.574 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.574 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.574 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.574 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.574 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.574 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.574 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.574 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.574 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.574 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.575 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.575 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.575 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.575 INFO:teuthology.orchestra.run.smithi067.stdout: "max_concurrent_clones": { 2024-07-27T03:19:09.575 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "max_concurrent_clones", 2024-07-27T03:19:09.575 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.575 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.575 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.575 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "4", 2024-07-27T03:19:09.575 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.575 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.575 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.575 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Number of asynchronous cloner threads", 2024-07-27T03:19:09.575 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.575 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.575 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.576 INFO:teuthology.orchestra.run.smithi067.stdout: "periodic_async_work": { 2024-07-27T03:19:09.576 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "periodic_async_work", 2024-07-27T03:19:09.576 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.576 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.576 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.576 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.576 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.576 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.576 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.576 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Periodically check for async work", 2024-07-27T03:19:09.576 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.576 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.576 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.577 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_delay": { 2024-07-27T03:19:09.577 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_delay", 2024-07-27T03:19:09.577 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.577 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.577 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.577 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.577 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.577 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.577 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.577 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-07-27T03:19:09.577 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.577 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.577 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.577 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.578 INFO:teuthology.orchestra.run.smithi067.stdout: "snapshot_clone_no_wait": { 2024-07-27T03:19:09.578 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "snapshot_clone_no_wait", 2024-07-27T03:19:09.578 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.578 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.578 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.578 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "True", 2024-07-27T03:19:09.578 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.578 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.578 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.578 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-07-27T03:19:09.578 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.578 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.578 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.579 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.579 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.579 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.579 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.579 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.579 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.579 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.579 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.579 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.579 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.579 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.579 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.579 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.579 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.580 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.580 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.580 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.580 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix", 2024-07-27T03:19:09.580 INFO:teuthology.orchestra.run.smithi067.stdout: "can_run": true, 2024-07-27T03:19:09.580 INFO:teuthology.orchestra.run.smithi067.stdout: "error_string": "", 2024-07-27T03:19:09.580 INFO:teuthology.orchestra.run.smithi067.stdout: "module_options": { 2024-07-27T03:19:09.580 INFO:teuthology.orchestra.run.smithi067.stdout: "discovery_interval": { 2024-07-27T03:19:09.580 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "discovery_interval", 2024-07-27T03:19:09.580 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "uint", 2024-07-27T03:19:09.580 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.580 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.580 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "100", 2024-07-27T03:19:09.580 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.581 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.581 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.581 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.581 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.581 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.581 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.581 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.581 INFO:teuthology.orchestra.run.smithi067.stdout: "identifier": { 2024-07-27T03:19:09.581 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "identifier", 2024-07-27T03:19:09.581 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.581 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.581 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.581 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.581 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.582 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.582 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.582 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.582 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.582 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.582 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.582 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.582 INFO:teuthology.orchestra.run.smithi067.stdout: "interval": { 2024-07-27T03:19:09.582 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "interval", 2024-07-27T03:19:09.582 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "secs", 2024-07-27T03:19:09.582 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.582 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.582 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "60", 2024-07-27T03:19:09.582 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.583 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.583 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.583 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.583 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.583 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.583 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.583 INFO:teuthology.orchestra.run.smithi067.stdout: "log_level": { 2024-07-27T03:19:09.583 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_level", 2024-07-27T03:19:09.583 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.583 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.583 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.583 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.583 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.584 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.584 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.584 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.584 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.584 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.584 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.584 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.584 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.584 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.584 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.584 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.584 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.584 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.584 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.585 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster": { 2024-07-27T03:19:09.585 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster", 2024-07-27T03:19:09.585 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.585 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.585 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.585 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.585 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.585 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.585 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.585 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.585 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.585 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.585 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.585 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_cluster_level": { 2024-07-27T03:19:09.586 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_cluster_level", 2024-07-27T03:19:09.586 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.586 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.586 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.586 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "info", 2024-07-27T03:19:09.586 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.586 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.586 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [ 2024-07-27T03:19:09.586 INFO:teuthology.orchestra.run.smithi067.stdout: "", 2024-07-27T03:19:09.586 INFO:teuthology.orchestra.run.smithi067.stdout: "critical", 2024-07-27T03:19:09.586 INFO:teuthology.orchestra.run.smithi067.stdout: "debug", 2024-07-27T03:19:09.586 INFO:teuthology.orchestra.run.smithi067.stdout: "error", 2024-07-27T03:19:09.586 INFO:teuthology.orchestra.run.smithi067.stdout: "info", 2024-07-27T03:19:09.586 INFO:teuthology.orchestra.run.smithi067.stdout: "warning" 2024-07-27T03:19:09.587 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.587 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.587 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.587 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.587 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.587 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.587 INFO:teuthology.orchestra.run.smithi067.stdout: "log_to_file": { 2024-07-27T03:19:09.587 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "log_to_file", 2024-07-27T03:19:09.587 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "bool", 2024-07-27T03:19:09.587 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.587 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.587 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "False", 2024-07-27T03:19:09.587 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.587 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.587 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.588 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.588 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.588 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.588 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.588 INFO:teuthology.orchestra.run.smithi067.stdout: "sqlite3_killpoint": { 2024-07-27T03:19:09.588 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "sqlite3_killpoint", 2024-07-27T03:19:09.588 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.588 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "dev", 2024-07-27T03:19:09.588 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 1, 2024-07-27T03:19:09.588 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "0", 2024-07-27T03:19:09.588 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.588 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.588 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.588 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.588 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.589 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.589 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.589 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_host": { 2024-07-27T03:19:09.589 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_host", 2024-07-27T03:19:09.589 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.589 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.589 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.589 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "", 2024-07-27T03:19:09.589 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.589 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.589 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.589 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.589 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.589 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.589 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.590 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_port": { 2024-07-27T03:19:09.590 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_port", 2024-07-27T03:19:09.590 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "int", 2024-07-27T03:19:09.590 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.590 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.590 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "10051", 2024-07-27T03:19:09.590 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.590 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.590 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.590 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.590 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.590 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.590 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.591 INFO:teuthology.orchestra.run.smithi067.stdout: "zabbix_sender": { 2024-07-27T03:19:09.591 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "zabbix_sender", 2024-07-27T03:19:09.591 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "str", 2024-07-27T03:19:09.591 INFO:teuthology.orchestra.run.smithi067.stdout: "level": "advanced", 2024-07-27T03:19:09.591 INFO:teuthology.orchestra.run.smithi067.stdout: "flags": 0, 2024-07-27T03:19:09.591 INFO:teuthology.orchestra.run.smithi067.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-07-27T03:19:09.591 INFO:teuthology.orchestra.run.smithi067.stdout: "min": "", 2024-07-27T03:19:09.591 INFO:teuthology.orchestra.run.smithi067.stdout: "max": "", 2024-07-27T03:19:09.591 INFO:teuthology.orchestra.run.smithi067.stdout: "enum_allowed": [], 2024-07-27T03:19:09.591 INFO:teuthology.orchestra.run.smithi067.stdout: "desc": "", 2024-07-27T03:19:09.591 INFO:teuthology.orchestra.run.smithi067.stdout: "long_desc": "", 2024-07-27T03:19:09.591 INFO:teuthology.orchestra.run.smithi067.stdout: "tags": [], 2024-07-27T03:19:09.591 INFO:teuthology.orchestra.run.smithi067.stdout: "see_also": [] 2024-07-27T03:19:09.591 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.591 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.592 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.592 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.592 INFO:teuthology.orchestra.run.smithi067.stdout: "services": { 2024-07-27T03:19:09.592 INFO:teuthology.orchestra.run.smithi067.stdout: "dashboard": "https://172.21.15.67:8443/" 2024-07-27T03:19:09.592 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.592 INFO:teuthology.orchestra.run.smithi067.stdout: "always_on_modules": { 2024-07-27T03:19:09.592 INFO:teuthology.orchestra.run.smithi067.stdout: "octopus": [ 2024-07-27T03:19:09.592 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-07-27T03:19:09.592 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-07-27T03:19:09.592 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-07-27T03:19:09.592 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-07-27T03:19:09.592 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-07-27T03:19:09.592 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-07-27T03:19:09.592 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-07-27T03:19:09.592 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-07-27T03:19:09.593 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-07-27T03:19:09.593 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-07-27T03:19:09.593 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.593 INFO:teuthology.orchestra.run.smithi067.stdout: "pacific": [ 2024-07-27T03:19:09.593 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-07-27T03:19:09.593 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-07-27T03:19:09.593 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-07-27T03:19:09.593 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-07-27T03:19:09.593 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-07-27T03:19:09.593 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-07-27T03:19:09.593 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-07-27T03:19:09.593 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-07-27T03:19:09.593 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-07-27T03:19:09.593 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-07-27T03:19:09.593 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.594 INFO:teuthology.orchestra.run.smithi067.stdout: "quincy": [ 2024-07-27T03:19:09.594 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-07-27T03:19:09.594 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-07-27T03:19:09.594 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-07-27T03:19:09.594 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-07-27T03:19:09.594 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-07-27T03:19:09.594 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-07-27T03:19:09.594 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-07-27T03:19:09.594 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-07-27T03:19:09.594 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-07-27T03:19:09.594 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-07-27T03:19:09.594 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.594 INFO:teuthology.orchestra.run.smithi067.stdout: "reef": [ 2024-07-27T03:19:09.594 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-07-27T03:19:09.595 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-07-27T03:19:09.595 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-07-27T03:19:09.595 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-07-27T03:19:09.595 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-07-27T03:19:09.595 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-07-27T03:19:09.595 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-07-27T03:19:09.595 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-07-27T03:19:09.595 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-07-27T03:19:09.595 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-07-27T03:19:09.595 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-07-27T03:19:09.595 INFO:teuthology.orchestra.run.smithi067.stdout: "squid": [ 2024-07-27T03:19:09.595 INFO:teuthology.orchestra.run.smithi067.stdout: "balancer", 2024-07-27T03:19:09.595 INFO:teuthology.orchestra.run.smithi067.stdout: "crash", 2024-07-27T03:19:09.595 INFO:teuthology.orchestra.run.smithi067.stdout: "devicehealth", 2024-07-27T03:19:09.595 INFO:teuthology.orchestra.run.smithi067.stdout: "orchestrator", 2024-07-27T03:19:09.596 INFO:teuthology.orchestra.run.smithi067.stdout: "pg_autoscaler", 2024-07-27T03:19:09.596 INFO:teuthology.orchestra.run.smithi067.stdout: "progress", 2024-07-27T03:19:09.596 INFO:teuthology.orchestra.run.smithi067.stdout: "rbd_support", 2024-07-27T03:19:09.596 INFO:teuthology.orchestra.run.smithi067.stdout: "status", 2024-07-27T03:19:09.596 INFO:teuthology.orchestra.run.smithi067.stdout: "telemetry", 2024-07-27T03:19:09.596 INFO:teuthology.orchestra.run.smithi067.stdout: "volumes" 2024-07-27T03:19:09.596 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-07-27T03:19:09.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.596 INFO:teuthology.orchestra.run.smithi067.stdout: "last_failure_osd_epoch": 3, 2024-07-27T03:19:09.596 INFO:teuthology.orchestra.run.smithi067.stdout: "active_clients": [ 2024-07-27T03:19:09.596 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.596 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "devicehealth", 2024-07-27T03:19:09.596 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-07-27T03:19:09.596 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.596 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-07-27T03:19:09.597 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-07-27T03:19:09.597 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 3926355641 2024-07-27T03:19:09.597 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.597 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-07-27T03:19:09.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.597 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.597 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "libcephsqlite", 2024-07-27T03:19:09.597 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-07-27T03:19:09.597 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.597 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-07-27T03:19:09.597 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-07-27T03:19:09.597 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 138992728 2024-07-27T03:19:09.597 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.597 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-07-27T03:19:09.598 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.598 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.598 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "rbd_support", 2024-07-27T03:19:09.598 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-07-27T03:19:09.598 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.598 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-07-27T03:19:09.598 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-07-27T03:19:09.598 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 247463021 2024-07-27T03:19:09.598 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.598 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-07-27T03:19:09.598 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:09.598 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.598 INFO:teuthology.orchestra.run.smithi067.stdout: "name": "volumes", 2024-07-27T03:19:09.598 INFO:teuthology.orchestra.run.smithi067.stdout: "addrvec": [ 2024-07-27T03:19:09.599 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-07-27T03:19:09.599 INFO:teuthology.orchestra.run.smithi067.stdout: "type": "v2", 2024-07-27T03:19:09.599 INFO:teuthology.orchestra.run.smithi067.stdout: "addr": "172.21.15.67:0", 2024-07-27T03:19:09.599 INFO:teuthology.orchestra.run.smithi067.stdout: "nonce": 1722417064 2024-07-27T03:19:09.599 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.599 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-07-27T03:19:09.599 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:09.599 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-07-27T03:19:09.599 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:19:09.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:08.098+0000 7f83c1b57640 1 -- 172.21.15.67:0/240751111 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f83a80b3bd0 msgr2=0x7f83a80b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:09.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:08.098+0000 7f83c1b57640 1 --2- 172.21.15.67:0/240751111 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f83a80b3bd0 0x7f83a80b60e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f83ac031440 tx=0x7f83ac0314a0 comp rx=0 tx=0).stop 2024-07-27T03:19:09.600 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:08.098+0000 7f83c1b57640 1 -- 172.21.15.67:0/240751111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83bc106260 msgr2=0x7f83bc19a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:09.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:08.098+0000 7f83c1b57640 1 --2- 172.21.15.67:0/240751111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83bc106260 0x7f83bc19a010 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f83a400d770 tx=0x7f83a400dc30 comp rx=0 tx=0).stop 2024-07-27T03:19:09.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:08.098+0000 7f83c1b57640 1 -- 172.21.15.67:0/240751111 shutdown_connections 2024-07-27T03:19:09.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:08.098+0000 7f83c1b57640 1 --2- 172.21.15.67:0/240751111 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f83a80b3bd0 0x7f83a80b60e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:09.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:08.098+0000 7f83c1b57640 1 --2- 172.21.15.67:0/240751111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f83bc108e20 0x7f83bc19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:09.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:08.098+0000 7f83c1b57640 1 --2- 172.21.15.67:0/240751111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83bc106260 0x7f83bc19a010 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:09.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:08.098+0000 7f83c1b57640 1 --2- 172.21.15.67:0/240751111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83bc06acc0 0x7f83bc1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:09.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:08.098+0000 7f83c1b57640 1 -- 172.21.15.67:0/240751111 >> 172.21.15.67:0/240751111 conn(0x7f83bc0fd920 msgr2=0x7f83bc1046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:09.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:08.098+0000 7f83c1b57640 1 -- 172.21.15.67:0/240751111 shutdown_connections 2024-07-27T03:19:09.601 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:08.098+0000 7f83c1b57640 1 -- 172.21.15.67:0/240751111 wait complete. 2024-07-27T03:19:09.659 DEBUG:tasks.check_counter:Getting stats from a 2024-07-27T03:19:09.659 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph daemon mds.a perf dump 2024-07-27T03:19:09.660 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:08 smithi067 ceph-mon[32814]: pgmap v507: 129 pgs: 129 active+clean; 257 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-07-27T03:19:09.660 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:08 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2437582541' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-07-27T03:19:09.660 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:08 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/240751111' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-07-27T03:19:09.661 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:08 smithi149 ceph-mon[37173]: pgmap v507: 129 pgs: 129 active+clean; 257 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-07-27T03:19:09.661 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:08 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2437582541' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-07-27T03:19:09.661 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:08 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/240751111' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-07-27T03:19:09.661 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:07 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/927115621' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-07-27T03:19:09.661 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:07 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1406994793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-07-27T03:19:09.661 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:07 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3516674236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-07-27T03:19:09.661 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:08 smithi175 ceph-mon[37125]: pgmap v507: 129 pgs: 129 active+clean; 257 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 14 MiB/s wr, 15 op/s 2024-07-27T03:19:09.661 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:08 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2437582541' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-07-27T03:19:09.661 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:08 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/240751111' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-07-27T03:19:09.937 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:19:10.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:10 smithi067 ceph-mon[32814]: pgmap v508: 129 pgs: 129 active+clean; 257 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-07-27T03:19:10.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:10 smithi149 ceph-mon[37173]: pgmap v508: 129 pgs: 129 active+clean; 257 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-07-27T03:19:11.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:10 smithi175 ceph-mon[37125]: pgmap v508: 129 pgs: 129 active+clean; 257 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-07-27T03:19:11.681 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:19:11.681 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-0": { 2024-07-27T03:19:11.681 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 456, 2024-07-27T03:19:11.681 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 451, 2024-07-27T03:19:11.681 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 630464, 2024-07-27T03:19:11.682 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 175450, 2024-07-27T03:19:11.682 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 5, 2024-07-27T03:19:11.682 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 1, 2024-07-27T03:19:11.682 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.060092897, 2024-07-27T03:19:11.682 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.027448190, 2024-07-27T03:19:11.682 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.022998533, 2024-07-27T03:19:11.682 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.010418752, 2024-07-27T03:19:11.682 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:11.682 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 450, 2024-07-27T03:19:11.682 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.018194718, 2024-07-27T03:19:11.682 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000040432 2024-07-27T03:19:11.682 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.683 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:11.683 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.683 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.683 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.683 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.683 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 519248, 2024-07-27T03:19:11.683 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 70192 2024-07-27T03:19:11.683 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.683 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-1": { 2024-07-27T03:19:11.683 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 4, 2024-07-27T03:19:11.683 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 89, 2024-07-27T03:19:11.683 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 1550, 2024-07-27T03:19:11.683 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 211518, 2024-07-27T03:19:11.684 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 5, 2024-07-27T03:19:11.684 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 4, 2024-07-27T03:19:11.684 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.009666787, 2024-07-27T03:19:11.684 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.006736195, 2024-07-27T03:19:11.684 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.002067888, 2024-07-27T03:19:11.684 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000066060, 2024-07-27T03:19:11.684 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:11.684 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 87, 2024-07-27T03:19:11.684 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.006212576, 2024-07-27T03:19:11.684 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000071408 2024-07-27T03:19:11.684 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.684 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:11.684 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.685 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.685 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.685 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.685 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 160, 2024-07-27T03:19:11.685 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 210496 2024-07-27T03:19:11.685 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.685 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-2": { 2024-07-27T03:19:11.685 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 338, 2024-07-27T03:19:11.686 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 338, 2024-07-27T03:19:11.686 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 91097, 2024-07-27T03:19:11.686 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 106661, 2024-07-27T03:19:11.686 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 4, 2024-07-27T03:19:11.686 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 3, 2024-07-27T03:19:11.686 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.031245479, 2024-07-27T03:19:11.686 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.012631496, 2024-07-27T03:19:11.686 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.008922067, 2024-07-27T03:19:11.686 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.007298108, 2024-07-27T03:19:11.686 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:11.686 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 337, 2024-07-27T03:19:11.686 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.013339781, 2024-07-27T03:19:11.687 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000039583 2024-07-27T03:19:11.687 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.687 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:11.687 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.687 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.687 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.687 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.687 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-07-27T03:19:11.687 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 448 2024-07-27T03:19:11.687 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.687 INFO:teuthology.orchestra.run.smithi067.stdout: "cct": { 2024-07-27T03:19:11.687 INFO:teuthology.orchestra.run.smithi067.stdout: "total_workers": 1, 2024-07-27T03:19:11.687 INFO:teuthology.orchestra.run.smithi067.stdout: "unhealthy_workers": 0 2024-07-27T03:19:11.687 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.688 INFO:teuthology.orchestra.run.smithi067.stdout: "finisher-MDSRank": { 2024-07-27T03:19:11.688 INFO:teuthology.orchestra.run.smithi067.stdout: "queue_len": 0, 2024-07-27T03:19:11.688 INFO:teuthology.orchestra.run.smithi067.stdout: "complete_latency": { 2024-07-27T03:19:11.688 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 1329, 2024-07-27T03:19:11.688 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.033209695, 2024-07-27T03:19:11.688 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000024988 2024-07-27T03:19:11.688 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:11.688 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.688 INFO:teuthology.orchestra.run.smithi067.stdout: "finisher-PurgeQueue": { 2024-07-27T03:19:11.688 INFO:teuthology.orchestra.run.smithi067.stdout: "queue_len": 0, 2024-07-27T03:19:11.688 INFO:teuthology.orchestra.run.smithi067.stdout: "complete_latency": { 2024-07-27T03:19:11.688 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.688 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.688 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.689 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:11.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.689 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": { 2024-07-27T03:19:11.689 INFO:teuthology.orchestra.run.smithi067.stdout: "request": 0, 2024-07-27T03:19:11.689 INFO:teuthology.orchestra.run.smithi067.stdout: "reply": 0, 2024-07-27T03:19:11.689 INFO:teuthology.orchestra.run.smithi067.stdout: "reply_latency": { 2024-07-27T03:19:11.689 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.689 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.689 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.689 INFO:teuthology.orchestra.run.smithi067.stdout: "slow_reply": 0, 2024-07-27T03:19:11.689 INFO:teuthology.orchestra.run.smithi067.stdout: "forward": 0, 2024-07-27T03:19:11.689 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_fetch_complete": 0, 2024-07-27T03:19:11.690 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_fetch_keys": 0, 2024-07-27T03:19:11.690 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_commit": 0, 2024-07-27T03:19:11.690 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_split": 0, 2024-07-27T03:19:11.690 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_merge": 0, 2024-07-27T03:19:11.690 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes": 0, 2024-07-27T03:19:11.690 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_top": 0, 2024-07-27T03:19:11.690 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_bottom": 0, 2024-07-27T03:19:11.690 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_pin_tail": 0, 2024-07-27T03:19:11.690 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_pinned": 0, 2024-07-27T03:19:11.690 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_expired": 0, 2024-07-27T03:19:11.690 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_with_caps": 0, 2024-07-27T03:19:11.690 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": 0, 2024-07-27T03:19:11.690 INFO:teuthology.orchestra.run.smithi067.stdout: "subtrees": 1, 2024-07-27T03:19:11.690 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse": 0, 2024-07-27T03:19:11.691 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_hit": 0, 2024-07-27T03:19:11.691 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_forward": 0, 2024-07-27T03:19:11.691 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_discover": 0, 2024-07-27T03:19:11.691 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_dir_fetch": 0, 2024-07-27T03:19:11.691 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_remote_ino": 0, 2024-07-27T03:19:11.691 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_lock": 0, 2024-07-27T03:19:11.691 INFO:teuthology.orchestra.run.smithi067.stdout: "load_cent": 0, 2024-07-27T03:19:11.691 INFO:teuthology.orchestra.run.smithi067.stdout: "q": 0, 2024-07-27T03:19:11.691 INFO:teuthology.orchestra.run.smithi067.stdout: "exported": 0, 2024-07-27T03:19:11.691 INFO:teuthology.orchestra.run.smithi067.stdout: "exported_inodes": 0, 2024-07-27T03:19:11.691 INFO:teuthology.orchestra.run.smithi067.stdout: "imported": 0, 2024-07-27T03:19:11.691 INFO:teuthology.orchestra.run.smithi067.stdout: "imported_inodes": 0, 2024-07-27T03:19:11.692 INFO:teuthology.orchestra.run.smithi067.stdout: "openino_dir_fetch": 0, 2024-07-27T03:19:11.692 INFO:teuthology.orchestra.run.smithi067.stdout: "openino_backtrace_fetch": 0, 2024-07-27T03:19:11.692 INFO:teuthology.orchestra.run.smithi067.stdout: "openino_peer_discover": 0, 2024-07-27T03:19:11.692 INFO:teuthology.orchestra.run.smithi067.stdout: "root_rfiles": 0, 2024-07-27T03:19:11.692 INFO:teuthology.orchestra.run.smithi067.stdout: "root_rbytes": 0, 2024-07-27T03:19:11.692 INFO:teuthology.orchestra.run.smithi067.stdout: "root_rsnaps": 0, 2024-07-27T03:19:11.692 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_backtrace_fetch": 0, 2024-07-27T03:19:11.692 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_set_tag": 0, 2024-07-27T03:19:11.692 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_backtrace_repaired": 0, 2024-07-27T03:19:11.692 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_inotable_repaired": 0, 2024-07-27T03:19:11.692 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_dir_inodes": 0, 2024-07-27T03:19:11.692 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_dir_base_inodes": 0, 2024-07-27T03:19:11.692 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_dirfrag_rstats": 0, 2024-07-27T03:19:11.692 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_file_inodes": 0, 2024-07-27T03:19:11.692 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_inode_file_caps": 0, 2024-07-27T03:19:11.693 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_revoke": 0, 2024-07-27T03:19:11.693 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_grant": 0, 2024-07-27T03:19:11.693 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_trunc": 0, 2024-07-27T03:19:11.693 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-27T03:19:11.693 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-27T03:19:11.693 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_caps": 0, 2024-07-27T03:19:11.693 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_caps_dirty": 0, 2024-07-27T03:19:11.693 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_cap_release": 0, 2024-07-27T03:19:11.693 INFO:teuthology.orchestra.run.smithi067.stdout: "process_request_cap_release": 0 2024-07-27T03:19:11.694 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.694 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cache": { 2024-07-27T03:19:11.694 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_update": 0, 2024-07-27T03:19:11.694 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_update_receipt": 0, 2024-07-27T03:19:11.694 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_try_discover": 0, 2024-07-27T03:19:11.694 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_send_discover": 0, 2024-07-27T03:19:11.694 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_handle_discover": 0, 2024-07-27T03:19:11.694 INFO:teuthology.orchestra.run.smithi067.stdout: "num_strays": 0, 2024-07-27T03:19:11.694 INFO:teuthology.orchestra.run.smithi067.stdout: "num_strays_delayed": 0, 2024-07-27T03:19:11.694 INFO:teuthology.orchestra.run.smithi067.stdout: "num_strays_enqueuing": 0, 2024-07-27T03:19:11.694 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_created": 0, 2024-07-27T03:19:11.694 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_enqueued": 0, 2024-07-27T03:19:11.694 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_reintegrated": 0, 2024-07-27T03:19:11.695 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_migrated": 0, 2024-07-27T03:19:11.695 INFO:teuthology.orchestra.run.smithi067.stdout: "num_recovering_processing": 0, 2024-07-27T03:19:11.695 INFO:teuthology.orchestra.run.smithi067.stdout: "num_recovering_enqueued": 0, 2024-07-27T03:19:11.695 INFO:teuthology.orchestra.run.smithi067.stdout: "num_recovering_prioritized": 0, 2024-07-27T03:19:11.695 INFO:teuthology.orchestra.run.smithi067.stdout: "recovery_started": 0, 2024-07-27T03:19:11.695 INFO:teuthology.orchestra.run.smithi067.stdout: "recovery_completed": 0, 2024-07-27T03:19:11.695 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_quiesce_path": 0, 2024-07-27T03:19:11.695 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_quiesce_inode": 0, 2024-07-27T03:19:11.695 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_enqueue_scrub": 0, 2024-07-27T03:19:11.695 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_exportdir": 0, 2024-07-27T03:19:11.696 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_flush": 0, 2024-07-27T03:19:11.696 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_fragmentdir": 0, 2024-07-27T03:19:11.696 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_fragstats": 0, 2024-07-27T03:19:11.696 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_inodestats": 0 2024-07-27T03:19:11.696 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.696 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log": { 2024-07-27T03:19:11.696 INFO:teuthology.orchestra.run.smithi067.stdout: "evlrg": 0, 2024-07-27T03:19:11.696 INFO:teuthology.orchestra.run.smithi067.stdout: "evadd": 0, 2024-07-27T03:19:11.696 INFO:teuthology.orchestra.run.smithi067.stdout: "evex": 0, 2024-07-27T03:19:11.696 INFO:teuthology.orchestra.run.smithi067.stdout: "evtrm": 0, 2024-07-27T03:19:11.696 INFO:teuthology.orchestra.run.smithi067.stdout: "ev": 2, 2024-07-27T03:19:11.696 INFO:teuthology.orchestra.run.smithi067.stdout: "evexg": 0, 2024-07-27T03:19:11.696 INFO:teuthology.orchestra.run.smithi067.stdout: "evexd": 0, 2024-07-27T03:19:11.697 INFO:teuthology.orchestra.run.smithi067.stdout: "segadd": 0, 2024-07-27T03:19:11.697 INFO:teuthology.orchestra.run.smithi067.stdout: "segex": 0, 2024-07-27T03:19:11.697 INFO:teuthology.orchestra.run.smithi067.stdout: "segtrm": 0, 2024-07-27T03:19:11.697 INFO:teuthology.orchestra.run.smithi067.stdout: "seg": 2, 2024-07-27T03:19:11.697 INFO:teuthology.orchestra.run.smithi067.stdout: "segmjr": 2, 2024-07-27T03:19:11.697 INFO:teuthology.orchestra.run.smithi067.stdout: "segexg": 0, 2024-07-27T03:19:11.697 INFO:teuthology.orchestra.run.smithi067.stdout: "segexd": 0, 2024-07-27T03:19:11.697 INFO:teuthology.orchestra.run.smithi067.stdout: "expos": 4194304, 2024-07-27T03:19:11.697 INFO:teuthology.orchestra.run.smithi067.stdout: "wrpos": 4194892, 2024-07-27T03:19:11.697 INFO:teuthology.orchestra.run.smithi067.stdout: "rdpos": 4194352, 2024-07-27T03:19:11.697 INFO:teuthology.orchestra.run.smithi067.stdout: "jlat": { 2024-07-27T03:19:11.698 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.698 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.698 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.698 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.698 INFO:teuthology.orchestra.run.smithi067.stdout: "replayed": 2 2024-07-27T03:19:11.698 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.698 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_mem": { 2024-07-27T03:19:11.698 INFO:teuthology.orchestra.run.smithi067.stdout: "ino": 3, 2024-07-27T03:19:11.698 INFO:teuthology.orchestra.run.smithi067.stdout: "ino+": 3, 2024-07-27T03:19:11.698 INFO:teuthology.orchestra.run.smithi067.stdout: "ino-": 0, 2024-07-27T03:19:11.698 INFO:teuthology.orchestra.run.smithi067.stdout: "dir": 1, 2024-07-27T03:19:11.698 INFO:teuthology.orchestra.run.smithi067.stdout: "dir+": 1, 2024-07-27T03:19:11.699 INFO:teuthology.orchestra.run.smithi067.stdout: "dir-": 0, 2024-07-27T03:19:11.699 INFO:teuthology.orchestra.run.smithi067.stdout: "dn": 0, 2024-07-27T03:19:11.699 INFO:teuthology.orchestra.run.smithi067.stdout: "dn+": 0, 2024-07-27T03:19:11.699 INFO:teuthology.orchestra.run.smithi067.stdout: "dn-": 0, 2024-07-27T03:19:11.699 INFO:teuthology.orchestra.run.smithi067.stdout: "cap": 0, 2024-07-27T03:19:11.699 INFO:teuthology.orchestra.run.smithi067.stdout: "cap+": 0, 2024-07-27T03:19:11.699 INFO:teuthology.orchestra.run.smithi067.stdout: "cap-": 0, 2024-07-27T03:19:11.699 INFO:teuthology.orchestra.run.smithi067.stdout: "rss": 116328, 2024-07-27T03:19:11.699 INFO:teuthology.orchestra.run.smithi067.stdout: "heap": 215348 2024-07-27T03:19:11.699 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.699 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_server": { 2024-07-27T03:19:11.699 INFO:teuthology.orchestra.run.smithi067.stdout: "dispatch_client_request": 0, 2024-07-27T03:19:11.699 INFO:teuthology.orchestra.run.smithi067.stdout: "dispatch_server_request": 0, 2024-07-27T03:19:11.699 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_request": 0, 2024-07-27T03:19:11.700 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_session": 0, 2024-07-27T03:19:11.700 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_peer_request": 0, 2024-07-27T03:19:11.700 INFO:teuthology.orchestra.run.smithi067.stdout: "req_create_latency": { 2024-07-27T03:19:11.700 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.700 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.700 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.701 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.701 INFO:teuthology.orchestra.run.smithi067.stdout: "req_getattr_latency": { 2024-07-27T03:19:11.701 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.701 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.701 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.701 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.701 INFO:teuthology.orchestra.run.smithi067.stdout: "req_getfilelock_latency": { 2024-07-27T03:19:11.701 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.701 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.701 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.701 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.701 INFO:teuthology.orchestra.run.smithi067.stdout: "req_link_latency": { 2024-07-27T03:19:11.701 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.702 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.702 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.702 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.702 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookup_latency": { 2024-07-27T03:19:11.702 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.702 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.702 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.702 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.702 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookuphash_latency": { 2024-07-27T03:19:11.702 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.702 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.702 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.702 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.702 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupino_latency": { 2024-07-27T03:19:11.703 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.703 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.703 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.703 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.703 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupname_latency": { 2024-07-27T03:19:11.703 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.703 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.703 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.703 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.703 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupparent_latency": { 2024-07-27T03:19:11.703 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.703 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.703 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.704 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.704 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupsnap_latency": { 2024-07-27T03:19:11.704 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.704 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.704 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.704 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.704 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lssnap_latency": { 2024-07-27T03:19:11.704 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.704 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.704 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.704 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.704 INFO:teuthology.orchestra.run.smithi067.stdout: "req_mkdir_latency": { 2024-07-27T03:19:11.704 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.705 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.705 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.705 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.705 INFO:teuthology.orchestra.run.smithi067.stdout: "req_mknod_latency": { 2024-07-27T03:19:11.705 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.706 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.706 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.706 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.706 INFO:teuthology.orchestra.run.smithi067.stdout: "req_mksnap_latency": { 2024-07-27T03:19:11.706 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.706 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.706 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.706 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.706 INFO:teuthology.orchestra.run.smithi067.stdout: "req_open_latency": { 2024-07-27T03:19:11.706 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.706 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.707 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.707 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.707 INFO:teuthology.orchestra.run.smithi067.stdout: "req_readdir_latency": { 2024-07-27T03:19:11.707 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.707 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.707 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.707 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.707 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rename_latency": { 2024-07-27T03:19:11.707 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.707 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.707 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.707 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.707 INFO:teuthology.orchestra.run.smithi067.stdout: "req_renamesnap_latency": { 2024-07-27T03:19:11.708 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.708 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.708 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.708 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.708 INFO:teuthology.orchestra.run.smithi067.stdout: "req_snapdiff_latency": { 2024-07-27T03:19:11.708 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.708 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.708 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.708 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.708 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rmdir_latency": { 2024-07-27T03:19:11.708 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.708 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.708 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.709 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.709 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rmsnap_latency": { 2024-07-27T03:19:11.709 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.709 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.709 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.709 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.709 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rmxattr_latency": { 2024-07-27T03:19:11.709 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.709 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.709 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.709 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.709 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setattr_latency": { 2024-07-27T03:19:11.710 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.710 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.710 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.710 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.710 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setdirlayout_latency": { 2024-07-27T03:19:11.710 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.710 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.710 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.710 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.710 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setfilelock_latency": { 2024-07-27T03:19:11.710 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.710 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.710 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.711 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.711 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setlayout_latency": { 2024-07-27T03:19:11.711 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.711 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.711 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.711 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.711 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setxattr_latency": { 2024-07-27T03:19:11.711 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.711 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.711 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.711 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.711 INFO:teuthology.orchestra.run.smithi067.stdout: "req_symlink_latency": { 2024-07-27T03:19:11.711 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.712 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.712 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.712 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.712 INFO:teuthology.orchestra.run.smithi067.stdout: "req_unlink_latency": { 2024-07-27T03:19:11.712 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.712 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.712 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.712 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.712 INFO:teuthology.orchestra.run.smithi067.stdout: "cap_revoke_eviction": 0, 2024-07-27T03:19:11.712 INFO:teuthology.orchestra.run.smithi067.stdout: "cap_acquisition_throttle": 0, 2024-07-27T03:19:11.712 INFO:teuthology.orchestra.run.smithi067.stdout: "req_getvxattr_latency": { 2024-07-27T03:19:11.712 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.712 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.712 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.713 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:11.713 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.713 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_sessions": { 2024-07-27T03:19:11.713 INFO:teuthology.orchestra.run.smithi067.stdout: "session_count": 0, 2024-07-27T03:19:11.713 INFO:teuthology.orchestra.run.smithi067.stdout: "session_add": 0, 2024-07-27T03:19:11.713 INFO:teuthology.orchestra.run.smithi067.stdout: "session_remove": 0, 2024-07-27T03:19:11.713 INFO:teuthology.orchestra.run.smithi067.stdout: "sessions_open": 0, 2024-07-27T03:19:11.713 INFO:teuthology.orchestra.run.smithi067.stdout: "sessions_stale": 0, 2024-07-27T03:19:11.713 INFO:teuthology.orchestra.run.smithi067.stdout: "total_load": 0, 2024-07-27T03:19:11.713 INFO:teuthology.orchestra.run.smithi067.stdout: "average_load": 0, 2024-07-27T03:19:11.713 INFO:teuthology.orchestra.run.smithi067.stdout: "avg_session_uptime": 0, 2024-07-27T03:19:11.713 INFO:teuthology.orchestra.run.smithi067.stdout: "mdthresh_evicted": 0 2024-07-27T03:19:11.713 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.714 INFO:teuthology.orchestra.run.smithi067.stdout: "mempool": { 2024-07-27T03:19:11.714 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_bytes": 0, 2024-07-27T03:19:11.714 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_items": 0, 2024-07-27T03:19:11.714 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_bytes": 0, 2024-07-27T03:19:11.714 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_items": 0, 2024-07-27T03:19:11.714 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_bytes": 0, 2024-07-27T03:19:11.714 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_items": 0, 2024-07-27T03:19:11.714 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-27T03:19:11.714 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_items": 0, 2024-07-27T03:19:11.714 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-27T03:19:11.714 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_items": 0, 2024-07-27T03:19:11.714 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_bytes": 0, 2024-07-27T03:19:11.714 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_items": 0, 2024-07-27T03:19:11.714 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-27T03:19:11.715 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_items": 0, 2024-07-27T03:19:11.715 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_bytes": 0, 2024-07-27T03:19:11.715 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_items": 0, 2024-07-27T03:19:11.715 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_bytes": 0, 2024-07-27T03:19:11.715 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_items": 0, 2024-07-27T03:19:11.715 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-27T03:19:11.715 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_items": 0, 2024-07-27T03:19:11.715 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-27T03:19:11.715 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_items": 0, 2024-07-27T03:19:11.715 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_bytes": 0, 2024-07-27T03:19:11.715 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_items": 0, 2024-07-27T03:19:11.715 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_bytes": 0, 2024-07-27T03:19:11.715 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_items": 0, 2024-07-27T03:19:11.715 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-27T03:19:11.716 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_items": 0, 2024-07-27T03:19:11.716 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_bytes": 0, 2024-07-27T03:19:11.716 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_items": 0, 2024-07-27T03:19:11.716 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_bytes": 0, 2024-07-27T03:19:11.716 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_items": 0, 2024-07-27T03:19:11.716 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_bytes": 0, 2024-07-27T03:19:11.716 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_items": 0, 2024-07-27T03:19:11.716 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_bytes": 0, 2024-07-27T03:19:11.716 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_items": 0, 2024-07-27T03:19:11.716 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_bytes": 189256, 2024-07-27T03:19:11.716 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_items": 49, 2024-07-27T03:19:11.716 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_bytes": 0, 2024-07-27T03:19:11.716 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_items": 0, 2024-07-27T03:19:11.717 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_bytes": 0, 2024-07-27T03:19:11.717 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_items": 0, 2024-07-27T03:19:11.717 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_bytes": 0, 2024-07-27T03:19:11.717 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_items": 0, 2024-07-27T03:19:11.717 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_bytes": 0, 2024-07-27T03:19:11.717 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_items": 0, 2024-07-27T03:19:11.717 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_bytes": 8952, 2024-07-27T03:19:11.717 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_items": 168, 2024-07-27T03:19:11.717 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_bytes": 0, 2024-07-27T03:19:11.717 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_items": 0, 2024-07-27T03:19:11.717 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_bytes": 0, 2024-07-27T03:19:11.717 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_items": 0, 2024-07-27T03:19:11.717 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_bytes": 8512, 2024-07-27T03:19:11.717 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_items": 13, 2024-07-27T03:19:11.718 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_bytes": 0, 2024-07-27T03:19:11.718 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_items": 0, 2024-07-27T03:19:11.718 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_bytes": 0, 2024-07-27T03:19:11.718 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_items": 0 2024-07-27T03:19:11.718 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.718 INFO:teuthology.orchestra.run.smithi067.stdout: "objecter": { 2024-07-27T03:19:11.718 INFO:teuthology.orchestra.run.smithi067.stdout: "op_active": 0, 2024-07-27T03:19:11.718 INFO:teuthology.orchestra.run.smithi067.stdout: "op_laggy": 0, 2024-07-27T03:19:11.718 INFO:teuthology.orchestra.run.smithi067.stdout: "op_send": 668, 2024-07-27T03:19:11.718 INFO:teuthology.orchestra.run.smithi067.stdout: "op_send_bytes": 21, 2024-07-27T03:19:11.718 INFO:teuthology.orchestra.run.smithi067.stdout: "op_resend": 3, 2024-07-27T03:19:11.718 INFO:teuthology.orchestra.run.smithi067.stdout: "op_reply": 668, 2024-07-27T03:19:11.718 INFO:teuthology.orchestra.run.smithi067.stdout: "op_latency": { 2024-07-27T03:19:11.719 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 668, 2024-07-27T03:19:11.719 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.454989730, 2024-07-27T03:19:11.719 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000681122 2024-07-27T03:19:11.719 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.719 INFO:teuthology.orchestra.run.smithi067.stdout: "op_inflight": 0, 2024-07-27T03:19:11.719 INFO:teuthology.orchestra.run.smithi067.stdout: "oplen_avg": { 2024-07-27T03:19:11.719 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 668, 2024-07-27T03:19:11.719 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 669 2024-07-27T03:19:11.719 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.719 INFO:teuthology.orchestra.run.smithi067.stdout: "op": 668, 2024-07-27T03:19:11.719 INFO:teuthology.orchestra.run.smithi067.stdout: "op_r": 668, 2024-07-27T03:19:11.719 INFO:teuthology.orchestra.run.smithi067.stdout: "op_w": 0, 2024-07-27T03:19:11.719 INFO:teuthology.orchestra.run.smithi067.stdout: "op_rmw": 0, 2024-07-27T03:19:11.719 INFO:teuthology.orchestra.run.smithi067.stdout: "op_pg": 0, 2024-07-27T03:19:11.720 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_stat": 331, 2024-07-27T03:19:11.720 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_create": 0, 2024-07-27T03:19:11.720 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_read": 335, 2024-07-27T03:19:11.720 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_write": 0, 2024-07-27T03:19:11.720 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_writefull": 0, 2024-07-27T03:19:11.720 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_writesame": 0, 2024-07-27T03:19:11.720 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_append": 0, 2024-07-27T03:19:11.720 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_zero": 0, 2024-07-27T03:19:11.720 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_truncate": 0, 2024-07-27T03:19:11.720 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_delete": 0, 2024-07-27T03:19:11.720 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_mapext": 0, 2024-07-27T03:19:11.720 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_sparse_read": 0, 2024-07-27T03:19:11.720 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_clonerange": 0, 2024-07-27T03:19:11.720 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_getxattr": 1, 2024-07-27T03:19:11.721 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_setxattr": 0, 2024-07-27T03:19:11.721 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_cmpxattr": 0, 2024-07-27T03:19:11.721 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_rmxattr": 0, 2024-07-27T03:19:11.721 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_resetxattrs": 0, 2024-07-27T03:19:11.721 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_call": 0, 2024-07-27T03:19:11.721 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_watch": 0, 2024-07-27T03:19:11.721 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_notify": 0, 2024-07-27T03:19:11.721 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_src_cmpxattr": 0, 2024-07-27T03:19:11.721 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_pgls": 0, 2024-07-27T03:19:11.721 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_pgls_filter": 0, 2024-07-27T03:19:11.721 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_other": 0, 2024-07-27T03:19:11.721 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_active": 0, 2024-07-27T03:19:11.721 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_send": 0, 2024-07-27T03:19:11.721 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_resend": 0, 2024-07-27T03:19:11.722 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_ping": 0, 2024-07-27T03:19:11.722 INFO:teuthology.orchestra.run.smithi067.stdout: "poolop_active": 0, 2024-07-27T03:19:11.722 INFO:teuthology.orchestra.run.smithi067.stdout: "poolop_send": 0, 2024-07-27T03:19:11.722 INFO:teuthology.orchestra.run.smithi067.stdout: "poolop_resend": 0, 2024-07-27T03:19:11.722 INFO:teuthology.orchestra.run.smithi067.stdout: "poolstat_active": 0, 2024-07-27T03:19:11.722 INFO:teuthology.orchestra.run.smithi067.stdout: "poolstat_send": 0, 2024-07-27T03:19:11.722 INFO:teuthology.orchestra.run.smithi067.stdout: "poolstat_resend": 0, 2024-07-27T03:19:11.722 INFO:teuthology.orchestra.run.smithi067.stdout: "statfs_active": 0, 2024-07-27T03:19:11.722 INFO:teuthology.orchestra.run.smithi067.stdout: "statfs_send": 0, 2024-07-27T03:19:11.722 INFO:teuthology.orchestra.run.smithi067.stdout: "statfs_resend": 0, 2024-07-27T03:19:11.722 INFO:teuthology.orchestra.run.smithi067.stdout: "command_active": 0, 2024-07-27T03:19:11.722 INFO:teuthology.orchestra.run.smithi067.stdout: "command_send": 0, 2024-07-27T03:19:11.722 INFO:teuthology.orchestra.run.smithi067.stdout: "command_resend": 0, 2024-07-27T03:19:11.723 INFO:teuthology.orchestra.run.smithi067.stdout: "map_epoch": 91, 2024-07-27T03:19:11.723 INFO:teuthology.orchestra.run.smithi067.stdout: "map_full": 0, 2024-07-27T03:19:11.723 INFO:teuthology.orchestra.run.smithi067.stdout: "map_inc": 2, 2024-07-27T03:19:11.723 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_sessions": 5, 2024-07-27T03:19:11.723 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_session_open": 5, 2024-07-27T03:19:11.723 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_session_close": 0, 2024-07-27T03:19:11.723 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_laggy": 0, 2024-07-27T03:19:11.723 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_wr": 0, 2024-07-27T03:19:11.723 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_rd": 2, 2024-07-27T03:19:11.723 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_del": 0 2024-07-27T03:19:11.723 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.723 INFO:teuthology.orchestra.run.smithi067.stdout: "oft": { 2024-07-27T03:19:11.723 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_objs": 0, 2024-07-27T03:19:11.723 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_kv_pairs": 0, 2024-07-27T03:19:11.723 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_updates": 0, 2024-07-27T03:19:11.724 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_removes": 0 2024-07-27T03:19:11.724 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.724 INFO:teuthology.orchestra.run.smithi067.stdout: "osd-slow-ops": { 2024-07-27T03:19:11.724 INFO:teuthology.orchestra.run.smithi067.stdout: "slow_ops_count": 0 2024-07-27T03:19:11.724 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.724 INFO:teuthology.orchestra.run.smithi067.stdout: "purge_queue": { 2024-07-27T03:19:11.724 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing_ops": 0, 2024-07-27T03:19:11.724 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing_ops_high_water": 0, 2024-07-27T03:19:11.724 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing": 0, 2024-07-27T03:19:11.724 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing_high_water": 0, 2024-07-27T03:19:11.724 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executed_ops": 0, 2024-07-27T03:19:11.724 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executed": 0, 2024-07-27T03:19:11.724 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_item_in_journal": 0 2024-07-27T03:19:11.724 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.725 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-27T03:19:11.725 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-27T03:19:11.725 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 104857600, 2024-07-27T03:19:11.725 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-27T03:19:11.725 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 794, 2024-07-27T03:19:11.725 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 642260, 2024-07-27T03:19:11.725 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:11.725 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 794, 2024-07-27T03:19:11.725 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-07-27T03:19:11.725 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-07-27T03:19:11.725 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 794, 2024-07-27T03:19:11.725 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 642260, 2024-07-27T03:19:11.725 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-27T03:19:11.725 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.726 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.726 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.726 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:11.726 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.726 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-objecter_bytes": { 2024-07-27T03:19:11.726 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-27T03:19:11.726 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 104857600, 2024-07-27T03:19:11.726 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-27T03:19:11.726 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 0, 2024-07-27T03:19:11.726 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 0, 2024-07-27T03:19:11.726 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:11.726 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:11.726 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 668, 2024-07-27T03:19:11.727 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 593, 2024-07-27T03:19:11.727 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 668, 2024-07-27T03:19:11.727 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 593, 2024-07-27T03:19:11.727 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-27T03:19:11.727 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.727 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.727 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.727 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:11.727 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.727 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-objecter_ops": { 2024-07-27T03:19:11.727 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-27T03:19:11.727 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1024, 2024-07-27T03:19:11.727 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-27T03:19:11.727 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 0, 2024-07-27T03:19:11.728 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 0, 2024-07-27T03:19:11.728 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:11.728 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:11.728 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 668, 2024-07-27T03:19:11.728 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 668, 2024-07-27T03:19:11.728 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 668, 2024-07-27T03:19:11.728 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 668, 2024-07-27T03:19:11.728 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-27T03:19:11.728 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.728 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.728 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.728 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:11.728 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.728 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-write_buf_throttle": { 2024-07-27T03:19:11.729 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-27T03:19:11.729 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 3758096384, 2024-07-27T03:19:11.729 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-27T03:19:11.729 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 0, 2024-07-27T03:19:11.729 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 0, 2024-07-27T03:19:11.729 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:11.729 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:11.729 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-07-27T03:19:11.729 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-07-27T03:19:11.729 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 0, 2024-07-27T03:19:11.729 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 0, 2024-07-27T03:19:11.729 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-27T03:19:11.729 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.730 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.730 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.730 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:11.730 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:11.730 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-write_buf_throttle-0x5588c5082c30": { 2024-07-27T03:19:11.730 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-27T03:19:11.730 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 3758096384, 2024-07-27T03:19:11.730 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-27T03:19:11.730 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 0, 2024-07-27T03:19:11.730 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 0, 2024-07-27T03:19:11.730 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:11.730 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:11.730 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-07-27T03:19:11.731 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-07-27T03:19:11.731 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 0, 2024-07-27T03:19:11.731 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 0, 2024-07-27T03:19:11.731 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-27T03:19:11.731 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:11.731 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:11.731 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:11.731 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:11.731 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:11.731 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:19:12.552 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-07-27T03:19:12.552 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-07-27T03:19:12.552 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-07-27T03:19:12.553 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-07-27T03:19:12.553 DEBUG:tasks.check_counter:Getting stats from d 2024-07-27T03:19:12.553 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph daemon mds.d perf dump 2024-07-27T03:19:12.826 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:19:12.948 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:12 smithi067 ceph-mon[32814]: pgmap v509: 129 pgs: 129 active+clean; 181 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 21 op/s 2024-07-27T03:19:13.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:12 smithi149 ceph-mon[37173]: pgmap v509: 129 pgs: 129 active+clean; 181 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 21 op/s 2024-07-27T03:19:13.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:12 smithi175 ceph-mon[37125]: pgmap v509: 129 pgs: 129 active+clean; 181 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 20 MiB/s wr, 21 op/s 2024-07-27T03:19:14.685 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:19:14.685 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-0": { 2024-07-27T03:19:14.685 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 0, 2024-07-27T03:19:14.686 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 2, 2024-07-27T03:19:14.686 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 0, 2024-07-27T03:19:14.686 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 0, 2024-07-27T03:19:14.686 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 4, 2024-07-27T03:19:14.686 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 0, 2024-07-27T03:19:14.686 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.001441239, 2024-07-27T03:19:14.686 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.000000000, 2024-07-27T03:19:14.686 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.000346874, 2024-07-27T03:19:14.686 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-27T03:19:14.686 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:14.686 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:14.686 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:14.687 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:14.687 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:14.687 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:14.687 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:14.687 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:14.687 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:14.687 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:14.687 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-27T03:19:14.687 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 0 2024-07-27T03:19:14.687 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:14.687 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-1": { 2024-07-27T03:19:14.687 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 123, 2024-07-27T03:19:14.687 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 120, 2024-07-27T03:19:14.687 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 490352, 2024-07-27T03:19:14.688 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 73008, 2024-07-27T03:19:14.688 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 2, 2024-07-27T03:19:14.688 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 1, 2024-07-27T03:19:14.688 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.022506063, 2024-07-27T03:19:14.688 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.010636993, 2024-07-27T03:19:14.688 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.016818654, 2024-07-27T03:19:14.688 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-27T03:19:14.688 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:14.688 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 119, 2024-07-27T03:19:14.688 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.006453064, 2024-07-27T03:19:14.688 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000054227 2024-07-27T03:19:14.688 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:14.688 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:14.688 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:14.689 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:14.689 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:14.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:14.689 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 490352, 2024-07-27T03:19:14.689 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 73008 2024-07-27T03:19:14.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:14.689 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-2": { 2024-07-27T03:19:14.689 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 5, 2024-07-27T03:19:14.689 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 93, 2024-07-27T03:19:14.689 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 3152, 2024-07-27T03:19:14.689 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 95888, 2024-07-27T03:19:14.689 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 3, 2024-07-27T03:19:14.689 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 2, 2024-07-27T03:19:14.690 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.011894525, 2024-07-27T03:19:14.690 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.007063317, 2024-07-27T03:19:14.690 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.003960766, 2024-07-27T03:19:14.690 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-27T03:19:14.690 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:14.690 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 92, 2024-07-27T03:19:14.690 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.008397610, 2024-07-27T03:19:14.690 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000091278 2024-07-27T03:19:14.690 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:14.690 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:14.690 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:14.690 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:14.690 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:14.691 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:14.691 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-07-27T03:19:14.691 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 95888 2024-07-27T03:19:14.691 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:14.691 INFO:teuthology.orchestra.run.smithi067.stdout: "cct": { 2024-07-27T03:19:14.691 INFO:teuthology.orchestra.run.smithi067.stdout: "total_workers": 0, 2024-07-27T03:19:14.691 INFO:teuthology.orchestra.run.smithi067.stdout: "unhealthy_workers": 0 2024-07-27T03:19:14.691 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:14.691 INFO:teuthology.orchestra.run.smithi067.stdout: "mempool": { 2024-07-27T03:19:14.691 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_bytes": 0, 2024-07-27T03:19:14.691 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_items": 0, 2024-07-27T03:19:14.691 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_bytes": 0, 2024-07-27T03:19:14.691 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_items": 0, 2024-07-27T03:19:14.691 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_bytes": 0, 2024-07-27T03:19:14.692 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_items": 0, 2024-07-27T03:19:14.692 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-27T03:19:14.692 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_items": 0, 2024-07-27T03:19:14.692 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-27T03:19:14.692 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_items": 0, 2024-07-27T03:19:14.692 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_bytes": 0, 2024-07-27T03:19:14.692 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_items": 0, 2024-07-27T03:19:14.692 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-27T03:19:14.692 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_items": 0, 2024-07-27T03:19:14.692 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_bytes": 0, 2024-07-27T03:19:14.692 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_items": 0, 2024-07-27T03:19:14.692 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_bytes": 0, 2024-07-27T03:19:14.693 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_items": 0, 2024-07-27T03:19:14.693 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-27T03:19:14.693 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_items": 0, 2024-07-27T03:19:14.693 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-27T03:19:14.693 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_items": 0, 2024-07-27T03:19:14.693 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_bytes": 0, 2024-07-27T03:19:14.693 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_items": 0, 2024-07-27T03:19:14.693 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_bytes": 0, 2024-07-27T03:19:14.693 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_items": 0, 2024-07-27T03:19:14.693 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-27T03:19:14.693 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_items": 0, 2024-07-27T03:19:14.693 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_bytes": 0, 2024-07-27T03:19:14.693 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_items": 0, 2024-07-27T03:19:14.693 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_bytes": 0, 2024-07-27T03:19:14.694 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_items": 0, 2024-07-27T03:19:14.694 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_bytes": 0, 2024-07-27T03:19:14.694 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_items": 0, 2024-07-27T03:19:14.694 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_bytes": 0, 2024-07-27T03:19:14.694 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_items": 0, 2024-07-27T03:19:14.694 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_bytes": 148823, 2024-07-27T03:19:14.694 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_items": 27, 2024-07-27T03:19:14.694 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_bytes": 0, 2024-07-27T03:19:14.694 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_items": 0, 2024-07-27T03:19:14.694 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_bytes": 0, 2024-07-27T03:19:14.694 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_items": 0, 2024-07-27T03:19:14.694 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_bytes": 0, 2024-07-27T03:19:14.694 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_items": 0, 2024-07-27T03:19:14.695 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_bytes": 0, 2024-07-27T03:19:14.695 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_items": 0, 2024-07-27T03:19:14.695 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_bytes": 0, 2024-07-27T03:19:14.695 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_items": 0, 2024-07-27T03:19:14.695 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_bytes": 0, 2024-07-27T03:19:14.695 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_items": 0, 2024-07-27T03:19:14.695 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_bytes": 0, 2024-07-27T03:19:14.695 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_items": 0, 2024-07-27T03:19:14.695 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_bytes": 928, 2024-07-27T03:19:14.695 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_items": 2, 2024-07-27T03:19:14.695 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_bytes": 0, 2024-07-27T03:19:14.695 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_items": 0, 2024-07-27T03:19:14.695 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_bytes": 0, 2024-07-27T03:19:14.695 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_items": 0 2024-07-27T03:19:14.696 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:14.696 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-27T03:19:14.696 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-27T03:19:14.696 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 104857600, 2024-07-27T03:19:14.696 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-27T03:19:14.696 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 124, 2024-07-27T03:19:14.696 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 472985, 2024-07-27T03:19:14.696 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:14.696 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 124, 2024-07-27T03:19:14.696 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-07-27T03:19:14.696 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-07-27T03:19:14.696 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 124, 2024-07-27T03:19:14.696 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 472985, 2024-07-27T03:19:14.696 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-27T03:19:14.697 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:14.697 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:14.697 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:14.697 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:14.697 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:14.697 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:19:14.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:14 smithi067 ceph-mon[32814]: pgmap v510: 129 pgs: 129 active+clean; 141 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 21 MiB/s wr, 18 op/s 2024-07-27T03:19:14.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:14 smithi149 ceph-mon[37173]: pgmap v510: 129 pgs: 129 active+clean; 141 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 21 MiB/s wr, 18 op/s 2024-07-27T03:19:15.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:14 smithi175 ceph-mon[37125]: pgmap v510: 129 pgs: 129 active+clean; 141 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 21 MiB/s wr, 18 op/s 2024-07-27T03:19:15.497 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-07-27T03:19:15.497 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-07-27T03:19:15.498 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-07-27T03:19:15.498 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-07-27T03:19:15.498 DEBUG:tasks.check_counter:Getting stats from g 2024-07-27T03:19:15.498 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph daemon mds.g perf dump 2024-07-27T03:19:15.787 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:19:15.917 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:15 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:19:15.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:15 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:19:16.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:15 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-07-27T03:19:16.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:16 smithi067 ceph-mon[32814]: pgmap v511: 129 pgs: 129 active+clean; 37 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 21 MiB/s wr, 23 op/s 2024-07-27T03:19:16.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:16 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:19:16.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:16 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:19:16.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:16 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:19:16.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:16 smithi149 ceph-mon[37173]: pgmap v511: 129 pgs: 129 active+clean; 37 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 21 MiB/s wr, 23 op/s 2024-07-27T03:19:17.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:16 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:19:17.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:16 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:19:17.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:16 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:19:17.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:16 smithi175 ceph-mon[37125]: pgmap v511: 129 pgs: 129 active+clean; 37 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 21 MiB/s wr, 23 op/s 2024-07-27T03:19:17.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:16 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-07-27T03:19:17.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:16 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-07-27T03:19:17.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:16 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' 2024-07-27T03:19:17.586 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:19:17.587 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-0": { 2024-07-27T03:19:17.587 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 465, 2024-07-27T03:19:17.587 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 460, 2024-07-27T03:19:17.587 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 633296, 2024-07-27T03:19:17.587 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 179278, 2024-07-27T03:19:17.587 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 5, 2024-07-27T03:19:17.587 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 1, 2024-07-27T03:19:17.587 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.069797106, 2024-07-27T03:19:17.587 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.035671311, 2024-07-27T03:19:17.587 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.023175944, 2024-07-27T03:19:17.587 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.010209855, 2024-07-27T03:19:17.588 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:17.588 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 459, 2024-07-27T03:19:17.588 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.023218471, 2024-07-27T03:19:17.588 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000050584 2024-07-27T03:19:17.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.588 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:17.588 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.588 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.588 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.588 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 520064, 2024-07-27T03:19:17.588 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 72112 2024-07-27T03:19:17.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.589 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-1": { 2024-07-27T03:19:17.589 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 344, 2024-07-27T03:19:17.589 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 429, 2024-07-27T03:19:17.589 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 92590, 2024-07-27T03:19:17.589 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 320771, 2024-07-27T03:19:17.589 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 5, 2024-07-27T03:19:17.589 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 4, 2024-07-27T03:19:17.589 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.040814996, 2024-07-27T03:19:17.589 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.019909954, 2024-07-27T03:19:17.589 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.010217825, 2024-07-27T03:19:17.589 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.007712735, 2024-07-27T03:19:17.589 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:17.589 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 428, 2024-07-27T03:19:17.590 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.017241553, 2024-07-27T03:19:17.590 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000040284 2024-07-27T03:19:17.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.590 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:17.590 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.590 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.590 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.590 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-07-27T03:19:17.590 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 212912 2024-07-27T03:19:17.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.590 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-2": { 2024-07-27T03:19:17.591 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 4, 2024-07-27T03:19:17.591 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 6, 2024-07-27T03:19:17.591 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 1629, 2024-07-27T03:19:17.591 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 1284, 2024-07-27T03:19:17.591 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 5, 2024-07-27T03:19:17.591 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 4, 2024-07-27T03:19:17.591 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.001577595, 2024-07-27T03:19:17.591 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.000113704, 2024-07-27T03:19:17.591 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.001264247, 2024-07-27T03:19:17.591 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000082668, 2024-07-27T03:19:17.591 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:17.591 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 4, 2024-07-27T03:19:17.592 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.002747699, 2024-07-27T03:19:17.592 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000686924 2024-07-27T03:19:17.592 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.592 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:17.592 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.592 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.592 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.592 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.592 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-27T03:19:17.592 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 0 2024-07-27T03:19:17.592 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.592 INFO:teuthology.orchestra.run.smithi067.stdout: "cct": { 2024-07-27T03:19:17.592 INFO:teuthology.orchestra.run.smithi067.stdout: "total_workers": 1, 2024-07-27T03:19:17.593 INFO:teuthology.orchestra.run.smithi067.stdout: "unhealthy_workers": 0 2024-07-27T03:19:17.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.593 INFO:teuthology.orchestra.run.smithi067.stdout: "finisher-MDSRank": { 2024-07-27T03:19:17.593 INFO:teuthology.orchestra.run.smithi067.stdout: "queue_len": 0, 2024-07-27T03:19:17.593 INFO:teuthology.orchestra.run.smithi067.stdout: "complete_latency": { 2024-07-27T03:19:17.593 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 1353, 2024-07-27T03:19:17.593 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.037294678, 2024-07-27T03:19:17.593 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000027564 2024-07-27T03:19:17.593 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:17.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.593 INFO:teuthology.orchestra.run.smithi067.stdout: "finisher-PurgeQueue": { 2024-07-27T03:19:17.593 INFO:teuthology.orchestra.run.smithi067.stdout: "queue_len": 0, 2024-07-27T03:19:17.593 INFO:teuthology.orchestra.run.smithi067.stdout: "complete_latency": { 2024-07-27T03:19:17.593 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.594 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.594 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.594 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:17.594 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.594 INFO:teuthology.orchestra.run.smithi067.stdout: "mds": { 2024-07-27T03:19:17.594 INFO:teuthology.orchestra.run.smithi067.stdout: "request": 0, 2024-07-27T03:19:17.594 INFO:teuthology.orchestra.run.smithi067.stdout: "reply": 0, 2024-07-27T03:19:17.594 INFO:teuthology.orchestra.run.smithi067.stdout: "reply_latency": { 2024-07-27T03:19:17.594 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.594 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.594 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.594 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.594 INFO:teuthology.orchestra.run.smithi067.stdout: "slow_reply": 0, 2024-07-27T03:19:17.595 INFO:teuthology.orchestra.run.smithi067.stdout: "forward": 0, 2024-07-27T03:19:17.595 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_fetch_complete": 0, 2024-07-27T03:19:17.595 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_fetch_keys": 0, 2024-07-27T03:19:17.595 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_commit": 0, 2024-07-27T03:19:17.595 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_split": 0, 2024-07-27T03:19:17.595 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_merge": 0, 2024-07-27T03:19:17.595 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes": 0, 2024-07-27T03:19:17.595 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_top": 0, 2024-07-27T03:19:17.595 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_bottom": 0, 2024-07-27T03:19:17.595 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_pin_tail": 0, 2024-07-27T03:19:17.595 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_pinned": 0, 2024-07-27T03:19:17.595 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_expired": 0, 2024-07-27T03:19:17.596 INFO:teuthology.orchestra.run.smithi067.stdout: "inodes_with_caps": 0, 2024-07-27T03:19:17.596 INFO:teuthology.orchestra.run.smithi067.stdout: "caps": 0, 2024-07-27T03:19:17.596 INFO:teuthology.orchestra.run.smithi067.stdout: "subtrees": 1, 2024-07-27T03:19:17.596 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse": 0, 2024-07-27T03:19:17.596 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_hit": 0, 2024-07-27T03:19:17.596 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_forward": 0, 2024-07-27T03:19:17.596 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_discover": 0, 2024-07-27T03:19:17.596 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_dir_fetch": 0, 2024-07-27T03:19:17.596 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_remote_ino": 0, 2024-07-27T03:19:17.596 INFO:teuthology.orchestra.run.smithi067.stdout: "traverse_lock": 0, 2024-07-27T03:19:17.596 INFO:teuthology.orchestra.run.smithi067.stdout: "load_cent": 0, 2024-07-27T03:19:17.596 INFO:teuthology.orchestra.run.smithi067.stdout: "q": 0, 2024-07-27T03:19:17.596 INFO:teuthology.orchestra.run.smithi067.stdout: "exported": 0, 2024-07-27T03:19:17.596 INFO:teuthology.orchestra.run.smithi067.stdout: "exported_inodes": 0, 2024-07-27T03:19:17.597 INFO:teuthology.orchestra.run.smithi067.stdout: "imported": 0, 2024-07-27T03:19:17.597 INFO:teuthology.orchestra.run.smithi067.stdout: "imported_inodes": 0, 2024-07-27T03:19:17.597 INFO:teuthology.orchestra.run.smithi067.stdout: "openino_dir_fetch": 0, 2024-07-27T03:19:17.597 INFO:teuthology.orchestra.run.smithi067.stdout: "openino_backtrace_fetch": 0, 2024-07-27T03:19:17.597 INFO:teuthology.orchestra.run.smithi067.stdout: "openino_peer_discover": 0, 2024-07-27T03:19:17.597 INFO:teuthology.orchestra.run.smithi067.stdout: "root_rfiles": 0, 2024-07-27T03:19:17.597 INFO:teuthology.orchestra.run.smithi067.stdout: "root_rbytes": 0, 2024-07-27T03:19:17.597 INFO:teuthology.orchestra.run.smithi067.stdout: "root_rsnaps": 0, 2024-07-27T03:19:17.597 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_backtrace_fetch": 0, 2024-07-27T03:19:17.597 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_set_tag": 0, 2024-07-27T03:19:17.597 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_backtrace_repaired": 0, 2024-07-27T03:19:17.597 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_inotable_repaired": 0, 2024-07-27T03:19:17.597 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_dir_inodes": 0, 2024-07-27T03:19:17.597 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_dir_base_inodes": 0, 2024-07-27T03:19:17.597 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_dirfrag_rstats": 0, 2024-07-27T03:19:17.598 INFO:teuthology.orchestra.run.smithi067.stdout: "scrub_file_inodes": 0, 2024-07-27T03:19:17.598 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_inode_file_caps": 0, 2024-07-27T03:19:17.598 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_revoke": 0, 2024-07-27T03:19:17.598 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_grant": 0, 2024-07-27T03:19:17.598 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_trunc": 0, 2024-07-27T03:19:17.598 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-27T03:19:17.598 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-27T03:19:17.598 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_caps": 0, 2024-07-27T03:19:17.598 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_caps_dirty": 0, 2024-07-27T03:19:17.598 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_cap_release": 0, 2024-07-27T03:19:17.598 INFO:teuthology.orchestra.run.smithi067.stdout: "process_request_cap_release": 0 2024-07-27T03:19:17.598 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.598 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_cache": { 2024-07-27T03:19:17.599 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_update": 0, 2024-07-27T03:19:17.599 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_update_receipt": 0, 2024-07-27T03:19:17.599 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_try_discover": 0, 2024-07-27T03:19:17.599 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_send_discover": 0, 2024-07-27T03:19:17.599 INFO:teuthology.orchestra.run.smithi067.stdout: "dir_handle_discover": 0, 2024-07-27T03:19:17.599 INFO:teuthology.orchestra.run.smithi067.stdout: "num_strays": 0, 2024-07-27T03:19:17.599 INFO:teuthology.orchestra.run.smithi067.stdout: "num_strays_delayed": 0, 2024-07-27T03:19:17.599 INFO:teuthology.orchestra.run.smithi067.stdout: "num_strays_enqueuing": 0, 2024-07-27T03:19:17.599 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_created": 0, 2024-07-27T03:19:17.599 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_enqueued": 0, 2024-07-27T03:19:17.599 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_reintegrated": 0, 2024-07-27T03:19:17.599 INFO:teuthology.orchestra.run.smithi067.stdout: "strays_migrated": 0, 2024-07-27T03:19:17.599 INFO:teuthology.orchestra.run.smithi067.stdout: "num_recovering_processing": 0, 2024-07-27T03:19:17.599 INFO:teuthology.orchestra.run.smithi067.stdout: "num_recovering_enqueued": 0, 2024-07-27T03:19:17.600 INFO:teuthology.orchestra.run.smithi067.stdout: "num_recovering_prioritized": 0, 2024-07-27T03:19:17.600 INFO:teuthology.orchestra.run.smithi067.stdout: "recovery_started": 0, 2024-07-27T03:19:17.600 INFO:teuthology.orchestra.run.smithi067.stdout: "recovery_completed": 0, 2024-07-27T03:19:17.600 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_quiesce_path": 0, 2024-07-27T03:19:17.600 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_quiesce_inode": 0, 2024-07-27T03:19:17.600 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_enqueue_scrub": 0, 2024-07-27T03:19:17.600 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_exportdir": 0, 2024-07-27T03:19:17.600 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_flush": 0, 2024-07-27T03:19:17.600 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_fragmentdir": 0, 2024-07-27T03:19:17.600 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_fragstats": 0, 2024-07-27T03:19:17.600 INFO:teuthology.orchestra.run.smithi067.stdout: "ireq_inodestats": 0 2024-07-27T03:19:17.600 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.600 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_log": { 2024-07-27T03:19:17.601 INFO:teuthology.orchestra.run.smithi067.stdout: "evlrg": 0, 2024-07-27T03:19:17.601 INFO:teuthology.orchestra.run.smithi067.stdout: "evadd": 0, 2024-07-27T03:19:17.601 INFO:teuthology.orchestra.run.smithi067.stdout: "evex": 0, 2024-07-27T03:19:17.601 INFO:teuthology.orchestra.run.smithi067.stdout: "evtrm": 0, 2024-07-27T03:19:17.601 INFO:teuthology.orchestra.run.smithi067.stdout: "ev": 2, 2024-07-27T03:19:17.601 INFO:teuthology.orchestra.run.smithi067.stdout: "evexg": 0, 2024-07-27T03:19:17.601 INFO:teuthology.orchestra.run.smithi067.stdout: "evexd": 0, 2024-07-27T03:19:17.601 INFO:teuthology.orchestra.run.smithi067.stdout: "segadd": 0, 2024-07-27T03:19:17.601 INFO:teuthology.orchestra.run.smithi067.stdout: "segex": 0, 2024-07-27T03:19:17.601 INFO:teuthology.orchestra.run.smithi067.stdout: "segtrm": 0, 2024-07-27T03:19:17.601 INFO:teuthology.orchestra.run.smithi067.stdout: "seg": 2, 2024-07-27T03:19:17.601 INFO:teuthology.orchestra.run.smithi067.stdout: "segmjr": 2, 2024-07-27T03:19:17.601 INFO:teuthology.orchestra.run.smithi067.stdout: "segexg": 0, 2024-07-27T03:19:17.601 INFO:teuthology.orchestra.run.smithi067.stdout: "segexd": 0, 2024-07-27T03:19:17.602 INFO:teuthology.orchestra.run.smithi067.stdout: "expos": 4194304, 2024-07-27T03:19:17.602 INFO:teuthology.orchestra.run.smithi067.stdout: "wrpos": 4194892, 2024-07-27T03:19:17.602 INFO:teuthology.orchestra.run.smithi067.stdout: "rdpos": 4194352, 2024-07-27T03:19:17.602 INFO:teuthology.orchestra.run.smithi067.stdout: "jlat": { 2024-07-27T03:19:17.602 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.602 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.602 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.602 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.602 INFO:teuthology.orchestra.run.smithi067.stdout: "replayed": 2 2024-07-27T03:19:17.602 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.602 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_mem": { 2024-07-27T03:19:17.602 INFO:teuthology.orchestra.run.smithi067.stdout: "ino": 3, 2024-07-27T03:19:17.602 INFO:teuthology.orchestra.run.smithi067.stdout: "ino+": 3, 2024-07-27T03:19:17.603 INFO:teuthology.orchestra.run.smithi067.stdout: "ino-": 0, 2024-07-27T03:19:17.603 INFO:teuthology.orchestra.run.smithi067.stdout: "dir": 1, 2024-07-27T03:19:17.603 INFO:teuthology.orchestra.run.smithi067.stdout: "dir+": 1, 2024-07-27T03:19:17.603 INFO:teuthology.orchestra.run.smithi067.stdout: "dir-": 0, 2024-07-27T03:19:17.603 INFO:teuthology.orchestra.run.smithi067.stdout: "dn": 0, 2024-07-27T03:19:17.603 INFO:teuthology.orchestra.run.smithi067.stdout: "dn+": 0, 2024-07-27T03:19:17.603 INFO:teuthology.orchestra.run.smithi067.stdout: "dn-": 0, 2024-07-27T03:19:17.603 INFO:teuthology.orchestra.run.smithi067.stdout: "cap": 0, 2024-07-27T03:19:17.603 INFO:teuthology.orchestra.run.smithi067.stdout: "cap+": 0, 2024-07-27T03:19:17.603 INFO:teuthology.orchestra.run.smithi067.stdout: "cap-": 0, 2024-07-27T03:19:17.603 INFO:teuthology.orchestra.run.smithi067.stdout: "rss": 116732, 2024-07-27T03:19:17.603 INFO:teuthology.orchestra.run.smithi067.stdout: "heap": 215348 2024-07-27T03:19:17.603 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.603 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_server": { 2024-07-27T03:19:17.604 INFO:teuthology.orchestra.run.smithi067.stdout: "dispatch_client_request": 0, 2024-07-27T03:19:17.604 INFO:teuthology.orchestra.run.smithi067.stdout: "dispatch_server_request": 0, 2024-07-27T03:19:17.604 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_request": 0, 2024-07-27T03:19:17.604 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_client_session": 0, 2024-07-27T03:19:17.604 INFO:teuthology.orchestra.run.smithi067.stdout: "handle_peer_request": 0, 2024-07-27T03:19:17.604 INFO:teuthology.orchestra.run.smithi067.stdout: "req_create_latency": { 2024-07-27T03:19:17.604 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.604 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.604 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.604 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.604 INFO:teuthology.orchestra.run.smithi067.stdout: "req_getattr_latency": { 2024-07-27T03:19:17.604 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.604 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.605 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.605 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.605 INFO:teuthology.orchestra.run.smithi067.stdout: "req_getfilelock_latency": { 2024-07-27T03:19:17.605 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.605 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.605 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.605 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.605 INFO:teuthology.orchestra.run.smithi067.stdout: "req_link_latency": { 2024-07-27T03:19:17.605 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.605 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.605 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.605 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.605 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookup_latency": { 2024-07-27T03:19:17.605 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.606 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.606 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.606 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.606 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookuphash_latency": { 2024-07-27T03:19:17.606 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.606 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.606 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.606 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.606 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupino_latency": { 2024-07-27T03:19:17.606 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.606 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.606 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.606 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.607 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupname_latency": { 2024-07-27T03:19:17.607 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.607 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.607 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.607 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.607 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupparent_latency": { 2024-07-27T03:19:17.607 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.607 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.607 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.607 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.607 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lookupsnap_latency": { 2024-07-27T03:19:17.607 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.607 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.607 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.608 INFO:teuthology.orchestra.run.smithi067.stdout: "req_lssnap_latency": { 2024-07-27T03:19:17.608 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.608 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.608 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.608 INFO:teuthology.orchestra.run.smithi067.stdout: "req_mkdir_latency": { 2024-07-27T03:19:17.608 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.608 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.609 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.609 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.609 INFO:teuthology.orchestra.run.smithi067.stdout: "req_mknod_latency": { 2024-07-27T03:19:17.609 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.609 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.609 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.609 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.609 INFO:teuthology.orchestra.run.smithi067.stdout: "req_mksnap_latency": { 2024-07-27T03:19:17.609 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.609 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.610 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.610 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.610 INFO:teuthology.orchestra.run.smithi067.stdout: "req_open_latency": { 2024-07-27T03:19:17.610 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.610 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.610 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.610 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.610 INFO:teuthology.orchestra.run.smithi067.stdout: "req_readdir_latency": { 2024-07-27T03:19:17.610 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.610 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.610 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.610 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.610 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rename_latency": { 2024-07-27T03:19:17.611 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.611 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.611 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.611 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.611 INFO:teuthology.orchestra.run.smithi067.stdout: "req_renamesnap_latency": { 2024-07-27T03:19:17.611 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.611 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.611 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.611 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.612 INFO:teuthology.orchestra.run.smithi067.stdout: "req_snapdiff_latency": { 2024-07-27T03:19:17.612 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.612 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.612 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.612 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.612 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rmdir_latency": { 2024-07-27T03:19:17.612 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.612 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.612 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.612 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.612 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rmsnap_latency": { 2024-07-27T03:19:17.612 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.612 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.613 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.613 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.613 INFO:teuthology.orchestra.run.smithi067.stdout: "req_rmxattr_latency": { 2024-07-27T03:19:17.613 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.613 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.613 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.613 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.613 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setattr_latency": { 2024-07-27T03:19:17.613 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.613 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.613 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.613 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.613 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setdirlayout_latency": { 2024-07-27T03:19:17.614 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.614 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.614 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.614 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.614 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setfilelock_latency": { 2024-07-27T03:19:17.614 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.614 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.614 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.614 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.614 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setlayout_latency": { 2024-07-27T03:19:17.614 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.614 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.615 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.615 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.615 INFO:teuthology.orchestra.run.smithi067.stdout: "req_setxattr_latency": { 2024-07-27T03:19:17.615 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.615 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.615 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.615 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.615 INFO:teuthology.orchestra.run.smithi067.stdout: "req_symlink_latency": { 2024-07-27T03:19:17.615 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.615 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.615 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.615 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.617 INFO:teuthology.orchestra.run.smithi067.stdout: "req_unlink_latency": { 2024-07-27T03:19:17.617 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.618 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.618 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.618 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.618 INFO:teuthology.orchestra.run.smithi067.stdout: "cap_revoke_eviction": 0, 2024-07-27T03:19:17.618 INFO:teuthology.orchestra.run.smithi067.stdout: "cap_acquisition_throttle": 0, 2024-07-27T03:19:17.618 INFO:teuthology.orchestra.run.smithi067.stdout: "req_getvxattr_latency": { 2024-07-27T03:19:17.618 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.618 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.618 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.618 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:17.618 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.618 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_sessions": { 2024-07-27T03:19:17.618 INFO:teuthology.orchestra.run.smithi067.stdout: "session_count": 0, 2024-07-27T03:19:17.618 INFO:teuthology.orchestra.run.smithi067.stdout: "session_add": 0, 2024-07-27T03:19:17.619 INFO:teuthology.orchestra.run.smithi067.stdout: "session_remove": 0, 2024-07-27T03:19:17.619 INFO:teuthology.orchestra.run.smithi067.stdout: "sessions_open": 0, 2024-07-27T03:19:17.619 INFO:teuthology.orchestra.run.smithi067.stdout: "sessions_stale": 0, 2024-07-27T03:19:17.619 INFO:teuthology.orchestra.run.smithi067.stdout: "total_load": 0, 2024-07-27T03:19:17.619 INFO:teuthology.orchestra.run.smithi067.stdout: "average_load": 0, 2024-07-27T03:19:17.619 INFO:teuthology.orchestra.run.smithi067.stdout: "avg_session_uptime": 0, 2024-07-27T03:19:17.619 INFO:teuthology.orchestra.run.smithi067.stdout: "mdthresh_evicted": 0 2024-07-27T03:19:17.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.619 INFO:teuthology.orchestra.run.smithi067.stdout: "mempool": { 2024-07-27T03:19:17.619 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_bytes": 0, 2024-07-27T03:19:17.619 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_items": 0, 2024-07-27T03:19:17.619 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_bytes": 0, 2024-07-27T03:19:17.619 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_items": 0, 2024-07-27T03:19:17.619 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_bytes": 0, 2024-07-27T03:19:17.620 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_items": 0, 2024-07-27T03:19:17.620 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-27T03:19:17.620 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_items": 0, 2024-07-27T03:19:17.620 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-27T03:19:17.620 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_items": 0, 2024-07-27T03:19:17.620 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_bytes": 0, 2024-07-27T03:19:17.620 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_items": 0, 2024-07-27T03:19:17.620 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-27T03:19:17.620 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_items": 0, 2024-07-27T03:19:17.620 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_bytes": 0, 2024-07-27T03:19:17.620 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_items": 0, 2024-07-27T03:19:17.620 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_bytes": 0, 2024-07-27T03:19:17.621 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_items": 0, 2024-07-27T03:19:17.621 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-27T03:19:17.621 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_items": 0, 2024-07-27T03:19:17.621 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-27T03:19:17.621 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_items": 0, 2024-07-27T03:19:17.621 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_bytes": 0, 2024-07-27T03:19:17.621 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_items": 0, 2024-07-27T03:19:17.621 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_bytes": 0, 2024-07-27T03:19:17.621 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_items": 0, 2024-07-27T03:19:17.622 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-27T03:19:17.622 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_items": 0, 2024-07-27T03:19:17.622 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_bytes": 0, 2024-07-27T03:19:17.622 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_items": 0, 2024-07-27T03:19:17.622 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_bytes": 0, 2024-07-27T03:19:17.622 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_items": 0, 2024-07-27T03:19:17.622 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_bytes": 0, 2024-07-27T03:19:17.622 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_items": 0, 2024-07-27T03:19:17.622 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_bytes": 0, 2024-07-27T03:19:17.622 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_items": 0, 2024-07-27T03:19:17.622 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_bytes": 197319, 2024-07-27T03:19:17.622 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_items": 52, 2024-07-27T03:19:17.622 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_bytes": 0, 2024-07-27T03:19:17.623 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_items": 0, 2024-07-27T03:19:17.623 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_bytes": 0, 2024-07-27T03:19:17.623 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_items": 0, 2024-07-27T03:19:17.623 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_bytes": 0, 2024-07-27T03:19:17.623 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_items": 0, 2024-07-27T03:19:17.623 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_bytes": 0, 2024-07-27T03:19:17.623 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_items": 0, 2024-07-27T03:19:17.623 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_bytes": 8952, 2024-07-27T03:19:17.623 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_items": 168, 2024-07-27T03:19:17.623 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_bytes": 0, 2024-07-27T03:19:17.623 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_items": 0, 2024-07-27T03:19:17.623 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_bytes": 0, 2024-07-27T03:19:17.623 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_items": 0, 2024-07-27T03:19:17.623 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_bytes": 8512, 2024-07-27T03:19:17.624 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_items": 13, 2024-07-27T03:19:17.624 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_bytes": 0, 2024-07-27T03:19:17.624 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_items": 0, 2024-07-27T03:19:17.624 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_bytes": 0, 2024-07-27T03:19:17.624 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_items": 0 2024-07-27T03:19:17.624 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.624 INFO:teuthology.orchestra.run.smithi067.stdout: "objecter": { 2024-07-27T03:19:17.624 INFO:teuthology.orchestra.run.smithi067.stdout: "op_active": 0, 2024-07-27T03:19:17.624 INFO:teuthology.orchestra.run.smithi067.stdout: "op_laggy": 0, 2024-07-27T03:19:17.624 INFO:teuthology.orchestra.run.smithi067.stdout: "op_send": 680, 2024-07-27T03:19:17.624 INFO:teuthology.orchestra.run.smithi067.stdout: "op_send_bytes": 21, 2024-07-27T03:19:17.624 INFO:teuthology.orchestra.run.smithi067.stdout: "op_resend": 3, 2024-07-27T03:19:17.624 INFO:teuthology.orchestra.run.smithi067.stdout: "op_reply": 680, 2024-07-27T03:19:17.625 INFO:teuthology.orchestra.run.smithi067.stdout: "op_latency": { 2024-07-27T03:19:17.625 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 680, 2024-07-27T03:19:17.625 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.441990038, 2024-07-27T03:19:17.625 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000649985 2024-07-27T03:19:17.625 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.625 INFO:teuthology.orchestra.run.smithi067.stdout: "op_inflight": 0, 2024-07-27T03:19:17.625 INFO:teuthology.orchestra.run.smithi067.stdout: "oplen_avg": { 2024-07-27T03:19:17.625 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 680, 2024-07-27T03:19:17.625 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 681 2024-07-27T03:19:17.625 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.625 INFO:teuthology.orchestra.run.smithi067.stdout: "op": 680, 2024-07-27T03:19:17.625 INFO:teuthology.orchestra.run.smithi067.stdout: "op_r": 680, 2024-07-27T03:19:17.625 INFO:teuthology.orchestra.run.smithi067.stdout: "op_w": 0, 2024-07-27T03:19:17.625 INFO:teuthology.orchestra.run.smithi067.stdout: "op_rmw": 0, 2024-07-27T03:19:17.626 INFO:teuthology.orchestra.run.smithi067.stdout: "op_pg": 0, 2024-07-27T03:19:17.626 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_stat": 337, 2024-07-27T03:19:17.626 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_create": 0, 2024-07-27T03:19:17.626 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_read": 341, 2024-07-27T03:19:17.626 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_write": 0, 2024-07-27T03:19:17.626 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_writefull": 0, 2024-07-27T03:19:17.626 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_writesame": 0, 2024-07-27T03:19:17.626 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_append": 0, 2024-07-27T03:19:17.626 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_zero": 0, 2024-07-27T03:19:17.626 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_truncate": 0, 2024-07-27T03:19:17.626 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_delete": 0, 2024-07-27T03:19:17.626 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_mapext": 0, 2024-07-27T03:19:17.626 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_sparse_read": 0, 2024-07-27T03:19:17.627 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_clonerange": 0, 2024-07-27T03:19:17.627 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_getxattr": 1, 2024-07-27T03:19:17.627 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_setxattr": 0, 2024-07-27T03:19:17.627 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_cmpxattr": 0, 2024-07-27T03:19:17.627 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_rmxattr": 0, 2024-07-27T03:19:17.627 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_resetxattrs": 0, 2024-07-27T03:19:17.627 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_call": 0, 2024-07-27T03:19:17.627 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_watch": 0, 2024-07-27T03:19:17.627 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_notify": 0, 2024-07-27T03:19:17.627 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_src_cmpxattr": 0, 2024-07-27T03:19:17.627 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_pgls": 0, 2024-07-27T03:19:17.627 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_pgls_filter": 0, 2024-07-27T03:19:17.627 INFO:teuthology.orchestra.run.smithi067.stdout: "osdop_other": 0, 2024-07-27T03:19:17.628 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_active": 0, 2024-07-27T03:19:17.628 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_send": 0, 2024-07-27T03:19:17.628 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_resend": 0, 2024-07-27T03:19:17.628 INFO:teuthology.orchestra.run.smithi067.stdout: "linger_ping": 0, 2024-07-27T03:19:17.628 INFO:teuthology.orchestra.run.smithi067.stdout: "poolop_active": 0, 2024-07-27T03:19:17.628 INFO:teuthology.orchestra.run.smithi067.stdout: "poolop_send": 0, 2024-07-27T03:19:17.628 INFO:teuthology.orchestra.run.smithi067.stdout: "poolop_resend": 0, 2024-07-27T03:19:17.628 INFO:teuthology.orchestra.run.smithi067.stdout: "poolstat_active": 0, 2024-07-27T03:19:17.628 INFO:teuthology.orchestra.run.smithi067.stdout: "poolstat_send": 0, 2024-07-27T03:19:17.628 INFO:teuthology.orchestra.run.smithi067.stdout: "poolstat_resend": 0, 2024-07-27T03:19:17.628 INFO:teuthology.orchestra.run.smithi067.stdout: "statfs_active": 0, 2024-07-27T03:19:17.628 INFO:teuthology.orchestra.run.smithi067.stdout: "statfs_send": 0, 2024-07-27T03:19:17.628 INFO:teuthology.orchestra.run.smithi067.stdout: "statfs_resend": 0, 2024-07-27T03:19:17.628 INFO:teuthology.orchestra.run.smithi067.stdout: "command_active": 0, 2024-07-27T03:19:17.629 INFO:teuthology.orchestra.run.smithi067.stdout: "command_send": 0, 2024-07-27T03:19:17.629 INFO:teuthology.orchestra.run.smithi067.stdout: "command_resend": 0, 2024-07-27T03:19:17.629 INFO:teuthology.orchestra.run.smithi067.stdout: "map_epoch": 91, 2024-07-27T03:19:17.629 INFO:teuthology.orchestra.run.smithi067.stdout: "map_full": 0, 2024-07-27T03:19:17.629 INFO:teuthology.orchestra.run.smithi067.stdout: "map_inc": 2, 2024-07-27T03:19:17.629 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_sessions": 6, 2024-07-27T03:19:17.629 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_session_open": 6, 2024-07-27T03:19:17.629 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_session_close": 0, 2024-07-27T03:19:17.629 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_laggy": 0, 2024-07-27T03:19:17.629 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_wr": 0, 2024-07-27T03:19:17.629 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_rd": 2, 2024-07-27T03:19:17.629 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_del": 0 2024-07-27T03:19:17.629 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.630 INFO:teuthology.orchestra.run.smithi067.stdout: "oft": { 2024-07-27T03:19:17.630 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_objs": 0, 2024-07-27T03:19:17.630 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_kv_pairs": 0, 2024-07-27T03:19:17.630 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_updates": 0, 2024-07-27T03:19:17.630 INFO:teuthology.orchestra.run.smithi067.stdout: "omap_total_removes": 0 2024-07-27T03:19:17.630 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.630 INFO:teuthology.orchestra.run.smithi067.stdout: "osd-slow-ops": { 2024-07-27T03:19:17.630 INFO:teuthology.orchestra.run.smithi067.stdout: "slow_ops_count": 0 2024-07-27T03:19:17.630 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.630 INFO:teuthology.orchestra.run.smithi067.stdout: "purge_queue": { 2024-07-27T03:19:17.630 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing_ops": 0, 2024-07-27T03:19:17.630 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing_ops_high_water": 0, 2024-07-27T03:19:17.630 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing": 0, 2024-07-27T03:19:17.631 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executing_high_water": 0, 2024-07-27T03:19:17.631 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executed_ops": 0, 2024-07-27T03:19:17.631 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_executed": 0, 2024-07-27T03:19:17.631 INFO:teuthology.orchestra.run.smithi067.stdout: "pq_item_in_journal": 0 2024-07-27T03:19:17.631 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.631 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-27T03:19:17.631 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-27T03:19:17.631 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 104857600, 2024-07-27T03:19:17.631 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-27T03:19:17.631 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 809, 2024-07-27T03:19:17.631 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 645158, 2024-07-27T03:19:17.631 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:17.631 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 809, 2024-07-27T03:19:17.631 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-07-27T03:19:17.632 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-07-27T03:19:17.632 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 809, 2024-07-27T03:19:17.632 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 645158, 2024-07-27T03:19:17.632 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-27T03:19:17.632 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.632 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.632 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.632 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:17.632 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.632 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-objecter_bytes": { 2024-07-27T03:19:17.632 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-27T03:19:17.632 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 104857600, 2024-07-27T03:19:17.632 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-27T03:19:17.632 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 0, 2024-07-27T03:19:17.633 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 0, 2024-07-27T03:19:17.633 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:17.633 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:17.633 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 680, 2024-07-27T03:19:17.633 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 593, 2024-07-27T03:19:17.633 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 680, 2024-07-27T03:19:17.633 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 593, 2024-07-27T03:19:17.633 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-27T03:19:17.633 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.633 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.633 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.633 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:17.634 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.634 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-objecter_ops": { 2024-07-27T03:19:17.634 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-27T03:19:17.634 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 1024, 2024-07-27T03:19:17.634 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-27T03:19:17.634 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 0, 2024-07-27T03:19:17.634 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 0, 2024-07-27T03:19:17.634 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:17.634 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:17.634 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 680, 2024-07-27T03:19:17.634 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 680, 2024-07-27T03:19:17.634 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 680, 2024-07-27T03:19:17.634 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 680, 2024-07-27T03:19:17.634 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-27T03:19:17.635 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.635 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.635 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.635 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:17.635 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.635 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-write_buf_throttle": { 2024-07-27T03:19:17.635 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-27T03:19:17.635 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 3758096384, 2024-07-27T03:19:17.635 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-27T03:19:17.635 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 0, 2024-07-27T03:19:17.635 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 0, 2024-07-27T03:19:17.635 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:17.635 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:17.636 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-07-27T03:19:17.636 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-07-27T03:19:17.636 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 0, 2024-07-27T03:19:17.636 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 0, 2024-07-27T03:19:17.636 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-27T03:19:17.636 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.636 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.636 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.636 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:17.636 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:17.636 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-write_buf_throttle-0x55ffd65c0c30": { 2024-07-27T03:19:17.636 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-27T03:19:17.636 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 3758096384, 2024-07-27T03:19:17.636 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-27T03:19:17.636 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 0, 2024-07-27T03:19:17.636 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 0, 2024-07-27T03:19:17.637 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:17.637 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:17.637 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-07-27T03:19:17.637 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-07-27T03:19:17.637 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 0, 2024-07-27T03:19:17.637 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 0, 2024-07-27T03:19:17.637 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-27T03:19:17.637 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:17.637 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:17.637 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:17.637 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:17.637 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:17.637 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:19:18.483 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-07-27T03:19:18.483 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-07-27T03:19:18.484 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-07-27T03:19:18.484 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-07-27T03:19:18.484 DEBUG:tasks.check_counter:Getting stats from j 2024-07-27T03:19:18.484 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph daemon mds.j perf dump 2024-07-27T03:19:18.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:18 smithi067 ceph-mon[32814]: pgmap v512: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 14 MiB/s wr, 19 op/s 2024-07-27T03:19:18.756 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:19:18.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:18 smithi149 ceph-mon[37173]: pgmap v512: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 14 MiB/s wr, 19 op/s 2024-07-27T03:19:19.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:18 smithi175 ceph-mon[37125]: pgmap v512: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 14 MiB/s wr, 19 op/s 2024-07-27T03:19:20.585 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-07-27T03:19:20.585 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-0": { 2024-07-27T03:19:20.585 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 4, 2024-07-27T03:19:20.586 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 4, 2024-07-27T03:19:20.586 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 2992, 2024-07-27T03:19:20.586 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 448, 2024-07-27T03:19:20.586 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 4, 2024-07-27T03:19:20.586 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 0, 2024-07-27T03:19:20.586 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.004217742, 2024-07-27T03:19:20.587 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.000051367, 2024-07-27T03:19:20.587 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.003229278, 2024-07-27T03:19:20.587 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-27T03:19:20.587 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:20.587 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 3, 2024-07-27T03:19:20.587 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.005870437, 2024-07-27T03:19:20.587 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.001956812 2024-07-27T03:19:20.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:20.588 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:20.588 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:20.588 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:20.588 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:20.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:20.589 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-07-27T03:19:20.589 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 448 2024-07-27T03:19:20.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:20.589 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-1": { 2024-07-27T03:19:20.589 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 1, 2024-07-27T03:19:20.589 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 93, 2024-07-27T03:19:20.590 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 160, 2024-07-27T03:19:20.590 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 95792, 2024-07-27T03:19:20.590 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 3, 2024-07-27T03:19:20.590 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 2, 2024-07-27T03:19:20.590 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.010288100, 2024-07-27T03:19:20.591 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.007671951, 2024-07-27T03:19:20.591 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.001633242, 2024-07-27T03:19:20.591 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-27T03:19:20.591 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:20.591 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 91, 2024-07-27T03:19:20.591 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.004580711, 2024-07-27T03:19:20.592 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000050337 2024-07-27T03:19:20.592 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:20.592 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:20.592 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:20.592 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:20.593 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:20.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:20.593 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 160, 2024-07-27T03:19:20.593 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 95792 2024-07-27T03:19:20.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:20.593 INFO:teuthology.orchestra.run.smithi067.stdout: "AsyncMessenger::Worker-2": { 2024-07-27T03:19:20.594 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_messages": 125, 2024-07-27T03:19:20.594 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages": 122, 2024-07-27T03:19:20.594 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_bytes": 490896, 2024-07-27T03:19:20.594 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_bytes": 74288, 2024-07-27T03:19:20.594 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_created_connections": 2, 2024-07-27T03:19:20.594 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_active_connections": 1, 2024-07-27T03:19:20.595 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_total_time": 0.022556688, 2024-07-27T03:19:20.595 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_send_time": 0.011050798, 2024-07-27T03:19:20.595 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_recv_time": 0.015730362, 2024-07-27T03:19:20.595 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-07-27T03:19:20.595 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:20.595 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 121, 2024-07-27T03:19:20.596 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.007244706, 2024-07-27T03:19:20.596 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000059873 2024-07-27T03:19:20.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:20.596 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:20.596 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:20.596 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:20.597 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:20.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:20.597 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_recv_encrypted_bytes": 490896, 2024-07-27T03:19:20.597 INFO:teuthology.orchestra.run.smithi067.stdout: "msgr_send_encrypted_bytes": 74288 2024-07-27T03:19:20.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:20.597 INFO:teuthology.orchestra.run.smithi067.stdout: "cct": { 2024-07-27T03:19:20.598 INFO:teuthology.orchestra.run.smithi067.stdout: "total_workers": 0, 2024-07-27T03:19:20.598 INFO:teuthology.orchestra.run.smithi067.stdout: "unhealthy_workers": 0 2024-07-27T03:19:20.598 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:20.598 INFO:teuthology.orchestra.run.smithi067.stdout: "mempool": { 2024-07-27T03:19:20.598 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_bytes": 0, 2024-07-27T03:19:20.598 INFO:teuthology.orchestra.run.smithi067.stdout: "bloom_filter_items": 0, 2024-07-27T03:19:20.599 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_bytes": 0, 2024-07-27T03:19:20.599 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_alloc_items": 0, 2024-07-27T03:19:20.599 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_bytes": 0, 2024-07-27T03:19:20.599 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_data_items": 0, 2024-07-27T03:19:20.599 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-27T03:19:20.599 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_onode_items": 0, 2024-07-27T03:19:20.600 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-27T03:19:20.600 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_meta_items": 0, 2024-07-27T03:19:20.600 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_bytes": 0, 2024-07-27T03:19:20.600 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_other_items": 0, 2024-07-27T03:19:20.600 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-27T03:19:20.601 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_cache_buffer_items": 0, 2024-07-27T03:19:20.601 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_bytes": 0, 2024-07-27T03:19:20.601 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_extent_items": 0, 2024-07-27T03:19:20.601 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_bytes": 0, 2024-07-27T03:19:20.601 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_blob_items": 0, 2024-07-27T03:19:20.601 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-27T03:19:20.602 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_shared_blob_items": 0, 2024-07-27T03:19:20.602 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-27T03:19:20.602 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_inline_bl_items": 0, 2024-07-27T03:19:20.602 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_bytes": 0, 2024-07-27T03:19:20.602 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_fsck_items": 0, 2024-07-27T03:19:20.603 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_bytes": 0, 2024-07-27T03:19:20.603 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_txc_items": 0, 2024-07-27T03:19:20.603 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-27T03:19:20.603 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_deferred_items": 0, 2024-07-27T03:19:20.603 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_bytes": 0, 2024-07-27T03:19:20.604 INFO:teuthology.orchestra.run.smithi067.stdout: "bluestore_writing_items": 0, 2024-07-27T03:19:20.604 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_bytes": 0, 2024-07-27T03:19:20.604 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_items": 0, 2024-07-27T03:19:20.604 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_bytes": 0, 2024-07-27T03:19:20.604 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_reader_items": 0, 2024-07-27T03:19:20.604 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_bytes": 0, 2024-07-27T03:19:20.605 INFO:teuthology.orchestra.run.smithi067.stdout: "bluefs_file_writer_items": 0, 2024-07-27T03:19:20.605 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_bytes": 148840, 2024-07-27T03:19:20.605 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_anon_items": 28, 2024-07-27T03:19:20.605 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_bytes": 0, 2024-07-27T03:19:20.605 INFO:teuthology.orchestra.run.smithi067.stdout: "buffer_meta_items": 0, 2024-07-27T03:19:20.605 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_bytes": 0, 2024-07-27T03:19:20.606 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_items": 0, 2024-07-27T03:19:20.606 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_bytes": 0, 2024-07-27T03:19:20.606 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_mapbl_items": 0, 2024-07-27T03:19:20.606 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_bytes": 0, 2024-07-27T03:19:20.606 INFO:teuthology.orchestra.run.smithi067.stdout: "osd_pglog_items": 0, 2024-07-27T03:19:20.606 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_bytes": 0, 2024-07-27T03:19:20.607 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_items": 0, 2024-07-27T03:19:20.607 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_bytes": 0, 2024-07-27T03:19:20.607 INFO:teuthology.orchestra.run.smithi067.stdout: "osdmap_mapping_items": 0, 2024-07-27T03:19:20.607 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_bytes": 0, 2024-07-27T03:19:20.607 INFO:teuthology.orchestra.run.smithi067.stdout: "pgmap_items": 0, 2024-07-27T03:19:20.607 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_bytes": 928, 2024-07-27T03:19:20.607 INFO:teuthology.orchestra.run.smithi067.stdout: "mds_co_items": 2, 2024-07-27T03:19:20.608 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_bytes": 0, 2024-07-27T03:19:20.608 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_1_items": 0, 2024-07-27T03:19:20.608 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_bytes": 0, 2024-07-27T03:19:20.608 INFO:teuthology.orchestra.run.smithi067.stdout: "unittest_2_items": 0 2024-07-27T03:19:20.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-07-27T03:19:20.608 INFO:teuthology.orchestra.run.smithi067.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-27T03:19:20.609 INFO:teuthology.orchestra.run.smithi067.stdout: "val": 0, 2024-07-27T03:19:20.609 INFO:teuthology.orchestra.run.smithi067.stdout: "max": 104857600, 2024-07-27T03:19:20.609 INFO:teuthology.orchestra.run.smithi067.stdout: "get_started": 0, 2024-07-27T03:19:20.609 INFO:teuthology.orchestra.run.smithi067.stdout: "get": 126, 2024-07-27T03:19:20.609 INFO:teuthology.orchestra.run.smithi067.stdout: "get_sum": 473245, 2024-07-27T03:19:20.609 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:20.610 INFO:teuthology.orchestra.run.smithi067.stdout: "get_or_fail_success": 126, 2024-07-27T03:19:20.610 INFO:teuthology.orchestra.run.smithi067.stdout: "take": 0, 2024-07-27T03:19:20.610 INFO:teuthology.orchestra.run.smithi067.stdout: "take_sum": 0, 2024-07-27T03:19:20.610 INFO:teuthology.orchestra.run.smithi067.stdout: "put": 126, 2024-07-27T03:19:20.610 INFO:teuthology.orchestra.run.smithi067.stdout: "put_sum": 473245, 2024-07-27T03:19:20.611 INFO:teuthology.orchestra.run.smithi067.stdout: "wait": { 2024-07-27T03:19:20.611 INFO:teuthology.orchestra.run.smithi067.stdout: "avgcount": 0, 2024-07-27T03:19:20.611 INFO:teuthology.orchestra.run.smithi067.stdout: "sum": 0.000000000, 2024-07-27T03:19:20.611 INFO:teuthology.orchestra.run.smithi067.stdout: "avgtime": 0.000000000 2024-07-27T03:19:20.611 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:20.611 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-07-27T03:19:20.612 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-07-27T03:19:20.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:20 smithi067 ceph-mon[32814]: pgmap v513: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 8.0 MiB/s wr, 17 op/s 2024-07-27T03:19:20.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:20 smithi149 ceph-mon[37173]: pgmap v513: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 8.0 MiB/s wr, 17 op/s 2024-07-27T03:19:21.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:20 smithi175 ceph-mon[37125]: pgmap v513: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 8.0 MiB/s wr, 17 op/s 2024-07-27T03:19:21.200 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-07-27T03:19:21.200 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-07-27T03:19:21.200 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-07-27T03:19:21.200 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-07-27T03:19:21.200 DEBUG:tasks.check_counter:Getting stats from b 2024-07-27T03:19:21.201 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph daemon mds.b perf dump 2024-07-27T03:19:21.465 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.b/config 2024-07-27T03:19:22.922 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:22 smithi149 ceph-mon[37173]: pgmap v514: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.3 MiB/s wr, 19 op/s 2024-07-27T03:19:22.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:22 smithi067 ceph-mon[32814]: pgmap v514: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.3 MiB/s wr, 19 op/s 2024-07-27T03:19:23.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:22 smithi175 ceph-mon[37125]: pgmap v514: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.3 MiB/s wr, 19 op/s 2024-07-27T03:19:23.368 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-07-27T03:19:23.368 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-07-27T03:19:23.368 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 1069, 2024-07-27T03:19:23.368 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 821, 2024-07-27T03:19:23.368 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 375466, 2024-07-27T03:19:23.369 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 953628, 2024-07-27T03:19:23.369 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 16, 2024-07-27T03:19:23.369 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 4, 2024-07-27T03:19:23.369 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.136620571, 2024-07-27T03:19:23.369 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.050453680, 2024-07-27T03:19:23.369 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.060055850, 2024-07-27T03:19:23.369 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.028022229, 2024-07-27T03:19:23.369 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:23.369 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 822, 2024-07-27T03:19:23.369 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.545265931, 2024-07-27T03:19:23.369 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000663340 2024-07-27T03:19:23.370 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.370 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:23.370 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1101, 2024-07-27T03:19:23.370 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000248504, 2024-07-27T03:19:23.370 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000225 2024-07-27T03:19:23.370 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.370 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 182528, 2024-07-27T03:19:23.370 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 95776 2024-07-27T03:19:23.370 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.370 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-07-27T03:19:23.370 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 761, 2024-07-27T03:19:23.370 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 713, 2024-07-27T03:19:23.371 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 700944, 2024-07-27T03:19:23.371 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 262043, 2024-07-27T03:19:23.371 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 9, 2024-07-27T03:19:23.371 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 7, 2024-07-27T03:19:23.371 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.080888287, 2024-07-27T03:19:23.371 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.035666671, 2024-07-27T03:19:23.371 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.052093777, 2024-07-27T03:19:23.371 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.007676754, 2024-07-27T03:19:23.371 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:23.371 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 713, 2024-07-27T03:19:23.371 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.077588011, 2024-07-27T03:19:23.371 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000108819 2024-07-27T03:19:23.371 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.372 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:23.372 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 477, 2024-07-27T03:19:23.372 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000141559, 2024-07-27T03:19:23.372 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000296 2024-07-27T03:19:23.372 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.372 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 566064, 2024-07-27T03:19:23.372 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 84016 2024-07-27T03:19:23.372 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.372 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-07-27T03:19:23.372 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 742, 2024-07-27T03:19:23.372 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 777, 2024-07-27T03:19:23.372 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 184599, 2024-07-27T03:19:23.373 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 799922, 2024-07-27T03:19:23.373 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 39, 2024-07-27T03:19:23.373 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 7, 2024-07-27T03:19:23.373 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.095362439, 2024-07-27T03:19:23.373 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.042672782, 2024-07-27T03:19:23.373 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.053144362, 2024-07-27T03:19:23.373 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.005807291, 2024-07-27T03:19:23.373 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:23.373 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 772, 2024-07-27T03:19:23.373 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.091605139, 2024-07-27T03:19:23.373 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000118659 2024-07-27T03:19:23.373 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.373 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:23.374 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1024, 2024-07-27T03:19:23.374 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000236924, 2024-07-27T03:19:23.374 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000231 2024-07-27T03:19:23.374 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.374 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 160, 2024-07-27T03:19:23.374 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 238240 2024-07-27T03:19:23.374 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.374 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-07-27T03:19:23.374 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-07-27T03:19:23.374 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-07-27T03:19:23.374 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.374 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-07-27T03:19:23.375 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-07-27T03:19:23.375 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-07-27T03:19:23.375 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 365, 2024-07-27T03:19:23.375 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.106422707, 2024-07-27T03:19:23.375 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000291569 2024-07-27T03:19:23.375 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:23.375 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.375 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-07-27T03:19:23.375 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-07-27T03:19:23.375 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-07-27T03:19:23.375 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 88, 2024-07-27T03:19:23.375 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.003651414, 2024-07-27T03:19:23.376 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000041493 2024-07-27T03:19:23.376 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:23.376 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.376 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-07-27T03:19:23.376 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 66, 2024-07-27T03:19:23.376 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 66, 2024-07-27T03:19:23.376 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-07-27T03:19:23.376 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 66, 2024-07-27T03:19:23.376 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.081958773, 2024-07-27T03:19:23.376 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001241799 2024-07-27T03:19:23.376 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.376 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-07-27T03:19:23.377 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 0, 2024-07-27T03:19:23.377 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-07-27T03:19:23.377 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-07-27T03:19:23.377 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 12, 2024-07-27T03:19:23.377 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-07-27T03:19:23.377 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-07-27T03:19:23.377 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 24, 2024-07-27T03:19:23.377 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 0, 2024-07-27T03:19:23.377 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 24, 2024-07-27T03:19:23.377 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 0, 2024-07-27T03:19:23.377 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 23, 2024-07-27T03:19:23.377 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 0, 2024-07-27T03:19:23.377 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 2, 2024-07-27T03:19:23.378 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 2, 2024-07-27T03:19:23.378 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 2, 2024-07-27T03:19:23.378 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 98, 2024-07-27T03:19:23.378 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 89, 2024-07-27T03:19:23.378 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2024-07-27T03:19:23.378 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 0, 2024-07-27T03:19:23.378 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-07-27T03:19:23.378 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-07-27T03:19:23.378 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-07-27T03:19:23.378 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 25, 2024-07-27T03:19:23.378 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-07-27T03:19:23.378 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2024-07-27T03:19:23.379 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2024-07-27T03:19:23.379 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 0, 2024-07-27T03:19:23.379 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 0, 2024-07-27T03:19:23.379 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-07-27T03:19:23.379 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 58, 2024-07-27T03:19:23.379 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-07-27T03:19:23.379 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 4, 2024-07-27T03:19:23.379 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 234, 2024-07-27T03:19:23.379 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 0, 2024-07-27T03:19:23.379 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 58, 2024-07-27T03:19:23.379 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-07-27T03:19:23.379 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-07-27T03:19:23.380 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-07-27T03:19:23.380 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 34, 2024-07-27T03:19:23.380 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 4, 2024-07-27T03:19:23.380 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 30, 2024-07-27T03:19:23.380 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 24, 2024-07-27T03:19:23.380 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-07-27T03:19:23.380 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 14, 2024-07-27T03:19:23.380 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 32, 2024-07-27T03:19:23.380 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-07-27T03:19:23.380 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-27T03:19:23.381 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-27T03:19:23.381 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 100, 2024-07-27T03:19:23.381 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 80, 2024-07-27T03:19:23.381 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 2, 2024-07-27T03:19:23.381 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 26 2024-07-27T03:19:23.381 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.381 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-07-27T03:19:23.381 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 17, 2024-07-27T03:19:23.381 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 0, 2024-07-27T03:19:23.381 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 0, 2024-07-27T03:19:23.381 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 0, 2024-07-27T03:19:23.381 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 32, 2024-07-27T03:19:23.381 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2024-07-27T03:19:23.382 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-07-27T03:19:23.382 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-07-27T03:19:23.382 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 6, 2024-07-27T03:19:23.382 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 6, 2024-07-27T03:19:23.382 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-07-27T03:19:23.382 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-07-27T03:19:23.382 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-07-27T03:19:23.382 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-07-27T03:19:23.382 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-07-27T03:19:23.382 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-07-27T03:19:23.382 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-07-27T03:19:23.382 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_quiesce_path": 0, 2024-07-27T03:19:23.383 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_quiesce_inode": 0, 2024-07-27T03:19:23.383 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 4, 2024-07-27T03:19:23.383 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2024-07-27T03:19:23.383 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-07-27T03:19:23.383 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-07-27T03:19:23.383 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-07-27T03:19:23.383 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-07-27T03:19:23.383 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.383 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-07-27T03:19:23.383 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-07-27T03:19:23.383 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 155, 2024-07-27T03:19:23.383 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-07-27T03:19:23.383 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-07-27T03:19:23.384 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 155, 2024-07-27T03:19:23.384 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-07-27T03:19:23.384 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-07-27T03:19:23.384 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 2, 2024-07-27T03:19:23.384 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-07-27T03:19:23.384 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-07-27T03:19:23.384 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 2, 2024-07-27T03:19:23.384 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 2, 2024-07-27T03:19:23.384 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-07-27T03:19:23.384 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-07-27T03:19:23.384 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 4194304, 2024-07-27T03:19:23.384 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 4799922, 2024-07-27T03:19:23.384 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 0, 2024-07-27T03:19:23.385 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-07-27T03:19:23.385 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 130, 2024-07-27T03:19:23.385 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.288071180, 2024-07-27T03:19:23.385 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002215932 2024-07-27T03:19:23.385 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.385 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 0 2024-07-27T03:19:23.385 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.385 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-07-27T03:19:23.385 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 23, 2024-07-27T03:19:23.385 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 33, 2024-07-27T03:19:23.385 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 10, 2024-07-27T03:19:23.385 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 18, 2024-07-27T03:19:23.385 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 20, 2024-07-27T03:19:23.385 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 2, 2024-07-27T03:19:23.386 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 24, 2024-07-27T03:19:23.386 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 35, 2024-07-27T03:19:23.386 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 11, 2024-07-27T03:19:23.386 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 2, 2024-07-27T03:19:23.386 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 20, 2024-07-27T03:19:23.386 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 18, 2024-07-27T03:19:23.386 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 121492, 2024-07-27T03:19:23.386 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 231732 2024-07-27T03:19:23.386 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.386 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-07-27T03:19:23.386 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 89, 2024-07-27T03:19:23.386 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2024-07-27T03:19:23.386 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 66, 2024-07-27T03:19:23.387 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 14, 2024-07-27T03:19:23.387 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 0, 2024-07-27T03:19:23.387 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-07-27T03:19:23.387 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 8, 2024-07-27T03:19:23.387 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.006436710, 2024-07-27T03:19:23.387 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000804588 2024-07-27T03:19:23.387 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.387 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-07-27T03:19:23.387 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 28, 2024-07-27T03:19:23.387 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.008493239, 2024-07-27T03:19:23.387 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000303329 2024-07-27T03:19:23.387 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.387 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-07-27T03:19:23.388 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.388 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.388 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.388 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.388 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-07-27T03:19:23.388 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.388 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.388 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.388 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.388 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-07-27T03:19:23.388 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.388 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.389 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.389 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.389 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-07-27T03:19:23.389 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.389 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.389 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.389 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.389 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-07-27T03:19:23.389 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1, 2024-07-27T03:19:23.389 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000275965, 2024-07-27T03:19:23.389 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000275965 2024-07-27T03:19:23.389 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.389 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-07-27T03:19:23.390 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.390 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.390 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.390 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.390 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-07-27T03:19:23.390 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.390 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.390 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.390 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.390 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-07-27T03:19:23.390 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.390 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.391 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.391 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.391 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-07-27T03:19:23.391 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.391 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.391 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.391 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.392 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-07-27T03:19:23.392 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 8, 2024-07-27T03:19:23.392 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.013514513, 2024-07-27T03:19:23.392 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001689314 2024-07-27T03:19:23.392 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.392 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-07-27T03:19:23.392 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.392 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.392 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.392 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.392 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-07-27T03:19:23.393 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.393 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.393 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.393 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.393 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-07-27T03:19:23.393 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1, 2024-07-27T03:19:23.393 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000208308, 2024-07-27T03:19:23.393 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000208308 2024-07-27T03:19:23.393 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.393 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-07-27T03:19:23.393 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2, 2024-07-27T03:19:23.393 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.004699346, 2024-07-27T03:19:23.393 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002349673 2024-07-27T03:19:23.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.394 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-07-27T03:19:23.394 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2, 2024-07-27T03:19:23.394 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.002182994, 2024-07-27T03:19:23.394 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001091497 2024-07-27T03:19:23.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.394 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-07-27T03:19:23.394 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.394 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.394 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.394 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-07-27T03:19:23.395 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.395 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.395 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.395 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-07-27T03:19:23.395 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2, 2024-07-27T03:19:23.395 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.015157612, 2024-07-27T03:19:23.395 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.007578806 2024-07-27T03:19:23.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.395 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-07-27T03:19:23.395 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.395 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.395 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.396 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.396 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-07-27T03:19:23.396 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.396 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.396 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.396 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.396 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-07-27T03:19:23.396 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2, 2024-07-27T03:19:23.396 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.001251320, 2024-07-27T03:19:23.396 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000625660 2024-07-27T03:19:23.397 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.397 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-07-27T03:19:23.397 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.397 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.397 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.397 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.397 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-07-27T03:19:23.398 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.398 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.398 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.398 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.398 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-07-27T03:19:23.398 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.398 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.398 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.398 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.398 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-07-27T03:19:23.398 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 8, 2024-07-27T03:19:23.398 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.023204133, 2024-07-27T03:19:23.399 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002900516 2024-07-27T03:19:23.399 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.399 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-07-27T03:19:23.399 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.399 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.399 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.399 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.399 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-07-27T03:19:23.399 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4, 2024-07-27T03:19:23.399 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.006534633, 2024-07-27T03:19:23.399 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001633658 2024-07-27T03:19:23.399 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.399 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-07-27T03:19:23.399 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-07-27T03:19:23.400 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-07-27T03:19:23.400 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.400 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.400 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.400 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:23.400 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.400 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-07-27T03:19:23.400 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 1, 2024-07-27T03:19:23.400 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 3, 2024-07-27T03:19:23.400 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 2, 2024-07-27T03:19:23.400 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 1, 2024-07-27T03:19:23.400 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-07-27T03:19:23.400 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 13, 2024-07-27T03:19:23.400 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 13, 2024-07-27T03:19:23.401 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 161, 2024-07-27T03:19:23.401 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-07-27T03:19:23.401 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.401 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-07-27T03:19:23.401 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-07-27T03:19:23.401 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-07-27T03:19:23.401 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-07-27T03:19:23.401 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-07-27T03:19:23.401 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-07-27T03:19:23.401 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-07-27T03:19:23.401 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-27T03:19:23.401 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-07-27T03:19:23.402 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-27T03:19:23.402 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-07-27T03:19:23.402 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-07-27T03:19:23.402 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-07-27T03:19:23.402 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-27T03:19:23.402 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-07-27T03:19:23.402 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-07-27T03:19:23.402 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-07-27T03:19:23.402 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-07-27T03:19:23.402 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-07-27T03:19:23.402 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-27T03:19:23.402 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-07-27T03:19:23.402 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-27T03:19:23.403 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-07-27T03:19:23.403 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-07-27T03:19:23.403 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-07-27T03:19:23.403 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-07-27T03:19:23.403 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-07-27T03:19:23.403 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-27T03:19:23.403 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-07-27T03:19:23.403 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-07-27T03:19:23.403 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-07-27T03:19:23.403 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-07-27T03:19:23.403 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-07-27T03:19:23.403 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-07-27T03:19:23.403 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-07-27T03:19:23.404 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-07-27T03:19:23.404 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-07-27T03:19:23.404 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 316419, 2024-07-27T03:19:23.404 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 133, 2024-07-27T03:19:23.404 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 528, 2024-07-27T03:19:23.404 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 6, 2024-07-27T03:19:23.404 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-07-27T03:19:23.404 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-07-27T03:19:23.404 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-07-27T03:19:23.404 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-07-27T03:19:23.404 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-07-27T03:19:23.404 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-07-27T03:19:23.404 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 8952, 2024-07-27T03:19:23.404 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 168, 2024-07-27T03:19:23.404 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-07-27T03:19:23.405 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-07-27T03:19:23.405 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-07-27T03:19:23.405 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-07-27T03:19:23.405 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 97594, 2024-07-27T03:19:23.405 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 705, 2024-07-27T03:19:23.405 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-07-27T03:19:23.405 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-07-27T03:19:23.405 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-07-27T03:19:23.405 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-07-27T03:19:23.405 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.405 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-07-27T03:19:23.405 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-07-27T03:19:23.405 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-07-27T03:19:23.405 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 853, 2024-07-27T03:19:23.405 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 622923, 2024-07-27T03:19:23.405 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 37, 2024-07-27T03:19:23.406 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 853, 2024-07-27T03:19:23.406 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-07-27T03:19:23.406 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 853, 2024-07-27T03:19:23.406 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 19.764602191, 2024-07-27T03:19:23.406 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.023170694 2024-07-27T03:19:23.406 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.406 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-07-27T03:19:23.406 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-07-27T03:19:23.406 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 853, 2024-07-27T03:19:23.406 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 898 2024-07-27T03:19:23.406 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.406 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 853, 2024-07-27T03:19:23.406 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 68, 2024-07-27T03:19:23.406 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 785, 2024-07-27T03:19:23.407 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-07-27T03:19:23.407 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-07-27T03:19:23.407 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 0, 2024-07-27T03:19:23.407 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 11, 2024-07-27T03:19:23.407 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 6, 2024-07-27T03:19:23.407 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 132, 2024-07-27T03:19:23.407 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 23, 2024-07-27T03:19:23.407 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-07-27T03:19:23.407 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-07-27T03:19:23.407 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-07-27T03:19:23.407 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-07-27T03:19:23.407 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 585, 2024-07-27T03:19:23.407 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-07-27T03:19:23.407 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-07-27T03:19:23.407 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-07-27T03:19:23.407 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 62, 2024-07-27T03:19:23.408 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 22, 2024-07-27T03:19:23.408 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-07-27T03:19:23.408 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-07-27T03:19:23.408 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-07-27T03:19:23.408 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-07-27T03:19:23.408 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-07-27T03:19:23.408 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-07-27T03:19:23.408 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-07-27T03:19:23.408 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-07-27T03:19:23.408 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-07-27T03:19:23.408 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-07-27T03:19:23.408 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-07-27T03:19:23.408 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-07-27T03:19:23.408 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-07-27T03:19:23.408 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-07-27T03:19:23.409 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-07-27T03:19:23.409 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-07-27T03:19:23.409 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-07-27T03:19:23.409 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-07-27T03:19:23.409 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-07-27T03:19:23.409 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-07-27T03:19:23.409 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-07-27T03:19:23.409 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-07-27T03:19:23.409 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-07-27T03:19:23.409 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-07-27T03:19:23.409 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-07-27T03:19:23.409 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-07-27T03:19:23.409 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 91, 2024-07-27T03:19:23.410 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-07-27T03:19:23.410 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 11, 2024-07-27T03:19:23.410 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 12, 2024-07-27T03:19:23.410 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 12, 2024-07-27T03:19:23.410 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-07-27T03:19:23.410 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-07-27T03:19:23.410 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 47, 2024-07-27T03:19:23.410 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 0, 2024-07-27T03:19:23.410 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 10 2024-07-27T03:19:23.410 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.410 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-07-27T03:19:23.410 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 1, 2024-07-27T03:19:23.410 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-07-27T03:19:23.411 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-07-27T03:19:23.411 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-07-27T03:19:23.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.411 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-07-27T03:19:23.411 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-07-27T03:19:23.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.411 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-07-27T03:19:23.411 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-07-27T03:19:23.411 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 65, 2024-07-27T03:19:23.411 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-07-27T03:19:23.411 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 5, 2024-07-27T03:19:23.411 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 73, 2024-07-27T03:19:23.411 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 6, 2024-07-27T03:19:23.412 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-07-27T03:19:23.412 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.412 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-27T03:19:23.412 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-07-27T03:19:23.412 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-07-27T03:19:23.412 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-07-27T03:19:23.412 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 2568, 2024-07-27T03:19:23.412 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 997397, 2024-07-27T03:19:23.412 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:23.412 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 2568, 2024-07-27T03:19:23.412 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-07-27T03:19:23.412 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-07-27T03:19:23.412 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 2568, 2024-07-27T03:19:23.412 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 997397, 2024-07-27T03:19:23.412 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-07-27T03:19:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-07-27T03:19:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-07-27T03:19:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-07-27T03:19:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-07-27T03:19:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-07-27T03:19:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-07-27T03:19:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 853, 2024-07-27T03:19:23.413 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 623561, 2024-07-27T03:19:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 853, 2024-07-27T03:19:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 623561, 2024-07-27T03:19:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-07-27T03:19:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-07-27T03:19:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-07-27T03:19:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-07-27T03:19:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-07-27T03:19:23.414 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-07-27T03:19:23.415 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-07-27T03:19:23.415 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:23.415 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:23.415 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 853, 2024-07-27T03:19:23.415 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 853, 2024-07-27T03:19:23.415 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 853, 2024-07-27T03:19:23.415 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 853, 2024-07-27T03:19:23.415 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-07-27T03:19:23.415 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.415 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.415 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.415 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:23.415 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.415 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-07-27T03:19:23.416 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-07-27T03:19:23.416 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-07-27T03:19:23.416 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-07-27T03:19:23.416 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 6, 2024-07-27T03:19:23.416 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 638, 2024-07-27T03:19:23.416 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:23.416 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 6, 2024-07-27T03:19:23.416 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-07-27T03:19:23.416 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-07-27T03:19:23.416 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 2, 2024-07-27T03:19:23.416 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 638, 2024-07-27T03:19:23.416 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-07-27T03:19:23.416 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.416 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.416 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.417 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:23.417 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:23.417 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x55704654c820": { 2024-07-27T03:19:23.417 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-07-27T03:19:23.417 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-07-27T03:19:23.417 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-07-27T03:19:23.417 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 155, 2024-07-27T03:19:23.417 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 605618, 2024-07-27T03:19:23.417 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:23.417 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 155, 2024-07-27T03:19:23.417 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-07-27T03:19:23.417 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-07-27T03:19:23.417 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 130, 2024-07-27T03:19:23.417 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 605618, 2024-07-27T03:19:23.418 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-07-27T03:19:23.418 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:23.418 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:23.418 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:23.418 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:23.418 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:23.418 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-07-27T03:19:24.074 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-07-27T03:19:24.074 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-07-27T03:19:24.074 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=17 2024-07-27T03:19:24.074 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-07-27T03:19:24.074 DEBUG:tasks.check_counter:Getting stats from e 2024-07-27T03:19:24.074 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph daemon mds.e perf dump 2024-07-27T03:19:24.336 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.b/config 2024-07-27T03:19:24.715 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:24 smithi149 ceph-mon[37173]: pgmap v515: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 MiB/s wr, 13 op/s 2024-07-27T03:19:24.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:24 smithi067 ceph-mon[32814]: pgmap v515: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 MiB/s wr, 13 op/s 2024-07-27T03:19:25.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:24 smithi175 ceph-mon[37125]: pgmap v515: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 MiB/s wr, 13 op/s 2024-07-27T03:19:25.971 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-07-27T03:19:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-07-27T03:19:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 38, 2024-07-27T03:19:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 44, 2024-07-27T03:19:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 15413, 2024-07-27T03:19:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 163133, 2024-07-27T03:19:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 23, 2024-07-27T03:19:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 8, 2024-07-27T03:19:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.012392758, 2024-07-27T03:19:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.001446258, 2024-07-27T03:19:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.008256518, 2024-07-27T03:19:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000301929, 2024-07-27T03:19:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 38, 2024-07-27T03:19:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.013832005, 2024-07-27T03:19:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000364000 2024-07-27T03:19:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 38, 2024-07-27T03:19:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000007268, 2024-07-27T03:19:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000191 2024-07-27T03:19:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-27T03:19:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 0 2024-07-27T03:19:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-07-27T03:19:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 255, 2024-07-27T03:19:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 370, 2024-07-27T03:19:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 88156, 2024-07-27T03:19:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 296321, 2024-07-27T03:19:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 8, 2024-07-27T03:19:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 4, 2024-07-27T03:19:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.047923710, 2024-07-27T03:19:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.026229045, 2024-07-27T03:19:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.019183255, 2024-07-27T03:19:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.001812166, 2024-07-27T03:19:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 372, 2024-07-27T03:19:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.036948966, 2024-07-27T03:19:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000099325 2024-07-27T03:19:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 502, 2024-07-27T03:19:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000115722, 2024-07-27T03:19:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000230 2024-07-27T03:19:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 160, 2024-07-27T03:19:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 216400 2024-07-27T03:19:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-07-27T03:19:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 200, 2024-07-27T03:19:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 191, 2024-07-27T03:19:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 552024, 2024-07-27T03:19:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 98812, 2024-07-27T03:19:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 7, 2024-07-27T03:19:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 5, 2024-07-27T03:19:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.032940383, 2024-07-27T03:19:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.013190815, 2024-07-27T03:19:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.027181860, 2024-07-27T03:19:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000450691, 2024-07-27T03:19:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 191, 2024-07-27T03:19:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.033289190, 2024-07-27T03:19:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000174288 2024-07-27T03:19:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 28, 2024-07-27T03:19:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000005829, 2024-07-27T03:19:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000208 2024-07-27T03:19:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 537216, 2024-07-27T03:19:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 80720 2024-07-27T03:19:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-07-27T03:19:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-07-27T03:19:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-07-27T03:19:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-07-27T03:19:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-07-27T03:19:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-07-27T03:19:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 32, 2024-07-27T03:19:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.001249725, 2024-07-27T03:19:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000039053 2024-07-27T03:19:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-07-27T03:19:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-07-27T03:19:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-07-27T03:19:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2, 2024-07-27T03:19:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000012719, 2024-07-27T03:19:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000006359 2024-07-27T03:19:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-07-27T03:19:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 0, 2024-07-27T03:19:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 0, 2024-07-27T03:19:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-07-27T03:19:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-07-27T03:19:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 0, 2024-07-27T03:19:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-07-27T03:19:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-07-27T03:19:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 11, 2024-07-27T03:19:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-07-27T03:19:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-07-27T03:19:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 16, 2024-07-27T03:19:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 0, 2024-07-27T03:19:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 16, 2024-07-27T03:19:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 0, 2024-07-27T03:19:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 13, 2024-07-27T03:19:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 4, 2024-07-27T03:19:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 0, 2024-07-27T03:19:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 0, 2024-07-27T03:19:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 3, 2024-07-27T03:19:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 7, 2024-07-27T03:19:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 4, 2024-07-27T03:19:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2024-07-27T03:19:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 3, 2024-07-27T03:19:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-07-27T03:19:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-07-27T03:19:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-07-27T03:19:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-07-27T03:19:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-07-27T03:19:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2024-07-27T03:19:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2024-07-27T03:19:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 0, 2024-07-27T03:19:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 0, 2024-07-27T03:19:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-07-27T03:19:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2024-07-27T03:19:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-07-27T03:19:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 7, 2024-07-27T03:19:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 269487290, 2024-07-27T03:19:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 0, 2024-07-27T03:19:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2024-07-27T03:19:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-07-27T03:19:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-07-27T03:19:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-07-27T03:19:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2024-07-27T03:19:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-07-27T03:19:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2024-07-27T03:19:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2024-07-27T03:19:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-07-27T03:19:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 0, 2024-07-27T03:19:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 0, 2024-07-27T03:19:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-07-27T03:19:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-27T03:19:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-27T03:19:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 0, 2024-07-27T03:19:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 0, 2024-07-27T03:19:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 0, 2024-07-27T03:19:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 0 2024-07-27T03:19:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-07-27T03:19:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2024-07-27T03:19:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 11, 2024-07-27T03:19:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 7, 2024-07-27T03:19:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 8, 2024-07-27T03:19:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 0, 2024-07-27T03:19:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2024-07-27T03:19:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-07-27T03:19:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-07-27T03:19:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 0, 2024-07-27T03:19:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 0, 2024-07-27T03:19:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-07-27T03:19:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-07-27T03:19:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-07-27T03:19:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-07-27T03:19:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-07-27T03:19:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-07-27T03:19:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-07-27T03:19:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_quiesce_path": 0, 2024-07-27T03:19:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_quiesce_inode": 0, 2024-07-27T03:19:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-07-27T03:19:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2024-07-27T03:19:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-07-27T03:19:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-07-27T03:19:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-07-27T03:19:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-07-27T03:19:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-07-27T03:19:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-07-27T03:19:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 2, 2024-07-27T03:19:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-07-27T03:19:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-07-27T03:19:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 2, 2024-07-27T03:19:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-07-27T03:19:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-07-27T03:19:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 2, 2024-07-27T03:19:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-07-27T03:19:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-07-27T03:19:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 2, 2024-07-27T03:19:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 2, 2024-07-27T03:19:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-07-27T03:19:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-07-27T03:19:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 4194304, 2024-07-27T03:19:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 4194892, 2024-07-27T03:19:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 0, 2024-07-27T03:19:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-07-27T03:19:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1, 2024-07-27T03:19:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.001328285, 2024-07-27T03:19:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001328285 2024-07-27T03:19:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 0 2024-07-27T03:19:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-07-27T03:19:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 20, 2024-07-27T03:19:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 22, 2024-07-27T03:19:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 2, 2024-07-27T03:19:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 19, 2024-07-27T03:19:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 23, 2024-07-27T03:19:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 4, 2024-07-27T03:19:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 16, 2024-07-27T03:19:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 20, 2024-07-27T03:19:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 4, 2024-07-27T03:19:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 0, 2024-07-27T03:19:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 0, 2024-07-27T03:19:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 0, 2024-07-27T03:19:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 119832, 2024-07-27T03:19:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 223540 2024-07-27T03:19:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-07-27T03:19:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 0, 2024-07-27T03:19:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2024-07-27T03:19:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 0, 2024-07-27T03:19:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 0, 2024-07-27T03:19:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 0, 2024-07-27T03:19:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-07-27T03:19:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-07-27T03:19:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-07-27T03:19:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-07-27T03:19:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-07-27T03:19:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-07-27T03:19:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-07-27T03:19:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-07-27T03:19:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-07-27T03:19:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-07-27T03:19:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-07-27T03:19:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-07-27T03:19:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-07-27T03:19:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-07-27T03:19:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-07-27T03:19:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-07-27T03:19:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-07-27T03:19:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-07-27T03:19:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-07-27T03:19:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-07-27T03:19:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-07-27T03:19:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-07-27T03:19:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-07-27T03:19:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-07-27T03:19:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-07-27T03:19:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-07-27T03:19:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-07-27T03:19:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-07-27T03:19:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-07-27T03:19:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-07-27T03:19:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-07-27T03:19:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-07-27T03:19:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-07-27T03:19:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 0, 2024-07-27T03:19:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 0, 2024-07-27T03:19:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 0, 2024-07-27T03:19:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 0, 2024-07-27T03:19:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-07-27T03:19:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 0, 2024-07-27T03:19:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 0, 2024-07-27T03:19:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 0, 2024-07-27T03:19:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-07-27T03:19:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-07-27T03:19:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-07-27T03:19:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-07-27T03:19:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-07-27T03:19:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-07-27T03:19:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-07-27T03:19:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-07-27T03:19:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-27T03:19:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-07-27T03:19:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-27T03:19:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-07-27T03:19:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-07-27T03:19:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-07-27T03:19:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-27T03:19:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-07-27T03:19:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-07-27T03:19:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-07-27T03:19:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-07-27T03:19:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-07-27T03:19:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-27T03:19:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-07-27T03:19:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-27T03:19:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-07-27T03:19:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-07-27T03:19:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-07-27T03:19:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-07-27T03:19:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-07-27T03:19:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-27T03:19:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-07-27T03:19:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-07-27T03:19:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-07-27T03:19:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-07-27T03:19:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-07-27T03:19:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-07-27T03:19:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-07-27T03:19:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-07-27T03:19:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-07-27T03:19:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 246951, 2024-07-27T03:19:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 82, 2024-07-27T03:19:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-07-27T03:19:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-07-27T03:19:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-07-27T03:19:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-07-27T03:19:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-07-27T03:19:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-07-27T03:19:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-07-27T03:19:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-07-27T03:19:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 8952, 2024-07-27T03:19:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 168, 2024-07-27T03:19:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-07-27T03:19:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-07-27T03:19:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-07-27T03:19:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-07-27T03:19:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 80909, 2024-07-27T03:19:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 320, 2024-07-27T03:19:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-07-27T03:19:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-07-27T03:19:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-07-27T03:19:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-07-27T03:19:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-07-27T03:19:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-07-27T03:19:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-07-27T03:19:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 35, 2024-07-27T03:19:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 10710, 2024-07-27T03:19:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 33, 2024-07-27T03:19:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 35, 2024-07-27T03:19:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-07-27T03:19:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 35, 2024-07-27T03:19:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.215995647, 2024-07-27T03:19:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.006171304 2024-07-27T03:19:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-07-27T03:19:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-07-27T03:19:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 35, 2024-07-27T03:19:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 57 2024-07-27T03:19:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 35, 2024-07-27T03:19:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 0, 2024-07-27T03:19:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 35, 2024-07-27T03:19:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-07-27T03:19:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-07-27T03:19:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 0, 2024-07-27T03:19:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 10, 2024-07-27T03:19:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 0, 2024-07-27T03:19:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 1, 2024-07-27T03:19:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 5, 2024-07-27T03:19:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-07-27T03:19:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-07-27T03:19:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-07-27T03:19:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-07-27T03:19:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 6, 2024-07-27T03:19:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-07-27T03:19:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-07-27T03:19:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-07-27T03:19:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 0, 2024-07-27T03:19:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 20, 2024-07-27T03:19:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-07-27T03:19:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-07-27T03:19:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-07-27T03:19:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-07-27T03:19:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-07-27T03:19:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-07-27T03:19:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-07-27T03:19:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-07-27T03:19:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-07-27T03:19:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-07-27T03:19:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-07-27T03:19:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-07-27T03:19:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-07-27T03:19:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-07-27T03:19:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-07-27T03:19:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-07-27T03:19:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-07-27T03:19:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-07-27T03:19:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-07-27T03:19:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-07-27T03:19:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-07-27T03:19:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-07-27T03:19:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-07-27T03:19:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-07-27T03:19:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-07-27T03:19:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-07-27T03:19:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 91, 2024-07-27T03:19:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-07-27T03:19:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 2, 2024-07-27T03:19:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 9, 2024-07-27T03:19:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 9, 2024-07-27T03:19:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-07-27T03:19:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-07-27T03:19:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 14, 2024-07-27T03:19:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 0, 2024-07-27T03:19:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 1 2024-07-27T03:19:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-07-27T03:19:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 1, 2024-07-27T03:19:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-07-27T03:19:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-07-27T03:19:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-07-27T03:19:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-07-27T03:19:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-07-27T03:19:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-07-27T03:19:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-07-27T03:19:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 0, 2024-07-27T03:19:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-07-27T03:19:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 0, 2024-07-27T03:19:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 0, 2024-07-27T03:19:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 0, 2024-07-27T03:19:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-07-27T03:19:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-27T03:19:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-07-27T03:19:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-07-27T03:19:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-07-27T03:19:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 489, 2024-07-27T03:19:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 600252, 2024-07-27T03:19:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 489, 2024-07-27T03:19:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-07-27T03:19:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-07-27T03:19:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 489, 2024-07-27T03:19:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 600252, 2024-07-27T03:19:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-07-27T03:19:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-07-27T03:19:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-07-27T03:19:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-07-27T03:19:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-07-27T03:19:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-07-27T03:19:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-07-27T03:19:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 35, 2024-07-27T03:19:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 10710, 2024-07-27T03:19:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 35, 2024-07-27T03:19:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 10710, 2024-07-27T03:19:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-07-27T03:19:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-07-27T03:19:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-07-27T03:19:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-07-27T03:19:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-07-27T03:19:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-07-27T03:19:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-07-27T03:19:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 35, 2024-07-27T03:19:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 35, 2024-07-27T03:19:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 35, 2024-07-27T03:19:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 35, 2024-07-27T03:19:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-07-27T03:19:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-07-27T03:19:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-07-27T03:19:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-07-27T03:19:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-07-27T03:19:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-07-27T03:19:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-07-27T03:19:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-07-27T03:19:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-07-27T03:19:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-07-27T03:19:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-07-27T03:19:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-07-27T03:19:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x55d5b27f8820": { 2024-07-27T03:19:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-07-27T03:19:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-07-27T03:19:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-07-27T03:19:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 2, 2024-07-27T03:19:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 588, 2024-07-27T03:19:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 2, 2024-07-27T03:19:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-07-27T03:19:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-07-27T03:19:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 1, 2024-07-27T03:19:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 588, 2024-07-27T03:19:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-07-27T03:19:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:26.018 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-07-27T03:19:26.855 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-07-27T03:19:26.855 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-07-27T03:19:26.855 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-07-27T03:19:26.855 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=11 2024-07-27T03:19:26.856 DEBUG:tasks.check_counter:Getting stats from h 2024-07-27T03:19:26.856 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph daemon mds.h perf dump 2024-07-27T03:19:26.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:26 smithi067 ceph-mon[32814]: pgmap v516: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 853 B/s wr, 14 op/s 2024-07-27T03:19:26.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:26 smithi149 ceph-mon[37173]: pgmap v516: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 853 B/s wr, 14 op/s 2024-07-27T03:19:27.108 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.b/config 2024-07-27T03:19:27.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:26 smithi175 ceph-mon[37125]: pgmap v516: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 853 B/s wr, 14 op/s 2024-07-27T03:19:28.850 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:28 smithi149 ceph-mon[37173]: pgmap v517: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-07-27T03:19:28.870 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-07-27T03:19:28.871 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-07-27T03:19:28.871 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 285, 2024-07-27T03:19:28.871 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 316, 2024-07-27T03:19:28.871 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 94391, 2024-07-27T03:19:28.871 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 217182, 2024-07-27T03:19:28.871 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 18, 2024-07-27T03:19:28.871 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 8, 2024-07-27T03:19:28.871 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.052184373, 2024-07-27T03:19:28.871 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.023185491, 2024-07-27T03:19:28.871 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.024030951, 2024-07-27T03:19:28.872 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.003056634, 2024-07-27T03:19:28.872 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:28.872 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 315, 2024-07-27T03:19:28.872 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.038737064, 2024-07-27T03:19:28.872 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000122974 2024-07-27T03:19:28.872 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.872 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:28.872 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 529, 2024-07-27T03:19:28.872 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000148869, 2024-07-27T03:19:28.872 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000281 2024-07-27T03:19:28.872 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.872 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-07-27T03:19:28.872 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 448 2024-07-27T03:19:28.873 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.873 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-07-27T03:19:28.873 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 198, 2024-07-27T03:19:28.873 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 185, 2024-07-27T03:19:28.873 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 563671, 2024-07-27T03:19:28.873 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 96419, 2024-07-27T03:19:28.873 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 9, 2024-07-27T03:19:28.873 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 6, 2024-07-27T03:19:28.873 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.032796810, 2024-07-27T03:19:28.873 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.014752653, 2024-07-27T03:19:28.873 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.023844528, 2024-07-27T03:19:28.873 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000324187, 2024-07-27T03:19:28.873 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:28.874 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 184, 2024-07-27T03:19:28.874 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.031569699, 2024-07-27T03:19:28.874 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000171574 2024-07-27T03:19:28.874 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.874 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:28.874 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 28, 2024-07-27T03:19:28.874 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000006556, 2024-07-27T03:19:28.874 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000234 2024-07-27T03:19:28.874 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.874 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 549856, 2024-07-27T03:19:28.874 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 80272 2024-07-27T03:19:28.874 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.875 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-07-27T03:19:28.875 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 35, 2024-07-27T03:19:28.875 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 126, 2024-07-27T03:19:28.875 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 15282, 2024-07-27T03:19:28.875 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 290270, 2024-07-27T03:19:28.875 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 14, 2024-07-27T03:19:28.875 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 6, 2024-07-27T03:19:28.875 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.019660866, 2024-07-27T03:19:28.875 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.010012271, 2024-07-27T03:19:28.875 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.008335289, 2024-07-27T03:19:28.875 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000320109, 2024-07-27T03:19:28.875 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:28.875 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 124, 2024-07-27T03:19:28.875 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.017772502, 2024-07-27T03:19:28.876 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000143326 2024-07-27T03:19:28.876 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.876 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:28.876 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 45, 2024-07-27T03:19:28.876 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000011852, 2024-07-27T03:19:28.876 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000263 2024-07-27T03:19:28.876 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.876 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 2048, 2024-07-27T03:19:28.876 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 221488 2024-07-27T03:19:28.876 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.876 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-07-27T03:19:28.876 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-07-27T03:19:28.876 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-07-27T03:19:28.877 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.877 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-07-27T03:19:28.877 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-07-27T03:19:28.877 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-07-27T03:19:28.877 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 40, 2024-07-27T03:19:28.877 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.001418987, 2024-07-27T03:19:28.877 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000035474 2024-07-27T03:19:28.877 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:28.877 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.877 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-07-27T03:19:28.877 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-07-27T03:19:28.877 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-07-27T03:19:28.877 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2, 2024-07-27T03:19:28.878 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000013668, 2024-07-27T03:19:28.878 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000006834 2024-07-27T03:19:28.878 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:28.878 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.878 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-07-27T03:19:28.878 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 1, 2024-07-27T03:19:28.878 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 1, 2024-07-27T03:19:28.878 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-07-27T03:19:28.878 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1, 2024-07-27T03:19:28.878 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000146531, 2024-07-27T03:19:28.878 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000146531 2024-07-27T03:19:28.878 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.878 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-07-27T03:19:28.879 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 0, 2024-07-27T03:19:28.879 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-07-27T03:19:28.879 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-07-27T03:19:28.879 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 11, 2024-07-27T03:19:28.879 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-07-27T03:19:28.879 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-07-27T03:19:28.879 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 16, 2024-07-27T03:19:28.879 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 0, 2024-07-27T03:19:28.879 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 16, 2024-07-27T03:19:28.879 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 0, 2024-07-27T03:19:28.879 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 13, 2024-07-27T03:19:28.879 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 4, 2024-07-27T03:19:28.879 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 1, 2024-07-27T03:19:28.880 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 1, 2024-07-27T03:19:28.880 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 3, 2024-07-27T03:19:28.880 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 7, 2024-07-27T03:19:28.880 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 4, 2024-07-27T03:19:28.880 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2024-07-27T03:19:28.880 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 3, 2024-07-27T03:19:28.880 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-07-27T03:19:28.880 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-07-27T03:19:28.880 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-07-27T03:19:28.880 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-07-27T03:19:28.880 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-07-27T03:19:28.880 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2024-07-27T03:19:28.880 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2024-07-27T03:19:28.880 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 0, 2024-07-27T03:19:28.881 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 0, 2024-07-27T03:19:28.881 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-07-27T03:19:28.881 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2024-07-27T03:19:28.881 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-07-27T03:19:28.881 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 7, 2024-07-27T03:19:28.881 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 269487290, 2024-07-27T03:19:28.881 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 0, 2024-07-27T03:19:28.881 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2024-07-27T03:19:28.881 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-07-27T03:19:28.881 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-07-27T03:19:28.881 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-07-27T03:19:28.881 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2024-07-27T03:19:28.882 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-07-27T03:19:28.882 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2024-07-27T03:19:28.882 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2024-07-27T03:19:28.882 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-07-27T03:19:28.882 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 0, 2024-07-27T03:19:28.882 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 0, 2024-07-27T03:19:28.882 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-07-27T03:19:28.882 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-27T03:19:28.882 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-27T03:19:28.882 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 0, 2024-07-27T03:19:28.882 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 0, 2024-07-27T03:19:28.883 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 0, 2024-07-27T03:19:28.883 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 0 2024-07-27T03:19:28.883 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.883 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-07-27T03:19:28.883 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2024-07-27T03:19:28.883 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 11, 2024-07-27T03:19:28.883 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 7, 2024-07-27T03:19:28.883 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 8, 2024-07-27T03:19:28.883 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 0, 2024-07-27T03:19:28.883 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2024-07-27T03:19:28.883 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-07-27T03:19:28.883 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-07-27T03:19:28.883 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 0, 2024-07-27T03:19:28.884 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 0, 2024-07-27T03:19:28.884 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-07-27T03:19:28.884 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-07-27T03:19:28.884 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-07-27T03:19:28.884 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-07-27T03:19:28.884 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-07-27T03:19:28.884 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-07-27T03:19:28.884 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-07-27T03:19:28.884 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_quiesce_path": 0, 2024-07-27T03:19:28.884 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_quiesce_inode": 0, 2024-07-27T03:19:28.884 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-07-27T03:19:28.884 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2024-07-27T03:19:28.885 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-07-27T03:19:28.885 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-07-27T03:19:28.885 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-07-27T03:19:28.885 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-07-27T03:19:28.885 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.885 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-07-27T03:19:28.885 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-07-27T03:19:28.885 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 5, 2024-07-27T03:19:28.885 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-07-27T03:19:28.885 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-07-27T03:19:28.885 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 5, 2024-07-27T03:19:28.885 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-07-27T03:19:28.885 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-07-27T03:19:28.885 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 2, 2024-07-27T03:19:28.886 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-07-27T03:19:28.886 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-07-27T03:19:28.886 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 2, 2024-07-27T03:19:28.886 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 2, 2024-07-27T03:19:28.886 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-07-27T03:19:28.886 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-07-27T03:19:28.886 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 4194304, 2024-07-27T03:19:28.886 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 4195732, 2024-07-27T03:19:28.886 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 0, 2024-07-27T03:19:28.886 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-07-27T03:19:28.886 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4, 2024-07-27T03:19:28.886 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.008711823, 2024-07-27T03:19:28.886 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002177955 2024-07-27T03:19:28.887 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.887 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 0 2024-07-27T03:19:28.887 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.887 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-07-27T03:19:28.887 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 20, 2024-07-27T03:19:28.887 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 22, 2024-07-27T03:19:28.887 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 2, 2024-07-27T03:19:28.887 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 19, 2024-07-27T03:19:28.887 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 23, 2024-07-27T03:19:28.887 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 4, 2024-07-27T03:19:28.887 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 16, 2024-07-27T03:19:28.887 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 20, 2024-07-27T03:19:28.887 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 4, 2024-07-27T03:19:28.888 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 1, 2024-07-27T03:19:28.888 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 1, 2024-07-27T03:19:28.888 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 0, 2024-07-27T03:19:28.888 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 117120, 2024-07-27T03:19:28.888 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 223540 2024-07-27T03:19:28.888 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.888 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-07-27T03:19:28.888 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 1, 2024-07-27T03:19:28.888 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2024-07-27T03:19:28.888 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 1, 2024-07-27T03:19:28.888 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 7, 2024-07-27T03:19:28.888 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 0, 2024-07-27T03:19:28.889 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-07-27T03:19:28.889 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.889 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.889 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.889 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.889 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-07-27T03:19:28.889 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.889 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.889 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.889 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.889 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-07-27T03:19:28.889 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.890 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.890 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.890 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.890 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-07-27T03:19:28.890 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.890 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.890 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.890 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.890 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-07-27T03:19:28.890 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.890 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.891 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.891 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.891 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-07-27T03:19:28.891 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.891 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.891 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.891 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.891 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-07-27T03:19:28.891 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1, 2024-07-27T03:19:28.891 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000146531, 2024-07-27T03:19:28.891 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000146531 2024-07-27T03:19:28.891 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.891 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-07-27T03:19:28.892 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.892 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.892 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.892 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.892 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-07-27T03:19:28.892 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.892 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.892 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.892 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.892 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-07-27T03:19:28.892 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.892 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.893 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.893 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.893 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-07-27T03:19:28.893 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.893 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.893 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.893 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.893 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-07-27T03:19:28.893 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.894 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.894 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.894 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.894 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-07-27T03:19:28.894 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.894 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.894 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.894 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.899 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-07-27T03:19:28.899 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.899 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.899 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.899 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.899 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-07-27T03:19:28.899 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.899 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.899 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.900 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.900 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-07-27T03:19:28.900 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.900 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.900 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.900 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.900 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-07-27T03:19:28.900 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.900 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.900 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.900 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.901 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-07-27T03:19:28.901 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.901 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.901 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.901 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.901 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-07-27T03:19:28.901 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.901 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.901 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.901 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.901 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-07-27T03:19:28.901 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.901 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.902 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.902 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.902 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-07-27T03:19:28.902 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.902 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.902 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.902 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.902 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-07-27T03:19:28.902 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.902 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.902 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.902 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.903 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-07-27T03:19:28.903 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.903 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.903 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.903 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.903 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-07-27T03:19:28.903 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.903 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.903 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.903 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.903 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-07-27T03:19:28.903 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.903 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.904 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.904 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.904 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-07-27T03:19:28.904 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.904 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.904 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.904 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.904 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-07-27T03:19:28.904 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.904 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.904 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.904 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.904 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-07-27T03:19:28.905 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.905 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.905 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.905 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.905 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-07-27T03:19:28.905 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.905 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.905 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.905 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.905 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-07-27T03:19:28.905 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-07-27T03:19:28.905 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-07-27T03:19:28.905 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.905 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.906 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.906 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:28.906 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.906 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-07-27T03:19:28.906 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 1, 2024-07-27T03:19:28.906 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 2, 2024-07-27T03:19:28.906 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 1, 2024-07-27T03:19:28.906 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 1, 2024-07-27T03:19:28.906 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-07-27T03:19:28.906 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 1, 2024-07-27T03:19:28.906 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 1, 2024-07-27T03:19:28.906 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 155, 2024-07-27T03:19:28.906 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-07-27T03:19:28.907 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.907 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-07-27T03:19:28.907 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-07-27T03:19:28.907 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-07-27T03:19:28.907 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-07-27T03:19:28.907 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-07-27T03:19:28.907 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-07-27T03:19:28.907 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-07-27T03:19:28.907 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-27T03:19:28.907 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-07-27T03:19:28.907 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-27T03:19:28.907 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-07-27T03:19:28.908 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-07-27T03:19:28.908 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-07-27T03:19:28.908 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-27T03:19:28.908 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-07-27T03:19:28.908 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-07-27T03:19:28.908 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-07-27T03:19:28.908 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-07-27T03:19:28.908 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-07-27T03:19:28.908 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-27T03:19:28.908 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-07-27T03:19:28.908 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-27T03:19:28.908 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-07-27T03:19:28.908 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-07-27T03:19:28.909 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-07-27T03:19:28.909 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-07-27T03:19:28.909 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-07-27T03:19:28.909 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-27T03:19:28.909 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-07-27T03:19:28.909 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-07-27T03:19:28.909 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-07-27T03:19:28.909 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-07-27T03:19:28.909 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-07-27T03:19:28.909 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-07-27T03:19:28.909 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-07-27T03:19:28.909 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-07-27T03:19:28.910 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-07-27T03:19:28.910 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 279687, 2024-07-27T03:19:28.910 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 101, 2024-07-27T03:19:28.910 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-07-27T03:19:28.910 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-07-27T03:19:28.910 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-07-27T03:19:28.910 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-07-27T03:19:28.910 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-07-27T03:19:28.910 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-07-27T03:19:28.910 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-07-27T03:19:28.910 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-07-27T03:19:28.910 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 8952, 2024-07-27T03:19:28.910 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 168, 2024-07-27T03:19:28.911 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-07-27T03:19:28.911 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-07-27T03:19:28.911 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-07-27T03:19:28.911 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-07-27T03:19:28.911 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 81285, 2024-07-27T03:19:28.911 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 323, 2024-07-27T03:19:28.911 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-07-27T03:19:28.911 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-07-27T03:19:28.911 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-07-27T03:19:28.911 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-07-27T03:19:28.911 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.911 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-07-27T03:19:28.911 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-07-27T03:19:28.911 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-07-27T03:19:28.911 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 40, 2024-07-27T03:19:28.912 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 11730, 2024-07-27T03:19:28.912 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 33, 2024-07-27T03:19:28.912 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 40, 2024-07-27T03:19:28.912 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-07-27T03:19:28.912 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 40, 2024-07-27T03:19:28.912 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.245995046, 2024-07-27T03:19:28.912 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.006149876 2024-07-27T03:19:28.912 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.912 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-07-27T03:19:28.912 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-07-27T03:19:28.912 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 40, 2024-07-27T03:19:28.912 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 62 2024-07-27T03:19:28.912 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.913 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 40, 2024-07-27T03:19:28.913 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 0, 2024-07-27T03:19:28.913 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 40, 2024-07-27T03:19:28.913 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-07-27T03:19:28.913 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-07-27T03:19:28.913 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 0, 2024-07-27T03:19:28.913 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 10, 2024-07-27T03:19:28.913 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 0, 2024-07-27T03:19:28.913 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 4, 2024-07-27T03:19:28.913 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 7, 2024-07-27T03:19:28.913 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-07-27T03:19:28.913 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-07-27T03:19:28.913 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-07-27T03:19:28.913 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-07-27T03:19:28.914 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 6, 2024-07-27T03:19:28.914 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-07-27T03:19:28.914 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-07-27T03:19:28.914 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-07-27T03:19:28.914 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 0, 2024-07-27T03:19:28.914 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 20, 2024-07-27T03:19:28.914 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-07-27T03:19:28.914 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-07-27T03:19:28.914 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-07-27T03:19:28.914 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-07-27T03:19:28.914 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-07-27T03:19:28.914 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-07-27T03:19:28.915 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-07-27T03:19:28.915 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-07-27T03:19:28.915 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-07-27T03:19:28.915 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-07-27T03:19:28.915 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-07-27T03:19:28.915 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-07-27T03:19:28.915 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-07-27T03:19:28.915 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-07-27T03:19:28.915 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-07-27T03:19:28.915 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-07-27T03:19:28.915 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-07-27T03:19:28.915 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-07-27T03:19:28.915 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-07-27T03:19:28.916 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-07-27T03:19:28.916 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-07-27T03:19:28.916 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-07-27T03:19:28.916 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-07-27T03:19:28.916 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-07-27T03:19:28.916 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-07-27T03:19:28.916 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-07-27T03:19:28.916 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 91, 2024-07-27T03:19:28.916 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-07-27T03:19:28.916 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 2, 2024-07-27T03:19:28.916 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 10, 2024-07-27T03:19:28.916 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 10, 2024-07-27T03:19:28.916 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-07-27T03:19:28.916 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-07-27T03:19:28.917 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 14, 2024-07-27T03:19:28.917 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 0, 2024-07-27T03:19:28.917 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 1 2024-07-27T03:19:28.917 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.917 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-07-27T03:19:28.917 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 1, 2024-07-27T03:19:28.917 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-07-27T03:19:28.917 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-07-27T03:19:28.917 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-07-27T03:19:28.917 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.917 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-07-27T03:19:28.917 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-07-27T03:19:28.917 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.918 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-07-27T03:19:28.918 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-07-27T03:19:28.918 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 0, 2024-07-27T03:19:28.918 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-07-27T03:19:28.918 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 0, 2024-07-27T03:19:28.918 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 0, 2024-07-27T03:19:28.918 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 0, 2024-07-27T03:19:28.918 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-07-27T03:19:28.918 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.918 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-27T03:19:28.918 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-07-27T03:19:28.918 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-07-27T03:19:28.919 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-07-27T03:19:28.919 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 514, 2024-07-27T03:19:28.919 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 615134, 2024-07-27T03:19:28.919 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:28.919 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 514, 2024-07-27T03:19:28.919 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-07-27T03:19:28.919 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-07-27T03:19:28.919 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 514, 2024-07-27T03:19:28.919 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 615134, 2024-07-27T03:19:28.919 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-07-27T03:19:28.919 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.919 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.919 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.920 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:28.920 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.920 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-07-27T03:19:28.920 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-07-27T03:19:28.920 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-07-27T03:19:28.920 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-07-27T03:19:28.920 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-07-27T03:19:28.920 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-07-27T03:19:28.920 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:28.920 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:28.920 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 40, 2024-07-27T03:19:28.920 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 11730, 2024-07-27T03:19:28.920 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 40, 2024-07-27T03:19:28.920 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 11730, 2024-07-27T03:19:28.921 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-07-27T03:19:28.921 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.921 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.921 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.921 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:28.921 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.921 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-07-27T03:19:28.921 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-07-27T03:19:28.921 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-07-27T03:19:28.921 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-07-27T03:19:28.921 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-07-27T03:19:28.921 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-07-27T03:19:28.921 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:28.922 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:28.922 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 40, 2024-07-27T03:19:28.922 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 40, 2024-07-27T03:19:28.922 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 40, 2024-07-27T03:19:28.922 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 40, 2024-07-27T03:19:28.922 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-07-27T03:19:28.922 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.922 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.922 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.922 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:28.922 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.922 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-07-27T03:19:28.922 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-07-27T03:19:28.923 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-07-27T03:19:28.923 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-07-27T03:19:28.923 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-07-27T03:19:28.923 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-07-27T03:19:28.923 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:28.923 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:28.923 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-07-27T03:19:28.923 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-07-27T03:19:28.923 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-07-27T03:19:28.923 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-07-27T03:19:28.923 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-07-27T03:19:28.923 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.923 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.923 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.924 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:28.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:28.924 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x55e08a2cc680": { 2024-07-27T03:19:28.924 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-07-27T03:19:28.924 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-07-27T03:19:28.924 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-07-27T03:19:28.924 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 5, 2024-07-27T03:19:28.924 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 1428, 2024-07-27T03:19:28.924 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:28.924 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 5, 2024-07-27T03:19:28.924 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-07-27T03:19:28.924 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-07-27T03:19:28.924 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 4, 2024-07-27T03:19:28.924 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 1428, 2024-07-27T03:19:28.924 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-07-27T03:19:28.925 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:28.925 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:28.925 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:28.925 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:28.925 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:28.925 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-07-27T03:19:28.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:28 smithi067 ceph-mon[32814]: pgmap v517: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-07-27T03:19:29.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:28 smithi175 ceph-mon[37125]: pgmap v517: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-07-27T03:19:29.687 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-07-27T03:19:29.687 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-07-27T03:19:29.688 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-07-27T03:19:29.688 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=11 2024-07-27T03:19:29.688 DEBUG:tasks.check_counter:Getting stats from k 2024-07-27T03:19:29.688 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph daemon mds.k perf dump 2024-07-27T03:19:29.942 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.b/config 2024-07-27T03:19:30.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:30 smithi067 ceph-mon[32814]: pgmap v518: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-07-27T03:19:31.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:30 smithi149 ceph-mon[37173]: pgmap v518: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-07-27T03:19:31.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:30 smithi175 ceph-mon[37125]: pgmap v518: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 9 op/s 2024-07-27T03:19:31.633 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-07-27T03:19:31.634 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-07-27T03:19:31.634 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 958, 2024-07-27T03:19:31.634 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 958, 2024-07-27T03:19:31.634 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 276933, 2024-07-27T03:19:31.634 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 303827, 2024-07-27T03:19:31.634 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 6, 2024-07-27T03:19:31.634 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 2, 2024-07-27T03:19:31.634 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.101556434, 2024-07-27T03:19:31.634 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.046433086, 2024-07-27T03:19:31.634 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.021457178, 2024-07-27T03:19:31.635 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.026947003, 2024-07-27T03:19:31.635 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:31.635 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 957, 2024-07-27T03:19:31.635 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.037609471, 2024-07-27T03:19:31.635 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000039299 2024-07-27T03:19:31.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.635 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:31.635 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.635 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.635 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.635 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-07-27T03:19:31.635 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 448 2024-07-27T03:19:31.636 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.636 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-07-27T03:19:31.636 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 140, 2024-07-27T03:19:31.636 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 135, 2024-07-27T03:19:31.636 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 523404, 2024-07-27T03:19:31.636 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 78730, 2024-07-27T03:19:31.636 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 3, 2024-07-27T03:19:31.636 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 2, 2024-07-27T03:19:31.636 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.027019485, 2024-07-27T03:19:31.636 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.012521053, 2024-07-27T03:19:31.636 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.019726897, 2024-07-27T03:19:31.636 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000056193, 2024-07-27T03:19:31.637 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:31.637 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 134, 2024-07-27T03:19:31.637 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.009219728, 2024-07-27T03:19:31.637 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000068803 2024-07-27T03:19:31.637 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.637 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:31.637 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.637 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.637 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.637 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.637 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 522848, 2024-07-27T03:19:31.637 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 78032 2024-07-27T03:19:31.637 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.638 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-07-27T03:19:31.638 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 2, 2024-07-27T03:19:31.638 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 96, 2024-07-27T03:19:31.638 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 428, 2024-07-27T03:19:31.638 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 218862, 2024-07-27T03:19:31.638 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 4, 2024-07-27T03:19:31.638 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 3, 2024-07-27T03:19:31.638 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.007571678, 2024-07-27T03:19:31.638 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.005597613, 2024-07-27T03:19:31.638 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.000976532, 2024-07-27T03:19:31.638 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000021959, 2024-07-27T03:19:31.638 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:31.638 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 94, 2024-07-27T03:19:31.639 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.005976427, 2024-07-27T03:19:31.639 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000063579 2024-07-27T03:19:31.639 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.639 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:31.639 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.639 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.639 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.639 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.639 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 160, 2024-07-27T03:19:31.639 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 218544 2024-07-27T03:19:31.639 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.639 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-07-27T03:19:31.640 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-07-27T03:19:31.640 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-07-27T03:19:31.640 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.640 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-07-27T03:19:31.640 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-07-27T03:19:31.640 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-07-27T03:19:31.640 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1659, 2024-07-27T03:19:31.640 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.046141057, 2024-07-27T03:19:31.640 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000027812 2024-07-27T03:19:31.640 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:31.640 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.640 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-07-27T03:19:31.641 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-07-27T03:19:31.641 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-07-27T03:19:31.641 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.641 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.641 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.641 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:31.641 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.641 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-07-27T03:19:31.641 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 0, 2024-07-27T03:19:31.641 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 0, 2024-07-27T03:19:31.642 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-07-27T03:19:31.642 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.642 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.642 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.642 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.642 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-07-27T03:19:31.642 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 0, 2024-07-27T03:19:31.642 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-07-27T03:19:31.642 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-07-27T03:19:31.642 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 0, 2024-07-27T03:19:31.642 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-07-27T03:19:31.642 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-07-27T03:19:31.642 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 0, 2024-07-27T03:19:31.643 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 0, 2024-07-27T03:19:31.643 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 0, 2024-07-27T03:19:31.643 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 0, 2024-07-27T03:19:31.643 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 0, 2024-07-27T03:19:31.643 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 0, 2024-07-27T03:19:31.643 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 0, 2024-07-27T03:19:31.643 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 0, 2024-07-27T03:19:31.643 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 1, 2024-07-27T03:19:31.643 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 0, 2024-07-27T03:19:31.643 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 0, 2024-07-27T03:19:31.643 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2024-07-27T03:19:31.643 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 0, 2024-07-27T03:19:31.643 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-07-27T03:19:31.644 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-07-27T03:19:31.644 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-07-27T03:19:31.644 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-07-27T03:19:31.644 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-07-27T03:19:31.644 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2024-07-27T03:19:31.644 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2024-07-27T03:19:31.644 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 0, 2024-07-27T03:19:31.644 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 0, 2024-07-27T03:19:31.644 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-07-27T03:19:31.644 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2024-07-27T03:19:31.644 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-07-27T03:19:31.644 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 0, 2024-07-27T03:19:31.644 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 0, 2024-07-27T03:19:31.644 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 0, 2024-07-27T03:19:31.645 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2024-07-27T03:19:31.645 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-07-27T03:19:31.645 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-07-27T03:19:31.645 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-07-27T03:19:31.645 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2024-07-27T03:19:31.645 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-07-27T03:19:31.645 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2024-07-27T03:19:31.645 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2024-07-27T03:19:31.645 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-07-27T03:19:31.645 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 0, 2024-07-27T03:19:31.645 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 0, 2024-07-27T03:19:31.645 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-07-27T03:19:31.645 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-27T03:19:31.645 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-27T03:19:31.646 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 0, 2024-07-27T03:19:31.646 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 0, 2024-07-27T03:19:31.646 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 0, 2024-07-27T03:19:31.646 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 0 2024-07-27T03:19:31.646 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.646 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-07-27T03:19:31.646 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2024-07-27T03:19:31.646 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 0, 2024-07-27T03:19:31.646 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 0, 2024-07-27T03:19:31.646 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 0, 2024-07-27T03:19:31.646 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 0, 2024-07-27T03:19:31.646 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2024-07-27T03:19:31.647 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-07-27T03:19:31.647 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-07-27T03:19:31.647 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 0, 2024-07-27T03:19:31.647 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 0, 2024-07-27T03:19:31.647 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-07-27T03:19:31.647 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-07-27T03:19:31.647 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-07-27T03:19:31.647 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-07-27T03:19:31.647 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-07-27T03:19:31.647 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-07-27T03:19:31.647 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-07-27T03:19:31.647 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_quiesce_path": 0, 2024-07-27T03:19:31.647 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_quiesce_inode": 0, 2024-07-27T03:19:31.647 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-07-27T03:19:31.648 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2024-07-27T03:19:31.648 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-07-27T03:19:31.648 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-07-27T03:19:31.648 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-07-27T03:19:31.648 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-07-27T03:19:31.648 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.648 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-07-27T03:19:31.648 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-07-27T03:19:31.648 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 0, 2024-07-27T03:19:31.648 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-07-27T03:19:31.648 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-07-27T03:19:31.648 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 5, 2024-07-27T03:19:31.648 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-07-27T03:19:31.649 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-07-27T03:19:31.649 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 0, 2024-07-27T03:19:31.649 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-07-27T03:19:31.649 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-07-27T03:19:31.649 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 2, 2024-07-27T03:19:31.649 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 2, 2024-07-27T03:19:31.649 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-07-27T03:19:31.649 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-07-27T03:19:31.649 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 4194304, 2024-07-27T03:19:31.649 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 4195732, 2024-07-27T03:19:31.649 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 4195424, 2024-07-27T03:19:31.649 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-07-27T03:19:31.649 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.650 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.650 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.650 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.650 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 5 2024-07-27T03:19:31.650 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.650 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-07-27T03:19:31.650 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 3, 2024-07-27T03:19:31.650 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 3, 2024-07-27T03:19:31.650 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 0, 2024-07-27T03:19:31.650 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 1, 2024-07-27T03:19:31.650 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 1, 2024-07-27T03:19:31.650 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 0, 2024-07-27T03:19:31.650 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 0, 2024-07-27T03:19:31.651 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 0, 2024-07-27T03:19:31.651 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 0, 2024-07-27T03:19:31.651 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 0, 2024-07-27T03:19:31.651 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 0, 2024-07-27T03:19:31.651 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 0, 2024-07-27T03:19:31.651 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 116964, 2024-07-27T03:19:31.651 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 215348 2024-07-27T03:19:31.651 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.651 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-07-27T03:19:31.651 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 0, 2024-07-27T03:19:31.651 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2024-07-27T03:19:31.651 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 0, 2024-07-27T03:19:31.652 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 0, 2024-07-27T03:19:31.652 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 0, 2024-07-27T03:19:31.652 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-07-27T03:19:31.652 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.652 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.652 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.652 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.652 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-07-27T03:19:31.652 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.652 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.652 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.652 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.652 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-07-27T03:19:31.653 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.653 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.653 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.653 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.653 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-07-27T03:19:31.653 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.653 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.653 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.653 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.653 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-07-27T03:19:31.653 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.653 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.653 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.654 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.654 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-07-27T03:19:31.654 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.654 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.654 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.654 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.654 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-07-27T03:19:31.654 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.654 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.654 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.654 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.654 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-07-27T03:19:31.654 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.655 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.655 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.655 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.655 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-07-27T03:19:31.655 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.655 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.655 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.655 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.655 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-07-27T03:19:31.655 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.655 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.655 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.655 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.656 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-07-27T03:19:31.656 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.656 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.656 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.656 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.656 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-07-27T03:19:31.656 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.657 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.657 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.657 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.657 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-07-27T03:19:31.657 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.657 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.657 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.657 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.657 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-07-27T03:19:31.657 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.657 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.657 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.658 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.658 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-07-27T03:19:31.658 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.658 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.658 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.658 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.658 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-07-27T03:19:31.658 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.658 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.658 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.658 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.658 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-07-27T03:19:31.659 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.659 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.659 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.659 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.659 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-07-27T03:19:31.659 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.659 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.659 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.659 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.659 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-07-27T03:19:31.659 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.659 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.659 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.660 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.660 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-07-27T03:19:31.660 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.660 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.660 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.660 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.660 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-07-27T03:19:31.660 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.660 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.660 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.660 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.660 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-07-27T03:19:31.660 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.661 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.661 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.661 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.661 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-07-27T03:19:31.661 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.661 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.661 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.661 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.661 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-07-27T03:19:31.661 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.661 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.661 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.661 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.662 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-07-27T03:19:31.662 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.662 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.662 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.662 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.662 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-07-27T03:19:31.662 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.662 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.662 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.662 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.662 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-07-27T03:19:31.662 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.662 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.663 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.663 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.663 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-07-27T03:19:31.663 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.663 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.663 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.663 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.663 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-07-27T03:19:31.663 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.663 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.663 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.663 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.663 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-07-27T03:19:31.664 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-07-27T03:19:31.664 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-07-27T03:19:31.664 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.664 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.664 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.664 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:31.664 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.664 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-07-27T03:19:31.664 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 1, 2024-07-27T03:19:31.664 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 2, 2024-07-27T03:19:31.664 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 1, 2024-07-27T03:19:31.664 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 1, 2024-07-27T03:19:31.664 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-07-27T03:19:31.664 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 0, 2024-07-27T03:19:31.665 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 0, 2024-07-27T03:19:31.665 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 1625, 2024-07-27T03:19:31.665 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-07-27T03:19:31.665 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.665 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-07-27T03:19:31.665 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-07-27T03:19:31.665 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-07-27T03:19:31.665 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-07-27T03:19:31.665 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-07-27T03:19:31.665 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-07-27T03:19:31.665 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-07-27T03:19:31.665 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-27T03:19:31.665 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-07-27T03:19:31.666 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-27T03:19:31.666 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-07-27T03:19:31.666 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-07-27T03:19:31.666 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-07-27T03:19:31.666 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-27T03:19:31.666 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-07-27T03:19:31.666 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-07-27T03:19:31.666 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-07-27T03:19:31.666 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-07-27T03:19:31.666 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-07-27T03:19:31.666 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-27T03:19:31.666 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-07-27T03:19:31.666 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-27T03:19:31.667 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-07-27T03:19:31.667 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-07-27T03:19:31.667 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-07-27T03:19:31.667 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-07-27T03:19:31.667 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-07-27T03:19:31.667 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-27T03:19:31.667 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-07-27T03:19:31.667 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-07-27T03:19:31.667 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-07-27T03:19:31.667 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-07-27T03:19:31.667 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-07-27T03:19:31.667 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-07-27T03:19:31.667 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-07-27T03:19:31.668 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-07-27T03:19:31.668 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-07-27T03:19:31.668 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 181115, 2024-07-27T03:19:31.668 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 44, 2024-07-27T03:19:31.668 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-07-27T03:19:31.668 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-07-27T03:19:31.668 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-07-27T03:19:31.668 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-07-27T03:19:31.668 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-07-27T03:19:31.668 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-07-27T03:19:31.668 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-07-27T03:19:31.668 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-07-27T03:19:31.669 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 8952, 2024-07-27T03:19:31.669 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 168, 2024-07-27T03:19:31.669 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-07-27T03:19:31.669 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-07-27T03:19:31.669 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-07-27T03:19:31.669 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-07-27T03:19:31.669 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 8512, 2024-07-27T03:19:31.669 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 13, 2024-07-27T03:19:31.669 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-07-27T03:19:31.669 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-07-27T03:19:31.669 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-07-27T03:19:31.669 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-07-27T03:19:31.670 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.670 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-07-27T03:19:31.670 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-07-27T03:19:31.670 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-07-27T03:19:31.670 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 957, 2024-07-27T03:19:31.670 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 21, 2024-07-27T03:19:31.670 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 3, 2024-07-27T03:19:31.670 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 957, 2024-07-27T03:19:31.670 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-07-27T03:19:31.670 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 957, 2024-07-27T03:19:31.670 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.520989476, 2024-07-27T03:19:31.670 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000544398 2024-07-27T03:19:31.670 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.671 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-07-27T03:19:31.671 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-07-27T03:19:31.671 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 957, 2024-07-27T03:19:31.671 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 958 2024-07-27T03:19:31.671 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.671 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 957, 2024-07-27T03:19:31.671 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 957, 2024-07-27T03:19:31.671 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 0, 2024-07-27T03:19:31.671 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-07-27T03:19:31.671 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-07-27T03:19:31.671 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 597, 2024-07-27T03:19:31.671 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 0, 2024-07-27T03:19:31.671 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 358, 2024-07-27T03:19:31.671 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 0, 2024-07-27T03:19:31.672 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 0, 2024-07-27T03:19:31.672 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-07-27T03:19:31.672 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-07-27T03:19:31.672 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-07-27T03:19:31.672 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-07-27T03:19:31.672 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 0, 2024-07-27T03:19:31.672 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-07-27T03:19:31.672 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-07-27T03:19:31.672 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-07-27T03:19:31.672 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 1, 2024-07-27T03:19:31.672 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 0, 2024-07-27T03:19:31.672 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-07-27T03:19:31.672 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-07-27T03:19:31.673 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-07-27T03:19:31.673 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-07-27T03:19:31.673 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-07-27T03:19:31.673 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-07-27T03:19:31.673 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-07-27T03:19:31.673 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-07-27T03:19:31.673 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-07-27T03:19:31.673 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-07-27T03:19:31.673 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-07-27T03:19:31.673 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-07-27T03:19:31.673 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-07-27T03:19:31.673 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-07-27T03:19:31.673 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-07-27T03:19:31.674 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-07-27T03:19:31.674 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-07-27T03:19:31.674 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-07-27T03:19:31.674 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-07-27T03:19:31.674 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-07-27T03:19:31.674 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-07-27T03:19:31.674 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-07-27T03:19:31.674 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-07-27T03:19:31.674 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-07-27T03:19:31.674 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-07-27T03:19:31.674 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-07-27T03:19:31.675 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 91, 2024-07-27T03:19:31.675 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-07-27T03:19:31.675 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 2, 2024-07-27T03:19:31.675 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 4, 2024-07-27T03:19:31.675 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 4, 2024-07-27T03:19:31.675 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-07-27T03:19:31.675 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-07-27T03:19:31.675 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 0, 2024-07-27T03:19:31.675 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 2, 2024-07-27T03:19:31.675 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 0 2024-07-27T03:19:31.675 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.675 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-07-27T03:19:31.675 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 0, 2024-07-27T03:19:31.676 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-07-27T03:19:31.676 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-07-27T03:19:31.676 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-07-27T03:19:31.676 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.676 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-07-27T03:19:31.676 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-07-27T03:19:31.676 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.676 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-07-27T03:19:31.676 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-07-27T03:19:31.676 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 0, 2024-07-27T03:19:31.677 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-07-27T03:19:31.677 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 0, 2024-07-27T03:19:31.677 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 0, 2024-07-27T03:19:31.677 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 0, 2024-07-27T03:19:31.677 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-07-27T03:19:31.677 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.677 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-27T03:19:31.677 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-07-27T03:19:31.677 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-07-27T03:19:31.678 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-07-27T03:19:31.678 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 1096, 2024-07-27T03:19:31.678 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 692058, 2024-07-27T03:19:31.678 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:31.678 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 1096, 2024-07-27T03:19:31.678 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-07-27T03:19:31.678 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-07-27T03:19:31.678 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 1096, 2024-07-27T03:19:31.678 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 692058, 2024-07-27T03:19:31.678 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-07-27T03:19:31.679 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.679 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.679 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.679 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:31.679 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.679 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-07-27T03:19:31.679 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-07-27T03:19:31.679 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-07-27T03:19:31.679 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-07-27T03:19:31.679 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-07-27T03:19:31.679 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-07-27T03:19:31.679 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:31.679 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:31.680 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 957, 2024-07-27T03:19:31.680 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 1433, 2024-07-27T03:19:31.680 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 957, 2024-07-27T03:19:31.680 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 1433, 2024-07-27T03:19:31.680 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-07-27T03:19:31.680 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.680 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.680 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.680 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:31.680 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.681 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-07-27T03:19:31.681 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-07-27T03:19:31.681 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-07-27T03:19:31.681 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-07-27T03:19:31.681 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-07-27T03:19:31.681 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-07-27T03:19:31.681 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:31.681 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:31.682 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 957, 2024-07-27T03:19:31.682 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 957, 2024-07-27T03:19:31.682 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 957, 2024-07-27T03:19:31.682 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 957, 2024-07-27T03:19:31.682 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-07-27T03:19:31.682 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.682 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.682 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.682 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:31.682 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.682 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-07-27T03:19:31.682 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-07-27T03:19:31.682 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-07-27T03:19:31.683 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-07-27T03:19:31.683 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-07-27T03:19:31.683 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-07-27T03:19:31.683 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:31.683 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:31.683 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-07-27T03:19:31.683 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-07-27T03:19:31.683 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-07-27T03:19:31.683 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-07-27T03:19:31.683 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-07-27T03:19:31.683 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.683 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.683 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.684 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:31.684 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-07-27T03:19:31.684 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x5633765428f0": { 2024-07-27T03:19:31.684 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-07-27T03:19:31.684 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-07-27T03:19:31.684 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-07-27T03:19:31.684 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-07-27T03:19:31.684 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-07-27T03:19:31.684 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:31.684 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:31.684 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-07-27T03:19:31.684 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-07-27T03:19:31.684 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-07-27T03:19:31.684 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-07-27T03:19:31.685 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-07-27T03:19:31.685 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-07-27T03:19:31.685 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-07-27T03:19:31.685 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-07-27T03:19:31.685 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:31.685 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-07-27T03:19:31.685 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-07-27T03:19:32.456 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-07-27T03:19:32.457 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-07-27T03:19:32.457 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-07-27T03:19:32.457 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-07-27T03:19:32.457 DEBUG:tasks.check_counter:Getting stats from c 2024-07-27T03:19:32.457 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph daemon mds.c perf dump 2024-07-27T03:19:32.720 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.c/config 2024-07-27T03:19:32.841 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:32 smithi175 ceph-mon[37125]: pgmap v519: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-07-27T03:19:32.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:32 smithi067 ceph-mon[32814]: pgmap v519: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-07-27T03:19:33.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:32 smithi149 ceph-mon[37173]: pgmap v519: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-07-27T03:19:34.464 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-27T03:19:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-07-27T03:19:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 46, 2024-07-27T03:19:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 50, 2024-07-27T03:19:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 19558, 2024-07-27T03:19:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 165796, 2024-07-27T03:19:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 23, 2024-07-27T03:19:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 8, 2024-07-27T03:19:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.014455093, 2024-07-27T03:19:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.001726562, 2024-07-27T03:19:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.010118601, 2024-07-27T03:19:34.465 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000411862, 2024-07-27T03:19:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 45, 2024-07-27T03:19:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.022332323, 2024-07-27T03:19:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000496273 2024-07-27T03:19:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 41, 2024-07-27T03:19:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000023297, 2024-07-27T03:19:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000568 2024-07-27T03:19:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-07-27T03:19:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 448 2024-07-27T03:19:34.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-07-27T03:19:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 262, 2024-07-27T03:19:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 381, 2024-07-27T03:19:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 88844, 2024-07-27T03:19:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 292927, 2024-07-27T03:19:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 8, 2024-07-27T03:19:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 4, 2024-07-27T03:19:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.046130676, 2024-07-27T03:19:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.024626613, 2024-07-27T03:19:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.018041664, 2024-07-27T03:19:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.002520906, 2024-07-27T03:19:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:34.467 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 383, 2024-07-27T03:19:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.038082615, 2024-07-27T03:19:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000099432 2024-07-27T03:19:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 515, 2024-07-27T03:19:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000129531, 2024-07-27T03:19:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000251 2024-07-27T03:19:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 160, 2024-07-27T03:19:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 219808 2024-07-27T03:19:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-07-27T03:19:34.468 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 193, 2024-07-27T03:19:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 184, 2024-07-27T03:19:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 554073, 2024-07-27T03:19:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 101545, 2024-07-27T03:19:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 7, 2024-07-27T03:19:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 5, 2024-07-27T03:19:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.031756135, 2024-07-27T03:19:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.013234638, 2024-07-27T03:19:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.025000010, 2024-07-27T03:19:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000306913, 2024-07-27T03:19:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 183, 2024-07-27T03:19:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.020014755, 2024-07-27T03:19:34.469 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000109370 2024-07-27T03:19:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 26, 2024-07-27T03:19:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000005943, 2024-07-27T03:19:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000228 2024-07-27T03:19:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 541200, 2024-07-27T03:19:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 81552 2024-07-27T03:19:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-07-27T03:19:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2024-07-27T03:19:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-07-27T03:19:34.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2024-07-27T03:19:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-07-27T03:19:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-07-27T03:19:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 38, 2024-07-27T03:19:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.001443165, 2024-07-27T03:19:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000037978 2024-07-27T03:19:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2024-07-27T03:19:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-07-27T03:19:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-07-27T03:19:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2, 2024-07-27T03:19:34.471 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000013561, 2024-07-27T03:19:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000006780 2024-07-27T03:19:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2024-07-27T03:19:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 0, 2024-07-27T03:19:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 0, 2024-07-27T03:19:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2024-07-27T03:19:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2024-07-27T03:19:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 0, 2024-07-27T03:19:34.472 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 0, 2024-07-27T03:19:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 0, 2024-07-27T03:19:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 11, 2024-07-27T03:19:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 0, 2024-07-27T03:19:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 0, 2024-07-27T03:19:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 16, 2024-07-27T03:19:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 0, 2024-07-27T03:19:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 16, 2024-07-27T03:19:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 0, 2024-07-27T03:19:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 13, 2024-07-27T03:19:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 4, 2024-07-27T03:19:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 0, 2024-07-27T03:19:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 0, 2024-07-27T03:19:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 3, 2024-07-27T03:19:34.473 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 8, 2024-07-27T03:19:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 4, 2024-07-27T03:19:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 0, 2024-07-27T03:19:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 4, 2024-07-27T03:19:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2024-07-27T03:19:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2024-07-27T03:19:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 0, 2024-07-27T03:19:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 1000, 2024-07-27T03:19:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 1, 2024-07-27T03:19:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 0, 2024-07-27T03:19:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 0, 2024-07-27T03:19:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 0, 2024-07-27T03:19:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 0, 2024-07-27T03:19:34.474 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2024-07-27T03:19:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 0, 2024-07-27T03:19:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2024-07-27T03:19:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 7, 2024-07-27T03:19:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 269487290, 2024-07-27T03:19:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 0, 2024-07-27T03:19:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 0, 2024-07-27T03:19:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2024-07-27T03:19:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2024-07-27T03:19:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2024-07-27T03:19:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 0, 2024-07-27T03:19:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 0, 2024-07-27T03:19:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 0, 2024-07-27T03:19:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 0, 2024-07-27T03:19:34.475 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 0, 2024-07-27T03:19:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 0, 2024-07-27T03:19:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 0, 2024-07-27T03:19:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2024-07-27T03:19:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-27T03:19:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-27T03:19:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 0, 2024-07-27T03:19:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 0, 2024-07-27T03:19:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 0, 2024-07-27T03:19:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 0 2024-07-27T03:19:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2024-07-27T03:19:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 0, 2024-07-27T03:19:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 13, 2024-07-27T03:19:34.476 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 8, 2024-07-27T03:19:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 8, 2024-07-27T03:19:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 0, 2024-07-27T03:19:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2024-07-27T03:19:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2024-07-27T03:19:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2024-07-27T03:19:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 0, 2024-07-27T03:19:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 0, 2024-07-27T03:19:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2024-07-27T03:19:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2024-07-27T03:19:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2024-07-27T03:19:34.477 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2024-07-27T03:19:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2024-07-27T03:19:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2024-07-27T03:19:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2024-07-27T03:19:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_path": 0, 2024-07-27T03:19:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_inode": 0, 2024-07-27T03:19:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 0, 2024-07-27T03:19:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 0, 2024-07-27T03:19:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2024-07-27T03:19:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 0, 2024-07-27T03:19:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2024-07-27T03:19:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2024-07-27T03:19:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2024-07-27T03:19:34.478 INFO:teuthology.orchestra.run.smithi175.stdout: "evlrg": 0, 2024-07-27T03:19:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 2, 2024-07-27T03:19:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 0, 2024-07-27T03:19:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 0, 2024-07-27T03:19:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 2, 2024-07-27T03:19:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2024-07-27T03:19:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 0, 2024-07-27T03:19:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 2, 2024-07-27T03:19:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 0, 2024-07-27T03:19:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 0, 2024-07-27T03:19:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 2, 2024-07-27T03:19:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "segmjr": 2, 2024-07-27T03:19:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2024-07-27T03:19:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 0, 2024-07-27T03:19:34.479 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 4194304, 2024-07-27T03:19:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 4194892, 2024-07-27T03:19:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 0, 2024-07-27T03:19:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2024-07-27T03:19:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1, 2024-07-27T03:19:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.001571168, 2024-07-27T03:19:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001571168 2024-07-27T03:19:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 0 2024-07-27T03:19:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2024-07-27T03:19:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 20, 2024-07-27T03:19:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 22, 2024-07-27T03:19:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 2, 2024-07-27T03:19:34.480 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 19, 2024-07-27T03:19:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 23, 2024-07-27T03:19:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 4, 2024-07-27T03:19:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 16, 2024-07-27T03:19:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 20, 2024-07-27T03:19:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 4, 2024-07-27T03:19:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 0, 2024-07-27T03:19:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 0, 2024-07-27T03:19:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 0, 2024-07-27T03:19:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 116656, 2024-07-27T03:19:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 223540 2024-07-27T03:19:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2024-07-27T03:19:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 0, 2024-07-27T03:19:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 0, 2024-07-27T03:19:34.481 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 0, 2024-07-27T03:19:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 0, 2024-07-27T03:19:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 0, 2024-07-27T03:19:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2024-07-27T03:19:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2024-07-27T03:19:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.482 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2024-07-27T03:19:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2024-07-27T03:19:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2024-07-27T03:19:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2024-07-27T03:19:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2024-07-27T03:19:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2024-07-27T03:19:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.484 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2024-07-27T03:19:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2024-07-27T03:19:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2024-07-27T03:19:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.485 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2024-07-27T03:19:34.486 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2024-07-27T03:19:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.487 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2024-07-27T03:19:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2024-07-27T03:19:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2024-07-27T03:19:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2024-07-27T03:19:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2024-07-27T03:19:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2024-07-27T03:19:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.489 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2024-07-27T03:19:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2024-07-27T03:19:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2024-07-27T03:19:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.490 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2024-07-27T03:19:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2024-07-27T03:19:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2024-07-27T03:19:34.491 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2024-07-27T03:19:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2024-07-27T03:19:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.492 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2024-07-27T03:19:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2024-07-27T03:19:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2024-07-27T03:19:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2024-07-27T03:19:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2024-07-27T03:19:34.493 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2024-07-27T03:19:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 0, 2024-07-27T03:19:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 0, 2024-07-27T03:19:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 0, 2024-07-27T03:19:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 0, 2024-07-27T03:19:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2024-07-27T03:19:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 0, 2024-07-27T03:19:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 0, 2024-07-27T03:19:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 0, 2024-07-27T03:19:34.494 INFO:teuthology.orchestra.run.smithi175.stdout: "mdthresh_evicted": 0 2024-07-27T03:19:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-07-27T03:19:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-07-27T03:19:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-07-27T03:19:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-07-27T03:19:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-07-27T03:19:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-07-27T03:19:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-07-27T03:19:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-27T03:19:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-07-27T03:19:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-27T03:19:34.495 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-07-27T03:19:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-07-27T03:19:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-07-27T03:19:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-27T03:19:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-07-27T03:19:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-07-27T03:19:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-07-27T03:19:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-07-27T03:19:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-07-27T03:19:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-27T03:19:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-07-27T03:19:34.496 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-27T03:19:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-07-27T03:19:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-07-27T03:19:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-07-27T03:19:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-07-27T03:19:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-07-27T03:19:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-27T03:19:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-07-27T03:19:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-07-27T03:19:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-07-27T03:19:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-07-27T03:19:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-07-27T03:19:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-07-27T03:19:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-07-27T03:19:34.497 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-07-27T03:19:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-07-27T03:19:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 230071, 2024-07-27T03:19:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 74, 2024-07-27T03:19:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2024-07-27T03:19:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2024-07-27T03:19:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-07-27T03:19:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-07-27T03:19:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-07-27T03:19:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-07-27T03:19:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-07-27T03:19:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-07-27T03:19:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 8952, 2024-07-27T03:19:34.498 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 168, 2024-07-27T03:19:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-07-27T03:19:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-07-27T03:19:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-07-27T03:19:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-07-27T03:19:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 80909, 2024-07-27T03:19:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 320, 2024-07-27T03:19:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-07-27T03:19:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-07-27T03:19:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-07-27T03:19:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-07-27T03:19:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2024-07-27T03:19:34.499 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2024-07-27T03:19:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2024-07-27T03:19:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 35, 2024-07-27T03:19:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 10710, 2024-07-27T03:19:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 33, 2024-07-27T03:19:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 35, 2024-07-27T03:19:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2024-07-27T03:19:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 35, 2024-07-27T03:19:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.219996368, 2024-07-27T03:19:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.006285610 2024-07-27T03:19:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2024-07-27T03:19:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2024-07-27T03:19:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 35, 2024-07-27T03:19:34.500 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 57 2024-07-27T03:19:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 35, 2024-07-27T03:19:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 0, 2024-07-27T03:19:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 35, 2024-07-27T03:19:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2024-07-27T03:19:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2024-07-27T03:19:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 0, 2024-07-27T03:19:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 10, 2024-07-27T03:19:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 0, 2024-07-27T03:19:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 1, 2024-07-27T03:19:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 5, 2024-07-27T03:19:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2024-07-27T03:19:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2024-07-27T03:19:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2024-07-27T03:19:34.501 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 0, 2024-07-27T03:19:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 6, 2024-07-27T03:19:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2024-07-27T03:19:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2024-07-27T03:19:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2024-07-27T03:19:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 0, 2024-07-27T03:19:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 20, 2024-07-27T03:19:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2024-07-27T03:19:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2024-07-27T03:19:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2024-07-27T03:19:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2024-07-27T03:19:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2024-07-27T03:19:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2024-07-27T03:19:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2024-07-27T03:19:34.502 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2024-07-27T03:19:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2024-07-27T03:19:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2024-07-27T03:19:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2024-07-27T03:19:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2024-07-27T03:19:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2024-07-27T03:19:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2024-07-27T03:19:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2024-07-27T03:19:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2024-07-27T03:19:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2024-07-27T03:19:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2024-07-27T03:19:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2024-07-27T03:19:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2024-07-27T03:19:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2024-07-27T03:19:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2024-07-27T03:19:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2024-07-27T03:19:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2024-07-27T03:19:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2024-07-27T03:19:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2024-07-27T03:19:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 91, 2024-07-27T03:19:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2024-07-27T03:19:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 2, 2024-07-27T03:19:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 9, 2024-07-27T03:19:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 9, 2024-07-27T03:19:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 0, 2024-07-27T03:19:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2024-07-27T03:19:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 14, 2024-07-27T03:19:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 0, 2024-07-27T03:19:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 1 2024-07-27T03:19:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2024-07-27T03:19:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 1, 2024-07-27T03:19:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2024-07-27T03:19:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2024-07-27T03:19:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2024-07-27T03:19:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2024-07-27T03:19:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 0 2024-07-27T03:19:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2024-07-27T03:19:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2024-07-27T03:19:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 0, 2024-07-27T03:19:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2024-07-27T03:19:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 0, 2024-07-27T03:19:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed_ops": 0, 2024-07-27T03:19:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 0, 2024-07-27T03:19:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2024-07-27T03:19:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-27T03:19:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-27T03:19:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-07-27T03:19:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-27T03:19:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 497, 2024-07-27T03:19:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 606224, 2024-07-27T03:19:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 497, 2024-07-27T03:19:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-07-27T03:19:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-07-27T03:19:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 497, 2024-07-27T03:19:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 606224, 2024-07-27T03:19:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-27T03:19:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2024-07-27T03:19:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-27T03:19:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-07-27T03:19:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-27T03:19:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-07-27T03:19:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-07-27T03:19:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:34.508 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 35, 2024-07-27T03:19:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 10710, 2024-07-27T03:19:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 35, 2024-07-27T03:19:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 10710, 2024-07-27T03:19:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-27T03:19:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2024-07-27T03:19:34.509 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-27T03:19:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2024-07-27T03:19:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-27T03:19:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-07-27T03:19:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-07-27T03:19:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 35, 2024-07-27T03:19:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 35, 2024-07-27T03:19:34.510 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 35, 2024-07-27T03:19:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 35, 2024-07-27T03:19:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-27T03:19:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2024-07-27T03:19:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-27T03:19:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-07-27T03:19:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-27T03:19:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-07-27T03:19:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-07-27T03:19:34.511 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-07-27T03:19:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-07-27T03:19:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-07-27T03:19:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-07-27T03:19:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-27T03:19:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.512 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x55c357b10750": { 2024-07-27T03:19:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-27T03:19:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-07-27T03:19:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-27T03:19:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 2, 2024-07-27T03:19:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 588, 2024-07-27T03:19:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 2, 2024-07-27T03:19:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-07-27T03:19:34.513 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-07-27T03:19:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 1, 2024-07-27T03:19:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 588, 2024-07-27T03:19:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-27T03:19:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:34.514 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:34.514 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-27T03:19:34.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:34 smithi067 ceph-mon[32814]: pgmap v520: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:19:35.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:34 smithi149 ceph-mon[37173]: pgmap v520: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:19:35.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:34 smithi175 ceph-mon[37125]: pgmap v520: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:19:35.294 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-07-27T03:19:35.294 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-07-27T03:19:35.294 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-07-27T03:19:35.295 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=13 2024-07-27T03:19:35.295 DEBUG:tasks.check_counter:Getting stats from f 2024-07-27T03:19:35.295 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph daemon mds.f perf dump 2024-07-27T03:19:35.554 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.c/config 2024-07-27T03:19:36.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:36 smithi067 ceph-mon[32814]: pgmap v521: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:19:37.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:36 smithi149 ceph-mon[37173]: pgmap v521: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:19:37.021 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:36 smithi175 ceph-mon[37125]: pgmap v521: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:19:37.281 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-27T03:19:37.281 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-07-27T03:19:37.281 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 7, 2024-07-27T03:19:37.281 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 7, 2024-07-27T03:19:37.281 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 3787, 2024-07-27T03:19:37.282 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 1408, 2024-07-27T03:19:37.282 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 6, 2024-07-27T03:19:37.282 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 2, 2024-07-27T03:19:37.282 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.004162799, 2024-07-27T03:19:37.282 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.000174420, 2024-07-27T03:19:37.282 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.003100844, 2024-07-27T03:19:37.282 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000078100, 2024-07-27T03:19:37.282 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:37.282 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 6, 2024-07-27T03:19:37.282 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.006322567, 2024-07-27T03:19:37.282 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.001053761 2024-07-27T03:19:37.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.283 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:37.283 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.283 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.283 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.283 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-07-27T03:19:37.283 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 448 2024-07-27T03:19:37.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.283 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-07-27T03:19:37.283 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 797, 2024-07-27T03:19:37.283 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 890, 2024-07-27T03:19:37.283 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 840086, 2024-07-27T03:19:37.284 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 473173, 2024-07-27T03:19:37.284 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 5, 2024-07-27T03:19:37.284 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 4, 2024-07-27T03:19:37.284 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.103207288, 2024-07-27T03:19:37.284 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.046504116, 2024-07-27T03:19:37.284 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.023109149, 2024-07-27T03:19:37.284 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.025607974, 2024-07-27T03:19:37.284 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:37.284 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 888, 2024-07-27T03:19:37.284 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.035244806, 2024-07-27T03:19:37.284 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000039690 2024-07-27T03:19:37.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.285 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:37.285 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.285 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.285 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.285 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 160, 2024-07-27T03:19:37.285 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 219984 2024-07-27T03:19:37.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.285 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-07-27T03:19:37.285 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 497, 2024-07-27T03:19:37.285 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 492, 2024-07-27T03:19:37.285 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 611681, 2024-07-27T03:19:37.286 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 191402, 2024-07-27T03:19:37.286 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 4, 2024-07-27T03:19:37.286 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 3, 2024-07-27T03:19:37.286 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.070325999, 2024-07-27T03:19:37.286 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.032459458, 2024-07-27T03:19:37.286 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.026807715, 2024-07-27T03:19:37.286 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.012950172, 2024-07-27T03:19:37.286 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:37.286 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 491, 2024-07-27T03:19:37.286 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.023520545, 2024-07-27T03:19:37.286 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000047903 2024-07-27T03:19:37.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.287 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:37.287 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.287 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.287 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.287 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 522464, 2024-07-27T03:19:37.287 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 76912 2024-07-27T03:19:37.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.287 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-07-27T03:19:37.287 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2024-07-27T03:19:37.287 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-07-27T03:19:37.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.288 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2024-07-27T03:19:37.288 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-07-27T03:19:37.288 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-07-27T03:19:37.288 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1898, 2024-07-27T03:19:37.288 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.056912434, 2024-07-27T03:19:37.288 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000029985 2024-07-27T03:19:37.288 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:37.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.288 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2024-07-27T03:19:37.288 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-07-27T03:19:37.288 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-07-27T03:19:37.288 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.288 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.289 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.289 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:37.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.289 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2024-07-27T03:19:37.289 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 0, 2024-07-27T03:19:37.289 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 0, 2024-07-27T03:19:37.289 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2024-07-27T03:19:37.289 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.289 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.289 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.289 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2024-07-27T03:19:37.289 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 0, 2024-07-27T03:19:37.289 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 0, 2024-07-27T03:19:37.290 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 0, 2024-07-27T03:19:37.290 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 0, 2024-07-27T03:19:37.290 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 0, 2024-07-27T03:19:37.290 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 0, 2024-07-27T03:19:37.290 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 20, 2024-07-27T03:19:37.290 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 0, 2024-07-27T03:19:37.290 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 20, 2024-07-27T03:19:37.290 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 0, 2024-07-27T03:19:37.290 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 20, 2024-07-27T03:19:37.290 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 0, 2024-07-27T03:19:37.290 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 0, 2024-07-27T03:19:37.290 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 0, 2024-07-27T03:19:37.290 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 2, 2024-07-27T03:19:37.291 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 0, 2024-07-27T03:19:37.291 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 0, 2024-07-27T03:19:37.291 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 0, 2024-07-27T03:19:37.291 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 0, 2024-07-27T03:19:37.291 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2024-07-27T03:19:37.291 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2024-07-27T03:19:37.291 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 0, 2024-07-27T03:19:37.291 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 0, 2024-07-27T03:19:37.291 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 0, 2024-07-27T03:19:37.291 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 0, 2024-07-27T03:19:37.291 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 0, 2024-07-27T03:19:37.291 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 0, 2024-07-27T03:19:37.291 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 0, 2024-07-27T03:19:37.291 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2024-07-27T03:19:37.292 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 0, 2024-07-27T03:19:37.292 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2024-07-27T03:19:37.292 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 4, 2024-07-27T03:19:37.292 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 234, 2024-07-27T03:19:37.292 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 0, 2024-07-27T03:19:37.292 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 0, 2024-07-27T03:19:37.293 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2024-07-27T03:19:37.293 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2024-07-27T03:19:37.293 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2024-07-27T03:19:37.293 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 0, 2024-07-27T03:19:37.293 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 0, 2024-07-27T03:19:37.293 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 0, 2024-07-27T03:19:37.293 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 0, 2024-07-27T03:19:37.293 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 0, 2024-07-27T03:19:37.293 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 0, 2024-07-27T03:19:37.293 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 0, 2024-07-27T03:19:37.293 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2024-07-27T03:19:37.293 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-27T03:19:37.293 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-27T03:19:37.294 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 0, 2024-07-27T03:19:37.294 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 0, 2024-07-27T03:19:37.294 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 0, 2024-07-27T03:19:37.294 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 0 2024-07-27T03:19:37.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.294 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2024-07-27T03:19:37.294 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 0, 2024-07-27T03:19:37.294 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 0, 2024-07-27T03:19:37.294 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 0, 2024-07-27T03:19:37.294 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 0, 2024-07-27T03:19:37.294 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 0, 2024-07-27T03:19:37.294 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2024-07-27T03:19:37.294 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2024-07-27T03:19:37.295 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2024-07-27T03:19:37.295 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 0, 2024-07-27T03:19:37.295 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 0, 2024-07-27T03:19:37.295 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2024-07-27T03:19:37.295 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2024-07-27T03:19:37.295 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2024-07-27T03:19:37.295 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2024-07-27T03:19:37.295 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2024-07-27T03:19:37.295 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2024-07-27T03:19:37.295 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2024-07-27T03:19:37.295 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_path": 0, 2024-07-27T03:19:37.295 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_inode": 0, 2024-07-27T03:19:37.295 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 0, 2024-07-27T03:19:37.295 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 0, 2024-07-27T03:19:37.295 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2024-07-27T03:19:37.296 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 0, 2024-07-27T03:19:37.296 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2024-07-27T03:19:37.296 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2024-07-27T03:19:37.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.296 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2024-07-27T03:19:37.296 INFO:teuthology.orchestra.run.smithi175.stdout: "evlrg": 0, 2024-07-27T03:19:37.296 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 0, 2024-07-27T03:19:37.296 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 0, 2024-07-27T03:19:37.296 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 0, 2024-07-27T03:19:37.296 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 155, 2024-07-27T03:19:37.296 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2024-07-27T03:19:37.296 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 0, 2024-07-27T03:19:37.296 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 0, 2024-07-27T03:19:37.296 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 0, 2024-07-27T03:19:37.297 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 0, 2024-07-27T03:19:37.297 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 2, 2024-07-27T03:19:37.297 INFO:teuthology.orchestra.run.smithi175.stdout: "segmjr": 2, 2024-07-27T03:19:37.297 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2024-07-27T03:19:37.297 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 0, 2024-07-27T03:19:37.297 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 4194304, 2024-07-27T03:19:37.297 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 4799922, 2024-07-27T03:19:37.297 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 4797425, 2024-07-27T03:19:37.297 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2024-07-27T03:19:37.297 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.297 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.297 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.297 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 155 2024-07-27T03:19:37.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.298 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2024-07-27T03:19:37.298 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 14, 2024-07-27T03:19:37.298 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 20, 2024-07-27T03:19:37.298 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 6, 2024-07-27T03:19:37.298 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 9, 2024-07-27T03:19:37.298 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 11, 2024-07-27T03:19:37.298 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 2, 2024-07-27T03:19:37.298 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 20, 2024-07-27T03:19:37.298 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 25, 2024-07-27T03:19:37.298 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 5, 2024-07-27T03:19:37.298 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 0, 2024-07-27T03:19:37.302 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 0, 2024-07-27T03:19:37.302 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 0, 2024-07-27T03:19:37.302 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 116936, 2024-07-27T03:19:37.302 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 215348 2024-07-27T03:19:37.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.302 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2024-07-27T03:19:37.302 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 0, 2024-07-27T03:19:37.302 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 0, 2024-07-27T03:19:37.302 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 0, 2024-07-27T03:19:37.303 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 0, 2024-07-27T03:19:37.303 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 0, 2024-07-27T03:19:37.303 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2024-07-27T03:19:37.303 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.303 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.303 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.303 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2024-07-27T03:19:37.303 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.303 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.303 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.303 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2024-07-27T03:19:37.303 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.304 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.304 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.304 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2024-07-27T03:19:37.304 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.304 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.304 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.305 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2024-07-27T03:19:37.306 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.306 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.306 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.306 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2024-07-27T03:19:37.306 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.306 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.306 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.307 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2024-07-27T03:19:37.307 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.307 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.307 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.307 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2024-07-27T03:19:37.308 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.308 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.308 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.308 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2024-07-27T03:19:37.309 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.309 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.309 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.309 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2024-07-27T03:19:37.309 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.309 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.310 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.310 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2024-07-27T03:19:37.310 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.310 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.311 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.311 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2024-07-27T03:19:37.311 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.312 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.312 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.313 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2024-07-27T03:19:37.313 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.313 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.313 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.314 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2024-07-27T03:19:37.314 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.314 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.314 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.314 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.315 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2024-07-27T03:19:37.315 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.315 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.315 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.315 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2024-07-27T03:19:37.316 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.316 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.316 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.316 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2024-07-27T03:19:37.317 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.317 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.317 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.317 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2024-07-27T03:19:37.317 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.318 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.318 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.318 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2024-07-27T03:19:37.318 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.318 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.319 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.319 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2024-07-27T03:19:37.319 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.319 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.319 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.320 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2024-07-27T03:19:37.320 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.320 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.320 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.321 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2024-07-27T03:19:37.321 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.321 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.321 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.322 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2024-07-27T03:19:37.322 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.322 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.322 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.322 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2024-07-27T03:19:37.323 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.323 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.323 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.323 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.323 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2024-07-27T03:19:37.323 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.324 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.324 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.324 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2024-07-27T03:19:37.324 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.324 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.325 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.325 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2024-07-27T03:19:37.325 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.325 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.326 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.326 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2024-07-27T03:19:37.326 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.326 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.326 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.327 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2024-07-27T03:19:37.327 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.327 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.327 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.328 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2024-07-27T03:19:37.328 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2024-07-27T03:19:37.328 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2024-07-27T03:19:37.328 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.329 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.329 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.329 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:37.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.329 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2024-07-27T03:19:37.329 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 1, 2024-07-27T03:19:37.330 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 3, 2024-07-27T03:19:37.330 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 2, 2024-07-27T03:19:37.330 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 1, 2024-07-27T03:19:37.330 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2024-07-27T03:19:37.330 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 0, 2024-07-27T03:19:37.330 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 0, 2024-07-27T03:19:37.331 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 1638, 2024-07-27T03:19:37.331 INFO:teuthology.orchestra.run.smithi175.stdout: "mdthresh_evicted": 0 2024-07-27T03:19:37.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.331 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-07-27T03:19:37.331 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-07-27T03:19:37.332 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-07-27T03:19:37.332 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-07-27T03:19:37.332 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-07-27T03:19:37.332 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-07-27T03:19:37.332 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-07-27T03:19:37.332 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-27T03:19:37.333 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-07-27T03:19:37.333 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-27T03:19:37.333 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-07-27T03:19:37.333 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-07-27T03:19:37.333 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-07-27T03:19:37.333 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-27T03:19:37.334 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-07-27T03:19:37.334 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-07-27T03:19:37.334 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-07-27T03:19:37.334 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-07-27T03:19:37.334 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-07-27T03:19:37.335 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-27T03:19:37.335 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-07-27T03:19:37.335 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-27T03:19:37.335 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-07-27T03:19:37.335 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-07-27T03:19:37.335 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-07-27T03:19:37.335 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-07-27T03:19:37.336 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-07-27T03:19:37.336 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-27T03:19:37.336 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-07-27T03:19:37.336 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-07-27T03:19:37.336 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-07-27T03:19:37.336 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-07-27T03:19:37.337 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-07-27T03:19:37.337 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-07-27T03:19:37.337 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-07-27T03:19:37.337 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-07-27T03:19:37.337 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-07-27T03:19:37.337 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 197336, 2024-07-27T03:19:37.338 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 53, 2024-07-27T03:19:37.338 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2024-07-27T03:19:37.338 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2024-07-27T03:19:37.338 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-07-27T03:19:37.338 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-07-27T03:19:37.339 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-07-27T03:19:37.339 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-07-27T03:19:37.339 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-07-27T03:19:37.339 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-07-27T03:19:37.339 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 8952, 2024-07-27T03:19:37.339 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 168, 2024-07-27T03:19:37.340 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-07-27T03:19:37.340 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-07-27T03:19:37.340 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-07-27T03:19:37.340 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-07-27T03:19:37.340 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 57090, 2024-07-27T03:19:37.340 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 273, 2024-07-27T03:19:37.341 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-07-27T03:19:37.341 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-07-27T03:19:37.341 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-07-27T03:19:37.341 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-07-27T03:19:37.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.341 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2024-07-27T03:19:37.342 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2024-07-27T03:19:37.342 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2024-07-27T03:19:37.342 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 1159, 2024-07-27T03:19:37.342 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 26, 2024-07-27T03:19:37.342 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 4, 2024-07-27T03:19:37.343 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 1159, 2024-07-27T03:19:37.343 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2024-07-27T03:19:37.343 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1159, 2024-07-27T03:19:37.343 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.580990384, 2024-07-27T03:19:37.343 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000501285 2024-07-27T03:19:37.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.344 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2024-07-27T03:19:37.344 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2024-07-27T03:19:37.344 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1159, 2024-07-27T03:19:37.344 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 1160 2024-07-27T03:19:37.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.345 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 1159, 2024-07-27T03:19:37.345 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 1159, 2024-07-27T03:19:37.345 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 0, 2024-07-27T03:19:37.345 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2024-07-27T03:19:37.346 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2024-07-27T03:19:37.346 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 714, 2024-07-27T03:19:37.346 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 0, 2024-07-27T03:19:37.346 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 442, 2024-07-27T03:19:37.346 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 0, 2024-07-27T03:19:37.346 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 0, 2024-07-27T03:19:37.347 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2024-07-27T03:19:37.347 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2024-07-27T03:19:37.347 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2024-07-27T03:19:37.347 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 0, 2024-07-27T03:19:37.347 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 0, 2024-07-27T03:19:37.348 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2024-07-27T03:19:37.348 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2024-07-27T03:19:37.348 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2024-07-27T03:19:37.348 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 2, 2024-07-27T03:19:37.348 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 0, 2024-07-27T03:19:37.348 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2024-07-27T03:19:37.349 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2024-07-27T03:19:37.349 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2024-07-27T03:19:37.349 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2024-07-27T03:19:37.349 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2024-07-27T03:19:37.349 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2024-07-27T03:19:37.350 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2024-07-27T03:19:37.350 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2024-07-27T03:19:37.350 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2024-07-27T03:19:37.350 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2024-07-27T03:19:37.350 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2024-07-27T03:19:37.351 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2024-07-27T03:19:37.351 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2024-07-27T03:19:37.351 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2024-07-27T03:19:37.351 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2024-07-27T03:19:37.351 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2024-07-27T03:19:37.352 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2024-07-27T03:19:37.352 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2024-07-27T03:19:37.352 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2024-07-27T03:19:37.352 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2024-07-27T03:19:37.352 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2024-07-27T03:19:37.353 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2024-07-27T03:19:37.353 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2024-07-27T03:19:37.353 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2024-07-27T03:19:37.353 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2024-07-27T03:19:37.353 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2024-07-27T03:19:37.354 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 91, 2024-07-27T03:19:37.354 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2024-07-27T03:19:37.354 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 2, 2024-07-27T03:19:37.354 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 6, 2024-07-27T03:19:37.354 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 6, 2024-07-27T03:19:37.354 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 0, 2024-07-27T03:19:37.355 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2024-07-27T03:19:37.355 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 0, 2024-07-27T03:19:37.355 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 2, 2024-07-27T03:19:37.355 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 0 2024-07-27T03:19:37.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.355 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2024-07-27T03:19:37.356 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 0, 2024-07-27T03:19:37.356 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2024-07-27T03:19:37.356 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2024-07-27T03:19:37.356 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2024-07-27T03:19:37.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.356 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2024-07-27T03:19:37.356 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 0 2024-07-27T03:19:37.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.357 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2024-07-27T03:19:37.357 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2024-07-27T03:19:37.357 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 0, 2024-07-27T03:19:37.357 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2024-07-27T03:19:37.357 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 0, 2024-07-27T03:19:37.357 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed_ops": 0, 2024-07-27T03:19:37.357 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 0, 2024-07-27T03:19:37.357 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2024-07-27T03:19:37.358 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.358 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-27T03:19:37.358 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-27T03:19:37.358 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-07-27T03:19:37.358 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-27T03:19:37.358 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 1297, 2024-07-27T03:19:37.358 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 1328815, 2024-07-27T03:19:37.358 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:37.359 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 1297, 2024-07-27T03:19:37.359 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-07-27T03:19:37.359 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-07-27T03:19:37.359 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 1297, 2024-07-27T03:19:37.359 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 1328815, 2024-07-27T03:19:37.359 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-27T03:19:37.360 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.360 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.360 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.360 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:37.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.360 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2024-07-27T03:19:37.360 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-27T03:19:37.360 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-07-27T03:19:37.360 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-27T03:19:37.361 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-07-27T03:19:37.361 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-07-27T03:19:37.361 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:37.361 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:37.361 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 1159, 2024-07-27T03:19:37.361 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 605628, 2024-07-27T03:19:37.361 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 1159, 2024-07-27T03:19:37.361 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 605628, 2024-07-27T03:19:37.361 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-27T03:19:37.361 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.361 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.361 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.362 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:37.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.362 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2024-07-27T03:19:37.362 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-27T03:19:37.362 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2024-07-27T03:19:37.362 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-27T03:19:37.362 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-07-27T03:19:37.362 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-07-27T03:19:37.362 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:37.362 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:37.362 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 1159, 2024-07-27T03:19:37.362 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 1159, 2024-07-27T03:19:37.362 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 1159, 2024-07-27T03:19:37.363 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 1159, 2024-07-27T03:19:37.363 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-27T03:19:37.363 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.363 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.363 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.363 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:37.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.363 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2024-07-27T03:19:37.363 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-27T03:19:37.363 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-07-27T03:19:37.363 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-27T03:19:37.363 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-07-27T03:19:37.363 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-07-27T03:19:37.363 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:37.364 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:37.364 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-07-27T03:19:37.364 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-07-27T03:19:37.364 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-07-27T03:19:37.364 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-07-27T03:19:37.364 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-27T03:19:37.364 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.364 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.364 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.364 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:37.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:37.364 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x55747caa0750": { 2024-07-27T03:19:37.364 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-27T03:19:37.364 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-07-27T03:19:37.365 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-27T03:19:37.365 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-07-27T03:19:37.365 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-07-27T03:19:37.365 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:37.365 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:37.365 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-07-27T03:19:37.365 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-07-27T03:19:37.365 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-07-27T03:19:37.365 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-07-27T03:19:37.365 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-27T03:19:37.365 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:37.365 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:37.365 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:37.365 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:37.365 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:37.366 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-27T03:19:38.047 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-07-27T03:19:38.047 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-07-27T03:19:38.047 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-07-27T03:19:38.047 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-07-27T03:19:38.047 DEBUG:tasks.check_counter:Getting stats from i 2024-07-27T03:19:38.048 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph daemon mds.i perf dump 2024-07-27T03:19:38.299 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.c/config 2024-07-27T03:19:38.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:38 smithi067 ceph-mon[32814]: pgmap v522: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:19:39.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:38 smithi149 ceph-mon[37173]: pgmap v522: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:19:39.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:38 smithi175 ceph-mon[37125]: pgmap v522: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:19:39.973 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-27T03:19:39.974 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-07-27T03:19:39.974 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 199, 2024-07-27T03:19:39.974 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 190, 2024-07-27T03:19:39.974 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 560281, 2024-07-27T03:19:39.974 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 98549, 2024-07-27T03:19:39.974 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 10, 2024-07-27T03:19:39.974 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 5, 2024-07-27T03:19:39.974 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.035319177, 2024-07-27T03:19:39.974 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.015236303, 2024-07-27T03:19:39.974 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.023937282, 2024-07-27T03:19:39.974 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000341320, 2024-07-27T03:19:39.975 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:39.975 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 187, 2024-07-27T03:19:39.975 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.029116477, 2024-07-27T03:19:39.975 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000155703 2024-07-27T03:19:39.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.975 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:39.975 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 26, 2024-07-27T03:19:39.975 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000005672, 2024-07-27T03:19:39.975 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000218 2024-07-27T03:19:39.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.975 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 545808, 2024-07-27T03:19:39.975 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 80752 2024-07-27T03:19:39.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.976 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-07-27T03:19:39.976 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 47, 2024-07-27T03:19:39.976 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 139, 2024-07-27T03:19:39.976 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 17697, 2024-07-27T03:19:39.976 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 392171, 2024-07-27T03:19:39.976 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 21, 2024-07-27T03:19:39.976 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 9, 2024-07-27T03:19:39.976 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.018894598, 2024-07-27T03:19:39.976 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.007936824, 2024-07-27T03:19:39.976 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.009424608, 2024-07-27T03:19:39.976 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.000356375, 2024-07-27T03:19:39.976 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:39.976 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 137, 2024-07-27T03:19:39.977 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.020942718, 2024-07-27T03:19:39.977 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000152866 2024-07-27T03:19:39.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.977 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:39.977 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 52, 2024-07-27T03:19:39.977 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000014250, 2024-07-27T03:19:39.977 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000274 2024-07-27T03:19:39.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.977 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 1744, 2024-07-27T03:19:39.977 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 222864 2024-07-27T03:19:39.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.977 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-07-27T03:19:39.977 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 273, 2024-07-27T03:19:39.978 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 297, 2024-07-27T03:19:39.978 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 92197, 2024-07-27T03:19:39.978 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 86519, 2024-07-27T03:19:39.978 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 8, 2024-07-27T03:19:39.978 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 4, 2024-07-27T03:19:39.978 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.040701170, 2024-07-27T03:19:39.978 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.017859289, 2024-07-27T03:19:39.978 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.020233484, 2024-07-27T03:19:39.978 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.002676441, 2024-07-27T03:19:39.978 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:39.978 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 298, 2024-07-27T03:19:39.978 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.031093417, 2024-07-27T03:19:39.979 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000104340 2024-07-27T03:19:39.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.979 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:39.979 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 532, 2024-07-27T03:19:39.979 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000145066, 2024-07-27T03:19:39.979 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000272 2024-07-27T03:19:39.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.979 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-07-27T03:19:39.979 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 448 2024-07-27T03:19:39.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.979 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-07-27T03:19:39.979 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2024-07-27T03:19:39.980 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-07-27T03:19:39.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.980 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2024-07-27T03:19:39.980 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-07-27T03:19:39.980 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-07-27T03:19:39.980 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 40, 2024-07-27T03:19:39.980 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.002238082, 2024-07-27T03:19:39.980 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000055952 2024-07-27T03:19:39.980 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:39.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.980 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2024-07-27T03:19:39.980 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-07-27T03:19:39.980 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-07-27T03:19:39.980 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 2, 2024-07-27T03:19:39.981 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000016843, 2024-07-27T03:19:39.981 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000008421 2024-07-27T03:19:39.981 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:39.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.981 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2024-07-27T03:19:39.981 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 2, 2024-07-27T03:19:39.981 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 1, 2024-07-27T03:19:39.981 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2024-07-27T03:19:39.981 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1, 2024-07-27T03:19:39.981 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000237832, 2024-07-27T03:19:39.981 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000237832 2024-07-27T03:19:39.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.981 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2024-07-27T03:19:39.982 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 1, 2024-07-27T03:19:39.982 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 0, 2024-07-27T03:19:39.982 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 0, 2024-07-27T03:19:39.982 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 11, 2024-07-27T03:19:39.982 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 0, 2024-07-27T03:19:39.982 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 0, 2024-07-27T03:19:39.982 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 16, 2024-07-27T03:19:39.982 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 0, 2024-07-27T03:19:39.982 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 16, 2024-07-27T03:19:39.982 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 0, 2024-07-27T03:19:39.982 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 13, 2024-07-27T03:19:39.982 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 4, 2024-07-27T03:19:39.983 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 0, 2024-07-27T03:19:39.983 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 0, 2024-07-27T03:19:39.983 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 3, 2024-07-27T03:19:39.983 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 11, 2024-07-27T03:19:39.983 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 5, 2024-07-27T03:19:39.983 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 0, 2024-07-27T03:19:39.983 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 5, 2024-07-27T03:19:39.983 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2024-07-27T03:19:39.983 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2024-07-27T03:19:39.983 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 0, 2024-07-27T03:19:39.983 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 2000, 2024-07-27T03:19:39.983 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 2, 2024-07-27T03:19:39.983 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 0, 2024-07-27T03:19:39.984 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 0, 2024-07-27T03:19:39.984 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 0, 2024-07-27T03:19:39.984 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 0, 2024-07-27T03:19:39.984 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2024-07-27T03:19:39.984 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 0, 2024-07-27T03:19:39.984 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2024-07-27T03:19:39.984 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 7, 2024-07-27T03:19:39.984 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 269487290, 2024-07-27T03:19:39.984 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 0, 2024-07-27T03:19:39.984 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 0, 2024-07-27T03:19:39.984 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2024-07-27T03:19:39.984 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2024-07-27T03:19:39.984 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2024-07-27T03:19:39.985 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 0, 2024-07-27T03:19:39.985 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 0, 2024-07-27T03:19:39.985 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 0, 2024-07-27T03:19:39.985 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 0, 2024-07-27T03:19:39.985 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 0, 2024-07-27T03:19:39.985 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 0, 2024-07-27T03:19:39.985 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 0, 2024-07-27T03:19:39.985 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2024-07-27T03:19:39.985 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-27T03:19:39.985 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-27T03:19:39.985 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 0, 2024-07-27T03:19:39.985 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 0, 2024-07-27T03:19:39.985 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 0, 2024-07-27T03:19:39.985 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 1 2024-07-27T03:19:39.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.986 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2024-07-27T03:19:39.986 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 0, 2024-07-27T03:19:39.986 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 11, 2024-07-27T03:19:39.986 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 7, 2024-07-27T03:19:39.986 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 8, 2024-07-27T03:19:39.986 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 0, 2024-07-27T03:19:39.986 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2024-07-27T03:19:39.986 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2024-07-27T03:19:39.986 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2024-07-27T03:19:39.986 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 0, 2024-07-27T03:19:39.986 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 0, 2024-07-27T03:19:39.986 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2024-07-27T03:19:39.987 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2024-07-27T03:19:39.987 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2024-07-27T03:19:39.987 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2024-07-27T03:19:39.987 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2024-07-27T03:19:39.987 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2024-07-27T03:19:39.987 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2024-07-27T03:19:39.987 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_path": 0, 2024-07-27T03:19:39.987 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_inode": 0, 2024-07-27T03:19:39.987 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 0, 2024-07-27T03:19:39.987 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 0, 2024-07-27T03:19:39.987 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2024-07-27T03:19:39.987 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 0, 2024-07-27T03:19:39.987 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2024-07-27T03:19:39.987 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2024-07-27T03:19:39.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.988 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2024-07-27T03:19:39.988 INFO:teuthology.orchestra.run.smithi175.stdout: "evlrg": 0, 2024-07-27T03:19:39.988 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 4, 2024-07-27T03:19:39.988 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 0, 2024-07-27T03:19:39.988 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 0, 2024-07-27T03:19:39.988 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 4, 2024-07-27T03:19:39.988 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2024-07-27T03:19:39.988 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 0, 2024-07-27T03:19:39.988 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 2, 2024-07-27T03:19:39.988 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 0, 2024-07-27T03:19:39.988 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 0, 2024-07-27T03:19:39.988 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 2, 2024-07-27T03:19:39.989 INFO:teuthology.orchestra.run.smithi175.stdout: "segmjr": 2, 2024-07-27T03:19:39.989 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2024-07-27T03:19:39.989 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 0, 2024-07-27T03:19:39.989 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 4194304, 2024-07-27T03:19:39.989 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 4195424, 2024-07-27T03:19:39.989 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 0, 2024-07-27T03:19:39.989 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2024-07-27T03:19:39.989 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 3, 2024-07-27T03:19:39.989 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.006432711, 2024-07-27T03:19:39.989 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.002144237 2024-07-27T03:19:39.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.989 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 0 2024-07-27T03:19:39.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.989 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2024-07-27T03:19:39.990 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 20, 2024-07-27T03:19:39.990 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 22, 2024-07-27T03:19:39.990 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 2, 2024-07-27T03:19:39.990 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 19, 2024-07-27T03:19:39.990 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 22, 2024-07-27T03:19:39.990 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 3, 2024-07-27T03:19:39.990 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 16, 2024-07-27T03:19:39.990 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 20, 2024-07-27T03:19:39.990 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 4, 2024-07-27T03:19:39.990 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 0, 2024-07-27T03:19:39.990 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 1, 2024-07-27T03:19:39.990 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 1, 2024-07-27T03:19:39.990 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 118308, 2024-07-27T03:19:39.991 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 223540 2024-07-27T03:19:39.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.991 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2024-07-27T03:19:39.991 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 3, 2024-07-27T03:19:39.991 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 0, 2024-07-27T03:19:39.991 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 2, 2024-07-27T03:19:39.991 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 4, 2024-07-27T03:19:39.991 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 0, 2024-07-27T03:19:39.991 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2024-07-27T03:19:39.991 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:39.991 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:39.991 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:39.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.992 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2024-07-27T03:19:39.992 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1, 2024-07-27T03:19:39.992 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000237832, 2024-07-27T03:19:39.992 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000237832 2024-07-27T03:19:39.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.992 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2024-07-27T03:19:39.992 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:39.992 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:39.992 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:39.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.993 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2024-07-27T03:19:39.993 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:39.993 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:39.993 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:39.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.993 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2024-07-27T03:19:39.993 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:39.993 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:39.993 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:39.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.993 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2024-07-27T03:19:39.993 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:39.993 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:39.994 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:39.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.994 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2024-07-27T03:19:39.994 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:39.994 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:39.994 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:39.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.994 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2024-07-27T03:19:39.994 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:39.994 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:39.994 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:39.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.994 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2024-07-27T03:19:39.995 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:39.995 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:39.995 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:39.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.995 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2024-07-27T03:19:39.995 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:39.995 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:39.995 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:39.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.995 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2024-07-27T03:19:39.995 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:39.995 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:39.995 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:39.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.996 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2024-07-27T03:19:39.996 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:39.996 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:39.996 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:39.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.997 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2024-07-27T03:19:39.997 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:39.997 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:39.997 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:39.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.998 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2024-07-27T03:19:39.998 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:39.998 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:39.998 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:39.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.998 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2024-07-27T03:19:39.998 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:39.998 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:39.998 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:39.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.999 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2024-07-27T03:19:39.999 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:39.999 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:39.999 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:39.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.999 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2024-07-27T03:19:39.999 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:39.999 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:39.999 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:39.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:39.999 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2024-07-27T03:19:39.999 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:39.999 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:39.999 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:40.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.000 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2024-07-27T03:19:40.000 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:40.000 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:40.000 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:40.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.000 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2024-07-27T03:19:40.000 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:40.000 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:40.000 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:40.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.000 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2024-07-27T03:19:40.000 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:40.001 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:40.001 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:40.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.001 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2024-07-27T03:19:40.001 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:40.001 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:40.001 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:40.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.001 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2024-07-27T03:19:40.001 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:40.001 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:40.001 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:40.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.001 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2024-07-27T03:19:40.002 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:40.002 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:40.002 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:40.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.002 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2024-07-27T03:19:40.002 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:40.002 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:40.002 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:40.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.002 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2024-07-27T03:19:40.002 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:40.002 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:40.002 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:40.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.003 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2024-07-27T03:19:40.003 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:40.003 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:40.003 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:40.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.003 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2024-07-27T03:19:40.003 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:40.003 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:40.003 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:40.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.003 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2024-07-27T03:19:40.003 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:40.003 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:40.004 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:40.004 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.004 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2024-07-27T03:19:40.004 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2024-07-27T03:19:40.004 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2024-07-27T03:19:40.004 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:40.004 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:40.004 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:40.004 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:40.004 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.004 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2024-07-27T03:19:40.004 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 0, 2024-07-27T03:19:40.004 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 1, 2024-07-27T03:19:40.005 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 1, 2024-07-27T03:19:40.005 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 0, 2024-07-27T03:19:40.005 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2024-07-27T03:19:40.005 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 1, 2024-07-27T03:19:40.005 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 1, 2024-07-27T03:19:40.005 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 95, 2024-07-27T03:19:40.005 INFO:teuthology.orchestra.run.smithi175.stdout: "mdthresh_evicted": 0 2024-07-27T03:19:40.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.005 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-07-27T03:19:40.005 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-07-27T03:19:40.005 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-07-27T03:19:40.005 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-07-27T03:19:40.005 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-07-27T03:19:40.006 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-07-27T03:19:40.006 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-07-27T03:19:40.006 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-27T03:19:40.006 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-07-27T03:19:40.006 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-27T03:19:40.006 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-07-27T03:19:40.006 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-07-27T03:19:40.006 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-07-27T03:19:40.006 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-27T03:19:40.006 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-07-27T03:19:40.006 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-07-27T03:19:40.006 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-07-27T03:19:40.006 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-07-27T03:19:40.007 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-07-27T03:19:40.007 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-27T03:19:40.007 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-07-27T03:19:40.007 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-27T03:19:40.007 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-07-27T03:19:40.007 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-07-27T03:19:40.007 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-07-27T03:19:40.007 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-07-27T03:19:40.007 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-07-27T03:19:40.007 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-27T03:19:40.007 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-07-27T03:19:40.007 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-07-27T03:19:40.007 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-07-27T03:19:40.007 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-07-27T03:19:40.008 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-07-27T03:19:40.008 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-07-27T03:19:40.008 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-07-27T03:19:40.008 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-07-27T03:19:40.008 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-07-27T03:19:40.008 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 247671, 2024-07-27T03:19:40.008 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 87, 2024-07-27T03:19:40.008 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2024-07-27T03:19:40.008 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2024-07-27T03:19:40.008 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-07-27T03:19:40.008 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-07-27T03:19:40.008 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-07-27T03:19:40.008 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-07-27T03:19:40.009 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-07-27T03:19:40.009 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-07-27T03:19:40.009 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 8952, 2024-07-27T03:19:40.009 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 168, 2024-07-27T03:19:40.009 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-07-27T03:19:40.009 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-07-27T03:19:40.009 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-07-27T03:19:40.009 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-07-27T03:19:40.009 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 80941, 2024-07-27T03:19:40.009 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 324, 2024-07-27T03:19:40.009 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-07-27T03:19:40.009 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-07-27T03:19:40.009 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-07-27T03:19:40.009 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-07-27T03:19:40.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.010 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2024-07-27T03:19:40.010 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2024-07-27T03:19:40.010 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2024-07-27T03:19:40.010 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 38, 2024-07-27T03:19:40.010 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 11332, 2024-07-27T03:19:40.010 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 33, 2024-07-27T03:19:40.010 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 38, 2024-07-27T03:19:40.010 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2024-07-27T03:19:40.010 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 38, 2024-07-27T03:19:40.010 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.231996166, 2024-07-27T03:19:40.010 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.006105162 2024-07-27T03:19:40.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.010 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2024-07-27T03:19:40.011 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2024-07-27T03:19:40.011 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 38, 2024-07-27T03:19:40.011 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 60 2024-07-27T03:19:40.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.011 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 38, 2024-07-27T03:19:40.011 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 0, 2024-07-27T03:19:40.011 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 38, 2024-07-27T03:19:40.011 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2024-07-27T03:19:40.011 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2024-07-27T03:19:40.011 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 0, 2024-07-27T03:19:40.011 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 10, 2024-07-27T03:19:40.011 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 0, 2024-07-27T03:19:40.011 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 3, 2024-07-27T03:19:40.011 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 6, 2024-07-27T03:19:40.012 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2024-07-27T03:19:40.012 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2024-07-27T03:19:40.012 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2024-07-27T03:19:40.012 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 0, 2024-07-27T03:19:40.012 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 6, 2024-07-27T03:19:40.012 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2024-07-27T03:19:40.012 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2024-07-27T03:19:40.012 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2024-07-27T03:19:40.012 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 0, 2024-07-27T03:19:40.012 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 20, 2024-07-27T03:19:40.012 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2024-07-27T03:19:40.012 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2024-07-27T03:19:40.012 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2024-07-27T03:19:40.012 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2024-07-27T03:19:40.012 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2024-07-27T03:19:40.013 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2024-07-27T03:19:40.013 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2024-07-27T03:19:40.013 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2024-07-27T03:19:40.013 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2024-07-27T03:19:40.013 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2024-07-27T03:19:40.013 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2024-07-27T03:19:40.013 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2024-07-27T03:19:40.013 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2024-07-27T03:19:40.013 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2024-07-27T03:19:40.013 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2024-07-27T03:19:40.013 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2024-07-27T03:19:40.013 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2024-07-27T03:19:40.013 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2024-07-27T03:19:40.013 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2024-07-27T03:19:40.014 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2024-07-27T03:19:40.014 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2024-07-27T03:19:40.014 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2024-07-27T03:19:40.014 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2024-07-27T03:19:40.014 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2024-07-27T03:19:40.014 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2024-07-27T03:19:40.014 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2024-07-27T03:19:40.014 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 91, 2024-07-27T03:19:40.014 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2024-07-27T03:19:40.014 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 2, 2024-07-27T03:19:40.014 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 9, 2024-07-27T03:19:40.014 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 9, 2024-07-27T03:19:40.014 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 0, 2024-07-27T03:19:40.014 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2024-07-27T03:19:40.015 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 14, 2024-07-27T03:19:40.015 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 0, 2024-07-27T03:19:40.015 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 1 2024-07-27T03:19:40.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.015 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2024-07-27T03:19:40.015 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 1, 2024-07-27T03:19:40.015 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2024-07-27T03:19:40.015 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2024-07-27T03:19:40.015 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2024-07-27T03:19:40.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.015 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2024-07-27T03:19:40.015 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 0 2024-07-27T03:19:40.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.015 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2024-07-27T03:19:40.016 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2024-07-27T03:19:40.016 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 0, 2024-07-27T03:19:40.016 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2024-07-27T03:19:40.016 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 0, 2024-07-27T03:19:40.016 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed_ops": 0, 2024-07-27T03:19:40.016 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 0, 2024-07-27T03:19:40.016 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2024-07-27T03:19:40.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.016 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-27T03:19:40.016 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-27T03:19:40.016 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-07-27T03:19:40.016 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-27T03:19:40.016 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 515, 2024-07-27T03:19:40.016 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 612051, 2024-07-27T03:19:40.017 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:40.017 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 515, 2024-07-27T03:19:40.017 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-07-27T03:19:40.017 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-07-27T03:19:40.017 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 515, 2024-07-27T03:19:40.017 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 612051, 2024-07-27T03:19:40.017 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-27T03:19:40.017 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:40.017 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:40.017 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:40.017 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:40.017 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.017 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2024-07-27T03:19:40.017 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-27T03:19:40.018 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-07-27T03:19:40.018 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-27T03:19:40.018 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-07-27T03:19:40.018 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-07-27T03:19:40.018 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:40.018 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:40.018 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 38, 2024-07-27T03:19:40.018 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 11332, 2024-07-27T03:19:40.018 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 38, 2024-07-27T03:19:40.018 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 11332, 2024-07-27T03:19:40.018 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-27T03:19:40.018 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:40.018 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:40.018 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:40.019 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:40.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.019 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2024-07-27T03:19:40.019 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-27T03:19:40.019 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2024-07-27T03:19:40.019 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-27T03:19:40.019 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-07-27T03:19:40.019 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-07-27T03:19:40.019 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:40.019 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:40.019 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 38, 2024-07-27T03:19:40.019 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 38, 2024-07-27T03:19:40.019 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 38, 2024-07-27T03:19:40.019 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 38, 2024-07-27T03:19:40.020 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-27T03:19:40.020 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:40.020 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:40.020 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:40.020 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:40.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.020 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2024-07-27T03:19:40.020 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-27T03:19:40.020 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-07-27T03:19:40.020 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-27T03:19:40.020 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-07-27T03:19:40.020 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-07-27T03:19:40.020 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:40.020 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:40.021 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-07-27T03:19:40.021 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-07-27T03:19:40.021 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-07-27T03:19:40.021 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-07-27T03:19:40.021 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-27T03:19:40.021 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:40.021 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:40.021 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:40.021 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:40.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:40.021 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x555f781a0750": { 2024-07-27T03:19:40.021 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-27T03:19:40.021 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-07-27T03:19:40.021 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-27T03:19:40.022 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 4, 2024-07-27T03:19:40.022 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 1120, 2024-07-27T03:19:40.022 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:40.022 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 4, 2024-07-27T03:19:40.022 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-07-27T03:19:40.022 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-07-27T03:19:40.022 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 3, 2024-07-27T03:19:40.022 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 1120, 2024-07-27T03:19:40.022 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-27T03:19:40.022 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:40.022 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:40.022 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:40.022 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:40.022 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:40.023 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-27T03:19:40.785 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-07-27T03:19:40.785 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-07-27T03:19:40.785 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-07-27T03:19:40.785 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=11 2024-07-27T03:19:40.785 DEBUG:tasks.check_counter:Getting stats from l 2024-07-27T03:19:40.785 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph daemon mds.l perf dump 2024-07-27T03:19:40.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:40 smithi067 ceph-mon[32814]: pgmap v523: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:19:41.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:40 smithi149 ceph-mon[37173]: pgmap v523: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:19:41.037 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.c/config 2024-07-27T03:19:41.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:40 smithi175 ceph-mon[37125]: pgmap v523: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:19:42.749 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-07-27T03:19:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-0": { 2024-07-27T03:19:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 368, 2024-07-27T03:19:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 370, 2024-07-27T03:19:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 98127, 2024-07-27T03:19:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 117088, 2024-07-27T03:19:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 6, 2024-07-27T03:19:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 2, 2024-07-27T03:19:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.043987143, 2024-07-27T03:19:42.749 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.018514226, 2024-07-27T03:19:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.010105887, 2024-07-27T03:19:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.011170987, 2024-07-27T03:19:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 368, 2024-07-27T03:19:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.016246262, 2024-07-27T03:19:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000044147 2024-07-27T03:19:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.750 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 0, 2024-07-27T03:19:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 0 2024-07-27T03:19:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-1": { 2024-07-27T03:19:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 260, 2024-07-27T03:19:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 357, 2024-07-27T03:19:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 63896, 2024-07-27T03:19:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 304671, 2024-07-27T03:19:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 5, 2024-07-27T03:19:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 4, 2024-07-27T03:19:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.040054887, 2024-07-27T03:19:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.018268200, 2024-07-27T03:19:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.007678930, 2024-07-27T03:19:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.010507870, 2024-07-27T03:19:42.751 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 355, 2024-07-27T03:19:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.016629007, 2024-07-27T03:19:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000046842 2024-07-27T03:19:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 160, 2024-07-27T03:19:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 222304 2024-07-27T03:19:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: "AsyncMessenger::Worker-2": { 2024-07-27T03:19:42.752 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_messages": 507, 2024-07-27T03:19:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages": 500, 2024-07-27T03:19:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_bytes": 648288, 2024-07-27T03:19:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_bytes": 195516, 2024-07-27T03:19:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_created_connections": 3, 2024-07-27T03:19:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_active_connections": 2, 2024-07-27T03:19:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_total_time": 0.083104620, 2024-07-27T03:19:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_send_time": 0.036222048, 2024-07-27T03:19:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_recv_time": 0.034155011, 2024-07-27T03:19:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_running_fast_dispatch_time": 0.015247895, 2024-07-27T03:19:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_messages_queue_lat": { 2024-07-27T03:19:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 500, 2024-07-27T03:19:42.753 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.035031296, 2024-07-27T03:19:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000070062 2024-07-27T03:19:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_handle_ack_lat": { 2024-07-27T03:19:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_recv_encrypted_bytes": 526656, 2024-07-27T03:19:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: "msgr_send_encrypted_bytes": 80400 2024-07-27T03:19:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.754 INFO:teuthology.orchestra.run.smithi175.stdout: "cct": { 2024-07-27T03:19:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: "total_workers": 1, 2024-07-27T03:19:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: "unhealthy_workers": 0 2024-07-27T03:19:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-MDSRank": { 2024-07-27T03:19:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-07-27T03:19:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-07-27T03:19:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 1686, 2024-07-27T03:19:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.058649790, 2024-07-27T03:19:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000034786 2024-07-27T03:19:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: "finisher-PurgeQueue": { 2024-07-27T03:19:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: "queue_len": 0, 2024-07-27T03:19:42.755 INFO:teuthology.orchestra.run.smithi175.stdout: "complete_latency": { 2024-07-27T03:19:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: "mds": { 2024-07-27T03:19:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: "request": 0, 2024-07-27T03:19:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: "reply": 0, 2024-07-27T03:19:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: "reply_latency": { 2024-07-27T03:19:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_reply": 0, 2024-07-27T03:19:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: "forward": 0, 2024-07-27T03:19:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_complete": 0, 2024-07-27T03:19:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_fetch_keys": 0, 2024-07-27T03:19:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_commit": 0, 2024-07-27T03:19:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_split": 0, 2024-07-27T03:19:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_merge": 0, 2024-07-27T03:19:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes": 0, 2024-07-27T03:19:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_top": 0, 2024-07-27T03:19:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_bottom": 0, 2024-07-27T03:19:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pin_tail": 0, 2024-07-27T03:19:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_pinned": 0, 2024-07-27T03:19:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_expired": 0, 2024-07-27T03:19:42.757 INFO:teuthology.orchestra.run.smithi175.stdout: "inodes_with_caps": 0, 2024-07-27T03:19:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: "caps": 0, 2024-07-27T03:19:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: "subtrees": 1, 2024-07-27T03:19:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse": 0, 2024-07-27T03:19:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_hit": 0, 2024-07-27T03:19:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_forward": 0, 2024-07-27T03:19:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_discover": 0, 2024-07-27T03:19:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_dir_fetch": 0, 2024-07-27T03:19:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_remote_ino": 0, 2024-07-27T03:19:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: "traverse_lock": 0, 2024-07-27T03:19:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: "load_cent": 0, 2024-07-27T03:19:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: "q": 0, 2024-07-27T03:19:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: "exported": 0, 2024-07-27T03:19:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: "exported_inodes": 0, 2024-07-27T03:19:42.758 INFO:teuthology.orchestra.run.smithi175.stdout: "imported": 0, 2024-07-27T03:19:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "imported_inodes": 0, 2024-07-27T03:19:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_dir_fetch": 0, 2024-07-27T03:19:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_backtrace_fetch": 0, 2024-07-27T03:19:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "openino_peer_discover": 0, 2024-07-27T03:19:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rfiles": 0, 2024-07-27T03:19:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rbytes": 0, 2024-07-27T03:19:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "root_rsnaps": 0, 2024-07-27T03:19:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_fetch": 0, 2024-07-27T03:19:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_set_tag": 0, 2024-07-27T03:19:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_backtrace_repaired": 0, 2024-07-27T03:19:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_inotable_repaired": 0, 2024-07-27T03:19:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_inodes": 0, 2024-07-27T03:19:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dir_base_inodes": 0, 2024-07-27T03:19:42.759 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_dirfrag_rstats": 0, 2024-07-27T03:19:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: "scrub_file_inodes": 0, 2024-07-27T03:19:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_inode_file_caps": 0, 2024-07-27T03:19:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_revoke": 0, 2024-07-27T03:19:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_grant": 0, 2024-07-27T03:19:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_trunc": 0, 2024-07-27T03:19:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-07-27T03:19:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_cap_op_flush_ack": 0, 2024-07-27T03:19:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps": 0, 2024-07-27T03:19:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_caps_dirty": 0, 2024-07-27T03:19:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_cap_release": 0, 2024-07-27T03:19:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: "process_request_cap_release": 0 2024-07-27T03:19:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_cache": { 2024-07-27T03:19:42.760 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update": 0, 2024-07-27T03:19:42.761 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_update_receipt": 0, 2024-07-27T03:19:42.761 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_try_discover": 0, 2024-07-27T03:19:42.761 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_send_discover": 0, 2024-07-27T03:19:42.761 INFO:teuthology.orchestra.run.smithi175.stdout: "dir_handle_discover": 0, 2024-07-27T03:19:42.761 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays": 0, 2024-07-27T03:19:42.761 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_delayed": 0, 2024-07-27T03:19:42.761 INFO:teuthology.orchestra.run.smithi175.stdout: "num_strays_enqueuing": 0, 2024-07-27T03:19:42.761 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_created": 0, 2024-07-27T03:19:42.761 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_enqueued": 0, 2024-07-27T03:19:42.761 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_reintegrated": 0, 2024-07-27T03:19:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: "strays_migrated": 0, 2024-07-27T03:19:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_processing": 0, 2024-07-27T03:19:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_enqueued": 0, 2024-07-27T03:19:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: "num_recovering_prioritized": 0, 2024-07-27T03:19:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_started": 0, 2024-07-27T03:19:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: "recovery_completed": 0, 2024-07-27T03:19:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_path": 0, 2024-07-27T03:19:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_quiesce_inode": 0, 2024-07-27T03:19:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_enqueue_scrub": 0, 2024-07-27T03:19:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_exportdir": 0, 2024-07-27T03:19:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_flush": 0, 2024-07-27T03:19:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragmentdir": 0, 2024-07-27T03:19:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_fragstats": 0, 2024-07-27T03:19:42.762 INFO:teuthology.orchestra.run.smithi175.stdout: "ireq_inodestats": 0 2024-07-27T03:19:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_log": { 2024-07-27T03:19:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: "evlrg": 0, 2024-07-27T03:19:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: "evadd": 0, 2024-07-27T03:19:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: "evex": 0, 2024-07-27T03:19:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: "evtrm": 0, 2024-07-27T03:19:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: "ev": 4, 2024-07-27T03:19:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: "evexg": 0, 2024-07-27T03:19:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: "evexd": 0, 2024-07-27T03:19:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: "segadd": 0, 2024-07-27T03:19:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: "segex": 0, 2024-07-27T03:19:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: "segtrm": 0, 2024-07-27T03:19:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: "seg": 2, 2024-07-27T03:19:42.763 INFO:teuthology.orchestra.run.smithi175.stdout: "segmjr": 2, 2024-07-27T03:19:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: "segexg": 0, 2024-07-27T03:19:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: "segexd": 0, 2024-07-27T03:19:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: "expos": 4194304, 2024-07-27T03:19:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: "wrpos": 4195424, 2024-07-27T03:19:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: "rdpos": 4195283, 2024-07-27T03:19:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: "jlat": { 2024-07-27T03:19:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: "replayed": 4 2024-07-27T03:19:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_mem": { 2024-07-27T03:19:42.764 INFO:teuthology.orchestra.run.smithi175.stdout: "ino": 3, 2024-07-27T03:19:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: "ino+": 3, 2024-07-27T03:19:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: "ino-": 0, 2024-07-27T03:19:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: "dir": 1, 2024-07-27T03:19:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: "dir+": 1, 2024-07-27T03:19:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: "dir-": 0, 2024-07-27T03:19:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: "dn": 0, 2024-07-27T03:19:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: "dn+": 0, 2024-07-27T03:19:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: "dn-": 0, 2024-07-27T03:19:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: "cap": 0, 2024-07-27T03:19:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: "cap+": 0, 2024-07-27T03:19:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: "cap-": 0, 2024-07-27T03:19:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: "rss": 116988, 2024-07-27T03:19:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: "heap": 215348 2024-07-27T03:19:42.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_server": { 2024-07-27T03:19:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_client_request": 0, 2024-07-27T03:19:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: "dispatch_server_request": 0, 2024-07-27T03:19:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_request": 0, 2024-07-27T03:19:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_client_session": 0, 2024-07-27T03:19:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: "handle_peer_request": 0, 2024-07-27T03:19:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: "req_create_latency": { 2024-07-27T03:19:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getattr_latency": { 2024-07-27T03:19:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.766 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.767 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.767 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getfilelock_latency": { 2024-07-27T03:19:42.767 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.767 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.767 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.767 INFO:teuthology.orchestra.run.smithi175.stdout: "req_link_latency": { 2024-07-27T03:19:42.767 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.767 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.767 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.767 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookup_latency": { 2024-07-27T03:19:42.767 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.768 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.768 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.768 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookuphash_latency": { 2024-07-27T03:19:42.768 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.768 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.768 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.768 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupino_latency": { 2024-07-27T03:19:42.768 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.768 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.768 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.768 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupname_latency": { 2024-07-27T03:19:42.768 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.769 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.769 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.769 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupparent_latency": { 2024-07-27T03:19:42.769 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.769 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.769 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.769 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lookupsnap_latency": { 2024-07-27T03:19:42.769 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.769 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.769 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.769 INFO:teuthology.orchestra.run.smithi175.stdout: "req_lssnap_latency": { 2024-07-27T03:19:42.770 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.770 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.770 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.770 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mkdir_latency": { 2024-07-27T03:19:42.770 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.770 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:42 smithi175 ceph-mon[37125]: pgmap v524: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-07-27T03:19:42.771 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.771 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.771 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mknod_latency": { 2024-07-27T03:19:42.771 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.771 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.771 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.771 INFO:teuthology.orchestra.run.smithi175.stdout: "req_mksnap_latency": { 2024-07-27T03:19:42.772 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.772 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.772 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.772 INFO:teuthology.orchestra.run.smithi175.stdout: "req_open_latency": { 2024-07-27T03:19:42.772 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.772 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.772 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.772 INFO:teuthology.orchestra.run.smithi175.stdout: "req_readdir_latency": { 2024-07-27T03:19:42.772 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.772 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.772 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.773 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rename_latency": { 2024-07-27T03:19:42.773 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.773 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.773 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.773 INFO:teuthology.orchestra.run.smithi175.stdout: "req_renamesnap_latency": { 2024-07-27T03:19:42.773 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.773 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.773 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.773 INFO:teuthology.orchestra.run.smithi175.stdout: "req_snapdiff_latency": { 2024-07-27T03:19:42.773 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.774 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.774 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.774 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmdir_latency": { 2024-07-27T03:19:42.774 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.774 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.774 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.774 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmsnap_latency": { 2024-07-27T03:19:42.774 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.774 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.774 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.774 INFO:teuthology.orchestra.run.smithi175.stdout: "req_rmxattr_latency": { 2024-07-27T03:19:42.775 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.775 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.775 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.775 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setattr_latency": { 2024-07-27T03:19:42.775 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.775 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.775 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.775 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setdirlayout_latency": { 2024-07-27T03:19:42.775 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.775 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.775 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.776 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setfilelock_latency": { 2024-07-27T03:19:42.776 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.776 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.776 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.776 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setlayout_latency": { 2024-07-27T03:19:42.776 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.776 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.776 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.776 INFO:teuthology.orchestra.run.smithi175.stdout: "req_setxattr_latency": { 2024-07-27T03:19:42.776 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.776 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.776 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.777 INFO:teuthology.orchestra.run.smithi175.stdout: "req_symlink_latency": { 2024-07-27T03:19:42.777 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.777 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.777 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.777 INFO:teuthology.orchestra.run.smithi175.stdout: "req_unlink_latency": { 2024-07-27T03:19:42.777 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.777 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.777 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.777 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_revoke_eviction": 0, 2024-07-27T03:19:42.777 INFO:teuthology.orchestra.run.smithi175.stdout: "cap_acquisition_throttle": 0, 2024-07-27T03:19:42.777 INFO:teuthology.orchestra.run.smithi175.stdout: "req_getvxattr_latency": { 2024-07-27T03:19:42.778 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.778 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.778 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.778 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:42.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.778 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_sessions": { 2024-07-27T03:19:42.778 INFO:teuthology.orchestra.run.smithi175.stdout: "session_count": 0, 2024-07-27T03:19:42.778 INFO:teuthology.orchestra.run.smithi175.stdout: "session_add": 1, 2024-07-27T03:19:42.778 INFO:teuthology.orchestra.run.smithi175.stdout: "session_remove": 1, 2024-07-27T03:19:42.778 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_open": 1, 2024-07-27T03:19:42.778 INFO:teuthology.orchestra.run.smithi175.stdout: "sessions_stale": 0, 2024-07-27T03:19:42.778 INFO:teuthology.orchestra.run.smithi175.stdout: "total_load": 0, 2024-07-27T03:19:42.778 INFO:teuthology.orchestra.run.smithi175.stdout: "average_load": 0, 2024-07-27T03:19:42.779 INFO:teuthology.orchestra.run.smithi175.stdout: "avg_session_uptime": 1388, 2024-07-27T03:19:42.779 INFO:teuthology.orchestra.run.smithi175.stdout: "mdthresh_evicted": 0 2024-07-27T03:19:42.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.779 INFO:teuthology.orchestra.run.smithi175.stdout: "mempool": { 2024-07-27T03:19:42.779 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_bytes": 0, 2024-07-27T03:19:42.779 INFO:teuthology.orchestra.run.smithi175.stdout: "bloom_filter_items": 0, 2024-07-27T03:19:42.779 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_bytes": 0, 2024-07-27T03:19:42.779 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_alloc_items": 0, 2024-07-27T03:19:42.779 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_bytes": 0, 2024-07-27T03:19:42.779 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_data_items": 0, 2024-07-27T03:19:42.779 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_bytes": 0, 2024-07-27T03:19:42.779 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_onode_items": 0, 2024-07-27T03:19:42.779 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_bytes": 0, 2024-07-27T03:19:42.779 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_meta_items": 0, 2024-07-27T03:19:42.780 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_bytes": 0, 2024-07-27T03:19:42.780 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_other_items": 0, 2024-07-27T03:19:42.780 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_bytes": 0, 2024-07-27T03:19:42.780 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_cache_buffer_items": 0, 2024-07-27T03:19:42.780 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_bytes": 0, 2024-07-27T03:19:42.780 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_extent_items": 0, 2024-07-27T03:19:42.780 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_bytes": 0, 2024-07-27T03:19:42.780 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_blob_items": 0, 2024-07-27T03:19:42.780 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_bytes": 0, 2024-07-27T03:19:42.780 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_shared_blob_items": 0, 2024-07-27T03:19:42.780 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_bytes": 0, 2024-07-27T03:19:42.780 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_inline_bl_items": 0, 2024-07-27T03:19:42.780 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_bytes": 0, 2024-07-27T03:19:42.780 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_fsck_items": 0, 2024-07-27T03:19:42.781 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_bytes": 0, 2024-07-27T03:19:42.781 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_txc_items": 0, 2024-07-27T03:19:42.781 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_bytes": 0, 2024-07-27T03:19:42.781 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_deferred_items": 0, 2024-07-27T03:19:42.781 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_bytes": 0, 2024-07-27T03:19:42.781 INFO:teuthology.orchestra.run.smithi175.stdout: "bluestore_writing_items": 0, 2024-07-27T03:19:42.781 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_bytes": 0, 2024-07-27T03:19:42.781 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_items": 0, 2024-07-27T03:19:42.781 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_bytes": 0, 2024-07-27T03:19:42.781 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_reader_items": 0, 2024-07-27T03:19:42.781 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_bytes": 0, 2024-07-27T03:19:42.781 INFO:teuthology.orchestra.run.smithi175.stdout: "bluefs_file_writer_items": 0, 2024-07-27T03:19:42.781 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_bytes": 190063, 2024-07-27T03:19:42.781 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_anon_items": 51, 2024-07-27T03:19:42.782 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_bytes": 0, 2024-07-27T03:19:42.782 INFO:teuthology.orchestra.run.smithi175.stdout: "buffer_meta_items": 0, 2024-07-27T03:19:42.782 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_bytes": 0, 2024-07-27T03:19:42.782 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_items": 0, 2024-07-27T03:19:42.782 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_bytes": 0, 2024-07-27T03:19:42.782 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_mapbl_items": 0, 2024-07-27T03:19:42.782 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_bytes": 0, 2024-07-27T03:19:42.782 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_pglog_items": 0, 2024-07-27T03:19:42.782 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_bytes": 8952, 2024-07-27T03:19:42.782 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_items": 168, 2024-07-27T03:19:42.782 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_bytes": 0, 2024-07-27T03:19:42.782 INFO:teuthology.orchestra.run.smithi175.stdout: "osdmap_mapping_items": 0, 2024-07-27T03:19:42.782 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_bytes": 0, 2024-07-27T03:19:42.783 INFO:teuthology.orchestra.run.smithi175.stdout: "pgmap_items": 0, 2024-07-27T03:19:42.783 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_bytes": 8512, 2024-07-27T03:19:42.783 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_co_items": 13, 2024-07-27T03:19:42.783 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_bytes": 0, 2024-07-27T03:19:42.783 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_1_items": 0, 2024-07-27T03:19:42.783 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_bytes": 0, 2024-07-27T03:19:42.783 INFO:teuthology.orchestra.run.smithi175.stdout: "unittest_2_items": 0 2024-07-27T03:19:42.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.783 INFO:teuthology.orchestra.run.smithi175.stdout: "objecter": { 2024-07-27T03:19:42.783 INFO:teuthology.orchestra.run.smithi175.stdout: "op_active": 0, 2024-07-27T03:19:42.783 INFO:teuthology.orchestra.run.smithi175.stdout: "op_laggy": 0, 2024-07-27T03:19:42.783 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send": 989, 2024-07-27T03:19:42.783 INFO:teuthology.orchestra.run.smithi175.stdout: "op_send_bytes": 21, 2024-07-27T03:19:42.783 INFO:teuthology.orchestra.run.smithi175.stdout: "op_resend": 3, 2024-07-27T03:19:42.784 INFO:teuthology.orchestra.run.smithi175.stdout: "op_reply": 989, 2024-07-27T03:19:42.784 INFO:teuthology.orchestra.run.smithi175.stdout: "op_latency": { 2024-07-27T03:19:42.784 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 989, 2024-07-27T03:19:42.784 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.484991974, 2024-07-27T03:19:42.784 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000490386 2024-07-27T03:19:42.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.784 INFO:teuthology.orchestra.run.smithi175.stdout: "op_inflight": 0, 2024-07-27T03:19:42.784 INFO:teuthology.orchestra.run.smithi175.stdout: "oplen_avg": { 2024-07-27T03:19:42.784 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 989, 2024-07-27T03:19:42.784 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 990 2024-07-27T03:19:42.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.784 INFO:teuthology.orchestra.run.smithi175.stdout: "op": 989, 2024-07-27T03:19:42.784 INFO:teuthology.orchestra.run.smithi175.stdout: "op_r": 989, 2024-07-27T03:19:42.784 INFO:teuthology.orchestra.run.smithi175.stdout: "op_w": 0, 2024-07-27T03:19:42.785 INFO:teuthology.orchestra.run.smithi175.stdout: "op_rmw": 0, 2024-07-27T03:19:42.785 INFO:teuthology.orchestra.run.smithi175.stdout: "op_pg": 0, 2024-07-27T03:19:42.785 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_stat": 619, 2024-07-27T03:19:42.785 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_create": 0, 2024-07-27T03:19:42.785 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_read": 368, 2024-07-27T03:19:42.785 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_write": 0, 2024-07-27T03:19:42.785 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writefull": 0, 2024-07-27T03:19:42.785 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_writesame": 0, 2024-07-27T03:19:42.785 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_append": 0, 2024-07-27T03:19:42.785 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_zero": 0, 2024-07-27T03:19:42.785 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_truncate": 0, 2024-07-27T03:19:42.785 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_delete": 0, 2024-07-27T03:19:42.785 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_mapext": 0, 2024-07-27T03:19:42.785 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_sparse_read": 0, 2024-07-27T03:19:42.786 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_clonerange": 0, 2024-07-27T03:19:42.786 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_getxattr": 1, 2024-07-27T03:19:42.786 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_setxattr": 0, 2024-07-27T03:19:42.786 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_cmpxattr": 0, 2024-07-27T03:19:42.786 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_rmxattr": 0, 2024-07-27T03:19:42.786 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_resetxattrs": 0, 2024-07-27T03:19:42.786 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_call": 0, 2024-07-27T03:19:42.786 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_watch": 0, 2024-07-27T03:19:42.786 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_notify": 0, 2024-07-27T03:19:42.786 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_src_cmpxattr": 0, 2024-07-27T03:19:42.786 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls": 0, 2024-07-27T03:19:42.786 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_pgls_filter": 0, 2024-07-27T03:19:42.786 INFO:teuthology.orchestra.run.smithi175.stdout: "osdop_other": 0, 2024-07-27T03:19:42.786 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_active": 0, 2024-07-27T03:19:42.787 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_send": 0, 2024-07-27T03:19:42.787 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_resend": 0, 2024-07-27T03:19:42.787 INFO:teuthology.orchestra.run.smithi175.stdout: "linger_ping": 0, 2024-07-27T03:19:42.787 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_active": 0, 2024-07-27T03:19:42.787 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_send": 0, 2024-07-27T03:19:42.787 INFO:teuthology.orchestra.run.smithi175.stdout: "poolop_resend": 0, 2024-07-27T03:19:42.787 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_active": 0, 2024-07-27T03:19:42.787 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_send": 0, 2024-07-27T03:19:42.787 INFO:teuthology.orchestra.run.smithi175.stdout: "poolstat_resend": 0, 2024-07-27T03:19:42.787 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_active": 0, 2024-07-27T03:19:42.787 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_send": 0, 2024-07-27T03:19:42.787 INFO:teuthology.orchestra.run.smithi175.stdout: "statfs_resend": 0, 2024-07-27T03:19:42.787 INFO:teuthology.orchestra.run.smithi175.stdout: "command_active": 0, 2024-07-27T03:19:42.788 INFO:teuthology.orchestra.run.smithi175.stdout: "command_send": 0, 2024-07-27T03:19:42.788 INFO:teuthology.orchestra.run.smithi175.stdout: "command_resend": 0, 2024-07-27T03:19:42.788 INFO:teuthology.orchestra.run.smithi175.stdout: "map_epoch": 91, 2024-07-27T03:19:42.788 INFO:teuthology.orchestra.run.smithi175.stdout: "map_full": 0, 2024-07-27T03:19:42.788 INFO:teuthology.orchestra.run.smithi175.stdout: "map_inc": 2, 2024-07-27T03:19:42.788 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_sessions": 5, 2024-07-27T03:19:42.788 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_open": 5, 2024-07-27T03:19:42.788 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_session_close": 0, 2024-07-27T03:19:42.788 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_laggy": 0, 2024-07-27T03:19:42.788 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_wr": 0, 2024-07-27T03:19:42.788 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_rd": 2, 2024-07-27T03:19:42.788 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_del": 0 2024-07-27T03:19:42.788 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.788 INFO:teuthology.orchestra.run.smithi175.stdout: "oft": { 2024-07-27T03:19:42.789 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_objs": 0, 2024-07-27T03:19:42.789 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_kv_pairs": 0, 2024-07-27T03:19:42.789 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_updates": 0, 2024-07-27T03:19:42.789 INFO:teuthology.orchestra.run.smithi175.stdout: "omap_total_removes": 0 2024-07-27T03:19:42.789 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.789 INFO:teuthology.orchestra.run.smithi175.stdout: "osd-slow-ops": { 2024-07-27T03:19:42.789 INFO:teuthology.orchestra.run.smithi175.stdout: "slow_ops_count": 0 2024-07-27T03:19:42.789 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.789 INFO:teuthology.orchestra.run.smithi175.stdout: "purge_queue": { 2024-07-27T03:19:42.789 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops": 0, 2024-07-27T03:19:42.789 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_ops_high_water": 0, 2024-07-27T03:19:42.789 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing": 0, 2024-07-27T03:19:42.789 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executing_high_water": 0, 2024-07-27T03:19:42.789 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed_ops": 0, 2024-07-27T03:19:42.790 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_executed": 0, 2024-07-27T03:19:42.790 INFO:teuthology.orchestra.run.smithi175.stdout: "pq_item_in_journal": 0 2024-07-27T03:19:42.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.790 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-07-27T03:19:42.790 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-27T03:19:42.790 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-07-27T03:19:42.790 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-27T03:19:42.790 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 1131, 2024-07-27T03:19:42.790 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 698298, 2024-07-27T03:19:42.790 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:42.790 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 1131, 2024-07-27T03:19:42.790 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-07-27T03:19:42.790 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-07-27T03:19:42.790 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 1131, 2024-07-27T03:19:42.790 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 698298, 2024-07-27T03:19:42.791 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-27T03:19:42.791 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.791 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.791 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.791 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:42.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.791 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_bytes": { 2024-07-27T03:19:42.791 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-27T03:19:42.791 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 104857600, 2024-07-27T03:19:42.791 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-27T03:19:42.791 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-07-27T03:19:42.791 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-07-27T03:19:42.791 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:42.791 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:42.792 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 989, 2024-07-27T03:19:42.792 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 1125, 2024-07-27T03:19:42.792 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 989, 2024-07-27T03:19:42.792 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 1125, 2024-07-27T03:19:42.792 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-27T03:19:42.792 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.792 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.792 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.792 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:42.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.792 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-objecter_ops": { 2024-07-27T03:19:42.792 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-27T03:19:42.792 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 1024, 2024-07-27T03:19:42.792 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-27T03:19:42.793 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-07-27T03:19:42.793 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-07-27T03:19:42.793 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:42.793 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:42.793 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 989, 2024-07-27T03:19:42.793 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 989, 2024-07-27T03:19:42.793 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 989, 2024-07-27T03:19:42.793 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 989, 2024-07-27T03:19:42.793 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-27T03:19:42.793 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.793 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.793 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.793 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:42.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.794 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle": { 2024-07-27T03:19:42.794 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-27T03:19:42.794 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-07-27T03:19:42.794 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-27T03:19:42.794 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-07-27T03:19:42.794 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-07-27T03:19:42.794 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:42.794 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:42.794 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-07-27T03:19:42.794 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-07-27T03:19:42.794 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-07-27T03:19:42.794 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-07-27T03:19:42.794 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-27T03:19:42.794 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.795 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.795 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.795 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:42.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-07-27T03:19:42.795 INFO:teuthology.orchestra.run.smithi175.stdout: "throttle-write_buf_throttle-0x5590f17668f0": { 2024-07-27T03:19:42.795 INFO:teuthology.orchestra.run.smithi175.stdout: "val": 0, 2024-07-27T03:19:42.795 INFO:teuthology.orchestra.run.smithi175.stdout: "max": 3758096384, 2024-07-27T03:19:42.795 INFO:teuthology.orchestra.run.smithi175.stdout: "get_started": 0, 2024-07-27T03:19:42.795 INFO:teuthology.orchestra.run.smithi175.stdout: "get": 0, 2024-07-27T03:19:42.795 INFO:teuthology.orchestra.run.smithi175.stdout: "get_sum": 0, 2024-07-27T03:19:42.795 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_fail": 0, 2024-07-27T03:19:42.795 INFO:teuthology.orchestra.run.smithi175.stdout: "get_or_fail_success": 0, 2024-07-27T03:19:42.795 INFO:teuthology.orchestra.run.smithi175.stdout: "take": 0, 2024-07-27T03:19:42.795 INFO:teuthology.orchestra.run.smithi175.stdout: "take_sum": 0, 2024-07-27T03:19:42.796 INFO:teuthology.orchestra.run.smithi175.stdout: "put": 0, 2024-07-27T03:19:42.796 INFO:teuthology.orchestra.run.smithi175.stdout: "put_sum": 0, 2024-07-27T03:19:42.796 INFO:teuthology.orchestra.run.smithi175.stdout: "wait": { 2024-07-27T03:19:42.796 INFO:teuthology.orchestra.run.smithi175.stdout: "avgcount": 0, 2024-07-27T03:19:42.796 INFO:teuthology.orchestra.run.smithi175.stdout: "sum": 0.000000000, 2024-07-27T03:19:42.796 INFO:teuthology.orchestra.run.smithi175.stdout: "avgtime": 0.000000000 2024-07-27T03:19:42.796 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:42.796 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-07-27T03:19:42.796 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-07-27T03:19:42.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:42 smithi067 ceph-mon[32814]: pgmap v524: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-07-27T03:19:43.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:42 smithi149 ceph-mon[37173]: pgmap v524: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-07-27T03:19:43.583 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-07-27T03:19:43.583 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-07-27T03:19:43.583 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-07-27T03:19:43.583 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-07-27T03:19:43.584 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-07-27T03:19:43.594 INFO:tasks.kclient:Unmounting kernel clients... 2024-07-27T03:19:43.594 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-27T03:19:43.595 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-27T03:19:43.617 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-07-27T03:19:43.618 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/proc/self/mounts of=/dev/stdout 2024-07-27T03:19:43.679 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd blocklist ls 2024-07-27T03:19:43.997 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:19:44.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:44 smithi067 ceph-mon[32814]: pgmap v525: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:19:44.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:44 smithi149 ceph-mon[37173]: pgmap v525: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:19:45.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:44 smithi175 ceph-mon[37125]: pgmap v525: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-07-27T03:19:45.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.811+0000 7f3a89f92640 1 -- 172.21.15.67:0/2097398051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a84106690 msgr2=0x7f3a841100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:45.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.811+0000 7f3a89f92640 1 --2- 172.21.15.67:0/2097398051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a84106690 0x7f3a841100e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009ca0 tx=0x7f3a7c02a8c0 comp rx=0 tx=0).stop 2024-07-27T03:19:45.815 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.811+0000 7f3a89f92640 1 -- 172.21.15.67:0/2097398051 shutdown_connections 2024-07-27T03:19:45.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.811+0000 7f3a89f92640 1 --2- 172.21.15.67:0/2097398051 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a84106690 0x7f3a841100e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:45.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.811+0000 7f3a89f92640 1 --2- 172.21.15.67:0/2097398051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a84105d40 0x7f3a84106150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:45.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.811+0000 7f3a89f92640 1 --2- 172.21.15.67:0/2097398051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a84101a60 0x7f3a84101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:45.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.811+0000 7f3a89f92640 1 -- 172.21.15.67:0/2097398051 >> 172.21.15.67:0/2097398051 conn(0x7f3a84077730 msgr2=0x7f3a84077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:45.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.811+0000 7f3a89f92640 1 -- 172.21.15.67:0/2097398051 shutdown_connections 2024-07-27T03:19:45.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.811+0000 7f3a89f92640 1 -- 172.21.15.67:0/2097398051 wait complete. 2024-07-27T03:19:45.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.811+0000 7f3a89f92640 1 Processor -- start 2024-07-27T03:19:45.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.811+0000 7f3a89f92640 1 -- start start 2024-07-27T03:19:45.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.812+0000 7f3a89f92640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a84101a60 0x7f3a84199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:45.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.812+0000 7f3a89f92640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a84105d40 0x7f3a8419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:45.816 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.812+0000 7f3a89f92640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a84106690 0x7f3a8419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:45.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.812+0000 7f3a89f92640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f3a84113550 con 0x7f3a84106690 2024-07-27T03:19:45.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.812+0000 7f3a89f92640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3a841133d0 con 0x7f3a84105d40 2024-07-27T03:19:45.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.812+0000 7f3a89f92640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3a841136d0 con 0x7f3a84101a60 2024-07-27T03:19:45.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.812+0000 7f3a83fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a84106690 0x7f3a8419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:45.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.812+0000 7f3a83fff640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a84106690 0x7f3a8419f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60258/0 (socket says 172.21.15.67:60258) 2024-07-27T03:19:45.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.812+0000 7f3a83fff640 1 -- 172.21.15.67:0/3309350721 learned_addr learned my addr 172.21.15.67:0/3309350721 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:19:45.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.812+0000 7f3a837fe640 1 --2- 172.21.15.67:0/3309350721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a84101a60 0x7f3a84199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:45.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.812+0000 7f3a7bfff640 1 --2- 172.21.15.67:0/3309350721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a84105d40 0x7f3a8419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:45.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.812+0000 7f3a83fff640 1 -- 172.21.15.67:0/3309350721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a84101a60 msgr2=0x7f3a84199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:45.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.812+0000 7f3a83fff640 1 --2- 172.21.15.67:0/3309350721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a84101a60 0x7f3a84199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:45.817 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.812+0000 7f3a83fff640 1 -- 172.21.15.67:0/3309350721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a84105d40 msgr2=0x7f3a8419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:45.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.812+0000 7f3a83fff640 1 --2- 172.21.15.67:0/3309350721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a84105d40 0x7f3a8419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:45.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.812+0000 7f3a83fff640 1 -- 172.21.15.67:0/3309350721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8419fdf0 con 0x7f3a84106690 2024-07-27T03:19:45.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.812+0000 7f3a83fff640 1 --2- 172.21.15.67:0/3309350721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a84106690 0x7f3a8419f780 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c005cc0 tx=0x7f3a7c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:45.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.813+0000 7f3a817fa640 1 -- 172.21.15.67:0/3309350721 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c003bf0 con 0x7f3a84106690 2024-07-27T03:19:45.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.813+0000 7f3a817fa640 1 -- 172.21.15.67:0/3309350721 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a7c003d90 con 0x7f3a84106690 2024-07-27T03:19:45.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.813+0000 7f3a89f92640 1 -- 172.21.15.67:0/3309350721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a841a0080 con 0x7f3a84106690 2024-07-27T03:19:45.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.813+0000 7f3a817fa640 1 -- 172.21.15.67:0/3309350721 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c02fbc0 con 0x7f3a84106690 2024-07-27T03:19:45.818 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.813+0000 7f3a89f92640 1 -- 172.21.15.67:0/3309350721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a841a6900 con 0x7f3a84106690 2024-07-27T03:19:45.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.814+0000 7f3a89f92640 1 -- 172.21.15.67:0/3309350721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a84101e70 con 0x7f3a84106690 2024-07-27T03:19:45.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.815+0000 7f3a817fa640 1 -- 172.21.15.67:0/3309350721 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f3a7c02fd60 con 0x7f3a84106690 2024-07-27T03:19:45.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.816+0000 7f3a817fa640 1 --2- 172.21.15.67:0/3309350721 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3a700b3c80 0x7f3a700b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:45.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.816+0000 7f3a837fe640 1 --2- 172.21.15.67:0/3309350721 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3a700b3c80 0x7f3a700b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:45.819 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.816+0000 7f3a817fa640 1 -- 172.21.15.67:0/3309350721 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f3a7c1056d0 con 0x7f3a84106690 2024-07-27T03:19:45.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.817+0000 7f3a837fe640 1 --2- 172.21.15.67:0/3309350721 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3a700b3c80 0x7f3a700b6190 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3a74005e10 tx=0x7f3a74005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:45.821 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:45.819+0000 7f3a817fa640 1 -- 172.21.15.67:0/3309350721 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f3a7c0cc970 con 0x7f3a84106690 2024-07-27T03:19:46.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:46.017+0000 7f3a89f92640 1 -- 172.21.15.67:0/3309350721 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3a8419b070 con 0x7f3a84106690 2024-07-27T03:19:46.019 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:46.017+0000 7f3a817fa640 1 -- 172.21.15.67:0/3309350721 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) ==== 81+0+590 (secure 0 0 0) 0x7f3a8419b070 con 0x7f3a84106690 2024-07-27T03:19:46.020 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/3477601235 2024-07-28T03:04:54.947985+0000 2024-07-27T03:19:46.020 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/4130679770 2024-07-28T03:04:30.600273+0000 2024-07-27T03:19:46.020 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/1921856974 2024-07-28T03:04:54.947985+0000 2024-07-27T03:19:46.020 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/1029332005 2024-07-28T03:04:54.947985+0000 2024-07-27T03:19:46.020 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/2346308006 2024-07-28T03:04:30.600273+0000 2024-07-27T03:19:46.020 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6801/1921856974 2024-07-28T03:04:54.947985+0000 2024-07-27T03:19:46.020 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/284078926 2024-07-28T03:04:54.947985+0000 2024-07-27T03:19:46.020 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:6800/2346308006 2024-07-28T03:04:30.600273+0000 2024-07-27T03:19:46.020 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/2559121542 2024-07-28T03:04:30.600273+0000 2024-07-27T03:19:46.020 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67:0/925406021 2024-07-28T03:04:30.600273+0000 2024-07-27T03:19:46.021 INFO:teuthology.orchestra.run.smithi067.stderr:listed 10 entries 2024-07-27T03:19:46.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:46.020+0000 7f3a89f92640 1 -- 172.21.15.67:0/3309350721 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3a700b3c80 msgr2=0x7f3a700b6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:46.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:46.020+0000 7f3a89f92640 1 --2- 172.21.15.67:0/3309350721 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3a700b3c80 0x7f3a700b6190 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3a74005e10 tx=0x7f3a74005da0 comp rx=0 tx=0).stop 2024-07-27T03:19:46.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:46.020+0000 7f3a89f92640 1 -- 172.21.15.67:0/3309350721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a84106690 msgr2=0x7f3a8419f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:46.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:46.020+0000 7f3a89f92640 1 --2- 172.21.15.67:0/3309350721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a84106690 0x7f3a8419f780 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c005cc0 tx=0x7f3a7c02f380 comp rx=0 tx=0).stop 2024-07-27T03:19:46.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:46.020+0000 7f3a89f92640 1 -- 172.21.15.67:0/3309350721 shutdown_connections 2024-07-27T03:19:46.022 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:46.020+0000 7f3a89f92640 1 --2- 172.21.15.67:0/3309350721 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f3a700b3c80 0x7f3a700b6190 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:46.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:46.020+0000 7f3a89f92640 1 --2- 172.21.15.67:0/3309350721 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a84106690 0x7f3a8419f780 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:46.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:46.020+0000 7f3a89f92640 1 --2- 172.21.15.67:0/3309350721 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a84105d40 0x7f3a8419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:46.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:46.020+0000 7f3a89f92640 1 --2- 172.21.15.67:0/3309350721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a84101a60 0x7f3a84199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:46.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:46.020+0000 7f3a89f92640 1 -- 172.21.15.67:0/3309350721 >> 172.21.15.67:0/3309350721 conn(0x7f3a84077730 msgr2=0x7f3a84106ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:46.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:46.020+0000 7f3a89f92640 1 -- 172.21.15.67:0/3309350721 shutdown_connections 2024-07-27T03:19:46.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:46.020+0000 7f3a89f92640 1 -- 172.21.15.67:0/3309350721 wait complete. 2024-07-27T03:19:46.771 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-07-27T03:19:46.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:19:46.772 DEBUG:teuthology.orchestra.run.smithi067:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-07-27T03:19:46.821 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi067.front.sepia.ceph.com 2024-07-27T03:19:46.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:19:46.822 DEBUG:teuthology.orchestra.run.smithi067:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-07-27T03:19:46.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:19:46.852 DEBUG:teuthology.orchestra.run.smithi067:> ip netns list 2024-07-27T03:19:46.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:46 smithi175 ceph-mon[37125]: pgmap v526: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:19:46.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:46 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3309350721' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-27T03:19:46.909 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-07-27T03:19:46.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:19:46.909 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-07-27T03:19:46.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:46 smithi067 ceph-mon[32814]: pgmap v526: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:19:46.971 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:46 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3309350721' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-27T03:19:46.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-07-27T03:19:46.976 DEBUG:teuthology.orchestra.run.smithi067:> sudo ip link delete ceph-brx 2024-07-27T03:19:46.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:46 smithi149 ceph-mon[37173]: pgmap v526: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-07-27T03:19:47.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:46 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3309350721' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-07-27T03:19:47.072 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-07-27T03:19:47.082 INFO:tasks.cephadm:Teardown begin 2024-07-27T03:19:47.083 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-07-27T03:19:47.083 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:19:47.351 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:19:48.846 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:48 smithi067 ceph-mon[32814]: pgmap v527: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:19:48.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:48 smithi149 ceph-mon[37173]: pgmap v527: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:19:49.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.094+0000 7f41c78f1640 1 -- 172.21.15.67:0/3194349629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c01024d0 msgr2=0x7f41c01028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:49.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.095+0000 7f41c78f1640 1 --2- 172.21.15.67:0/3194349629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c01024d0 0x7f41c01028e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009ca0 tx=0x7f41b002a8c0 comp rx=0 tx=0).stop 2024-07-27T03:19:49.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.095+0000 7f41c78f1640 1 -- 172.21.15.67:0/3194349629 shutdown_connections 2024-07-27T03:19:49.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.095+0000 7f41c78f1640 1 --2- 172.21.15.67:0/3194349629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41c0108760 0x7f41c010ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:49.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.095+0000 7f41c78f1640 1 --2- 172.21.15.67:0/3194349629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41c0107e10 0x7f41c0108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:49.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.095+0000 7f41c78f1640 1 --2- 172.21.15.67:0/3194349629 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c01024d0 0x7f41c01028e0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:49.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.095+0000 7f41c78f1640 1 -- 172.21.15.67:0/3194349629 >> 172.21.15.67:0/3194349629 conn(0x7f41c00791a0 msgr2=0x7f41c00795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.095+0000 7f41c78f1640 1 -- 172.21.15.67:0/3194349629 shutdown_connections 2024-07-27T03:19:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.095+0000 7f41c78f1640 1 -- 172.21.15.67:0/3194349629 wait complete. 2024-07-27T03:19:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.095+0000 7f41c78f1640 1 Processor -- start 2024-07-27T03:19:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.096+0000 7f41c78f1640 1 -- start start 2024-07-27T03:19:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.096+0000 7f41c78f1640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c01024d0 0x7f41c019acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.096+0000 7f41c78f1640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41c0107e10 0x7f41c019b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.096+0000 7f41c78f1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41c0108760 0x7f41c019b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.096+0000 7f41c78f1640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f41c01123c0 con 0x7f41c01024d0 2024-07-27T03:19:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.096+0000 7f41c78f1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f41c0112240 con 0x7f41c0108760 2024-07-27T03:19:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.096+0000 7f41c78f1640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f41c0112540 con 0x7f41c0107e10 2024-07-27T03:19:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.096+0000 7f41c568d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c01024d0 0x7f41c019acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:49.100 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.096+0000 7f41c568d640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c01024d0 0x7f41c019acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60276/0 (socket says 172.21.15.67:60276) 2024-07-27T03:19:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.096+0000 7f41c568d640 1 -- 172.21.15.67:0/719176387 learned_addr learned my addr 172.21.15.67:0/719176387 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:19:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.096+0000 7f41c4e8c640 1 --2- 172.21.15.67:0/719176387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41c0107e10 0x7f41c019b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.096+0000 7f41c5e8e640 1 --2- 172.21.15.67:0/719176387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41c0108760 0x7f41c019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.096+0000 7f41c568d640 1 -- 172.21.15.67:0/719176387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41c0107e10 msgr2=0x7f41c019b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.096+0000 7f41c568d640 1 --2- 172.21.15.67:0/719176387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41c0107e10 0x7f41c019b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.096+0000 7f41c568d640 1 -- 172.21.15.67:0/719176387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41c0108760 msgr2=0x7f41c019b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.096+0000 7f41c568d640 1 --2- 172.21.15.67:0/719176387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41c0108760 0x7f41c019b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.096+0000 7f41c568d640 1 -- 172.21.15.67:0/719176387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c01a5f90 con 0x7f41c01024d0 2024-07-27T03:19:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.096+0000 7f41c5e8e640 1 --2- 172.21.15.67:0/719176387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41c0108760 0x7f41c019b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-07-27T03:19:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.097+0000 7f41c568d640 1 --2- 172.21.15.67:0/719176387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c01024d0 0x7f41c019acd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f41b0006010 tx=0x7f41b00319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.097+0000 7f41be7fc640 1 -- 172.21.15.67:0/719176387 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b0003bf0 con 0x7f41c01024d0 2024-07-27T03:19:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.097+0000 7f41c78f1640 1 -- 172.21.15.67:0/719176387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c01a6220 con 0x7f41c01024d0 2024-07-27T03:19:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.097+0000 7f41be7fc640 1 -- 172.21.15.67:0/719176387 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41b0003d90 con 0x7f41c01024d0 2024-07-27T03:19:49.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.097+0000 7f41be7fc640 1 -- 172.21.15.67:0/719176387 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b002fc20 con 0x7f41c01024d0 2024-07-27T03:19:49.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.097+0000 7f41c78f1640 1 -- 172.21.15.67:0/719176387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c01a67b0 con 0x7f41c01024d0 2024-07-27T03:19:49.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.098+0000 7f41c78f1640 1 -- 172.21.15.67:0/719176387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c01028e0 con 0x7f41c01024d0 2024-07-27T03:19:49.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.099+0000 7f41be7fc640 1 -- 172.21.15.67:0/719176387 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f41b00046a0 con 0x7f41c01024d0 2024-07-27T03:19:49.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.100+0000 7f41be7fc640 1 --2- 172.21.15.67:0/719176387 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f41a80b3bd0 0x7f41a80b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:49.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.100+0000 7f41c4e8c640 1 --2- 172.21.15.67:0/719176387 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f41a80b3bd0 0x7f41a80b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:49.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.100+0000 7f41be7fc640 1 -- 172.21.15.67:0/719176387 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f41b0105520 con 0x7f41c01024d0 2024-07-27T03:19:49.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.101+0000 7f41c4e8c640 1 --2- 172.21.15.67:0/719176387 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f41a80b3bd0 0x7f41a80b60e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f41c0100730 tx=0x7f41b400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:49.107 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.105+0000 7f41be7fc640 1 -- 172.21.15.67:0/719176387 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f41b00cc7c0 con 0x7f41c01024d0 2024-07-27T03:19:49.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:48 smithi175 ceph-mon[37125]: pgmap v527: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-07-27T03:19:49.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.323+0000 7f41c78f1640 1 -- 172.21.15.67:0/719176387 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41c002e100 con 0x7f41c01024d0 2024-07-27T03:19:49.326 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.324+0000 7f41be7fc640 1 -- 172.21.15.67:0/719176387 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11983 (secure 0 0 0) 0x7f41b00cc0c0 con 0x7f41c01024d0 2024-07-27T03:19:49.327 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:19:49.327 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"btime":"2024-07-27T03:16:40:350577+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14679,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/252686369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":252686369},{"type":"v1","addr":"172.21.15.67:6835","nonce":252686369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/3358832762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3358832762},{"type":"v1","addr":"172.21.15.67:6837","nonce":3358832762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-27T03:12:34.320208+0000","modified":"2024-07-27T03:16:39.741149+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24620,"mds_1":24587,"mds_2":24583,"mds_3":24581,"mds_4":24563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14736":{"gid":14736,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/2162708317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2162708317},{"type":"v1","addr":"172.21.15.67:6839","nonce":2162708317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14748":{"gid":14748,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/795704244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":795704244},{"type":"v1","addr":"172.21.15.67:6841","nonce":795704244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/3868325215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3868325215},{"type":"v1","addr":"172.21.15.149:6833","nonce":3868325215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2462749027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2462749027},{"type":"v1","addr":"172.21.15.175:6835","nonce":2462749027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.149:6835/3730964806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3730964806},{"type":"v1","addr":"172.21.15.149:6835","nonce":3730964806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.175:6837/1304803502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1304803502},{"type":"v1","addr":"172.21.15.175:6837","nonce":1304803502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.175:6839/1670568853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1670568853},{"type":"v1","addr":"172.21.15.175:6839","nonce":1670568853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24587":{"gid":24587,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.149:6837/2932951845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2932951845},{"type":"v1","addr":"172.21.15.149:6837","nonce":2932951845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.149:6839/281746498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":281746498},{"type":"v1","addr":"172.21.15.149:6839","nonce":281746498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24620,"qdb_cluster":[24587,24583,24620,24581,24563]},"id":1}]} 2024-07-27T03:19:49.327 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 30 2024-07-27T03:19:49.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.326+0000 7f41c78f1640 1 -- 172.21.15.67:0/719176387 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f41a80b3bd0 msgr2=0x7f41a80b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:49.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.326+0000 7f41c78f1640 1 --2- 172.21.15.67:0/719176387 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f41a80b3bd0 0x7f41a80b60e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f41c0100730 tx=0x7f41b400b410 comp rx=0 tx=0).stop 2024-07-27T03:19:49.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.326+0000 7f41c78f1640 1 -- 172.21.15.67:0/719176387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c01024d0 msgr2=0x7f41c019acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:49.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.326+0000 7f41c78f1640 1 --2- 172.21.15.67:0/719176387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c01024d0 0x7f41c019acd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f41b0006010 tx=0x7f41b00319f0 comp rx=0 tx=0).stop 2024-07-27T03:19:49.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.326+0000 7f41c78f1640 1 -- 172.21.15.67:0/719176387 shutdown_connections 2024-07-27T03:19:49.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.326+0000 7f41c78f1640 1 --2- 172.21.15.67:0/719176387 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f41a80b3bd0 0x7f41a80b60e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:49.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.326+0000 7f41c78f1640 1 --2- 172.21.15.67:0/719176387 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f41c0108760 0x7f41c019b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:49.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.326+0000 7f41c78f1640 1 --2- 172.21.15.67:0/719176387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41c0107e10 0x7f41c019b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:49.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.326+0000 7f41c78f1640 1 --2- 172.21.15.67:0/719176387 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f41c01024d0 0x7f41c019acd0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:49.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.326+0000 7f41c78f1640 1 -- 172.21.15.67:0/719176387 >> 172.21.15.67:0/719176387 conn(0x7f41c00791a0 msgr2=0x7f41c0100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:49.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.326+0000 7f41c78f1640 1 -- 172.21.15.67:0/719176387 shutdown_connections 2024-07-27T03:19:49.329 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:49.326+0000 7f41c78f1640 1 -- 172.21.15.67:0/719176387 wait complete. 2024-07-27T03:19:49.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:49 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/719176387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:19:49.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:49 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/719176387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:19:50.027 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs dump --format=json 2024-07-27T03:19:50.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:49 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/719176387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:19:50.298 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:19:50.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:50 smithi149 ceph-mon[37173]: pgmap v528: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-07-27T03:19:51.101 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:50 smithi067 ceph-mon[32814]: pgmap v528: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-07-27T03:19:51.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:50 smithi175 ceph-mon[37125]: pgmap v528: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-07-27T03:19:52.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.034+0000 7f7a43a90640 1 -- 172.21.15.67:0/905920189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a3c107e10 msgr2=0x7f7a3c108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:52.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.034+0000 7f7a43a90640 1 --2- 172.21.15.67:0/905920189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a3c107e10 0x7f7a3c108220 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009900 tx=0x7f7a2c02a410 comp rx=0 tx=0).stop 2024-07-27T03:19:52.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.034+0000 7f7a43a90640 1 -- 172.21.15.67:0/905920189 shutdown_connections 2024-07-27T03:19:52.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.034+0000 7f7a43a90640 1 --2- 172.21.15.67:0/905920189 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a3c108760 0x7f7a3c10ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:52.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.034+0000 7f7a43a90640 1 --2- 172.21.15.67:0/905920189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a3c107e10 0x7f7a3c108220 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:52.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.034+0000 7f7a43a90640 1 --2- 172.21.15.67:0/905920189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a3c1024d0 0x7f7a3c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:52.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.034+0000 7f7a43a90640 1 -- 172.21.15.67:0/905920189 >> 172.21.15.67:0/905920189 conn(0x7f7a3c079190 msgr2=0x7f7a3c0795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:52.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.034+0000 7f7a43a90640 1 -- 172.21.15.67:0/905920189 shutdown_connections 2024-07-27T03:19:52.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.034+0000 7f7a43a90640 1 -- 172.21.15.67:0/905920189 wait complete. 2024-07-27T03:19:52.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.034+0000 7f7a43a90640 1 Processor -- start 2024-07-27T03:19:52.037 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.034+0000 7f7a43a90640 1 -- start start 2024-07-27T03:19:52.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.035+0000 7f7a43a90640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a3c1024d0 0x7f7a3c10fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:52.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.035+0000 7f7a43a90640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a3c107e10 0x7f7a3c10ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:52.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.035+0000 7f7a43a90640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a3c108760 0x7f7a3c1104a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:52.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.035+0000 7f7a43a90640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c1134f0 con 0x7f7a3c1024d0 2024-07-27T03:19:52.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.035+0000 7f7a43a90640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c113370 con 0x7f7a3c108760 2024-07-27T03:19:52.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.035+0000 7f7a43a90640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c113670 con 0x7f7a3c107e10 2024-07-27T03:19:52.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.035+0000 7f7a4182c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a3c1024d0 0x7f7a3c10fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:52.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.035+0000 7f7a4182c640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a3c1024d0 0x7f7a3c10fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:60288/0 (socket says 172.21.15.67:60288) 2024-07-27T03:19:52.039 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.035+0000 7f7a4182c640 1 -- 172.21.15.67:0/3361703373 learned_addr learned my addr 172.21.15.67:0/3361703373 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:19:52.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.035+0000 7f7a4202d640 1 --2- 172.21.15.67:0/3361703373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a3c108760 0x7f7a3c1104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:52.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.035+0000 7f7a4102b640 1 --2- 172.21.15.67:0/3361703373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a3c107e10 0x7f7a3c10ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:52.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.035+0000 7f7a4182c640 1 -- 172.21.15.67:0/3361703373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a3c107e10 msgr2=0x7f7a3c10ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:52.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.035+0000 7f7a4182c640 1 --2- 172.21.15.67:0/3361703373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a3c107e10 0x7f7a3c10ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:52.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.035+0000 7f7a4182c640 1 -- 172.21.15.67:0/3361703373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a3c108760 msgr2=0x7f7a3c1104a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:52.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.035+0000 7f7a4182c640 1 --2- 172.21.15.67:0/3361703373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a3c108760 0x7f7a3c1104a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:52.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.035+0000 7f7a4182c640 1 -- 172.21.15.67:0/3361703373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a3c1a80d0 con 0x7f7a3c1024d0 2024-07-27T03:19:52.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.035+0000 7f7a4102b640 1 --2- 172.21.15.67:0/3361703373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a3c107e10 0x7f7a3c10ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:19:52.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.036+0000 7f7a4182c640 1 --2- 172.21.15.67:0/3361703373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a3c1024d0 0x7f7a3c10fa20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800c890 tx=0x7f7a3800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:52.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.036+0000 7f7a32ffd640 1 -- 172.21.15.67:0/3361703373 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38004220 con 0x7f7a3c1024d0 2024-07-27T03:19:52.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.036+0000 7f7a32ffd640 1 -- 172.21.15.67:0/3361703373 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a380043c0 con 0x7f7a3c1024d0 2024-07-27T03:19:52.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.036+0000 7f7a43a90640 1 -- 172.21.15.67:0/3361703373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a3c1a83c0 con 0x7f7a3c1024d0 2024-07-27T03:19:52.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.036+0000 7f7a32ffd640 1 -- 172.21.15.67:0/3361703373 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38004a00 con 0x7f7a3c1024d0 2024-07-27T03:19:52.040 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.036+0000 7f7a43a90640 1 -- 172.21.15.67:0/3361703373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a3c1a8920 con 0x7f7a3c1024d0 2024-07-27T03:19:52.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.037+0000 7f7a43a90640 1 -- 172.21.15.67:0/3361703373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a3c103750 con 0x7f7a3c1024d0 2024-07-27T03:19:52.041 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.039+0000 7f7a32ffd640 1 -- 172.21.15.67:0/3361703373 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f7a380079b0 con 0x7f7a3c1024d0 2024-07-27T03:19:52.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.039+0000 7f7a32ffd640 1 --2- 172.21.15.67:0/3361703373 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7a280b3c50 0x7f7a280b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:52.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.040+0000 7f7a4102b640 1 --2- 172.21.15.67:0/3361703373 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7a280b3c50 0x7f7a280b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:52.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.040+0000 7f7a32ffd640 1 -- 172.21.15.67:0/3361703373 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f7a380e40c0 con 0x7f7a3c1024d0 2024-07-27T03:19:52.042 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.040+0000 7f7a4102b640 1 --2- 172.21.15.67:0/3361703373 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7a280b3c50 0x7f7a280b6160 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c000c00 tx=0x7f7a2c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:52.045 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.043+0000 7f7a32ffd640 1 -- 172.21.15.67:0/3361703373 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f7a38014030 con 0x7f7a3c1024d0 2024-07-27T03:19:52.268 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.266+0000 7f7a43a90640 1 -- 172.21.15.67:0/3361703373 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a3c1028e0 con 0x7f7a3c1024d0 2024-07-27T03:19:52.269 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.267+0000 7f7a32ffd640 1 -- 172.21.15.67:0/3361703373 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11983 (secure 0 0 0) 0x7f7a380ab3e0 con 0x7f7a3c1024d0 2024-07-27T03:19:52.269 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:19:52.270 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":30,"btime":"2024-07-27T03:16:40:350577+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14679,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6835/252686369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6834","nonce":252686369},{"type":"v1","addr":"172.21.15.67:6835","nonce":252686369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14685,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.67:6837/3358832762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6836","nonce":3358832762},{"type":"v1","addr":"172.21.15.67:6837","nonce":3358832762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-07-27T03:12:34.320208+0000","modified":"2024-07-27T03:16:39.741149+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24620,"mds_1":24587,"mds_2":24583,"mds_3":24581,"mds_4":24563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14736":{"gid":14736,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6839/2162708317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6838","nonce":2162708317},{"type":"v1","addr":"172.21.15.67:6839","nonce":2162708317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14748":{"gid":14748,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.67:6841/795704244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6840","nonce":795704244},{"type":"v1","addr":"172.21.15.67:6841","nonce":795704244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6833/3804638701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":3804638701},{"type":"v1","addr":"172.21.15.175:6833","nonce":3804638701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24533":{"gid":24533,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6833/3868325215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":3868325215},{"type":"v1","addr":"172.21.15.149:6833","nonce":3868325215}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.175:6835/2462749027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2462749027},{"type":"v1","addr":"172.21.15.175:6835","nonce":2462749027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24563":{"gid":24563,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.149:6835/3730964806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3730964806},{"type":"v1","addr":"172.21.15.149:6835","nonce":3730964806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24581":{"gid":24581,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.175:6837/1304803502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1304803502},{"type":"v1","addr":"172.21.15.175:6837","nonce":1304803502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24583":{"gid":24583,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.175:6839/1670568853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6838","nonce":1670568853},{"type":"v1","addr":"172.21.15.175:6839","nonce":1670568853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24587":{"gid":24587,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":24,"addr":"172.21.15.149:6837/2932951845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":2932951845},{"type":"v1","addr":"172.21.15.149:6837","nonce":2932951845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.149:6839/281746498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":281746498},{"type":"v1","addr":"172.21.15.149:6839","nonce":281746498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24620,"qdb_cluster":[24587,24583,24620,24581,24563]},"id":1}]} 2024-07-27T03:19:52.270 INFO:teuthology.orchestra.run.smithi067.stderr:dumped fsmap epoch 30 2024-07-27T03:19:52.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.269+0000 7f7a43a90640 1 -- 172.21.15.67:0/3361703373 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7a280b3c50 msgr2=0x7f7a280b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:52.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.269+0000 7f7a43a90640 1 --2- 172.21.15.67:0/3361703373 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7a280b3c50 0x7f7a280b6160 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c000c00 tx=0x7f7a2c031400 comp rx=0 tx=0).stop 2024-07-27T03:19:52.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.269+0000 7f7a43a90640 1 -- 172.21.15.67:0/3361703373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a3c1024d0 msgr2=0x7f7a3c10fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:52.271 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.269+0000 7f7a43a90640 1 --2- 172.21.15.67:0/3361703373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a3c1024d0 0x7f7a3c10fa20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800c890 tx=0x7f7a3800cd50 comp rx=0 tx=0).stop 2024-07-27T03:19:52.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.269+0000 7f7a43a90640 1 -- 172.21.15.67:0/3361703373 shutdown_connections 2024-07-27T03:19:52.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.269+0000 7f7a43a90640 1 --2- 172.21.15.67:0/3361703373 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f7a280b3c50 0x7f7a280b6160 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:52.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.269+0000 7f7a43a90640 1 --2- 172.21.15.67:0/3361703373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a3c108760 0x7f7a3c1104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:52.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.269+0000 7f7a43a90640 1 --2- 172.21.15.67:0/3361703373 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a3c107e10 0x7f7a3c10ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:52.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.269+0000 7f7a43a90640 1 --2- 172.21.15.67:0/3361703373 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7a3c1024d0 0x7f7a3c10fa20 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:52.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.269+0000 7f7a43a90640 1 -- 172.21.15.67:0/3361703373 >> 172.21.15.67:0/3361703373 conn(0x7f7a3c079190 msgr2=0x7f7a3c103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:52.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.269+0000 7f7a43a90640 1 -- 172.21.15.67:0/3361703373 shutdown_connections 2024-07-27T03:19:52.272 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:52.269+0000 7f7a43a90640 1 -- 172.21.15.67:0/3361703373 wait complete. 2024-07-27T03:19:52.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:52 smithi149 ceph-mon[37173]: pgmap v529: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-07-27T03:19:53.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:52 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3361703373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:19:53.003 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:52 smithi067 ceph-mon[32814]: pgmap v529: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-07-27T03:19:53.003 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:52 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3361703373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:19:53.035 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd dump --format=json 2024-07-27T03:19:53.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:52 smithi175 ceph-mon[37125]: pgmap v529: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-07-27T03:19:53.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:52 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3361703373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-07-27T03:19:53.304 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:19:54.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:54 smithi067 ceph-mon[32814]: pgmap v530: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-07-27T03:19:54.999 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:54 smithi149 ceph-mon[37173]: pgmap v530: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-07-27T03:19:55.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.092+0000 7f22c7f3a640 1 -- 172.21.15.67:0/1515956620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c0108f10 msgr2=0x7f22c0109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:55.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.092+0000 7f22c7f3a640 1 --2- 172.21.15.67:0/1515956620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c0108f10 0x7f22c0109320 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009900 tx=0x7f22b002a410 comp rx=0 tx=0).stop 2024-07-27T03:19:55.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.093+0000 7f22c7f3a640 1 -- 172.21.15.67:0/1515956620 shutdown_connections 2024-07-27T03:19:55.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.093+0000 7f22c7f3a640 1 --2- 172.21.15.67:0/1515956620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22c0109860 0x7f22c0110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:55.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.093+0000 7f22c7f3a640 1 --2- 172.21.15.67:0/1515956620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c0108f10 0x7f22c0109320 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:55.110 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.093+0000 7f22c7f3a640 1 --2- 172.21.15.67:0/1515956620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22c01035d0 0x7f22c01039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:55.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.093+0000 7f22c7f3a640 1 -- 172.21.15.67:0/1515956620 >> 172.21.15.67:0/1515956620 conn(0x7f22c007b2c0 msgr2=0x7f22c007b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:55.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.093+0000 7f22c7f3a640 1 -- 172.21.15.67:0/1515956620 shutdown_connections 2024-07-27T03:19:55.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.093+0000 7f22c7f3a640 1 -- 172.21.15.67:0/1515956620 wait complete. 2024-07-27T03:19:55.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.093+0000 7f22c7f3a640 1 Processor -- start 2024-07-27T03:19:55.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.093+0000 7f22c7f3a640 1 -- start start 2024-07-27T03:19:55.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.094+0000 7f22c7f3a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22c01035d0 0x7f22c0199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:55.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.094+0000 7f22c7f3a640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22c0108f10 0x7f22c019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:55.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.094+0000 7f22c7f3a640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c0109860 0x7f22c019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:55.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.094+0000 7f22c7f3a640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f22c01134d0 con 0x7f22c0109860 2024-07-27T03:19:55.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.094+0000 7f22c7f3a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f22c0113350 con 0x7f22c01035d0 2024-07-27T03:19:55.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.094+0000 7f22c7f3a640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22c0113650 con 0x7f22c0108f10 2024-07-27T03:19:55.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.094+0000 7f22c64d7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c0109860 0x7f22c019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:55.111 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.094+0000 7f22c64d7640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c0109860 0x7f22c019f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56432/0 (socket says 172.21.15.67:56432) 2024-07-27T03:19:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.094+0000 7f22c64d7640 1 -- 172.21.15.67:0/1669846593 learned_addr learned my addr 172.21.15.67:0/1669846593 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:19:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.094+0000 7f22c54d5640 1 --2- 172.21.15.67:0/1669846593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22c0108f10 0x7f22c019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.094+0000 7f22c5cd6640 1 --2- 172.21.15.67:0/1669846593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22c01035d0 0x7f22c0199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.094+0000 7f22c64d7640 1 -- 172.21.15.67:0/1669846593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22c0108f10 msgr2=0x7f22c019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.094+0000 7f22c64d7640 1 --2- 172.21.15.67:0/1669846593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22c0108f10 0x7f22c019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.094+0000 7f22c64d7640 1 -- 172.21.15.67:0/1669846593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22c01035d0 msgr2=0x7f22c0199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.094+0000 7f22c64d7640 1 --2- 172.21.15.67:0/1669846593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22c01035d0 0x7f22c0199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.094+0000 7f22c64d7640 1 -- 172.21.15.67:0/1669846593 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c019ff10 con 0x7f22c0109860 2024-07-27T03:19:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.094+0000 7f22c64d7640 1 --2- 172.21.15.67:0/1669846593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c0109860 0x7f22c019f7e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f22b800cad0 tx=0x7f22b800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.095+0000 7f22b6ffd640 1 -- 172.21.15.67:0/1669846593 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b800ae20 con 0x7f22c0109860 2024-07-27T03:19:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.095+0000 7f22b6ffd640 1 -- 172.21.15.67:0/1669846593 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22b8005020 con 0x7f22c0109860 2024-07-27T03:19:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.095+0000 7f22b6ffd640 1 -- 172.21.15.67:0/1669846593 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b800f6b0 con 0x7f22c0109860 2024-07-27T03:19:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.095+0000 7f22c7f3a640 1 -- 172.21.15.67:0/1669846593 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22c01a0200 con 0x7f22c0109860 2024-07-27T03:19:55.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.095+0000 7f22c7f3a640 1 -- 172.21.15.67:0/1669846593 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22c01a6960 con 0x7f22c0109860 2024-07-27T03:19:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.096+0000 7f22c7f3a640 1 -- 172.21.15.67:0/1669846593 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22c01039e0 con 0x7f22c0109860 2024-07-27T03:19:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.098+0000 7f22b6ffd640 1 -- 172.21.15.67:0/1669846593 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f22b80051d0 con 0x7f22c0109860 2024-07-27T03:19:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.099+0000 7f22b6ffd640 1 --2- 172.21.15.67:0/1669846593 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f22ac0b3c50 0x7f22ac0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.099+0000 7f22c5cd6640 1 --2- 172.21.15.67:0/1669846593 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f22ac0b3c50 0x7f22ac0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.099+0000 7f22b6ffd640 1 -- 172.21.15.67:0/1669846593 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7f22b80e4370 con 0x7f22c0109860 2024-07-27T03:19:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.100+0000 7f22c5cd6640 1 --2- 172.21.15.67:0/1669846593 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f22ac0b3c50 0x7f22ac0b6160 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0058c0 tx=0x7f22bc00a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:55.113 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.102+0000 7f22b6ffd640 1 -- 172.21.15.67:0/1669846593 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f22b8014030 con 0x7f22c0109860 2024-07-27T03:19:55.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:54 smithi175 ceph-mon[37125]: pgmap v530: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-07-27T03:19:55.291 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.288+0000 7f22c7f3a640 1 -- 172.21.15.67:0/1669846593 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22c019b0d0 con 0x7f22c0109860 2024-07-27T03:19:55.292 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.290+0000 7f22b6ffd640 1 -- 172.21.15.67:0/1669846593 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24829 (secure 0 0 0) 0x7f22c019b0d0 con 0x7f22c0109860 2024-07-27T03:19:55.292 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-07-27T03:19:55.293 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":91,"fsid":"c0782a00-4bc4-11ef-bcc7-c7b262605968","created":"2024-07-27T03:04:08.096824+0000","modified":"2024-07-27T03:13:57.294556+0000","last_up_change":"2024-07-27T03:11:33.314800+0000","last_in_change":"2024-07-27T03:11:14.028095+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-07-27T03:08:01.042479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-07-27T03:12:27.565948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-07-27T03:12:31.353726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-07-27T03:12:40.599486+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9f614272-18e8-4562-812c-4f43e40efd2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6803","nonce":2648129674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6805","nonce":2648129674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6809","nonce":2648129674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2648129674},{"type":"v1","addr":"172.21.15.67:6807","nonce":2648129674}]},"public_addr":"172.21.15.67:6803/2648129674","cluster_addr":"172.21.15.67:6805/2648129674","heartbeat_back_addr":"172.21.15.67:6809/2648129674","heartbeat_front_addr":"172.21.15.67:6807/2648129674","state":["exists","up"]},{"osd":1,"uuid":"f59f5879-163a-4269-bbe7-1200eea65341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6801","nonce":2561753152}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6803","nonce":2561753152}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6807","nonce":2561753152}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2561753152},{"type":"v1","addr":"172.21.15.149:6805","nonce":2561753152}]},"public_addr":"172.21.15.149:6801/2561753152","cluster_addr":"172.21.15.149:6803/2561753152","heartbeat_back_addr":"172.21.15.149:6807/2561753152","heartbeat_front_addr":"172.21.15.149:6805/2561753152","state":["exists","up"]},{"osd":2,"uuid":"444e0ebe-7ca3-4be9-8bd5-db859f824445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6801","nonce":3770940078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6803","nonce":3770940078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6807","nonce":3770940078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":3770940078},{"type":"v1","addr":"172.21.15.175:6805","nonce":3770940078}]},"public_addr":"172.21.15.175:6801/3770940078","cluster_addr":"172.21.15.175:6803/3770940078","heartbeat_back_addr":"172.21.15.175:6807/3770940078","heartbeat_front_addr":"172.21.15.175:6805/3770940078","state":["exists","up"]},{"osd":3,"uuid":"d554db3d-9604-427c-8736-b1bb379f8d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6811","nonce":3159144693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6813","nonce":3159144693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6817","nonce":3159144693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":3159144693},{"type":"v1","addr":"172.21.15.67:6815","nonce":3159144693}]},"public_addr":"172.21.15.67:6811/3159144693","cluster_addr":"172.21.15.67:6813/3159144693","heartbeat_back_addr":"172.21.15.67:6817/3159144693","heartbeat_front_addr":"172.21.15.67:6815/3159144693","state":["exists","up"]},{"osd":4,"uuid":"c317dbe5-3b49-47a6-8408-7042a3dac345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6809","nonce":3393503532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6811","nonce":3393503532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6815","nonce":3393503532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":3393503532},{"type":"v1","addr":"172.21.15.149:6813","nonce":3393503532}]},"public_addr":"172.21.15.149:6809/3393503532","cluster_addr":"172.21.15.149:6811/3393503532","heartbeat_back_addr":"172.21.15.149:6815/3393503532","heartbeat_front_addr":"172.21.15.149:6813/3393503532","state":["exists","up"]},{"osd":5,"uuid":"5c85abd6-2b42-4957-a9b7-59a9ee36b841","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6809","nonce":2291204784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6811","nonce":2291204784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6815","nonce":2291204784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":2291204784},{"type":"v1","addr":"172.21.15.175:6813","nonce":2291204784}]},"public_addr":"172.21.15.175:6809/2291204784","cluster_addr":"172.21.15.175:6811/2291204784","heartbeat_back_addr":"172.21.15.175:6815/2291204784","heartbeat_front_addr":"172.21.15.175:6813/2291204784","state":["exists","up"]},{"osd":6,"uuid":"bf6c277d-ec24-4470-98de-cf6f28876ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6819","nonce":2660182879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6821","nonce":2660182879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6825","nonce":2660182879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":2660182879},{"type":"v1","addr":"172.21.15.67:6823","nonce":2660182879}]},"public_addr":"172.21.15.67:6819/2660182879","cluster_addr":"172.21.15.67:6821/2660182879","heartbeat_back_addr":"172.21.15.67:6825/2660182879","heartbeat_front_addr":"172.21.15.67:6823/2660182879","state":["exists","up"]},{"osd":7,"uuid":"fea2c7cf-d5cb-45ae-9b29-f6e56c903891","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6817","nonce":2289624404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6819","nonce":2289624404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6823","nonce":2289624404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2289624404},{"type":"v1","addr":"172.21.15.149:6821","nonce":2289624404}]},"public_addr":"172.21.15.149:6817/2289624404","cluster_addr":"172.21.15.149:6819/2289624404","heartbeat_back_addr":"172.21.15.149:6823/2289624404","heartbeat_front_addr":"172.21.15.149:6821/2289624404","state":["exists","up"]},{"osd":8,"uuid":"14d04d96-579a-4dc9-a4d4-50e4850b48e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6817","nonce":2286032466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6819","nonce":2286032466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6823","nonce":2286032466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":2286032466},{"type":"v1","addr":"172.21.15.175:6821","nonce":2286032466}]},"public_addr":"172.21.15.175:6817/2286032466","cluster_addr":"172.21.15.175:6819/2286032466","heartbeat_back_addr":"172.21.15.175:6823/2286032466","heartbeat_front_addr":"172.21.15.175:6821/2286032466","state":["exists","up"]},{"osd":9,"uuid":"b65dfb6b-1efe-4314-a8be-ce6a295e99fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6827","nonce":3411081513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6829","nonce":3411081513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6833","nonce":3411081513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3411081513},{"type":"v1","addr":"172.21.15.67:6831","nonce":3411081513}]},"public_addr":"172.21.15.67:6827/3411081513","cluster_addr":"172.21.15.67:6829/3411081513","heartbeat_back_addr":"172.21.15.67:6833/3411081513","heartbeat_front_addr":"172.21.15.67:6831/3411081513","state":["exists","up"]},{"osd":10,"uuid":"e3dabae0-0d64-4222-984c-fc1cce883456","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6825","nonce":2011495212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6827","nonce":2011495212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6831","nonce":2011495212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2011495212},{"type":"v1","addr":"172.21.15.149:6829","nonce":2011495212}]},"public_addr":"172.21.15.149:6825/2011495212","cluster_addr":"172.21.15.149:6827/2011495212","heartbeat_back_addr":"172.21.15.149:6831/2011495212","heartbeat_front_addr":"172.21.15.149:6829/2011495212","state":["exists","up"]},{"osd":11,"uuid":"406c082c-9b5f-4781-8cbf-3c04fa49e410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6825","nonce":2565514213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6827","nonce":2565514213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6831","nonce":2565514213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2565514213},{"type":"v1","addr":"172.21.15.175:6829","nonce":2565514213}]},"public_addr":"172.21.15.175:6825/2565514213","cluster_addr":"172.21.15.175:6827/2565514213","heartbeat_back_addr":"172.21.15.175:6831/2565514213","heartbeat_front_addr":"172.21.15.175:6829/2565514213","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:11.399380+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:33.728518+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:07:56.411181+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:19.871731+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:08:42.994576+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:06.712815+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:31.321967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:09:55.070898+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:18.686108+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:10:43.187795+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:07.318538+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-07-27T03:11:31.374843+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/3477601235":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/4130679770":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6800/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/1029332005":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6801/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:6801/1921856974":"2024-07-28T03:04:54.947985+0000","172.21.15.67:0/284078926":"2024-07-28T03:04:54.947985+0000","172.21.15.67:6800/2346308006":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/2559121542":"2024-07-28T03:04:30.600273+0000","172.21.15.67:0/925406021":"2024-07-28T03:04:30.600273+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-07-27T03:19:55.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.292+0000 7f22c7f3a640 1 -- 172.21.15.67:0/1669846593 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f22ac0b3c50 msgr2=0x7f22ac0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:55.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.292+0000 7f22c7f3a640 1 --2- 172.21.15.67:0/1669846593 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f22ac0b3c50 0x7f22ac0b6160 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0058c0 tx=0x7f22bc00a4c0 comp rx=0 tx=0).stop 2024-07-27T03:19:55.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.292+0000 7f22c7f3a640 1 -- 172.21.15.67:0/1669846593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c0109860 msgr2=0x7f22c019f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:55.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.292+0000 7f22c7f3a640 1 --2- 172.21.15.67:0/1669846593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c0109860 0x7f22c019f7e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f22b800cad0 tx=0x7f22b800cf90 comp rx=0 tx=0).stop 2024-07-27T03:19:55.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.292+0000 7f22c7f3a640 1 -- 172.21.15.67:0/1669846593 shutdown_connections 2024-07-27T03:19:55.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.292+0000 7f22c7f3a640 1 --2- 172.21.15.67:0/1669846593 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f22ac0b3c50 0x7f22ac0b6160 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:55.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.292+0000 7f22c7f3a640 1 --2- 172.21.15.67:0/1669846593 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f22c0109860 0x7f22c019f7e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:55.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.292+0000 7f22c7f3a640 1 --2- 172.21.15.67:0/1669846593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22c0108f10 0x7f22c019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:55.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.292+0000 7f22c7f3a640 1 --2- 172.21.15.67:0/1669846593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22c01035d0 0x7f22c0199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:55.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.292+0000 7f22c7f3a640 1 -- 172.21.15.67:0/1669846593 >> 172.21.15.67:0/1669846593 conn(0x7f22c007b2c0 msgr2=0x7f22c0107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:55.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.292+0000 7f22c7f3a640 1 -- 172.21.15.67:0/1669846593 shutdown_connections 2024-07-27T03:19:55.295 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:55.292+0000 7f22c7f3a640 1 -- 172.21.15.67:0/1669846593 wait complete. 2024-07-27T03:19:55.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:55 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1669846593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:19:56.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:55 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1669846593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:19:56.067 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs fail cephfs 2024-07-27T03:19:56.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:55 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1669846593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-07-27T03:19:56.341 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:19:57.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:56 smithi149 ceph-mon[37173]: pgmap v531: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-07-27T03:19:57.108 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:56 smithi067 ceph-mon[32814]: pgmap v531: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-07-27T03:19:57.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:56 smithi175 ceph-mon[37125]: pgmap v531: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-07-27T03:19:58.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.117+0000 7ff35ce87640 1 -- 172.21.15.67:0/3948542563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3581024d0 msgr2=0x7ff3581028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:58.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.117+0000 7ff35ce87640 1 --2- 172.21.15.67:0/3948542563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3581024d0 0x7ff3581028e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff348009ca0 tx=0x7ff34802a8c0 comp rx=0 tx=0).stop 2024-07-27T03:19:58.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.117+0000 7ff35ce87640 1 -- 172.21.15.67:0/3948542563 shutdown_connections 2024-07-27T03:19:58.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.117+0000 7ff35ce87640 1 --2- 172.21.15.67:0/3948542563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff358108760 0x7ff35810ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:58.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.117+0000 7ff35ce87640 1 --2- 172.21.15.67:0/3948542563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff358107e10 0x7ff358108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:58.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.117+0000 7ff35ce87640 1 --2- 172.21.15.67:0/3948542563 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff3581024d0 0x7ff3581028e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:58.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.117+0000 7ff35ce87640 1 -- 172.21.15.67:0/3948542563 >> 172.21.15.67:0/3948542563 conn(0x7ff3580791a0 msgr2=0x7ff3580795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:58.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.117+0000 7ff35ce87640 1 -- 172.21.15.67:0/3948542563 shutdown_connections 2024-07-27T03:19:58.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.117+0000 7ff35ce87640 1 -- 172.21.15.67:0/3948542563 wait complete. 2024-07-27T03:19:58.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.118+0000 7ff35ce87640 1 Processor -- start 2024-07-27T03:19:58.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.118+0000 7ff35ce87640 1 -- start start 2024-07-27T03:19:58.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.119+0000 7ff35ce87640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3581024d0 0x7ff358199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:58.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.119+0000 7ff35ce87640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff358107e10 0x7ff35819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:58.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.119+0000 7ff35ce87640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff358108760 0x7ff35819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:58.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.119+0000 7ff35ce87640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff358113530 con 0x7ff358107e10 2024-07-27T03:19:58.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.119+0000 7ff35ce87640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff3581133b0 con 0x7ff358108760 2024-07-27T03:19:58.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.119+0000 7ff35ce87640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff3581136b0 con 0x7ff3581024d0 2024-07-27T03:19:58.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.119+0000 7ff355d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff358107e10 0x7ff35819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:58.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.119+0000 7ff355d9b640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff358107e10 0x7ff35819a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56454/0 (socket says 172.21.15.67:56454) 2024-07-27T03:19:58.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.119+0000 7ff355d9b640 1 -- 172.21.15.67:0/2587199696 learned_addr learned my addr 172.21.15.67:0/2587199696 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:19:58.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.119+0000 7ff35659c640 1 --2- 172.21.15.67:0/2587199696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3581024d0 0x7ff358199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:58.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.119+0000 7ff356d9d640 1 --2- 172.21.15.67:0/2587199696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff358108760 0x7ff35819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:58.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.119+0000 7ff355d9b640 1 -- 172.21.15.67:0/2587199696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3581024d0 msgr2=0x7ff358199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:58.124 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.119+0000 7ff355d9b640 1 --2- 172.21.15.67:0/2587199696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3581024d0 0x7ff358199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:58.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.119+0000 7ff355d9b640 1 -- 172.21.15.67:0/2587199696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff358108760 msgr2=0x7ff35819f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:58.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.119+0000 7ff355d9b640 1 --2- 172.21.15.67:0/2587199696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff358108760 0x7ff35819f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:58.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.119+0000 7ff355d9b640 1 -- 172.21.15.67:0/2587199696 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35819ff10 con 0x7ff358107e10 2024-07-27T03:19:58.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.119+0000 7ff355d9b640 1 --2- 172.21.15.67:0/2587199696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff358107e10 0x7ff35819a540 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff34000d770 tx=0x7ff34000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:58.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.119+0000 7ff34f7fe640 1 -- 172.21.15.67:0/2587199696 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff340017070 con 0x7ff358107e10 2024-07-27T03:19:58.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.120+0000 7ff34f7fe640 1 -- 172.21.15.67:0/2587199696 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff340009c30 con 0x7ff358107e10 2024-07-27T03:19:58.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.120+0000 7ff35ce87640 1 -- 172.21.15.67:0/2587199696 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3581a0200 con 0x7ff358107e10 2024-07-27T03:19:58.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.120+0000 7ff34f7fe640 1 -- 172.21.15.67:0/2587199696 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34000f620 con 0x7ff358107e10 2024-07-27T03:19:58.125 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.120+0000 7ff35ce87640 1 -- 172.21.15.67:0/2587199696 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3581a6960 con 0x7ff358107e10 2024-07-27T03:19:58.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.121+0000 7ff35ce87640 1 -- 172.21.15.67:0/2587199696 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3581028e0 con 0x7ff358107e10 2024-07-27T03:19:58.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.122+0000 7ff34f7fe640 1 -- 172.21.15.67:0/2587199696 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7ff34001a070 con 0x7ff358107e10 2024-07-27T03:19:58.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.123+0000 7ff34f7fe640 1 --2- 172.21.15.67:0/2587199696 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff3440b3c50 0x7ff3440b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:19:58.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.123+0000 7ff35659c640 1 --2- 172.21.15.67:0/2587199696 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff3440b3c50 0x7ff3440b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:19:58.126 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.123+0000 7ff34f7fe640 1 -- 172.21.15.67:0/2587199696 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9179+0+0 (secure 0 0 0) 0x7ff3400e3a80 con 0x7ff358107e10 2024-07-27T03:19:58.127 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.124+0000 7ff35659c640 1 --2- 172.21.15.67:0/2587199696 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff3440b3c50 0x7ff3440b6160 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff348000f80 tx=0x7ff348009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:19:58.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.126+0000 7ff34f7fe640 1 -- 172.21.15.67:0/2587199696 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7ff340009de0 con 0x7ff358107e10 2024-07-27T03:19:58.353 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.350+0000 7ff35ce87640 1 -- 172.21.15.67:0/2587199696 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff35810a590 con 0x7ff358107e10 2024-07-27T03:19:58.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.352+0000 7ff34f7fe640 1 -- 172.21.15.67:0/2587199696 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) ==== 161+0+0 (secure 0 0 0) 0x7ff3400aada0 con 0x7ff358107e10 2024-07-27T03:19:58.355 INFO:teuthology.orchestra.run.smithi067.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-07-27T03:19:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.356+0000 7ff35ce87640 1 -- 172.21.15.67:0/2587199696 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff3440b3c50 msgr2=0x7ff3440b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.356+0000 7ff35ce87640 1 --2- 172.21.15.67:0/2587199696 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff3440b3c50 0x7ff3440b6160 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff348000f80 tx=0x7ff348009360 comp rx=0 tx=0).stop 2024-07-27T03:19:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.356+0000 7ff35ce87640 1 -- 172.21.15.67:0/2587199696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff358107e10 msgr2=0x7ff35819a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:19:58.359 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.356+0000 7ff35ce87640 1 --2- 172.21.15.67:0/2587199696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff358107e10 0x7ff35819a540 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff34000d770 tx=0x7ff34000dc30 comp rx=0 tx=0).stop 2024-07-27T03:19:58.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.356+0000 7ff35ce87640 1 -- 172.21.15.67:0/2587199696 shutdown_connections 2024-07-27T03:19:58.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.356+0000 7ff35ce87640 1 --2- 172.21.15.67:0/2587199696 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff3440b3c50 0x7ff3440b6160 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:58.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.356+0000 7ff35ce87640 1 --2- 172.21.15.67:0/2587199696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff358108760 0x7ff35819f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:58.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.356+0000 7ff35ce87640 1 --2- 172.21.15.67:0/2587199696 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff358107e10 0x7ff35819a540 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:58.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.356+0000 7ff35ce87640 1 --2- 172.21.15.67:0/2587199696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3581024d0 0x7ff358199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:19:58.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.356+0000 7ff35ce87640 1 -- 172.21.15.67:0/2587199696 >> 172.21.15.67:0/2587199696 conn(0x7ff3580791a0 msgr2=0x7ff358100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:19:58.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.356+0000 7ff35ce87640 1 -- 172.21.15.67:0/2587199696 shutdown_connections 2024-07-27T03:19:58.360 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:19:58.356+0000 7ff35ce87640 1 -- 172.21.15.67:0/2587199696 wait complete. 2024-07-27T03:19:58.672 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:58 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-a[32809]: 2024-07-27T03:19:58.350+0000 7efcfe4cc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-27T03:19:58.673 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:19:58 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-a[78470]: ignoring --setuser ceph since I am not root 2024-07-27T03:19:58.673 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:19:58 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-a[78470]: ignoring --setgroup ceph since I am not root 2024-07-27T03:19:58.673 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:19:58 smithi067 ceph-mds[78474]: -- 172.21.15.67:0/4103499337 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5636f331fc20 con 0x5636f74c5600 2024-07-27T03:19:58.673 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:19:58 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-a[78470]: starting mds.a at 2024-07-27T03:19:58.673 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 27 03:19:58 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-g[78048]: ignoring --setuser ceph since I am not root 2024-07-27T03:19:58.673 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 27 03:19:58 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-g[78048]: ignoring --setgroup ceph since I am not root 2024-07-27T03:19:58.673 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 27 03:19:58 smithi067 ceph-mds[78058]: -- 172.21.15.67:0/948984718 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5624f1e6d860 con 0x5624f6013a80 2024-07-27T03:19:58.673 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 27 03:19:58 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-g[78048]: starting mds.g at 2024-07-27T03:19:58.749 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-b[67306]: ignoring --setuser ceph since I am not root 2024-07-27T03:19:58.749 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-b[67306]: ignoring --setgroup ceph since I am not root 2024-07-27T03:19:58.749 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-mds[67310]: -- 172.21.15.149:0/3395271204 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5587de1c3c20 con 0x5587e2369600 2024-07-27T03:19:58.750 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-b[67306]: starting mds.b at 2024-07-27T03:19:58.750 INFO:journalctl@ceph.mds.h.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-h[67002]: ignoring --setuser ceph since I am not root 2024-07-27T03:19:58.750 INFO:journalctl@ceph.mds.h.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-h[67002]: ignoring --setgroup ceph since I am not root 2024-07-27T03:19:58.750 INFO:journalctl@ceph.mds.h.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-mds[67006]: -- 172.21.15.149:0/3351527360 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d1136d5c20 con 0x55d11787ba80 2024-07-27T03:19:58.750 INFO:journalctl@ceph.mds.h.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-h[67002]: starting mds.h at 2024-07-27T03:19:58.750 INFO:journalctl@ceph.mds.k.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-k[66374]: ignoring --setuser ceph since I am not root 2024-07-27T03:19:58.751 INFO:journalctl@ceph.mds.k.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-k[66374]: ignoring --setgroup ceph since I am not root 2024-07-27T03:19:58.751 INFO:journalctl@ceph.mds.k.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-mds[66378]: -- 172.21.15.149:0/886815413 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555d55639860 con 0x555d597dfa80 2024-07-27T03:19:58.751 INFO:journalctl@ceph.mds.k.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-k[66374]: starting mds.k at 2024-07-27T03:19:58.751 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-e[66689]: ignoring --setuser ceph since I am not root 2024-07-27T03:19:58.751 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-e[66689]: ignoring --setgroup ceph since I am not root 2024-07-27T03:19:58.751 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-mds[66693]: -- 172.21.15.149:0/2806905372 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561786927c20 con 0x56178aacd600 2024-07-27T03:19:58.751 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-e[66689]: starting mds.e at 2024-07-27T03:19:58.752 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jul 27 03:19:58 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-i[67827]: ignoring --setuser ceph since I am not root 2024-07-27T03:19:58.752 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jul 27 03:19:58 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-i[67827]: ignoring --setgroup ceph since I am not root 2024-07-27T03:19:58.752 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jul 27 03:19:58 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-i[67827]: starting mds.i at 2024-07-27T03:19:58.752 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jul 27 03:19:58 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-c[67516]: ignoring --setuser ceph since I am not root 2024-07-27T03:19:58.752 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jul 27 03:19:58 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-c[67516]: ignoring --setgroup ceph since I am not root 2024-07-27T03:19:58.752 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jul 27 03:19:58 smithi175 ceph-mds[67525]: -- 172.21.15.175:0/106155911 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f233d6f860 con 0x55f237f15600 2024-07-27T03:19:58.752 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jul 27 03:19:58 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-c[67516]: starting mds.c at 2024-07-27T03:19:58.753 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jul 27 03:19:58 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-f[67211]: ignoring --setuser ceph since I am not root 2024-07-27T03:19:58.753 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jul 27 03:19:58 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-f[67211]: ignoring --setgroup ceph since I am not root 2024-07-27T03:19:58.753 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jul 27 03:19:58 smithi175 ceph-mds[67215]: -- 172.21.15.175:0/2183713686 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559be3a29c20 con 0x559be7bcfa80 2024-07-27T03:19:58.753 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jul 27 03:19:58 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-f[67211]: starting mds.f at 2024-07-27T03:19:58.753 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jul 27 03:19:58 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-l[66909]: ignoring --setuser ceph since I am not root 2024-07-27T03:19:58.753 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jul 27 03:19:58 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-l[66909]: ignoring --setgroup ceph since I am not root 2024-07-27T03:19:58.753 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jul 27 03:19:58 smithi175 ceph-mds[66913]: -- 172.21.15.175:0/413470220 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5645e3ad9860 con 0x5645e7c7f600 2024-07-27T03:19:58.753 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jul 27 03:19:58 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-l[66909]: starting mds.l at 2024-07-27T03:19:58.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:58 smithi067 ceph-mon[32814]: pgmap v532: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-07-27T03:19:58.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:58 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2587199696' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-07-27T03:19:58.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:58 smithi067 ceph-mon[32814]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-27T03:19:58.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:58 smithi067 ceph-mon[32814]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-27T03:19:58.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:58 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2587199696' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-07-27T03:19:58.969 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:58 smithi067 ceph-mon[32814]: osdmap e92: 12 total, 12 up, 12 in 2024-07-27T03:19:58.970 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:19:58 smithi067 ceph-mon[32814]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-07-27T03:19:59.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-mon[37173]: pgmap v532: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-07-27T03:19:59.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2587199696' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-07-27T03:19:59.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-mon[37173]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-27T03:19:59.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-mon[37173]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-27T03:19:59.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2587199696' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-07-27T03:19:59.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-mon[37173]: osdmap e92: 12 total, 12 up, 12 in 2024-07-27T03:19:59.000 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:19:58 smithi149 ceph-mon[37173]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-07-27T03:19:59.020 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-07-27T03:19:59.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:58 smithi175 ceph-mon[37125]: pgmap v532: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-07-27T03:19:59.124 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:58 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2587199696' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-07-27T03:19:59.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:58 smithi175 ceph-mon[37125]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-07-27T03:19:59.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:58 smithi175 ceph-mon[37125]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-07-27T03:19:59.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:58 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2587199696' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-07-27T03:19:59.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:58 smithi175 ceph-mon[37125]: osdmap e92: 12 total, 12 up, 12 in 2024-07-27T03:19:59.125 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:19:58 smithi175 ceph-mon[37125]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-07-27T03:19:59.287 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:20:00.360 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-a[32809]: 2024-07-27T03:19:59.999+0000 7efd00cd1640 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-07-27T03:20:00.360 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-a[32809]: 2024-07-27T03:19:59.999+0000 7efd00cd1640 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-07-27T03:20:00.361 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-a[32809]: 2024-07-27T03:19:59.999+0000 7efd00cd1640 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-07-27T03:20:00.361 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-a[32809]: 2024-07-27T03:19:59.999+0000 7efd00cd1640 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-07-27T03:20:00.361 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-a[32809]: 2024-07-27T03:19:59.999+0000 7efd00cd1640 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-07-27T03:20:00.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: pgmap v534: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 409 B/s wr, 10 op/s 2024-07-27T03:20:00.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.175:6833/1334549389,v1:172.21.15.175:6835/1334549389] up:boot 2024-07-27T03:20:00.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.175:6832/1676160331,v1:172.21.15.175:6834/1676160331] up:boot 2024-07-27T03:20:00.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.67:6839/1722507060,v1:172.21.15.67:6841/1722507060] up:boot 2024-07-27T03:20:00.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.67:6838/846482796,v1:172.21.15.67:6840/846482796] up:boot 2024-07-27T03:20:00.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.149:6833/2775616736,v1:172.21.15.149:6835/2775616736] up:boot 2024-07-27T03:20:00.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.149:6832/1556765136,v1:172.21.15.149:6834/1556765136] up:boot 2024-07-27T03:20:00.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.149:6836/620433683,v1:172.21.15.149:6838/620433683] up:boot 2024-07-27T03:20:00.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.149:6837/1841204281,v1:172.21.15.149:6839/1841204281] up:boot 2024-07-27T03:20:00.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.175:6836/3857025307,v1:172.21.15.175:6838/3857025307] up:boot 2024-07-27T03:20:00.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: mds.? [v2:172.21.15.175:6837/344320809,v1:172.21.15.175:6839/344320809] up:boot 2024-07-27T03:20:00.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-07-27T03:20:00.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-27T03:20:00.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-27T03:20:00.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-27T03:20:00.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-27T03:20:00.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-27T03:20:00.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-27T03:20:00.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-27T03:20:00.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-27T03:20:00.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-27T03:20:00.626 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-27T03:20:00.627 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-07-27T03:20:00.627 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-07-27T03:20:00.627 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: fs cephfs is degraded 2024-07-27T03:20:00.627 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-07-27T03:20:00.627 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:00 smithi175 ceph-mon[37125]: fs cephfs is offline because no MDS is active for it. 2024-07-27T03:20:00.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: pgmap v534: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 409 B/s wr, 10 op/s 2024-07-27T03:20:00.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.175:6833/1334549389,v1:172.21.15.175:6835/1334549389] up:boot 2024-07-27T03:20:00.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.175:6832/1676160331,v1:172.21.15.175:6834/1676160331] up:boot 2024-07-27T03:20:00.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.67:6839/1722507060,v1:172.21.15.67:6841/1722507060] up:boot 2024-07-27T03:20:00.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.67:6838/846482796,v1:172.21.15.67:6840/846482796] up:boot 2024-07-27T03:20:00.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.149:6833/2775616736,v1:172.21.15.149:6835/2775616736] up:boot 2024-07-27T03:20:00.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.149:6832/1556765136,v1:172.21.15.149:6834/1556765136] up:boot 2024-07-27T03:20:00.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.149:6836/620433683,v1:172.21.15.149:6838/620433683] up:boot 2024-07-27T03:20:00.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.149:6837/1841204281,v1:172.21.15.149:6839/1841204281] up:boot 2024-07-27T03:20:00.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.175:6836/3857025307,v1:172.21.15.175:6838/3857025307] up:boot 2024-07-27T03:20:00.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: mds.? [v2:172.21.15.175:6837/344320809,v1:172.21.15.175:6839/344320809] up:boot 2024-07-27T03:20:00.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-07-27T03:20:00.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-27T03:20:00.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-27T03:20:00.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-27T03:20:00.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-27T03:20:00.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-27T03:20:00.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-27T03:20:00.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-27T03:20:00.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-27T03:20:00.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-27T03:20:00.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-27T03:20:00.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-07-27T03:20:00.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-07-27T03:20:00.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: fs cephfs is degraded 2024-07-27T03:20:00.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-07-27T03:20:00.721 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:00 smithi067 ceph-mon[32814]: fs cephfs is offline because no MDS is active for it. 2024-07-27T03:20:00.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: pgmap v534: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 409 B/s wr, 10 op/s 2024-07-27T03:20:00.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.175:6833/1334549389,v1:172.21.15.175:6835/1334549389] up:boot 2024-07-27T03:20:00.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.175:6832/1676160331,v1:172.21.15.175:6834/1676160331] up:boot 2024-07-27T03:20:00.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.67:6839/1722507060,v1:172.21.15.67:6841/1722507060] up:boot 2024-07-27T03:20:00.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.67:6838/846482796,v1:172.21.15.67:6840/846482796] up:boot 2024-07-27T03:20:00.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.149:6833/2775616736,v1:172.21.15.149:6835/2775616736] up:boot 2024-07-27T03:20:00.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.149:6832/1556765136,v1:172.21.15.149:6834/1556765136] up:boot 2024-07-27T03:20:00.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.149:6836/620433683,v1:172.21.15.149:6838/620433683] up:boot 2024-07-27T03:20:00.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.149:6837/1841204281,v1:172.21.15.149:6839/1841204281] up:boot 2024-07-27T03:20:00.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.175:6836/3857025307,v1:172.21.15.175:6838/3857025307] up:boot 2024-07-27T03:20:00.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: mds.? [v2:172.21.15.175:6837/344320809,v1:172.21.15.175:6839/344320809] up:boot 2024-07-27T03:20:00.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-07-27T03:20:00.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-07-27T03:20:00.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-07-27T03:20:00.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-07-27T03:20:00.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-07-27T03:20:00.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-07-27T03:20:00.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-07-27T03:20:00.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-07-27T03:20:00.751 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-07-27T03:20:00.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-07-27T03:20:00.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: from='mgr.14150 172.21.15.67:0/272714214' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-07-27T03:20:00.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-07-27T03:20:00.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-07-27T03:20:00.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: fs cephfs is degraded 2024-07-27T03:20:00.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-07-27T03:20:00.752 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:00 smithi149 ceph-mon[37173]: fs cephfs is offline because no MDS is active for it. 2024-07-27T03:20:01.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.121+0000 7fa293ab8640 1 -- 172.21.15.67:0/1792111066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa28c1098b0 msgr2=0x7fa28c1100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:20:01.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.121+0000 7fa293ab8640 1 --2- 172.21.15.67:0/1792111066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa28c1098b0 0x7fa28c1100e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa284009ca0 tx=0x7fa28402a8c0 comp rx=0 tx=0).stop 2024-07-27T03:20:01.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.121+0000 7fa293ab8640 1 -- 172.21.15.67:0/1792111066 shutdown_connections 2024-07-27T03:20:01.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.121+0000 7fa293ab8640 1 --2- 172.21.15.67:0/1792111066 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa28c1098b0 0x7fa28c1100e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:01.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.121+0000 7fa293ab8640 1 --2- 172.21.15.67:0/1792111066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa28c108f60 0x7fa28c109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:01.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.121+0000 7fa293ab8640 1 --2- 172.21.15.67:0/1792111066 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa28c103620 0x7fa28c103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:01.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.121+0000 7fa293ab8640 1 -- 172.21.15.67:0/1792111066 >> 172.21.15.67:0/1792111066 conn(0x7fa28c0ff070 msgr2=0x7fa28c1014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:20:01.129 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.121+0000 7fa293ab8640 1 -- 172.21.15.67:0/1792111066 shutdown_connections 2024-07-27T03:20:01.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.121+0000 7fa293ab8640 1 -- 172.21.15.67:0/1792111066 wait complete. 2024-07-27T03:20:01.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.122+0000 7fa293ab8640 1 Processor -- start 2024-07-27T03:20:01.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.122+0000 7fa293ab8640 1 -- start start 2024-07-27T03:20:01.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.122+0000 7fa293ab8640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa28c103620 0x7fa28c075180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:20:01.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.122+0000 7fa293ab8640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa28c108f60 0x7fa28c0756c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:20:01.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.122+0000 7fa293ab8640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa28c1098b0 0x7fa28c075c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:20:01.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.122+0000 7fa293ab8640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c113580 con 0x7fa28c1098b0 2024-07-27T03:20:01.130 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.122+0000 7fa293ab8640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c113400 con 0x7fa28c103620 2024-07-27T03:20:01.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.122+0000 7fa293ab8640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c113700 con 0x7fa28c108f60 2024-07-27T03:20:01.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.122+0000 7fa292055640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa28c1098b0 0x7fa28c075c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:20:01.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.122+0000 7fa292055640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa28c1098b0 0x7fa28c075c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56506/0 (socket says 172.21.15.67:56506) 2024-07-27T03:20:01.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.122+0000 7fa292055640 1 -- 172.21.15.67:0/2458819822 learned_addr learned my addr 172.21.15.67:0/2458819822 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:20:01.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.123+0000 7fa291854640 1 --2- 172.21.15.67:0/2458819822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa28c103620 0x7fa28c075180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:20:01.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.123+0000 7fa291053640 1 --2- 172.21.15.67:0/2458819822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa28c108f60 0x7fa28c0756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:20:01.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.123+0000 7fa292055640 1 -- 172.21.15.67:0/2458819822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa28c108f60 msgr2=0x7fa28c0756c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:20:01.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.123+0000 7fa292055640 1 --2- 172.21.15.67:0/2458819822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa28c108f60 0x7fa28c0756c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:01.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.123+0000 7fa292055640 1 -- 172.21.15.67:0/2458819822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa28c103620 msgr2=0x7fa28c075180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:20:01.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.123+0000 7fa292055640 1 --2- 172.21.15.67:0/2458819822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa28c103620 0x7fa28c075180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:01.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.123+0000 7fa292055640 1 -- 172.21.15.67:0/2458819822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28c1a81f0 con 0x7fa28c1098b0 2024-07-27T03:20:01.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.123+0000 7fa292055640 1 --2- 172.21.15.67:0/2458819822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa28c1098b0 0x7fa28c075c00 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa284005cc0 tx=0x7fa28402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:20:01.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.123+0000 7fa282ffd640 1 -- 172.21.15.67:0/2458819822 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa284003bf0 con 0x7fa28c1098b0 2024-07-27T03:20:01.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.123+0000 7fa282ffd640 1 -- 172.21.15.67:0/2458819822 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa284003d90 con 0x7fa28c1098b0 2024-07-27T03:20:01.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.123+0000 7fa293ab8640 1 -- 172.21.15.67:0/2458819822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28c1a8480 con 0x7fa28c1098b0 2024-07-27T03:20:01.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.123+0000 7fa282ffd640 1 -- 172.21.15.67:0/2458819822 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28402fbc0 con 0x7fa28c1098b0 2024-07-27T03:20:01.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.123+0000 7fa293ab8640 1 -- 172.21.15.67:0/2458819822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa28c1a89c0 con 0x7fa28c1098b0 2024-07-27T03:20:01.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.124+0000 7fa293ab8640 1 -- 172.21.15.67:0/2458819822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa28c103a30 con 0x7fa28c1098b0 2024-07-27T03:20:01.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.126+0000 7fa282ffd640 1 -- 172.21.15.67:0/2458819822 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7fa28402fd60 con 0x7fa28c1098b0 2024-07-27T03:20:01.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.127+0000 7fa282ffd640 1 --2- 172.21.15.67:0/2458819822 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa2780b3bd0 0x7fa2780b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:20:01.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.127+0000 7fa282ffd640 1 -- 172.21.15.67:0/2458819822 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9609+0+0 (secure 0 0 0) 0x7fa284105a20 con 0x7fa28c1098b0 2024-07-27T03:20:01.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.127+0000 7fa291854640 1 --2- 172.21.15.67:0/2458819822 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa2780b3bd0 0x7fa2780b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:20:01.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.128+0000 7fa291854640 1 --2- 172.21.15.67:0/2458819822 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa2780b3bd0 0x7fa2780b60e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa28c076bd0 tx=0x7fa28800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:20:01.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.130+0000 7fa282ffd640 1 -- 172.21.15.67:0/2458819822 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7fa2840ccb10 con 0x7fa28c1098b0 2024-07-27T03:20:01.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.352+0000 7fa293ab8640 1 -- 172.21.15.67:0/2458819822 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa28c02e100 con 0x7fa28c1098b0 2024-07-27T03:20:01.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.364+0000 7fa282ffd640 1 -- 172.21.15.67:0/2458819822 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v33) ==== 104+0+0 (secure 0 0 0) 0x7fa2840cc410 con 0x7fa28c1098b0 2024-07-27T03:20:01.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.367+0000 7fa293ab8640 1 -- 172.21.15.67:0/2458819822 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa2780b3bd0 msgr2=0x7fa2780b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:20:01.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.367+0000 7fa293ab8640 1 --2- 172.21.15.67:0/2458819822 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa2780b3bd0 0x7fa2780b60e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa28c076bd0 tx=0x7fa28800b390 comp rx=0 tx=0).stop 2024-07-27T03:20:01.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.367+0000 7fa293ab8640 1 -- 172.21.15.67:0/2458819822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa28c1098b0 msgr2=0x7fa28c075c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:20:01.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.367+0000 7fa293ab8640 1 --2- 172.21.15.67:0/2458819822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa28c1098b0 0x7fa28c075c00 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa284005cc0 tx=0x7fa28402f380 comp rx=0 tx=0).stop 2024-07-27T03:20:01.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.367+0000 7fa293ab8640 1 -- 172.21.15.67:0/2458819822 shutdown_connections 2024-07-27T03:20:01.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.367+0000 7fa293ab8640 1 --2- 172.21.15.67:0/2458819822 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7fa2780b3bd0 0x7fa2780b60e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:01.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.367+0000 7fa293ab8640 1 --2- 172.21.15.67:0/2458819822 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa28c1098b0 0x7fa28c075c00 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:01.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.367+0000 7fa293ab8640 1 --2- 172.21.15.67:0/2458819822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa28c108f60 0x7fa28c0756c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:01.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.367+0000 7fa293ab8640 1 --2- 172.21.15.67:0/2458819822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa28c103620 0x7fa28c075180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:01.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.367+0000 7fa293ab8640 1 -- 172.21.15.67:0/2458819822 >> 172.21.15.67:0/2458819822 conn(0x7fa28c0ff070 msgr2=0x7fa28c101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:20:01.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.367+0000 7fa293ab8640 1 -- 172.21.15.67:0/2458819822 shutdown_connections 2024-07-27T03:20:01.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:01.367+0000 7fa293ab8640 1 -- 172.21.15.67:0/2458819822 wait complete. 2024-07-27T03:20:01.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:01 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2458819822' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-07-27T03:20:01.691 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:01 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2458819822' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-07-27T03:20:01.749 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:01 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2458819822' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-07-27T03:20:02.119 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-07-27T03:20:02.391 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:20:02.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:02 smithi175 ceph-mon[37125]: pgmap v535: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-07-27T03:20:02.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:02 smithi175 ceph-mon[37125]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-27T03:20:02.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:02 smithi175 ceph-mon[37125]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-27T03:20:02.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:02 smithi175 ceph-mon[37125]: Cluster is now healthy 2024-07-27T03:20:02.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:02 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2458819822' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-07-27T03:20:02.625 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:02 smithi175 ceph-mon[37125]: fsmap 12 up:standby 2024-07-27T03:20:02.645 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:02 smithi067 ceph-mon[32814]: pgmap v535: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-07-27T03:20:02.645 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:02 smithi067 ceph-mon[32814]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-27T03:20:02.645 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:02 smithi067 ceph-mon[32814]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-27T03:20:02.645 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:02 smithi067 ceph-mon[32814]: Cluster is now healthy 2024-07-27T03:20:02.645 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:02 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2458819822' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-07-27T03:20:02.645 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:02 smithi067 ceph-mon[32814]: fsmap 12 up:standby 2024-07-27T03:20:02.749 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:02 smithi149 ceph-mon[37173]: pgmap v535: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-07-27T03:20:02.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:02 smithi149 ceph-mon[37173]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-07-27T03:20:02.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:02 smithi149 ceph-mon[37173]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-07-27T03:20:02.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:02 smithi149 ceph-mon[37173]: Cluster is now healthy 2024-07-27T03:20:02.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:02 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2458819822' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-07-27T03:20:02.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:02 smithi149 ceph-mon[37173]: fsmap 12 up:standby 2024-07-27T03:20:04.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.127+0000 7ffb66ced640 1 -- 172.21.15.67:0/80159041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb601024d0 msgr2=0x7ffb601028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:20:04.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.127+0000 7ffb66ced640 1 --2- 172.21.15.67:0/80159041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb601024d0 0x7ffb601028e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009900 tx=0x7ffb5402a410 comp rx=0 tx=0).stop 2024-07-27T03:20:04.132 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.127+0000 7ffb66ced640 1 -- 172.21.15.67:0/80159041 shutdown_connections 2024-07-27T03:20:04.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.127+0000 7ffb66ced640 1 --2- 172.21.15.67:0/80159041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb60108760 0x7ffb6010ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:04.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.127+0000 7ffb66ced640 1 --2- 172.21.15.67:0/80159041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb60107e10 0x7ffb60108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:04.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.127+0000 7ffb66ced640 1 --2- 172.21.15.67:0/80159041 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb601024d0 0x7ffb601028e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:04.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.127+0000 7ffb66ced640 1 -- 172.21.15.67:0/80159041 >> 172.21.15.67:0/80159041 conn(0x7ffb600791a0 msgr2=0x7ffb600795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:20:04.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.127+0000 7ffb66ced640 1 -- 172.21.15.67:0/80159041 shutdown_connections 2024-07-27T03:20:04.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.127+0000 7ffb66ced640 1 -- 172.21.15.67:0/80159041 wait complete. 2024-07-27T03:20:04.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.128+0000 7ffb66ced640 1 Processor -- start 2024-07-27T03:20:04.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.128+0000 7ffb66ced640 1 -- start start 2024-07-27T03:20:04.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.128+0000 7ffb66ced640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb601024d0 0x7ffb6019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:20:04.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.128+0000 7ffb66ced640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb60107e10 0x7ffb6019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:20:04.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.128+0000 7ffb66ced640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb60108760 0x7ffb6019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:20:04.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.128+0000 7ffb66ced640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ffb601123c0 con 0x7ffb601024d0 2024-07-27T03:20:04.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.128+0000 7ffb66ced640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffb60112240 con 0x7ffb60107e10 2024-07-27T03:20:04.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.128+0000 7ffb66ced640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffb60112540 con 0x7ffb60108760 2024-07-27T03:20:04.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.129+0000 7ffb64a89640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb601024d0 0x7ffb6019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:20:04.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.129+0000 7ffb64a89640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb601024d0 0x7ffb6019ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56524/0 (socket says 172.21.15.67:56524) 2024-07-27T03:20:04.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.129+0000 7ffb64a89640 1 -- 172.21.15.67:0/1005060898 learned_addr learned my addr 172.21.15.67:0/1005060898 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:20:04.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.129+0000 7ffb6528a640 1 --2- 172.21.15.67:0/1005060898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb60108760 0x7ffb6019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:20:04.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.129+0000 7ffb5ffff640 1 --2- 172.21.15.67:0/1005060898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb60107e10 0x7ffb6019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:20:04.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.129+0000 7ffb64a89640 1 -- 172.21.15.67:0/1005060898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb60108760 msgr2=0x7ffb6019b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:20:04.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.129+0000 7ffb64a89640 1 --2- 172.21.15.67:0/1005060898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb60108760 0x7ffb6019b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:04.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.129+0000 7ffb64a89640 1 -- 172.21.15.67:0/1005060898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb60107e10 msgr2=0x7ffb6019b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:20:04.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.129+0000 7ffb64a89640 1 --2- 172.21.15.67:0/1005060898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb60107e10 0x7ffb6019b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:04.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.129+0000 7ffb64a89640 1 -- 172.21.15.67:0/1005060898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb601a6020 con 0x7ffb601024d0 2024-07-27T03:20:04.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.129+0000 7ffb64a89640 1 --2- 172.21.15.67:0/1005060898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb601024d0 0x7ffb6019ad90 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ffb54005940 tx=0x7ffb540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:20:04.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.129+0000 7ffb5dffb640 1 -- 172.21.15.67:0/1005060898 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5403f030 con 0x7ffb601024d0 2024-07-27T03:20:04.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.129+0000 7ffb5dffb640 1 -- 172.21.15.67:0/1005060898 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb54003cb0 con 0x7ffb601024d0 2024-07-27T03:20:04.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.129+0000 7ffb66ced640 1 -- 172.21.15.67:0/1005060898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb601a62b0 con 0x7ffb601024d0 2024-07-27T03:20:04.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.130+0000 7ffb5dffb640 1 -- 172.21.15.67:0/1005060898 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb54035070 con 0x7ffb601024d0 2024-07-27T03:20:04.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.130+0000 7ffb66ced640 1 -- 172.21.15.67:0/1005060898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb601a6790 con 0x7ffb601024d0 2024-07-27T03:20:04.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.131+0000 7ffb66ced640 1 -- 172.21.15.67:0/1005060898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb601028e0 con 0x7ffb601024d0 2024-07-27T03:20:04.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.132+0000 7ffb5dffb640 1 -- 172.21.15.67:0/1005060898 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7ffb54042030 con 0x7ffb601024d0 2024-07-27T03:20:04.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.133+0000 7ffb5dffb640 1 --2- 172.21.15.67:0/1005060898 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ffb500b3c00 0x7ffb500b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:20:04.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.133+0000 7ffb5ffff640 1 --2- 172.21.15.67:0/1005060898 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ffb500b3c00 0x7ffb500b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:20:04.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.133+0000 7ffb5dffb640 1 -- 172.21.15.67:0/1005060898 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9609+0+0 (secure 0 0 0) 0x7ffb54104820 con 0x7ffb601024d0 2024-07-27T03:20:04.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.134+0000 7ffb5ffff640 1 --2- 172.21.15.67:0/1005060898 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ffb500b3c00 0x7ffb500b6110 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ffb60100730 tx=0x7ffb4c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:20:04.138 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.136+0000 7ffb5dffb640 1 -- 172.21.15.67:0/1005060898 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7ffb540cb2c0 con 0x7ffb601024d0 2024-07-27T03:20:04.323 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.321+0000 7ffb66ced640 1 -- 172.21.15.67:0/1005060898 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ffb6006ace0 con 0x7ffb601024d0 2024-07-27T03:20:04.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.374+0000 7ffb5dffb640 1 -- 172.21.15.67:0/1005060898 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) ==== 181+0+0 (secure 0 0 0) 0x7ffb540cf810 con 0x7ffb601024d0 2024-07-27T03:20:04.377 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_metadata' removed 2024-07-27T03:20:04.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.376+0000 7ffb66ced640 1 -- 172.21.15.67:0/1005060898 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ffb500b3c00 msgr2=0x7ffb500b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:20:04.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.376+0000 7ffb66ced640 1 --2- 172.21.15.67:0/1005060898 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ffb500b3c00 0x7ffb500b6110 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ffb60100730 tx=0x7ffb4c00b410 comp rx=0 tx=0).stop 2024-07-27T03:20:04.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.376+0000 7ffb66ced640 1 -- 172.21.15.67:0/1005060898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb601024d0 msgr2=0x7ffb6019ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:20:04.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.376+0000 7ffb66ced640 1 --2- 172.21.15.67:0/1005060898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb601024d0 0x7ffb6019ad90 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ffb54005940 tx=0x7ffb540042f0 comp rx=0 tx=0).stop 2024-07-27T03:20:04.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.376+0000 7ffb66ced640 1 -- 172.21.15.67:0/1005060898 shutdown_connections 2024-07-27T03:20:04.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.376+0000 7ffb66ced640 1 --2- 172.21.15.67:0/1005060898 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ffb500b3c00 0x7ffb500b6110 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:04.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.376+0000 7ffb66ced640 1 --2- 172.21.15.67:0/1005060898 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb60108760 0x7ffb6019b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:04.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.376+0000 7ffb66ced640 1 --2- 172.21.15.67:0/1005060898 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb60107e10 0x7ffb6019b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:04.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.376+0000 7ffb66ced640 1 --2- 172.21.15.67:0/1005060898 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb601024d0 0x7ffb6019ad90 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:04.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.377+0000 7ffb66ced640 1 -- 172.21.15.67:0/1005060898 >> 172.21.15.67:0/1005060898 conn(0x7ffb600791a0 msgr2=0x7ffb60100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:20:04.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.377+0000 7ffb66ced640 1 -- 172.21.15.67:0/1005060898 shutdown_connections 2024-07-27T03:20:04.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:04.377+0000 7ffb66ced640 1 -- 172.21.15.67:0/1005060898 wait complete. 2024-07-27T03:20:04.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:04 smithi175 ceph-mon[37125]: pgmap v536: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-07-27T03:20:04.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:04 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1005060898' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-27T03:20:04.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:04 smithi067 ceph-mon[32814]: pgmap v536: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-07-27T03:20:04.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:04 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1005060898' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-27T03:20:04.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:04 smithi149 ceph-mon[37173]: pgmap v536: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-07-27T03:20:04.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:04 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1005060898' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-27T03:20:05.176 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-07-27T03:20:05.447 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:20:05.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:05 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/1005060898' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-07-27T03:20:05.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:05 smithi067 ceph-mon[32814]: osdmap e93: 12 total, 12 up, 12 in 2024-07-27T03:20:05.702 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:05 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/1005060898' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-07-27T03:20:05.703 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:05 smithi175 ceph-mon[37125]: osdmap e93: 12 total, 12 up, 12 in 2024-07-27T03:20:05.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:05 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/1005060898' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-07-27T03:20:05.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:05 smithi149 ceph-mon[37173]: osdmap e93: 12 total, 12 up, 12 in 2024-07-27T03:20:06.645 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:06 smithi067 ceph-mon[32814]: pgmap v538: 97 pgs: 97 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:20:06.749 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:06 smithi149 ceph-mon[37173]: pgmap v538: 97 pgs: 97 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:20:06.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:06 smithi175 ceph-mon[37125]: pgmap v538: 97 pgs: 97 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:20:07.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.171+0000 7ff2856cd640 1 -- 172.21.15.67:0/4180986826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff280108760 msgr2=0x7ff28010ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:20:07.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.171+0000 7ff2856cd640 1 --2- 172.21.15.67:0/4180986826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff280108760 0x7ff28010ef90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff270009ca0 tx=0x7ff27002a8c0 comp rx=0 tx=0).stop 2024-07-27T03:20:07.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.171+0000 7ff2856cd640 1 -- 172.21.15.67:0/4180986826 shutdown_connections 2024-07-27T03:20:07.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.171+0000 7ff2856cd640 1 --2- 172.21.15.67:0/4180986826 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff280108760 0x7ff28010ef90 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:07.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.171+0000 7ff2856cd640 1 --2- 172.21.15.67:0/4180986826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff280107e10 0x7ff280108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:07.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.171+0000 7ff2856cd640 1 --2- 172.21.15.67:0/4180986826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2801024d0 0x7ff2801028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:07.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.171+0000 7ff2856cd640 1 -- 172.21.15.67:0/4180986826 >> 172.21.15.67:0/4180986826 conn(0x7ff2800791a0 msgr2=0x7ff2800795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:20:07.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.171+0000 7ff2856cd640 1 -- 172.21.15.67:0/4180986826 shutdown_connections 2024-07-27T03:20:07.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.171+0000 7ff2856cd640 1 -- 172.21.15.67:0/4180986826 wait complete. 2024-07-27T03:20:07.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.172+0000 7ff2856cd640 1 Processor -- start 2024-07-27T03:20:07.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.172+0000 7ff2856cd640 1 -- start start 2024-07-27T03:20:07.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.172+0000 7ff2856cd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2801024d0 0x7ff2800ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:20:07.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.172+0000 7ff2856cd640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff280107e10 0x7ff2800ff620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:20:07.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.172+0000 7ff2856cd640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff280108760 0x7ff2800ffb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:20:07.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.172+0000 7ff2856cd640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801123c0 con 0x7ff280108760 2024-07-27T03:20:07.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.172+0000 7ff2856cd640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff280112240 con 0x7ff2801024d0 2024-07-27T03:20:07.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.172+0000 7ff2856cd640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff280112540 con 0x7ff280107e10 2024-07-27T03:20:07.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.172+0000 7ff27f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff280108760 0x7ff2800ffb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:20:07.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.173+0000 7ff27f7fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff280108760 0x7ff2800ffb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56498/0 (socket says 172.21.15.67:56498) 2024-07-27T03:20:07.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.173+0000 7ff27f7fe640 1 -- 172.21.15.67:0/3716216246 learned_addr learned my addr 172.21.15.67:0/3716216246 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:20:07.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.173+0000 7ff27e7fc640 1 --2- 172.21.15.67:0/3716216246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff280107e10 0x7ff2800ff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:20:07.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.173+0000 7ff27effd640 1 --2- 172.21.15.67:0/3716216246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2801024d0 0x7ff2800ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:20:07.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.173+0000 7ff27f7fe640 1 -- 172.21.15.67:0/3716216246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff280107e10 msgr2=0x7ff2800ff620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:20:07.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.173+0000 7ff27f7fe640 1 --2- 172.21.15.67:0/3716216246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff280107e10 0x7ff2800ff620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:07.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.173+0000 7ff27f7fe640 1 -- 172.21.15.67:0/3716216246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2801024d0 msgr2=0x7ff2800ff0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:20:07.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.173+0000 7ff27f7fe640 1 --2- 172.21.15.67:0/3716216246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2801024d0 0x7ff2800ff0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:07.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.173+0000 7ff27f7fe640 1 -- 172.21.15.67:0/3716216246 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2801a8200 con 0x7ff280108760 2024-07-27T03:20:07.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.173+0000 7ff27f7fe640 1 --2- 172.21.15.67:0/3716216246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff280108760 0x7ff2800ffb60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff270005cc0 tx=0x7ff27002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:20:07.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.173+0000 7ff277fff640 1 -- 172.21.15.67:0/3716216246 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff270003bf0 con 0x7ff280108760 2024-07-27T03:20:07.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.173+0000 7ff2856cd640 1 -- 172.21.15.67:0/3716216246 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2801a8490 con 0x7ff280108760 2024-07-27T03:20:07.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.173+0000 7ff277fff640 1 -- 172.21.15.67:0/3716216246 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff270003d90 con 0x7ff280108760 2024-07-27T03:20:07.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.173+0000 7ff277fff640 1 -- 172.21.15.67:0/3716216246 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27002fbc0 con 0x7ff280108760 2024-07-27T03:20:07.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.173+0000 7ff2856cd640 1 -- 172.21.15.67:0/3716216246 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2801a8a20 con 0x7ff280108760 2024-07-27T03:20:07.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.175+0000 7ff2856cd640 1 -- 172.21.15.67:0/3716216246 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2801028e0 con 0x7ff280108760 2024-07-27T03:20:07.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.176+0000 7ff277fff640 1 -- 172.21.15.67:0/3716216246 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7ff27002fd60 con 0x7ff280108760 2024-07-27T03:20:07.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.177+0000 7ff277fff640 1 --2- 172.21.15.67:0/3716216246 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff26c0b3bd0 0x7ff26c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:20:07.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.177+0000 7ff27effd640 1 --2- 172.21.15.67:0/3716216246 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff26c0b3bd0 0x7ff26c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:20:07.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.177+0000 7ff277fff640 1 -- 172.21.15.67:0/3716216246 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9113+0+0 (secure 0 0 0) 0x7ff270105780 con 0x7ff280108760 2024-07-27T03:20:07.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.177+0000 7ff27effd640 1 --2- 172.21.15.67:0/3716216246 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff26c0b3bd0 0x7ff26c0b60e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff2780058c0 tx=0x7ff27800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:20:07.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.180+0000 7ff277fff640 1 -- 172.21.15.67:0/3716216246 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7ff2700cca60 con 0x7ff280108760 2024-07-27T03:20:07.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.364+0000 7ff2856cd640 1 -- 172.21.15.67:0/3716216246 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff2801003e0 con 0x7ff280108760 2024-07-27T03:20:07.393 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.391+0000 7ff277fff640 1 -- 172.21.15.67:0/3716216246 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) ==== 169+0+0 (secure 0 0 0) 0x7ff2801003e0 con 0x7ff280108760 2024-07-27T03:20:07.393 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data' removed 2024-07-27T03:20:07.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.393+0000 7ff2856cd640 1 -- 172.21.15.67:0/3716216246 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff26c0b3bd0 msgr2=0x7ff26c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:20:07.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.393+0000 7ff2856cd640 1 --2- 172.21.15.67:0/3716216246 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff26c0b3bd0 0x7ff26c0b60e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff2780058c0 tx=0x7ff27800a2d0 comp rx=0 tx=0).stop 2024-07-27T03:20:07.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.393+0000 7ff2856cd640 1 -- 172.21.15.67:0/3716216246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff280108760 msgr2=0x7ff2800ffb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:20:07.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.394+0000 7ff2856cd640 1 --2- 172.21.15.67:0/3716216246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff280108760 0x7ff2800ffb60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff270005cc0 tx=0x7ff27002f380 comp rx=0 tx=0).stop 2024-07-27T03:20:07.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.394+0000 7ff2856cd640 1 -- 172.21.15.67:0/3716216246 shutdown_connections 2024-07-27T03:20:07.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.394+0000 7ff2856cd640 1 --2- 172.21.15.67:0/3716216246 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7ff26c0b3bd0 0x7ff26c0b60e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:07.397 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.394+0000 7ff2856cd640 1 --2- 172.21.15.67:0/3716216246 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff280108760 0x7ff2800ffb60 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:07.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.394+0000 7ff2856cd640 1 --2- 172.21.15.67:0/3716216246 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff280107e10 0x7ff2800ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:07.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.394+0000 7ff2856cd640 1 --2- 172.21.15.67:0/3716216246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2801024d0 0x7ff2800ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:07.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.394+0000 7ff2856cd640 1 -- 172.21.15.67:0/3716216246 >> 172.21.15.67:0/3716216246 conn(0x7ff2800791a0 msgr2=0x7ff28006acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:20:07.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.394+0000 7ff2856cd640 1 -- 172.21.15.67:0/3716216246 shutdown_connections 2024-07-27T03:20:07.398 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:07.394+0000 7ff2856cd640 1 -- 172.21.15.67:0/3716216246 wait complete. 2024-07-27T03:20:07.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:07 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3716216246' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-27T03:20:07.749 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:07 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3716216246' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-27T03:20:07.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:07 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3716216246' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-27T03:20:08.129 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8 shell --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-07-27T03:20:08.409 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/mon.a/config 2024-07-27T03:20:08.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:08 smithi067 ceph-mon[32814]: pgmap v539: 97 pgs: 97 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:20:08.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:08 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/3716216246' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-07-27T03:20:08.668 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:08 smithi067 ceph-mon[32814]: osdmap e94: 12 total, 12 up, 12 in 2024-07-27T03:20:08.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:08 smithi149 ceph-mon[37173]: pgmap v539: 97 pgs: 97 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:20:08.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:08 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/3716216246' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-07-27T03:20:08.750 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:08 smithi149 ceph-mon[37173]: osdmap e94: 12 total, 12 up, 12 in 2024-07-27T03:20:08.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:08 smithi175 ceph-mon[37125]: pgmap v539: 97 pgs: 97 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:20:08.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:08 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/3716216246' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-07-27T03:20:08.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:08 smithi175 ceph-mon[37125]: osdmap e94: 12 total, 12 up, 12 in 2024-07-27T03:20:10.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.240+0000 7f60ce25f640 1 -- 172.21.15.67:0/3209904375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60c8075220 msgr2=0x7f60c8075630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:20:10.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.240+0000 7f60ce25f640 1 --2- 172.21.15.67:0/3209904375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60c8075220 0x7f60c8075630 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f60b8009900 tx=0x7f60b802a3f0 comp rx=0 tx=0).stop 2024-07-27T03:20:10.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.240+0000 7f60ce25f640 1 -- 172.21.15.67:0/3209904375 shutdown_connections 2024-07-27T03:20:10.243 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.240+0000 7f60ce25f640 1 --2- 172.21.15.67:0/3209904375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60c810fb30 0x7f60c8111f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.240+0000 7f60ce25f640 1 --2- 172.21.15.67:0/3209904375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60c8075b70 0x7f60c810f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.240+0000 7f60ce25f640 1 --2- 172.21.15.67:0/3209904375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60c8075220 0x7f60c8075630 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.240+0000 7f60ce25f640 1 -- 172.21.15.67:0/3209904375 >> 172.21.15.67:0/3209904375 conn(0x7f60c807b2c0 msgr2=0x7f60c807b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:20:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.240+0000 7f60ce25f640 1 -- 172.21.15.67:0/3209904375 shutdown_connections 2024-07-27T03:20:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.240+0000 7f60ce25f640 1 -- 172.21.15.67:0/3209904375 wait complete. 2024-07-27T03:20:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.241+0000 7f60ce25f640 1 Processor -- start 2024-07-27T03:20:10.244 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.241+0000 7f60ce25f640 1 -- start start 2024-07-27T03:20:10.245 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.242+0000 7f60ce25f640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60c8075220 0x7f60c819c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:20:10.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.242+0000 7f60ce25f640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60c8075b70 0x7f60c819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:20:10.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.242+0000 7f60ce25f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60c810fb30 0x7f60c81a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:20:10.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.242+0000 7f60ce25f640 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 -- 0x7f60c81156c0 con 0x7f60c8075220 2024-07-27T03:20:10.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.242+0000 7f60ce25f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60c8115540 con 0x7f60c810fb30 2024-07-27T03:20:10.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.242+0000 7f60ce25f640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60c8115840 con 0x7f60c8075b70 2024-07-27T03:20:10.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.242+0000 7f60c77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60c8075220 0x7f60c819c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:20:10.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.242+0000 7f60c77fe640 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60c8075220 0x7f60c819c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:56508/0 (socket says 172.21.15.67:56508) 2024-07-27T03:20:10.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.242+0000 7f60c77fe640 1 -- 172.21.15.67:0/2476138687 learned_addr learned my addr 172.21.15.67:0/2476138687 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-07-27T03:20:10.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.242+0000 7f60c6ffd640 1 --2- 172.21.15.67:0/2476138687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60c8075b70 0x7f60c819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:20:10.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.242+0000 7f60c7fff640 1 --2- 172.21.15.67:0/2476138687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60c810fb30 0x7f60c81a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:20:10.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.242+0000 7f60c77fe640 1 -- 172.21.15.67:0/2476138687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60c8075b70 msgr2=0x7f60c819c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:20:10.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.242+0000 7f60c77fe640 1 --2- 172.21.15.67:0/2476138687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60c8075b70 0x7f60c819c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:10.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.242+0000 7f60c77fe640 1 -- 172.21.15.67:0/2476138687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60c810fb30 msgr2=0x7f60c81a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:20:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.242+0000 7f60c77fe640 1 --2- 172.21.15.67:0/2476138687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60c810fb30 0x7f60c81a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.242+0000 7f60c77fe640 1 -- 172.21.15.67:0/2476138687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c81a20a0 con 0x7f60c8075220 2024-07-27T03:20:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.242+0000 7f60c7fff640 1 --2- 172.21.15.67:0/2476138687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60c810fb30 0x7f60c81a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-07-27T03:20:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.242+0000 7f60c77fe640 1 --2- 172.21.15.67:0/2476138687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60c8075220 0x7f60c819c1f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f60b8006010 tx=0x7f60b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:20:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.243+0000 7f60c4ff9640 1 -- 172.21.15.67:0/2476138687 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b803f030 con 0x7f60c8075220 2024-07-27T03:20:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.243+0000 7f60ce25f640 1 -- 172.21.15.67:0/2476138687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c81a2330 con 0x7f60c8075220 2024-07-27T03:20:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.243+0000 7f60c4ff9640 1 -- 172.21.15.67:0/2476138687 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60b8003800 con 0x7f60c8075220 2024-07-27T03:20:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.243+0000 7f60c4ff9640 1 -- 172.21.15.67:0/2476138687 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b8035070 con 0x7f60c8075220 2024-07-27T03:20:10.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.243+0000 7f60ce25f640 1 -- 172.21.15.67:0/2476138687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c81a2860 con 0x7f60c8075220 2024-07-27T03:20:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.244+0000 7f60ce25f640 1 -- 172.21.15.67:0/2476138687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60c8075630 con 0x7f60c8075220 2024-07-27T03:20:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.245+0000 7f60c4ff9640 1 -- 172.21.15.67:0/2476138687 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 18) ==== 151059+0+0 (secure 0 0 0) 0x7f60b8003cb0 con 0x7f60c8075220 2024-07-27T03:20:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.246+0000 7f60c4ff9640 1 --2- 172.21.15.67:0/2476138687 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f60b00b3c00 0x7f60b00b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-07-27T03:20:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.246+0000 7f60c6ffd640 1 --2- 172.21.15.67:0/2476138687 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f60b00b3c00 0x7f60b00b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-07-27T03:20:10.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.246+0000 7f60c4ff9640 1 -- 172.21.15.67:0/2476138687 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8496+0+0 (secure 0 0 0) 0x7f60b81047f0 con 0x7f60c8075220 2024-07-27T03:20:10.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.247+0000 7f60c6ffd640 1 --2- 172.21.15.67:0/2476138687 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f60b00b3c00 0x7f60b00b6110 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f60c8111830 tx=0x7f60bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-07-27T03:20:10.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.249+0000 7f60c4ff9640 1 -- 172.21.15.67:0/2476138687 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+194024 (secure 0 0 0) 0x7f60c81a2860 con 0x7f60c8075220 2024-07-27T03:20:10.436 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:10.433+0000 7f60ce25f640 1 -- 172.21.15.67:0/2476138687 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f60c81a8d30 con 0x7f60c8075220 2024-07-27T03:20:10.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:10 smithi067 ceph-mon[32814]: pgmap v541: 33 pgs: 33 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:20:10.749 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:10 smithi149 ceph-mon[37173]: pgmap v541: 33 pgs: 33 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:20:10.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:10 smithi175 ceph-mon[37125]: pgmap v541: 33 pgs: 33 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:20:11.406 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:11.403+0000 7f60c4ff9640 1 -- 172.21.15.67:0/2476138687 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v95) ==== 178+0+0 (secure 0 0 0) 0x7f60b80cbd40 con 0x7f60c8075220 2024-07-27T03:20:11.406 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'cephfs_data_ec' removed 2024-07-27T03:20:11.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:11.406+0000 7f60ce25f640 1 -- 172.21.15.67:0/2476138687 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f60b00b3c00 msgr2=0x7f60b00b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:20:11.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:11.406+0000 7f60ce25f640 1 --2- 172.21.15.67:0/2476138687 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f60b00b3c00 0x7f60b00b6110 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f60c8111830 tx=0x7f60bc005f90 comp rx=0 tx=0).stop 2024-07-27T03:20:11.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:11.406+0000 7f60ce25f640 1 -- 172.21.15.67:0/2476138687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60c8075220 msgr2=0x7f60c819c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-07-27T03:20:11.408 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:11.406+0000 7f60ce25f640 1 --2- 172.21.15.67:0/2476138687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60c8075220 0x7f60c819c1f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f60b8006010 tx=0x7f60b80042f0 comp rx=0 tx=0).stop 2024-07-27T03:20:11.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:11.406+0000 7f60ce25f640 1 -- 172.21.15.67:0/2476138687 shutdown_connections 2024-07-27T03:20:11.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:11.406+0000 7f60ce25f640 1 --2- 172.21.15.67:0/2476138687 >> [v2:172.21.15.67:6800/1534270904,v1:172.21.15.67:6801/1534270904] conn(0x7f60b00b3c00 0x7f60b00b6110 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:11.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:11.406+0000 7f60ce25f640 1 --2- 172.21.15.67:0/2476138687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60c810fb30 0x7f60c81a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:11.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:11.406+0000 7f60ce25f640 1 --2- 172.21.15.67:0/2476138687 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60c8075b70 0x7f60c819c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:11.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:11.406+0000 7f60ce25f640 1 --2- 172.21.15.67:0/2476138687 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f60c8075220 0x7f60c819c1f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-07-27T03:20:11.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:11.406+0000 7f60ce25f640 1 -- 172.21.15.67:0/2476138687 >> 172.21.15.67:0/2476138687 conn(0x7f60c807b2c0 msgr2=0x7f60c8108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-07-27T03:20:11.409 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:11.406+0000 7f60ce25f640 1 -- 172.21.15.67:0/2476138687 shutdown_connections 2024-07-27T03:20:11.410 INFO:teuthology.orchestra.run.smithi067.stderr:2024-07-27T03:20:11.406+0000 7f60ce25f640 1 -- 172.21.15.67:0/2476138687 wait complete. 2024-07-27T03:20:11.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:11 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2476138687' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-27T03:20:11.749 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:11 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2476138687' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-27T03:20:11.874 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:11 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2476138687' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-07-27T03:20:12.127 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-27T03:20:12.159 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-27T03:20:12.192 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-27T03:20:12.226 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-07-27T03:20:12.226 DEBUG:teuthology.orchestra.run.smithi067:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-07-27T03:20:12.245 DEBUG:teuthology.orchestra.run.smithi149:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-07-27T03:20:12.262 DEBUG:teuthology.orchestra.run.smithi175:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-07-27T03:20:12.283 INFO:tasks.cephadm:Stopping all daemons... 2024-07-27T03:20:12.284 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-07-27T03:20:12.284 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mon.a 2024-07-27T03:20:12.407 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:12 smithi067 systemd[1]: Stopping Ceph mon.a for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:20:12.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:12 smithi149 ceph-mon[37173]: pgmap v542: 33 pgs: 33 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:20:12.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:12 smithi149 ceph-mon[37173]: from='client.? 172.21.15.67:0/2476138687' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-07-27T03:20:12.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:12 smithi149 ceph-mon[37173]: osdmap e95: 12 total, 12 up, 12 in 2024-07-27T03:20:12.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:12 smithi175 ceph-mon[37125]: pgmap v542: 33 pgs: 33 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:20:12.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:12 smithi175 ceph-mon[37125]: from='client.? 172.21.15.67:0/2476138687' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-07-27T03:20:12.624 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:12 smithi175 ceph-mon[37125]: osdmap e95: 12 total, 12 up, 12 in 2024-07-27T03:20:12.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:12 smithi067 ceph-mon[32814]: pgmap v542: 33 pgs: 33 active+clean; 577 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail 2024-07-27T03:20:12.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:12 smithi067 ceph-mon[32814]: from='client.? 172.21.15.67:0/2476138687' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-07-27T03:20:12.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:12 smithi067 ceph-mon[32814]: osdmap e95: 12 total, 12 up, 12 in 2024-07-27T03:20:12.719 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:12 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-a[32809]: 2024-07-27T03:20:12.625+0000 7efd03cd7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:20:12.720 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:12 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-a[32809]: 2024-07-27T03:20:12.625+0000 7efd03cd7640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-07-27T03:20:13.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:12 smithi067 podman[105724]: 2024-07-27 03:20:12.820699121 +0000 UTC m=+0.335103328 container died 23a97e917213294314c4e6b62ead3c7165eb08c17308bf9c3d514b912dacad6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-27T03:20:13.469 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:13 smithi067 podman[105724]: 2024-07-27 03:20:13.309549133 +0000 UTC m=+0.823953339 container cleanup 23a97e917213294314c4e6b62ead3c7165eb08c17308bf9c3d514b912dacad6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-a, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-27T03:20:13.470 INFO:journalctl@ceph.mon.a.smithi067.stdout:Jul 27 03:20:13 smithi067 bash[105724]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-a 2024-07-27T03:20:13.704 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mon.a.service' 2024-07-27T03:20:13.755 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:20:13.756 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-07-27T03:20:13.756 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-07-27T03:20:13.756 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mon.b 2024-07-27T03:20:14.088 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:13 smithi149 systemd[1]: Stopping Ceph mon.b for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:20:14.499 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:14 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-b[37164]: 2024-07-27T03:20:14.085+0000 7f7171baa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:20:14.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:14 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-b[37164]: 2024-07-27T03:20:14.085+0000 7f7171baa640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-07-27T03:20:14.500 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:14 smithi149 podman[70555]: 2024-07-27 03:20:14.290825483 +0000 UTC m=+0.334980981 container died caccb2bcb96e35cf409d8bd047e3a0dcc030005a4207d39be2ba59cbb6a719fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-b, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0) 2024-07-27T03:20:14.956 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:14 smithi149 podman[70555]: 2024-07-27 03:20:14.67840721 +0000 UTC m=+0.722562654 container cleanup caccb2bcb96e35cf409d8bd047e3a0dcc030005a4207d39be2ba59cbb6a719fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-07-27T03:20:14.957 INFO:journalctl@ceph.mon.b.smithi149.stdout:Jul 27 03:20:14 smithi149 bash[70555]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-b 2024-07-27T03:20:15.083 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mon.b.service' 2024-07-27T03:20:15.146 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:20:15.147 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-07-27T03:20:15.147 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-07-27T03:20:15.147 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mon.c 2024-07-27T03:20:15.464 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:15 smithi175 systemd[1]: Stopping Ceph mon.c for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:20:15.815 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:15 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-c[37121]: 2024-07-27T03:20:15.461+0000 7f340762d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:20:15.815 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:15 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-c[37121]: 2024-07-27T03:20:15.461+0000 7f340762d640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-07-27T03:20:15.815 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:15 smithi175 podman[71153]: 2024-07-27 03:20:15.502845086 +0000 UTC m=+0.164570237 container died f9196a2808bcb3837bbb7580848001c39e79464a83850e0ebe482cfad09ac90b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-07-27T03:20:16.121 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:15 smithi175 podman[71153]: 2024-07-27 03:20:15.912945273 +0000 UTC m=+0.574670428 container cleanup f9196a2808bcb3837bbb7580848001c39e79464a83850e0ebe482cfad09ac90b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-07-27T03:20:16.121 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:15 smithi175 bash[71153]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-c 2024-07-27T03:20:16.374 INFO:journalctl@ceph.mon.c.smithi175.stdout:Jul 27 03:20:16 smithi175 podman[71165]: 2024-07-27 03:20:16.172724165 +0000 UTC m=+0.673251466 container remove f9196a2808bcb3837bbb7580848001c39e79464a83850e0ebe482cfad09ac90b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf) 2024-07-27T03:20:16.397 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mon.c.service' 2024-07-27T03:20:16.444 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:20:16.444 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-07-27T03:20:16.444 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-07-27T03:20:16.444 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mgr.x 2024-07-27T03:20:16.768 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:20:16 smithi067 systemd[1]: Stopping Ceph mgr.x for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:20:17.110 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:20:16 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x[33058]: 2024-07-27T03:20:16.764+0000 7f45135c0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:20:17.110 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:20:16 smithi067 podman[105845]: 2024-07-27 03:20:16.806284234 +0000 UTC m=+0.164561692 container died b8e00c316ed9dfe7584904853eb2a5a0f312309c8ace088c7d1945c64fb04acb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240724) 2024-07-27T03:20:17.469 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:20:17 smithi067 podman[105845]: 2024-07-27 03:20:17.315743756 +0000 UTC m=+0.674021213 container cleanup b8e00c316ed9dfe7584904853eb2a5a0f312309c8ace088c7d1945c64fb04acb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, ceph=True, org.label-schema.schema-version=1.0) 2024-07-27T03:20:17.469 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:20:17 smithi067 bash[105845]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x 2024-07-27T03:20:17.777 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Jul 27 03:20:17 smithi067 podman[105858]: 2024-07-27 03:20:17.525907822 +0000 UTC m=+0.723303430 container remove b8e00c316ed9dfe7584904853eb2a5a0f312309c8ace088c7d1945c64fb04acb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:af8fdbf079811fe158ce62927bd145c35b74d4b8, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-x, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-07-27T03:20:17.781 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mgr.x.service' 2024-07-27T03:20:17.820 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:20:17.820 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-07-27T03:20:17.820 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-07-27T03:20:17.821 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mgr.y 2024-07-27T03:20:18.165 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Jul 27 03:20:17 smithi149 systemd[1]: Stopping Ceph mgr.y for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:20:18.470 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Jul 27 03:20:18 smithi149 podman[70681]: 2024-07-27 03:20:18.163248309 +0000 UTC m=+0.153148789 container died 7c5e4c54a309023ff7ba4abc1261d8f80e3c637d098ee5fecfa906fe748123f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-y, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_CLEAN=True, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-27T03:20:18.745 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Jul 27 03:20:18 smithi149 podman[70681]: 2024-07-27 03:20:18.551235731 +0000 UTC m=+0.541136212 container cleanup 7c5e4c54a309023ff7ba4abc1261d8f80e3c637d098ee5fecfa906fe748123f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-27T03:20:18.746 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Jul 27 03:20:18 smithi149 bash[70681]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-y 2024-07-27T03:20:18.975 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mgr.y.service' 2024-07-27T03:20:18.997 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Jul 27 03:20:18 smithi149 podman[70695]: 2024-07-27 03:20:18.744271031 +0000 UTC m=+0.568940959 container remove 7c5e4c54a309023ff7ba4abc1261d8f80e3c637d098ee5fecfa906fe748123f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-27T03:20:18.997 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Jul 27 03:20:18 smithi149 systemd[1]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-07-27T03:20:18.997 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Jul 27 03:20:18 smithi149 systemd[1]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-07-27T03:20:18.997 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Jul 27 03:20:18 smithi149 systemd[1]: Stopped Ceph mgr.y for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:20:18.997 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Jul 27 03:20:18 smithi149 systemd[1]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mgr.y.service: Consumed 6.663s CPU time. 2024-07-27T03:20:19.008 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:20:19.008 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-07-27T03:20:19.008 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-07-27T03:20:19.008 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mgr.z 2024-07-27T03:20:19.374 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jul 27 03:20:19 smithi175 systemd[1]: Stopping Ceph mgr.z for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:20:19.670 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jul 27 03:20:19 smithi175 podman[71276]: 2024-07-27 03:20:19.388244733 +0000 UTC m=+0.180204412 container died 8d5ca39b6dcd1596f12d71e6442642aba73e1d94e61fd0512391c315ca14bd5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-27T03:20:20.037 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jul 27 03:20:19 smithi175 podman[71276]: 2024-07-27 03:20:19.768500535 +0000 UTC m=+0.560460272 container cleanup 8d5ca39b6dcd1596f12d71e6442642aba73e1d94e61fd0512391c315ca14bd5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0) 2024-07-27T03:20:20.038 INFO:journalctl@ceph.mgr.z.smithi175.stdout:Jul 27 03:20:19 smithi175 bash[71276]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mgr-z 2024-07-27T03:20:20.190 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mgr.z.service' 2024-07-27T03:20:20.223 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:20:20.224 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-07-27T03:20:20.224 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-07-27T03:20:20.224 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.0 2024-07-27T03:20:20.548 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:20:20 smithi067 systemd[1]: Stopping Ceph osd.0 for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:20:20.969 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:20:20 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-0[50496]: 2024-07-27T03:20:20.545+0000 7fd6e742d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:20:20.969 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:20:20 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-0[50496]: 2024-07-27T03:20:20.545+0000 7fd6e742d640 -1 osd.0 95 *** Got signal Terminated *** 2024-07-27T03:20:20.969 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:20:20 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-0[50496]: 2024-07-27T03:20:20.545+0000 7fd6e742d640 -1 osd.0 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-27T03:20:26.199 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:20:25 smithi067 podman[105974]: 2024-07-27 03:20:25.907683001 +0000 UTC m=+5.479715545 container died d36756a1846655ba4bca52977df0e0d7aab292e687e2deccf96b6a5e974b9650 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_BRANCH=HEAD) 2024-07-27T03:20:26.469 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:20:26 smithi067 podman[105974]: 2024-07-27 03:20:26.405165457 +0000 UTC m=+5.977198000 container cleanup d36756a1846655ba4bca52977df0e0d7aab292e687e2deccf96b6a5e974b9650 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, ceph=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-27T03:20:26.469 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:20:26 smithi067 bash[105974]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-0 2024-07-27T03:20:26.969 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:20:26 smithi067 podman[105989]: 2024-07-27 03:20:26.665277979 +0000 UTC m=+0.753039557 container remove d36756a1846655ba4bca52977df0e0d7aab292e687e2deccf96b6a5e974b9650 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-27T03:20:27.295 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:20:27 smithi067 podman[106076]: 2024-07-27 03:20:26.940250421 +0000 UTC m=+0.018169533 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c 2024-07-27T03:20:27.295 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:20:27 smithi067 podman[106076]: 2024-07-27 03:20:27.083072695 +0000 UTC m=+0.160991793 container create 29bb6e8ba066b3d0e0d06f1da7ff26ae0c7a251eddc2da5272d35b504834e761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-0-deactivate, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_CLEAN=True, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-07-27T03:20:27.719 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:20:27 smithi067 podman[106076]: 2024-07-27 03:20:27.293913618 +0000 UTC m=+0.371832729 container init 29bb6e8ba066b3d0e0d06f1da7ff26ae0c7a251eddc2da5272d35b504834e761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-07-27T03:20:27.719 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:20:27 smithi067 podman[106076]: 2024-07-27 03:20:27.29760079 +0000 UTC m=+0.375519888 container start 29bb6e8ba066b3d0e0d06f1da7ff26ae0c7a251eddc2da5272d35b504834e761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-07-27T03:20:27.720 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:20:27 smithi067 podman[106076]: 2024-07-27 03:20:27.394070951 +0000 UTC m=+0.471990050 container attach 29bb6e8ba066b3d0e0d06f1da7ff26ae0c7a251eddc2da5272d35b504834e761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-07-27T03:20:27.720 INFO:journalctl@ceph.osd.0.smithi067.stdout:Jul 27 03:20:27 smithi067 podman[106076]: 2024-07-27 03:20:27.49990292 +0000 UTC m=+0.577822036 container died 29bb6e8ba066b3d0e0d06f1da7ff26ae0c7a251eddc2da5272d35b504834e761 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-0-deactivate, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-07-27T03:20:28.178 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.0.service' 2024-07-27T03:20:28.219 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:20:28.219 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-07-27T03:20:28.220 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-07-27T03:20:28.220 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.1 2024-07-27T03:20:28.499 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:20:28 smithi149 systemd[1]: Stopping Ceph osd.1 for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:20:28.999 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:20:28 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-1[41680]: 2024-07-27T03:20:28.522+0000 7fce2f7f0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:20:29.000 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:20:28 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-1[41680]: 2024-07-27T03:20:28.522+0000 7fce2f7f0640 -1 osd.1 95 *** Got signal Terminated *** 2024-07-27T03:20:29.000 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:20:28 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-1[41680]: 2024-07-27T03:20:28.522+0000 7fce2f7f0640 -1 osd.1 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-27T03:20:34.283 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:20:34 smithi149 podman[70802]: 2024-07-27 03:20:34.025045234 +0000 UTC m=+5.617566139 container died 956ef7224651158a17e6277914f3eb82e7e0eee8884cd0f93c1b35e5742eeb5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-1, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, ceph=True) 2024-07-27T03:20:34.700 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:20:34 smithi149 podman[70802]: 2024-07-27 03:20:34.38051376 +0000 UTC m=+5.973034661 container cleanup 956ef7224651158a17e6277914f3eb82e7e0eee8884cd0f93c1b35e5742eeb5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-1, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-07-27T03:20:34.700 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:20:34 smithi149 bash[70802]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-1 2024-07-27T03:20:34.999 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:20:34 smithi149 podman[70817]: 2024-07-27 03:20:34.698948443 +0000 UTC m=+0.668106098 container remove 956ef7224651158a17e6277914f3eb82e7e0eee8884cd0f93c1b35e5742eeb5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240724) 2024-07-27T03:20:35.346 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:20:35 smithi149 podman[70908]: 2024-07-27 03:20:34.960394221 +0000 UTC m=+0.017186181 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c 2024-07-27T03:20:35.346 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:20:35 smithi149 podman[70908]: 2024-07-27 03:20:35.091727124 +0000 UTC m=+0.148519077 container create 5c52dad8c3622c5e09ecd57b067b0066dc97662a57e0f23a1a361bf42e4b9b2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724) 2024-07-27T03:20:35.750 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:20:35 smithi149 podman[70908]: 2024-07-27 03:20:35.344827252 +0000 UTC m=+0.401619217 container init 5c52dad8c3622c5e09ecd57b067b0066dc97662a57e0f23a1a361bf42e4b9b2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2) 2024-07-27T03:20:35.750 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:20:35 smithi149 podman[70908]: 2024-07-27 03:20:35.347947474 +0000 UTC m=+0.404739434 container start 5c52dad8c3622c5e09ecd57b067b0066dc97662a57e0f23a1a361bf42e4b9b2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-1-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-27T03:20:35.750 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:20:35 smithi149 podman[70908]: 2024-07-27 03:20:35.436395055 +0000 UTC m=+0.493187022 container attach 5c52dad8c3622c5e09ecd57b067b0066dc97662a57e0f23a1a361bf42e4b9b2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-27T03:20:35.750 INFO:journalctl@ceph.osd.1.smithi149.stdout:Jul 27 03:20:35 smithi149 podman[70908]: 2024-07-27 03:20:35.534970317 +0000 UTC m=+0.591762279 container died 5c52dad8c3622c5e09ecd57b067b0066dc97662a57e0f23a1a361bf42e4b9b2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-07-27T03:20:36.201 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.1.service' 2024-07-27T03:20:36.240 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:20:36.241 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-07-27T03:20:36.241 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-07-27T03:20:36.241 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.2 2024-07-27T03:20:36.565 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:20:36 smithi175 systemd[1]: Stopping Ceph osd.2 for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:20:36.818 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:20:36 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-2[42269]: 2024-07-27T03:20:36.562+0000 7f09b71ed640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:20:36.818 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:20:36 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-2[42269]: 2024-07-27T03:20:36.562+0000 7f09b71ed640 -1 osd.2 95 *** Got signal Terminated *** 2024-07-27T03:20:36.818 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:20:36 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-2[42269]: 2024-07-27T03:20:36.562+0000 7f09b71ed640 -1 osd.2 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-27T03:20:42.158 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:20:41 smithi175 podman[71404]: 2024-07-27 03:20:41.875601915 +0000 UTC m=+5.440505979 container died 404238dc1400c7ee3e18d61c4b24f0551d3dcc0266eeb4a81a816082af1bdf06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724) 2024-07-27T03:20:42.558 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:20:42 smithi175 podman[71404]: 2024-07-27 03:20:42.238131814 +0000 UTC m=+5.803035882 container cleanup 404238dc1400c7ee3e18d61c4b24f0551d3dcc0266eeb4a81a816082af1bdf06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2) 2024-07-27T03:20:42.558 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:20:42 smithi175 bash[71404]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-2 2024-07-27T03:20:42.874 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:20:42 smithi175 podman[71430]: 2024-07-27 03:20:42.556381197 +0000 UTC m=+0.677262346 container remove 404238dc1400c7ee3e18d61c4b24f0551d3dcc0266eeb4a81a816082af1bdf06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf) 2024-07-27T03:20:43.153 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:20:42 smithi175 podman[71520]: 2024-07-27 03:20:42.777961732 +0000 UTC m=+0.017780521 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c 2024-07-27T03:20:43.153 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:20:42 smithi175 podman[71520]: 2024-07-27 03:20:42.907418872 +0000 UTC m=+0.147237708 container create 873c58ea6bd5578aeec76e8a3f1af1204979698e43c058aa7aad4d1b74674075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0) 2024-07-27T03:20:43.624 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:20:43 smithi175 podman[71520]: 2024-07-27 03:20:43.151869524 +0000 UTC m=+0.391688299 container init 873c58ea6bd5578aeec76e8a3f1af1204979698e43c058aa7aad4d1b74674075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0) 2024-07-27T03:20:43.625 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:20:43 smithi175 podman[71520]: 2024-07-27 03:20:43.155301033 +0000 UTC m=+0.395119809 container start 873c58ea6bd5578aeec76e8a3f1af1204979698e43c058aa7aad4d1b74674075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-2-deactivate, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=) 2024-07-27T03:20:43.625 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:20:43 smithi175 podman[71520]: 2024-07-27 03:20:43.260312768 +0000 UTC m=+0.500131560 container attach 873c58ea6bd5578aeec76e8a3f1af1204979698e43c058aa7aad4d1b74674075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240724, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-07-27T03:20:43.625 INFO:journalctl@ceph.osd.2.smithi175.stdout:Jul 27 03:20:43 smithi175 podman[71520]: 2024-07-27 03:20:43.338198208 +0000 UTC m=+0.578016993 container died 873c58ea6bd5578aeec76e8a3f1af1204979698e43c058aa7aad4d1b74674075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-07-27T03:20:43.979 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.2.service' 2024-07-27T03:20:44.019 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:20:44.019 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-07-27T03:20:44.019 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-07-27T03:20:44.020 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.3 2024-07-27T03:20:44.345 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 27 03:20:44 smithi067 systemd[1]: Stopping Ceph osd.3 for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:20:44.719 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 27 03:20:44 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-3[57396]: 2024-07-27T03:20:44.342+0000 7f9e6b679640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:20:44.719 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 27 03:20:44 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-3[57396]: 2024-07-27T03:20:44.342+0000 7f9e6b679640 -1 osd.3 95 *** Got signal Terminated *** 2024-07-27T03:20:44.719 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 27 03:20:44 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-3[57396]: 2024-07-27T03:20:44.342+0000 7f9e6b679640 -1 osd.3 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-27T03:20:48.874 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:20:48 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:20:48.489+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.493385+0000 front 2024-07-27T03:20:23.493461+0000 (oldest deadline 2024-07-27T03:20:48.193306+0000) 2024-07-27T03:20:49.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:20:48 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:20:48.886+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:20:49.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:20:48 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:20:48.771+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:20:49.874 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:20:49 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:20:49.473+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.493385+0000 front 2024-07-27T03:20:23.493461+0000 (oldest deadline 2024-07-27T03:20:48.193306+0000) 2024-07-27T03:20:50.137 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:20:49 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:20:49.924+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:20:50.137 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 27 03:20:49 smithi067 podman[106193]: 2024-07-27 03:20:49.867702253 +0000 UTC m=+5.634429491 container died 09a91e0de36fbafb77d61491555e3b385b8ab6b526a917eca8e343939c466570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724) 2024-07-27T03:20:50.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:20:49 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:20:49.792+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:20:50.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:20:49 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:20:49.983+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:20:50.469 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 27 03:20:50 smithi067 podman[106193]: 2024-07-27 03:20:50.335151526 +0000 UTC m=+6.101878763 container cleanup 09a91e0de36fbafb77d61491555e3b385b8ab6b526a917eca8e343939c466570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-07-27T03:20:50.469 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 27 03:20:50 smithi067 bash[106193]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-3 2024-07-27T03:20:50.469 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:20:50 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:20:50.262+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:20:50.874 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:20:50 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:20:50.489+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.493385+0000 front 2024-07-27T03:20:23.493461+0000 (oldest deadline 2024-07-27T03:20:48.193306+0000) 2024-07-27T03:20:50.949 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 27 03:20:50 smithi067 podman[106221]: 2024-07-27 03:20:50.662047967 +0000 UTC m=+0.791955004 container remove 09a91e0de36fbafb77d61491555e3b385b8ab6b526a917eca8e343939c466570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-07-27T03:20:51.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:20:50 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:20:50.946+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:20:51.219 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 27 03:20:51 smithi067 podman[106308]: 2024-07-27 03:20:51.021395826 +0000 UTC m=+0.112395968 container create 3c8c5b6ddd0d96e1545dda58b1d6ca7b07a13553f7929b028cfcc717363e58f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-07-27T03:20:51.219 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 27 03:20:51 smithi067 podman[106308]: 2024-07-27 03:20:50.927036293 +0000 UTC m=+0.018036440 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c 2024-07-27T03:20:51.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:20:50 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:20:50.812+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:20:51.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:20:50 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:20:50.963+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:20:51.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:20:51 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:20:51.311+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:20:51.720 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 27 03:20:51 smithi067 podman[106308]: 2024-07-27 03:20:51.257660364 +0000 UTC m=+0.348660526 container init 3c8c5b6ddd0d96e1545dda58b1d6ca7b07a13553f7929b028cfcc717363e58f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-07-27T03:20:51.720 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 27 03:20:51 smithi067 podman[106308]: 2024-07-27 03:20:51.261317254 +0000 UTC m=+0.352317395 container start 3c8c5b6ddd0d96e1545dda58b1d6ca7b07a13553f7929b028cfcc717363e58f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-3-deactivate, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-27T03:20:51.720 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 27 03:20:51 smithi067 podman[106308]: 2024-07-27 03:20:51.399035796 +0000 UTC m=+0.490035949 container attach 3c8c5b6ddd0d96e1545dda58b1d6ca7b07a13553f7929b028cfcc717363e58f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf) 2024-07-27T03:20:51.720 INFO:journalctl@ceph.osd.3.smithi067.stdout:Jul 27 03:20:51 smithi067 podman[106308]: 2024-07-27 03:20:51.464476979 +0000 UTC m=+0.555477119 container died 3c8c5b6ddd0d96e1545dda58b1d6ca7b07a13553f7929b028cfcc717363e58f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-07-27T03:20:51.874 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:20:51 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:20:51.447+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.493385+0000 front 2024-07-27T03:20:23.493461+0000 (oldest deadline 2024-07-27T03:20:48.193306+0000) 2024-07-27T03:20:52.029 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:20:51 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:20:51.899+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:20:52.046 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.3.service' 2024-07-27T03:20:52.088 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:20:52.088 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-07-27T03:20:52.089 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-07-27T03:20:52.089 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.4 2024-07-27T03:20:52.115 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:20:51 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:20:51.816+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:20:52.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:20:51 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:20:51.952+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:20:52.403 INFO:journalctl@ceph.osd.4.smithi149.stdout:Jul 27 03:20:52 smithi149 systemd[1]: Stopping Ceph osd.4 for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:20:52.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:20:52 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:20:52.329+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:20:52.749 INFO:journalctl@ceph.osd.4.smithi149.stdout:Jul 27 03:20:52 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-4[48622]: 2024-07-27T03:20:52.400+0000 7fce8d15f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:20:52.749 INFO:journalctl@ceph.osd.4.smithi149.stdout:Jul 27 03:20:52 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-4[48622]: 2024-07-27T03:20:52.400+0000 7fce8d15f640 -1 osd.4 95 *** Got signal Terminated *** 2024-07-27T03:20:52.749 INFO:journalctl@ceph.osd.4.smithi149.stdout:Jul 27 03:20:52 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-4[48622]: 2024-07-27T03:20:52.400+0000 7fce8d15f640 -1 osd.4 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-27T03:20:52.874 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:20:52 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:20:52.429+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.493385+0000 front 2024-07-27T03:20:23.493461+0000 (oldest deadline 2024-07-27T03:20:48.193306+0000) 2024-07-27T03:20:53.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:20:52 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:20:52.945+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:20:53.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:20:52 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:20:52.798+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:20:53.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:20:52 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:20:52.944+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:20:53.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:20:53 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:20:53.343+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:20:53.874 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:20:53 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:20:53.415+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.493385+0000 front 2024-07-27T03:20:23.493461+0000 (oldest deadline 2024-07-27T03:20:48.193306+0000) 2024-07-27T03:20:54.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:20:53 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:20:53.966+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:20:54.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:20:53 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:20:53.779+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:20:54.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:20:53 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:20:53.915+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:20:54.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:20:54 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:20:54.295+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:20:54.874 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:20:54 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:20:54.461+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.493385+0000 front 2024-07-27T03:20:23.493461+0000 (oldest deadline 2024-07-27T03:20:48.193306+0000) 2024-07-27T03:20:55.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:20:54 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:20:54.961+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:20:55.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:20:54 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:20:54.817+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:20:55.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:20:54 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:20:54.900+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:20:55.374 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:20:54 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:20:54.993+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:20:55.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:20:55 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:20:55.283+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:20:55.874 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:20:55 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:20:55.447+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.493385+0000 front 2024-07-27T03:20:23.493461+0000 (oldest deadline 2024-07-27T03:20:48.193306+0000) 2024-07-27T03:20:55.999 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:20:55 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:20:55.842+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:20:56.000 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:20:55 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:20:55.626+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:20:56.306 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:20:55 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:20:55.990+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:20:56.374 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:20:55 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:20:55.989+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:20:56.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:20:55 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:20:55.910+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:20:56.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:20:55 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:20:55.910+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:20:56.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:20:56 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:20:56.304+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:20:56.874 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:20:56 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:20:56.454+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.493385+0000 front 2024-07-27T03:20:23.493461+0000 (oldest deadline 2024-07-27T03:20:48.193306+0000) 2024-07-27T03:20:56.874 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:20:56 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:20:56.454+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.595118+0000 front 2024-07-27T03:20:31.595105+0000 (oldest deadline 2024-07-27T03:20:56.294979+0000) 2024-07-27T03:20:56.999 INFO:journalctl@ceph.osd.4.smithi149.stdout:Jul 27 03:20:56 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-4[48622]: 2024-07-27T03:20:56.635+0000 7fce85742640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:30.216961+0000 front 2024-07-27T03:20:30.216995+0000 (oldest deadline 2024-07-27T03:20:56.117125+0000) 2024-07-27T03:20:57.000 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:20:56 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:20:56.580+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:20:57.000 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:20:56 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:20:56.806+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:20:57.315 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:20:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:20:57.023+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:20:57.315 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:20:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:20:57.023+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:20:57.374 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:20:56 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:20:56.988+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:20:57.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:20:56 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:20:56.886+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:20:57.375 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:20:56 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:20:56.886+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:20:57.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:20:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:20:57.312+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:20:57.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:20:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:20:57.312+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.158947+0000 front 2024-07-27T03:20:31.158989+0000 (oldest deadline 2024-07-27T03:20:57.059004+0000) 2024-07-27T03:20:57.874 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:20:57 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:20:57.471+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.493385+0000 front 2024-07-27T03:20:23.493461+0000 (oldest deadline 2024-07-27T03:20:48.193306+0000) 2024-07-27T03:20:57.874 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:20:57 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:20:57.471+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.595118+0000 front 2024-07-27T03:20:31.595105+0000 (oldest deadline 2024-07-27T03:20:56.294979+0000) 2024-07-27T03:20:58.000 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:20:57 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:20:57.605+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:20:58.000 INFO:journalctl@ceph.osd.4.smithi149.stdout:Jul 27 03:20:57 smithi149 podman[71022]: 2024-07-27 03:20:57.82526417 +0000 UTC m=+5.539911425 container died 0fc36efd99f8d8bc7eaae9399fda4aeadedc8b86c8fc916babbe821be1192a89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, ceph=True, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-27T03:20:58.000 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:20:57 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:20:57.847+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:20:58.000 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:20:57 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:20:57.847+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:20:58.292 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:20:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:20:57.989+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:20:58.292 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:20:57 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:20:57.989+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:20:58.374 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:20:57 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:20:57.940+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:20:58.375 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:20:57 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:20:57.931+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:20:58.375 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:20:57 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:20:57.931+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:20:58.463 INFO:journalctl@ceph.osd.4.smithi149.stdout:Jul 27 03:20:58 smithi149 podman[71022]: 2024-07-27 03:20:58.226707304 +0000 UTC m=+5.941354569 container cleanup 0fc36efd99f8d8bc7eaae9399fda4aeadedc8b86c8fc916babbe821be1192a89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-27T03:20:58.463 INFO:journalctl@ceph.osd.4.smithi149.stdout:Jul 27 03:20:58 smithi149 bash[71022]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-4 2024-07-27T03:20:58.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:20:58 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:20:58.289+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:20:58.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:20:58 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:20:58.289+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.158947+0000 front 2024-07-27T03:20:31.158989+0000 (oldest deadline 2024-07-27T03:20:57.059004+0000) 2024-07-27T03:20:58.750 INFO:journalctl@ceph.osd.4.smithi149.stdout:Jul 27 03:20:58 smithi149 podman[71042]: 2024-07-27 03:20:58.4614303 +0000 UTC m=+0.635561132 container remove 0fc36efd99f8d8bc7eaae9399fda4aeadedc8b86c8fc916babbe821be1192a89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-07-27T03:20:58.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:20:58 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:20:58.572+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:20:58.874 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:20:58 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:20:58.483+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.493385+0000 front 2024-07-27T03:20:23.493461+0000 (oldest deadline 2024-07-27T03:20:48.193306+0000) 2024-07-27T03:20:58.874 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:20:58 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:20:58.483+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.595118+0000 front 2024-07-27T03:20:31.595105+0000 (oldest deadline 2024-07-27T03:20:56.294979+0000) 2024-07-27T03:20:59.083 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:20:58 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:20:58.880+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:20:59.083 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:20:58 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:20:58.880+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:20:59.083 INFO:journalctl@ceph.osd.4.smithi149.stdout:Jul 27 03:20:58 smithi149 podman[71134]: 2024-07-27 03:20:58.701124863 +0000 UTC m=+0.017336506 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c 2024-07-27T03:20:59.083 INFO:journalctl@ceph.osd.4.smithi149.stdout:Jul 27 03:20:58 smithi149 podman[71134]: 2024-07-27 03:20:58.837787081 +0000 UTC m=+0.153998748 container create 45195d5b8b9dc925c74df6547bcf3db4a82bf9d62a7639aab115f7a619a5c224 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-27T03:20:59.083 INFO:journalctl@ceph.osd.4.smithi149.stdout:Jul 27 03:20:59 smithi149 podman[71134]: 2024-07-27 03:20:59.08173529 +0000 UTC m=+0.397946931 container init 45195d5b8b9dc925c74df6547bcf3db4a82bf9d62a7639aab115f7a619a5c224 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-07-27T03:20:59.318 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:20:58 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:20:58.975+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:20:59.318 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:20:58 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:20:58.975+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:20:59.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:20:58 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:20:58.883+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:20:59.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:20:58 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:20:58.883+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:20:59.374 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:20:58 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:20:58.900+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:20:59.500 INFO:journalctl@ceph.osd.4.smithi149.stdout:Jul 27 03:20:59 smithi149 podman[71134]: 2024-07-27 03:20:59.085076892 +0000 UTC m=+0.401288519 container start 45195d5b8b9dc925c74df6547bcf3db4a82bf9d62a7639aab115f7a619a5c224 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0) 2024-07-27T03:20:59.500 INFO:journalctl@ceph.osd.4.smithi149.stdout:Jul 27 03:20:59 smithi149 podman[71134]: 2024-07-27 03:20:59.173714756 +0000 UTC m=+0.489926382 container attach 45195d5b8b9dc925c74df6547bcf3db4a82bf9d62a7639aab115f7a619a5c224 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724) 2024-07-27T03:20:59.500 INFO:journalctl@ceph.osd.4.smithi149.stdout:Jul 27 03:20:59 smithi149 podman[71134]: 2024-07-27 03:20:59.269666337 +0000 UTC m=+0.585877970 container died 45195d5b8b9dc925c74df6547bcf3db4a82bf9d62a7639aab115f7a619a5c224 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240724, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-07-27T03:20:59.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:20:59 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:20:59.315+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:20:59.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:20:59 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:20:59.315+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.158947+0000 front 2024-07-27T03:20:31.158989+0000 (oldest deadline 2024-07-27T03:20:57.059004+0000) 2024-07-27T03:20:59.860 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:20:59 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:20:59.604+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:20:59.870 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:20:59 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:20:59.472+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.493385+0000 front 2024-07-27T03:20:23.493461+0000 (oldest deadline 2024-07-27T03:20:48.193306+0000) 2024-07-27T03:20:59.870 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:20:59 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:20:59.472+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.595118+0000 front 2024-07-27T03:20:31.595105+0000 (oldest deadline 2024-07-27T03:20:56.294979+0000) 2024-07-27T03:20:59.936 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.4.service' 2024-07-27T03:21:00.016 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:21:00.016 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-07-27T03:21:00.017 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-07-27T03:21:00.017 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.5 2024-07-27T03:21:00.124 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:00 smithi175 systemd[1]: Stopping Ceph osd.5 for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:21:00.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:20:59 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:20:59.867+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:00.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:20:59 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:20:59.868+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:00.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:20:59 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:20:59.868+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:00.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:20:59 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:20:59.959+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:00.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:20:59 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:20:59.959+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:00.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:20:59 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:20:59.858+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:00.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:20:59 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:20:59.858+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:00.624 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:00 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:21:00.332+0000 7fcb811e5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:21:00.624 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:00 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:21:00.332+0000 7fcb811e5640 -1 osd.5 95 *** Got signal Terminated *** 2024-07-27T03:21:00.624 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:00 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:21:00.332+0000 7fcb811e5640 -1 osd.5 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-27T03:21:00.624 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:00 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:21:00.453+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.493385+0000 front 2024-07-27T03:20:23.493461+0000 (oldest deadline 2024-07-27T03:20:48.193306+0000) 2024-07-27T03:21:00.624 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:00 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:21:00.453+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.595118+0000 front 2024-07-27T03:20:31.595105+0000 (oldest deadline 2024-07-27T03:20:56.294979+0000) 2024-07-27T03:21:00.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:00 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:00.354+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:21:00.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:00 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:00.354+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.158947+0000 front 2024-07-27T03:20:31.158989+0000 (oldest deadline 2024-07-27T03:20:57.059004+0000) 2024-07-27T03:21:00.861 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:00 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:00.575+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:21:01.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:00 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:00.840+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:01.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:00 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:00.820+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:01.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:00 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:00.820+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:01.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:00 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:00.922+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:01.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:00 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:00.922+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:01.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:00 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:00.859+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:01.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:00 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:00.859+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:01.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:01 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:01.370+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:21:01.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:01 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:01.370+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.158947+0000 front 2024-07-27T03:20:31.158989+0000 (oldest deadline 2024-07-27T03:20:57.059004+0000) 2024-07-27T03:21:01.790 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:01 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:21:01.501+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.493385+0000 front 2024-07-27T03:20:23.493461+0000 (oldest deadline 2024-07-27T03:20:48.193306+0000) 2024-07-27T03:21:01.791 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:01 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:21:01.501+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.595118+0000 front 2024-07-27T03:20:31.595105+0000 (oldest deadline 2024-07-27T03:20:56.294979+0000) 2024-07-27T03:21:01.881 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:01 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:01.527+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:21:02.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:01 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:01.854+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:02.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:01 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:01.788+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:02.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:01 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:01.788+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:02.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:01 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:01.930+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:02.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:01 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:01.930+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:02.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:01 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:01.879+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:02.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:01 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:01.879+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:02.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:02 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:02.361+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:21:02.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:02 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:02.361+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.158947+0000 front 2024-07-27T03:20:31.158989+0000 (oldest deadline 2024-07-27T03:20:57.059004+0000) 2024-07-27T03:21:02.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:02 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:02.484+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:21:02.808 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:02 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:21:02.522+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.493385+0000 front 2024-07-27T03:20:23.493461+0000 (oldest deadline 2024-07-27T03:20:48.193306+0000) 2024-07-27T03:21:02.809 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:02 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:21:02.522+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.595118+0000 front 2024-07-27T03:20:31.595105+0000 (oldest deadline 2024-07-27T03:20:56.294979+0000) 2024-07-27T03:21:02.809 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:02 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:02.761+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:02.809 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:02 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:02.761+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:02.809 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:02 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:02.761+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:03.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:02 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:02.806+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:03.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:02 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:02.917+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:03.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:02 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:02.917+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:03.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:02 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:02.846+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:03.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:02 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:02.846+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:03.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:02 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:02.846+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:03.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:03 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:03.330+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:21:03.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:03 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:03.330+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.158947+0000 front 2024-07-27T03:20:31.158989+0000 (oldest deadline 2024-07-27T03:20:57.059004+0000) 2024-07-27T03:21:03.749 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:03 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:21:03.482+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.493385+0000 front 2024-07-27T03:20:23.493461+0000 (oldest deadline 2024-07-27T03:20:48.193306+0000) 2024-07-27T03:21:03.749 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:03 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:21:03.482+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.595118+0000 front 2024-07-27T03:20:31.595105+0000 (oldest deadline 2024-07-27T03:20:56.294979+0000) 2024-07-27T03:21:03.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:03 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:03.437+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:21:04.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:03 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:03.765+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:04.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:03 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:03.747+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:04.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:03 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:03.747+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:04.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:03 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:03.747+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:04.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:03 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:03.952+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:04.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:03 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:03.952+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:04.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:03 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:03.846+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:04.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:03 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:03.846+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:04.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:03 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:03.846+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:04.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:04 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:04.371+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:21:04.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:04 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:04.371+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.158947+0000 front 2024-07-27T03:20:31.158989+0000 (oldest deadline 2024-07-27T03:20:57.059004+0000) 2024-07-27T03:21:04.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:04 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:04.439+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:21:04.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:04 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:04.439+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:40.885574+0000 front 2024-07-27T03:20:40.885405+0000 (oldest deadline 2024-07-27T03:21:04.385408+0000) 2024-07-27T03:21:04.769 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:04 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:21:04.468+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.493385+0000 front 2024-07-27T03:20:23.493461+0000 (oldest deadline 2024-07-27T03:20:48.193306+0000) 2024-07-27T03:21:04.769 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:04 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5[49156]: 2024-07-27T03:21:04.468+0000 7fcb7d7d0640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.595118+0000 front 2024-07-27T03:20:31.595105+0000 (oldest deadline 2024-07-27T03:20:56.294979+0000) 2024-07-27T03:21:05.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:04 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:04.767+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:05.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:04 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:04.790+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:05.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:04 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:04.790+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:05.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:04 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:04.790+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:05.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:04 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:04.856+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:05.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:04 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:04.856+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:05.251 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:04 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:04.856+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:05.404 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:04 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:04.980+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:05.405 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:04 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:04.980+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:05.405 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:04 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:04.980+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:05.520 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:05 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:05.452+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:21:05.520 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:05 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:05.452+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:40.885574+0000 front 2024-07-27T03:20:40.885405+0000 (oldest deadline 2024-07-27T03:21:04.385408+0000) 2024-07-27T03:21:05.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:05 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:05.402+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:21:05.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:05 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:05.402+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.158947+0000 front 2024-07-27T03:20:31.158989+0000 (oldest deadline 2024-07-27T03:20:57.059004+0000) 2024-07-27T03:21:05.985 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:05 smithi175 podman[71622]: 2024-07-27 03:21:05.70328385 +0000 UTC m=+5.493788655 container died 9ad61b1b22df1f5c9db8a5917d4b02a6b65f2949800637a2c6f9e3eff7689f6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0) 2024-07-27T03:21:05.985 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:05 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:05.810+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:05.985 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:05 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:05.749+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:05.985 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:05 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:05.749+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:05.985 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:05 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:05.749+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:06.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:05 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:05.940+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:06.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:05 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:05.940+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:06.220 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:05 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:05.940+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:06.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:05 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:05.889+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:06.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:05 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:05.890+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:06.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:05 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:05.890+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:06.260 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:06 smithi175 podman[71622]: 2024-07-27 03:21:06.074003264 +0000 UTC m=+5.864508064 container cleanup 9ad61b1b22df1f5c9db8a5917d4b02a6b65f2949800637a2c6f9e3eff7689f6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=) 2024-07-27T03:21:06.260 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:06 smithi175 bash[71622]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5 2024-07-27T03:21:06.624 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:06 smithi175 podman[71639]: 2024-07-27 03:21:06.258471453 +0000 UTC m=+0.551885852 container remove 9ad61b1b22df1f5c9db8a5917d4b02a6b65f2949800637a2c6f9e3eff7689f6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=) 2024-07-27T03:21:06.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:06 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:06.434+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:21:06.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:06 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:06.434+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.158947+0000 front 2024-07-27T03:20:31.158989+0000 (oldest deadline 2024-07-27T03:20:57.059004+0000) 2024-07-27T03:21:06.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:06 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:06.498+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:21:06.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:06 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:06.498+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:40.885574+0000 front 2024-07-27T03:20:40.885405+0000 (oldest deadline 2024-07-27T03:21:04.385408+0000) 2024-07-27T03:21:06.980 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:06 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:06.768+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:06.981 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:06 smithi175 podman[71731]: 2024-07-27 03:21:06.595826849 +0000 UTC m=+0.017299146 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c 2024-07-27T03:21:06.981 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:06 smithi175 podman[71731]: 2024-07-27 03:21:06.718019238 +0000 UTC m=+0.139491533 container create 7a87fffd50721a18da51ad3bf7d40ea96a54446338bfca8cde03f16c3a30a2bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-27T03:21:06.981 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:06 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:06.734+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:06.981 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:06 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:06.734+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:06.982 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:06 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:06.734+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:07.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:06 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:06.896+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:07.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:06 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:06.896+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:07.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:06 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:06.896+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:07.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:06 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:06.931+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:07.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:06 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:06.931+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:07.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:06 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:06.931+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:07.374 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:06 smithi175 podman[71731]: 2024-07-27 03:21:06.978776129 +0000 UTC m=+0.400248436 container init 7a87fffd50721a18da51ad3bf7d40ea96a54446338bfca8cde03f16c3a30a2bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf) 2024-07-27T03:21:07.374 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:06 smithi175 podman[71731]: 2024-07-27 03:21:06.981892065 +0000 UTC m=+0.403364368 container start 7a87fffd50721a18da51ad3bf7d40ea96a54446338bfca8cde03f16c3a30a2bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-07-27T03:21:07.374 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:07 smithi175 podman[71731]: 2024-07-27 03:21:07.079163436 +0000 UTC m=+0.500635728 container attach 7a87fffd50721a18da51ad3bf7d40ea96a54446338bfca8cde03f16c3a30a2bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-27T03:21:07.375 INFO:journalctl@ceph.osd.5.smithi175.stdout:Jul 27 03:21:07 smithi175 podman[71731]: 2024-07-27 03:21:07.171057364 +0000 UTC m=+0.592529657 container died 7a87fffd50721a18da51ad3bf7d40ea96a54446338bfca8cde03f16c3a30a2bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf) 2024-07-27T03:21:07.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:07 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:07.403+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:21:07.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:07 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:07.403+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.158947+0000 front 2024-07-27T03:20:31.158989+0000 (oldest deadline 2024-07-27T03:20:57.059004+0000) 2024-07-27T03:21:07.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:07 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:07.459+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:21:07.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:07 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:07.459+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:40.885574+0000 front 2024-07-27T03:20:40.885405+0000 (oldest deadline 2024-07-27T03:21:04.385408+0000) 2024-07-27T03:21:07.793 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.5.service' 2024-07-27T03:21:07.832 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:21:07.832 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-07-27T03:21:07.832 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-07-27T03:21:07.832 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.6 2024-07-27T03:21:08.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:07 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:07.810+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:08.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:07 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:07.811+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:36.377872+0000 front 2024-07-27T03:20:36.378048+0000 (oldest deadline 2024-07-27T03:21:07.578572+0000) 2024-07-27T03:21:08.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:07 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:07.761+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:08.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:07 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:07.761+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:08.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:07 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:07.761+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:08.148 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:07 smithi067 systemd[1]: Stopping Ceph osd.6 for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:21:08.148 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:07 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:07.851+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:08.149 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:07 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:07.851+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:08.149 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:07 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:07.851+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:08.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:07 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:07.912+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:08.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:07 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:07.912+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:08.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:07 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:07.912+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:08.431 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:08 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:08.146+0000 7fb1a86f3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:21:08.431 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:08 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:08.146+0000 7fb1a86f3640 -1 osd.6 95 *** Got signal Terminated *** 2024-07-27T03:21:08.431 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:08 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:08.146+0000 7fb1a86f3640 -1 osd.6 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-27T03:21:08.431 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:08 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:08.428+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:21:08.431 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:08 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:08.428+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.158947+0000 front 2024-07-27T03:20:31.158989+0000 (oldest deadline 2024-07-27T03:20:57.059004+0000) 2024-07-27T03:21:08.431 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:08 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:08.428+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.059424+0000 front 2024-07-27T03:20:37.059783+0000 (oldest deadline 2024-07-27T03:21:07.660289+0000) 2024-07-27T03:21:08.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:08 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:08.417+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:21:08.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:08 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:08.417+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:40.885574+0000 front 2024-07-27T03:20:40.885405+0000 (oldest deadline 2024-07-27T03:21:04.385408+0000) 2024-07-27T03:21:09.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:08 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:08.844+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:09.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:08 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:08.844+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:36.377872+0000 front 2024-07-27T03:20:36.378048+0000 (oldest deadline 2024-07-27T03:21:07.578572+0000) 2024-07-27T03:21:09.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:08 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:08.791+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:09.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:08 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:08.791+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:09.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:08 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:08.791+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:09.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:08 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:08.812+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:09.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:08 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:08.812+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:09.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:08 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:08.812+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:09.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:08 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:08.914+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:09.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:08 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:08.914+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:09.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:08 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:08.914+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:09.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:09 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:09.409+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:21:09.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:09 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:09.409+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.158947+0000 front 2024-07-27T03:20:31.158989+0000 (oldest deadline 2024-07-27T03:20:57.059004+0000) 2024-07-27T03:21:09.720 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:09 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:09.409+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.059424+0000 front 2024-07-27T03:20:37.059783+0000 (oldest deadline 2024-07-27T03:21:07.660289+0000) 2024-07-27T03:21:09.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:09 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:09.458+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:21:09.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:09 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:09.458+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:40.885574+0000 front 2024-07-27T03:20:40.885405+0000 (oldest deadline 2024-07-27T03:21:04.385408+0000) 2024-07-27T03:21:10.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:09 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:09.808+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:10.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:09 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:09.808+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:36.377872+0000 front 2024-07-27T03:20:36.378048+0000 (oldest deadline 2024-07-27T03:21:07.578572+0000) 2024-07-27T03:21:10.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:09 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:09.807+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:10.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:09 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:09.807+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:10.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:09 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:09.807+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:10.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:09 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:09.842+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:10.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:09 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:09.842+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:10.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:09 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:09.842+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:10.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:09 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:09.911+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:10.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:09 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:09.911+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:10.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:09 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:09.911+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:10.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:10 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:10.387+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:21:10.720 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:10 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:10.387+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.158947+0000 front 2024-07-27T03:20:31.158989+0000 (oldest deadline 2024-07-27T03:20:57.059004+0000) 2024-07-27T03:21:10.720 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:10 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:10.387+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.059424+0000 front 2024-07-27T03:20:37.059783+0000 (oldest deadline 2024-07-27T03:21:07.660289+0000) 2024-07-27T03:21:10.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:10 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:10.488+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:21:10.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:10 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:10.488+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:40.885574+0000 front 2024-07-27T03:20:40.885405+0000 (oldest deadline 2024-07-27T03:21:04.385408+0000) 2024-07-27T03:21:11.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:10 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:10.774+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:11.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:10 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:10.774+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:36.377872+0000 front 2024-07-27T03:20:36.378048+0000 (oldest deadline 2024-07-27T03:21:07.578572+0000) 2024-07-27T03:21:11.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:10 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:10.774+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.579090+0000 front 2024-07-27T03:20:47.579104+0000 (oldest deadline 2024-07-27T03:21:10.479029+0000) 2024-07-27T03:21:11.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:10 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:10.833+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:11.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:10 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:10.833+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:11.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:10 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:10.833+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:11.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:10 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:10.865+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:11.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:10 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:10.865+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:11.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:10 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:10.865+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:11.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:10 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:10.865+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:11.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:10 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:10.914+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:11.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:10 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:10.914+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:11.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:10 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:10.914+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:11.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:11 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:11.400+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:21:11.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:11 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:11.400+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.158947+0000 front 2024-07-27T03:20:31.158989+0000 (oldest deadline 2024-07-27T03:20:57.059004+0000) 2024-07-27T03:21:11.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:11 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:11.400+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.059424+0000 front 2024-07-27T03:20:37.059783+0000 (oldest deadline 2024-07-27T03:21:07.660289+0000) 2024-07-27T03:21:11.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:11 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:11.400+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.660929+0000 front 2024-07-27T03:20:47.660765+0000 (oldest deadline 2024-07-27T03:21:10.560932+0000) 2024-07-27T03:21:11.915 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:11 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:11.505+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:21:11.915 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:11 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:11.505+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:40.885574+0000 front 2024-07-27T03:20:40.885405+0000 (oldest deadline 2024-07-27T03:21:04.385408+0000) 2024-07-27T03:21:11.915 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:11 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:11.505+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.686495+0000 front 2024-07-27T03:20:46.686568+0000 (oldest deadline 2024-07-27T03:21:10.687168+0000) 2024-07-27T03:21:12.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:11 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:11.769+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:12.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:11 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:11.769+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:36.377872+0000 front 2024-07-27T03:20:36.378048+0000 (oldest deadline 2024-07-27T03:21:07.578572+0000) 2024-07-27T03:21:12.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:11 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:11.769+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.579090+0000 front 2024-07-27T03:20:47.579104+0000 (oldest deadline 2024-07-27T03:21:10.479029+0000) 2024-07-27T03:21:12.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:11 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:11.871+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:12.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:11 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:11.871+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:12.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:11 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:11.871+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:12.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:11 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:11.836+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:12.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:11 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:11.836+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:12.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:11 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:11.836+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:12.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:11 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:11.836+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:12.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:11 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:11.913+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:12.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:11 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:11.913+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:12.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:11 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:11.913+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:12.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:12 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:12.415+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.557764+0000 front 2024-07-27T03:20:23.557852+0000 (oldest deadline 2024-07-27T03:20:49.457943+0000) 2024-07-27T03:21:12.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:12 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:12.415+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.158947+0000 front 2024-07-27T03:20:31.158989+0000 (oldest deadline 2024-07-27T03:20:57.059004+0000) 2024-07-27T03:21:12.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:12 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:12.415+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.059424+0000 front 2024-07-27T03:20:37.059783+0000 (oldest deadline 2024-07-27T03:21:07.660289+0000) 2024-07-27T03:21:12.719 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:12 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6[64302]: 2024-07-27T03:21:12.415+0000 7fb1a4cde640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.660929+0000 front 2024-07-27T03:20:47.660765+0000 (oldest deadline 2024-07-27T03:21:10.560932+0000) 2024-07-27T03:21:12.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:12 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:12.482+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:21:12.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:12 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:12.482+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:40.885574+0000 front 2024-07-27T03:20:40.885405+0000 (oldest deadline 2024-07-27T03:21:04.385408+0000) 2024-07-27T03:21:12.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:12 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:12.482+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.686495+0000 front 2024-07-27T03:20:46.686568+0000 (oldest deadline 2024-07-27T03:21:10.687168+0000) 2024-07-27T03:21:13.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:12 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:12.816+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:13.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:12 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:12.816+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:36.377872+0000 front 2024-07-27T03:20:36.378048+0000 (oldest deadline 2024-07-27T03:21:07.578572+0000) 2024-07-27T03:21:13.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:12 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:12.816+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.579090+0000 front 2024-07-27T03:20:47.579104+0000 (oldest deadline 2024-07-27T03:21:10.479029+0000) 2024-07-27T03:21:13.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:12 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:12.832+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:13.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:12 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:12.832+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:13.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:12 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:12.832+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:13.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:12 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:12.864+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:13.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:12 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:12.864+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:13.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:12 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:12.864+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:13.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:12 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:12.864+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:13.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:12 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:12.897+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:13.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:12 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:12.897+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:13.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:12 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:12.897+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:13.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:12 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:12.897+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:13.707 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:13 smithi067 podman[106411]: 2024-07-27 03:21:13.454149566 +0000 UTC m=+5.418145701 container died 7def2980f6bfd523014c77caeb011baa03462c58eb0b37b31597620ceb0c1f8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240724, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf) 2024-07-27T03:21:13.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:13 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:13.495+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:21:13.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:13 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:13.495+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:40.885574+0000 front 2024-07-27T03:20:40.885405+0000 (oldest deadline 2024-07-27T03:21:04.385408+0000) 2024-07-27T03:21:13.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:13 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:13.495+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.686495+0000 front 2024-07-27T03:20:46.686568+0000 (oldest deadline 2024-07-27T03:21:10.687168+0000) 2024-07-27T03:21:13.969 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:13 smithi067 podman[106411]: 2024-07-27 03:21:13.872003954 +0000 UTC m=+5.836000079 container cleanup 7def2980f6bfd523014c77caeb011baa03462c58eb0b37b31597620ceb0c1f8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240724) 2024-07-27T03:21:13.969 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:13 smithi067 bash[106411]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6 2024-07-27T03:21:13.970 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:13 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:13.820+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:13.970 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:13 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:13.820+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:13.970 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:13 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:13.820+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:13.970 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:13 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:13.820+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:14.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:13 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:13.863+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:14.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:13 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:13.863+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:36.377872+0000 front 2024-07-27T03:20:36.378048+0000 (oldest deadline 2024-07-27T03:21:07.578572+0000) 2024-07-27T03:21:14.125 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:13 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:13.863+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.579090+0000 front 2024-07-27T03:20:47.579104+0000 (oldest deadline 2024-07-27T03:21:10.479029+0000) 2024-07-27T03:21:14.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:13 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:13.856+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:14.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:13 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:13.856+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:14.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:13 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:13.856+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:14.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:13 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:13.867+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:14.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:13 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:13.867+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:14.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:13 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:13.867+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:14.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:13 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:13.867+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:14.469 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:14 smithi067 podman[106429]: 2024-07-27 03:21:14.090129364 +0000 UTC m=+0.633792096 container remove 7def2980f6bfd523014c77caeb011baa03462c58eb0b37b31597620ceb0c1f8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-27T03:21:14.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:14 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:14.466+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:21:14.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:14 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:14.466+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:40.885574+0000 front 2024-07-27T03:20:40.885405+0000 (oldest deadline 2024-07-27T03:21:04.385408+0000) 2024-07-27T03:21:14.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:14 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:14.466+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.686495+0000 front 2024-07-27T03:20:46.686568+0000 (oldest deadline 2024-07-27T03:21:10.687168+0000) 2024-07-27T03:21:14.814 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:14 smithi067 podman[106515]: 2024-07-27 03:21:14.410410107 +0000 UTC m=+0.018068966 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c 2024-07-27T03:21:14.814 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:14 smithi067 podman[106515]: 2024-07-27 03:21:14.516573891 +0000 UTC m=+0.124232785 container create 7e459cf7573d1af7b94328d07f5a7b585a995db851b10a8febc574fc28f76d3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-07-27T03:21:14.814 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:14 smithi067 podman[106515]: 2024-07-27 03:21:14.735411215 +0000 UTC m=+0.343070065 container init 7e459cf7573d1af7b94328d07f5a7b585a995db851b10a8febc574fc28f76d3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-07-27T03:21:14.814 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:14 smithi067 podman[106515]: 2024-07-27 03:21:14.738873425 +0000 UTC m=+0.346532274 container start 7e459cf7573d1af7b94328d07f5a7b585a995db851b10a8febc574fc28f76d3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.license=GPLv2) 2024-07-27T03:21:14.814 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:14 smithi067 podman[106515]: 2024-07-27 03:21:14.810550511 +0000 UTC m=+0.418209373 container attach 7e459cf7573d1af7b94328d07f5a7b585a995db851b10a8febc574fc28f76d3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.36.0) 2024-07-27T03:21:15.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:14 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:14.835+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:15.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:14 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:14.835+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:36.377872+0000 front 2024-07-27T03:20:36.378048+0000 (oldest deadline 2024-07-27T03:21:07.578572+0000) 2024-07-27T03:21:15.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:14 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:14.835+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.579090+0000 front 2024-07-27T03:20:47.579104+0000 (oldest deadline 2024-07-27T03:21:10.479029+0000) 2024-07-27T03:21:15.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:14 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:14.827+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:15.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:14 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:14.827+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:15.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:14 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:14.827+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:15.215 INFO:journalctl@ceph.osd.6.smithi067.stdout:Jul 27 03:21:14 smithi067 podman[106515]: 2024-07-27 03:21:14.931179671 +0000 UTC m=+0.538838524 container died 7e459cf7573d1af7b94328d07f5a7b585a995db851b10a8febc574fc28f76d3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-07-27T03:21:15.215 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:14 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:14.860+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:15.215 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:14 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:14.860+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:15.215 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:14 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:14.860+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:15.215 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:14 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:14.860+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:15.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:14 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:14.821+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:15.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:14 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:14.821+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:15.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:14 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:14.821+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:15.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:14 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:14.821+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:15.606 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.6.service' 2024-07-27T03:21:15.650 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:21:15.650 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-07-27T03:21:15.650 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-07-27T03:21:15.651 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.7 2024-07-27T03:21:15.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:15 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:15.467+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:21:15.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:15 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:15.467+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:40.885574+0000 front 2024-07-27T03:20:40.885405+0000 (oldest deadline 2024-07-27T03:21:04.385408+0000) 2024-07-27T03:21:15.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:15 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:15.467+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.686495+0000 front 2024-07-27T03:20:46.686568+0000 (oldest deadline 2024-07-27T03:21:10.687168+0000) 2024-07-27T03:21:15.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:15 smithi149 systemd[1]: Stopping Ceph osd.7 for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:21:15.872 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:15 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:15.824+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:15.872 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:15 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:15.824+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:15.872 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:15 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:15.824+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:15.872 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:15 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:15.824+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:16.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:15 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:15.838+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:16.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:15 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:15.838+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:36.377872+0000 front 2024-07-27T03:20:36.378048+0000 (oldest deadline 2024-07-27T03:21:07.578572+0000) 2024-07-27T03:21:16.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:15 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:15.838+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.579090+0000 front 2024-07-27T03:20:47.579104+0000 (oldest deadline 2024-07-27T03:21:10.479029+0000) 2024-07-27T03:21:16.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:15 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:15.842+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:16.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:15 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:15.842+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:16.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:15 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:15.842+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:16.250 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:15 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:15.954+0000 7fe7d00a8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:21:16.250 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:15 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:15.954+0000 7fe7d00a8640 -1 osd.7 95 *** Got signal Terminated *** 2024-07-27T03:21:16.250 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:15 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:15.954+0000 7fe7d00a8640 -1 osd.7 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-27T03:21:16.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:15 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:15.851+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:16.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:15 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:15.851+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:16.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:15 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:15.851+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:16.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:15 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:15.851+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:16.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:16 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:16.485+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:21:16.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:16 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:16.486+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:40.885574+0000 front 2024-07-27T03:20:40.885405+0000 (oldest deadline 2024-07-27T03:21:04.385408+0000) 2024-07-27T03:21:16.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:16 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:16.486+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.686495+0000 front 2024-07-27T03:20:46.686568+0000 (oldest deadline 2024-07-27T03:21:10.687168+0000) 2024-07-27T03:21:17.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:16 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:16.821+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:17.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:16 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:16.821+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:36.377872+0000 front 2024-07-27T03:20:36.378048+0000 (oldest deadline 2024-07-27T03:21:07.578572+0000) 2024-07-27T03:21:17.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:16 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:16.821+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.579090+0000 front 2024-07-27T03:20:47.579104+0000 (oldest deadline 2024-07-27T03:21:10.479029+0000) 2024-07-27T03:21:17.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:16 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:16.805+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:17.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:16 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:16.805+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:17.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:16 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:16.805+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:17.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:16 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:16.797+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:17.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:16 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:16.797+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:17.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:16 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:16.797+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:17.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:16 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:16.797+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:17.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:16 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:16.817+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:17.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:16 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:16.817+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:17.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:16 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:16.817+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:17.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:16 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:16.817+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:17.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:17 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:17.478+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:21:17.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:17 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:17.478+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:40.885574+0000 front 2024-07-27T03:20:40.885405+0000 (oldest deadline 2024-07-27T03:21:04.385408+0000) 2024-07-27T03:21:17.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:17 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:17.478+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.686495+0000 front 2024-07-27T03:20:46.686568+0000 (oldest deadline 2024-07-27T03:21:10.687168+0000) 2024-07-27T03:21:18.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:17 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:17.849+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:18.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:17 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:17.849+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:36.377872+0000 front 2024-07-27T03:20:36.378048+0000 (oldest deadline 2024-07-27T03:21:07.578572+0000) 2024-07-27T03:21:18.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:17 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:17.849+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.579090+0000 front 2024-07-27T03:20:47.579104+0000 (oldest deadline 2024-07-27T03:21:10.479029+0000) 2024-07-27T03:21:18.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:17 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:17.851+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:18.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:17 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:17.851+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:18.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:17 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:17.851+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:18.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:17 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:17.824+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:18.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:17 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:17.824+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:18.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:17 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:17.824+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:18.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:17 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:17.824+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:18.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:17 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:17.773+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:18.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:17 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:17.773+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:18.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:17 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:17.773+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:18.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:17 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:17.773+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:18.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:18 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:18.449+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:21:18.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:18 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:18.449+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:40.885574+0000 front 2024-07-27T03:20:40.885405+0000 (oldest deadline 2024-07-27T03:21:04.385408+0000) 2024-07-27T03:21:18.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:18 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:18.449+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.686495+0000 front 2024-07-27T03:20:46.686568+0000 (oldest deadline 2024-07-27T03:21:10.687168+0000) 2024-07-27T03:21:18.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:18 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:18.449+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:54.187990+0000 front 2024-07-27T03:20:54.187963+0000 (oldest deadline 2024-07-27T03:21:18.288045+0000) 2024-07-27T03:21:19.120 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:18 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:18.864+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:19.121 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:18 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:18.864+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:19.121 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:18 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:18.864+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:19.121 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:18 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:18.864+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:19.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:18 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:18.882+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:19.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:18 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:18.882+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:36.377872+0000 front 2024-07-27T03:20:36.378048+0000 (oldest deadline 2024-07-27T03:21:07.578572+0000) 2024-07-27T03:21:19.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:18 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:18.882+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.579090+0000 front 2024-07-27T03:20:47.579104+0000 (oldest deadline 2024-07-27T03:21:10.479029+0000) 2024-07-27T03:21:19.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:18 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:18.869+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:19.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:18 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:18.869+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:19.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:18 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:18.869+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:19.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:18 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:18.869+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:19.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:18 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:18.792+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:19.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:18 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:18.792+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:19.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:18 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:18.792+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:19.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:18 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:18.792+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:19.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:19 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:19.404+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:21:19.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:19 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:19.404+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:40.885574+0000 front 2024-07-27T03:20:40.885405+0000 (oldest deadline 2024-07-27T03:21:04.385408+0000) 2024-07-27T03:21:19.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:19 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:19.404+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.686495+0000 front 2024-07-27T03:20:46.686568+0000 (oldest deadline 2024-07-27T03:21:10.687168+0000) 2024-07-27T03:21:19.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:19 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:19.404+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:54.187990+0000 front 2024-07-27T03:20:54.187963+0000 (oldest deadline 2024-07-27T03:21:18.288045+0000) 2024-07-27T03:21:20.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:19 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:19.853+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:20.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:19 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:19.853+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:36.377872+0000 front 2024-07-27T03:20:36.378048+0000 (oldest deadline 2024-07-27T03:21:07.578572+0000) 2024-07-27T03:21:20.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:19 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:19.853+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.579090+0000 front 2024-07-27T03:20:47.579104+0000 (oldest deadline 2024-07-27T03:21:10.479029+0000) 2024-07-27T03:21:20.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:19 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:19.911+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:20.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:19 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:19.911+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:20.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:19 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:19.911+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:20.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:19 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:19.911+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:20.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:19 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:19.911+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:20.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:19 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:19.911+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:20.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:19 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:19.911+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:20.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:19 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:19.911+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:20.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:19 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:19.822+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:20.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:19 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:19.822+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:20.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:19 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:19.822+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:20.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:19 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:19.822+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:20.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:20 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:20.365+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.483927+0000 front 2024-07-27T03:20:31.483872+0000 (oldest deadline 2024-07-27T03:20:54.984083+0000) 2024-07-27T03:21:20.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:20 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:20.365+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:40.885574+0000 front 2024-07-27T03:20:40.885405+0000 (oldest deadline 2024-07-27T03:21:04.385408+0000) 2024-07-27T03:21:20.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:20 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:20.365+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.686495+0000 front 2024-07-27T03:20:46.686568+0000 (oldest deadline 2024-07-27T03:21:10.687168+0000) 2024-07-27T03:21:20.750 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:20 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7[55580]: 2024-07-27T03:21:20.365+0000 7fe7cc693640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:54.187990+0000 front 2024-07-27T03:20:54.187963+0000 (oldest deadline 2024-07-27T03:21:18.288045+0000) 2024-07-27T03:21:21.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:20 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:20.891+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:21.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:20 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:20.891+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:36.377872+0000 front 2024-07-27T03:20:36.378048+0000 (oldest deadline 2024-07-27T03:21:07.578572+0000) 2024-07-27T03:21:21.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:20 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:20.891+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.579090+0000 front 2024-07-27T03:20:47.579104+0000 (oldest deadline 2024-07-27T03:21:10.479029+0000) 2024-07-27T03:21:21.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:20 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:20.870+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:21.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:20 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:20.870+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:21.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:20 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:20.870+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:21.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:20 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:20.870+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:21.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:20 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:20.893+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:21.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:20 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:20.893+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:21.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:20 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:20.893+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:21.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:20 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:20.893+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:21.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:20 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:20.840+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:21.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:20 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:20.840+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:21.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:20 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:20.840+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:21.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:20 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:20.840+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:21.749 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:21 smithi149 podman[71250]: 2024-07-27 03:21:21.449186937 +0000 UTC m=+5.608378482 container died e61acfa75d8d109431e25086fcaece809c82672fa1edcee56f4c10c8d1efdb31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-07-27T03:21:22.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:21 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:21.890+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:22.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:21 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:21.890+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:36.377872+0000 front 2024-07-27T03:20:36.378048+0000 (oldest deadline 2024-07-27T03:21:07.578572+0000) 2024-07-27T03:21:22.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:21 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:21.890+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.579090+0000 front 2024-07-27T03:20:47.579104+0000 (oldest deadline 2024-07-27T03:21:10.479029+0000) 2024-07-27T03:21:22.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:21 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:21.890+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.380104+0000 front 2024-07-27T03:20:56.380077+0000 (oldest deadline 2024-07-27T03:21:21.680070+0000) 2024-07-27T03:21:22.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:21 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:21.858+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:22.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:21 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:21.858+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:22.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:21 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:21.858+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:22.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:21 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:21.858+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:22.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:21 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:21.868+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:22.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:21 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:21.868+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:22.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:21 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:21.868+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:22.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:21 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:21.868+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:22.220 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:21 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:21.868+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.876660+0000 front 2024-07-27T03:20:55.876705+0000 (oldest deadline 2024-07-27T03:21:21.776437+0000) 2024-07-27T03:21:22.250 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:21 smithi149 podman[71250]: 2024-07-27 03:21:21.762828762 +0000 UTC m=+5.922020307 container cleanup e61acfa75d8d109431e25086fcaece809c82672fa1edcee56f4c10c8d1efdb31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-07-27T03:21:22.250 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:21 smithi149 bash[71250]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7 2024-07-27T03:21:22.250 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:22 smithi149 podman[71266]: 2024-07-27 03:21:22.005782885 +0000 UTC m=+0.550653174 container remove e61acfa75d8d109431e25086fcaece809c82672fa1edcee56f4c10c8d1efdb31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-07-27T03:21:22.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:21 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:21.870+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:22.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:21 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:21.870+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:22.251 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:21 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:21.870+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:22.251 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:21 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:21.870+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:22.636 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:22 smithi149 podman[71355]: 2024-07-27 03:21:22.235358929 +0000 UTC m=+0.017377910 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c 2024-07-27T03:21:22.636 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:22 smithi149 podman[71355]: 2024-07-27 03:21:22.348391773 +0000 UTC m=+0.130410740 container create 3da6b9157a95f45f2005e008473529b0d693859c294c2789bc2524223fe552fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7-deactivate, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-07-27T03:21:23.000 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:22 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:22.842+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:23.000 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:22 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:22.842+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:23.000 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:22 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:22.842+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:23.000 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:22 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:22.842+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:23.000 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:22 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:22.842+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.097429+0000 front 2024-07-27T03:20:56.097464+0000 (oldest deadline 2024-07-27T03:21:21.997518+0000) 2024-07-27T03:21:23.000 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:22 smithi149 podman[71355]: 2024-07-27 03:21:22.634664362 +0000 UTC m=+0.416683334 container init 3da6b9157a95f45f2005e008473529b0d693859c294c2789bc2524223fe552fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, io.buildah.version=1.36.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-07-27T03:21:23.001 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:22 smithi149 podman[71355]: 2024-07-27 03:21:22.638497138 +0000 UTC m=+0.420516105 container start 3da6b9157a95f45f2005e008473529b0d693859c294c2789bc2524223fe552fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-07-27T03:21:23.001 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:22 smithi149 podman[71355]: 2024-07-27 03:21:22.742949928 +0000 UTC m=+0.524968921 container attach 3da6b9157a95f45f2005e008473529b0d693859c294c2789bc2524223fe552fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-07-27T03:21:23.001 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:22 smithi149 podman[71355]: 2024-07-27 03:21:22.830966377 +0000 UTC m=+0.612985356 container died 3da6b9157a95f45f2005e008473529b0d693859c294c2789bc2524223fe552fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-07-27T03:21:23.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:22 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:22.853+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:23.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:22 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:22.853+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:36.377872+0000 front 2024-07-27T03:20:36.378048+0000 (oldest deadline 2024-07-27T03:21:07.578572+0000) 2024-07-27T03:21:23.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:22 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:22.853+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.579090+0000 front 2024-07-27T03:20:47.579104+0000 (oldest deadline 2024-07-27T03:21:10.479029+0000) 2024-07-27T03:21:23.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:22 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:22.853+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.380104+0000 front 2024-07-27T03:20:56.380077+0000 (oldest deadline 2024-07-27T03:21:21.680070+0000) 2024-07-27T03:21:23.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:22 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:22.855+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:23.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:22 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:22.855+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:23.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:22 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:22.856+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:23.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:22 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:22.856+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:23.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:22 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:22.880+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:23.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:22 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:22.880+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:23.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:22 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:22.880+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:23.220 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:22 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:22.880+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:23.220 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:22 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:22.880+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.876660+0000 front 2024-07-27T03:20:55.876705+0000 (oldest deadline 2024-07-27T03:21:21.776437+0000) 2024-07-27T03:21:23.389 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.7.service' 2024-07-27T03:21:23.391 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:23 smithi149 podman[71378]: 2024-07-27 03:21:23.363150089 +0000 UTC m=+0.526737297 container remove 3da6b9157a95f45f2005e008473529b0d693859c294c2789bc2524223fe552fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2) 2024-07-27T03:21:23.391 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:23 smithi149 systemd[1]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.7.service: Deactivated successfully. 2024-07-27T03:21:23.391 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:23 smithi149 systemd[1]: Stopped Ceph osd.7 for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:21:23.391 INFO:journalctl@ceph.osd.7.smithi149.stdout:Jul 27 03:21:23 smithi149 systemd[1]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.7.service: Consumed 7.704s CPU time. 2024-07-27T03:21:23.428 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:21:23.428 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-07-27T03:21:23.429 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-07-27T03:21:23.429 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.8 2024-07-27T03:21:23.736 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:23 smithi175 systemd[1]: Stopping Ceph osd.8 for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:21:23.736 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:23 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:23.734+0000 7f8c24937640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:21:24.000 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:23 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:23.831+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:24.000 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:23 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:23.831+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:24.000 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:23 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:23.831+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:24.000 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:23 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:23.831+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:24.000 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:23 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:23.831+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.097429+0000 front 2024-07-27T03:20:56.097464+0000 (oldest deadline 2024-07-27T03:21:21.997518+0000) 2024-07-27T03:21:24.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:23 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:23.858+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:24.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:23 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:23.858+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:24.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:23 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:23.858+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:24.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:23 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:23.858+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:24.125 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:23 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:23.734+0000 7f8c24937640 -1 osd.8 95 *** Got signal Terminated *** 2024-07-27T03:21:24.125 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:23 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:23.734+0000 7f8c24937640 -1 osd.8 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-27T03:21:24.125 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:23 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:23.872+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:24.125 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:23 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:23.872+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:36.377872+0000 front 2024-07-27T03:20:36.378048+0000 (oldest deadline 2024-07-27T03:21:07.578572+0000) 2024-07-27T03:21:24.125 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:23 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:23.872+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.579090+0000 front 2024-07-27T03:20:47.579104+0000 (oldest deadline 2024-07-27T03:21:10.479029+0000) 2024-07-27T03:21:24.125 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:23 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:23.872+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.380104+0000 front 2024-07-27T03:20:56.380077+0000 (oldest deadline 2024-07-27T03:21:21.680070+0000) 2024-07-27T03:21:24.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:23 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:23.889+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:24.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:23 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:23.889+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:24.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:23 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:23.889+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:24.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:23 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:23.889+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:24.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:23 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:23.889+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.876660+0000 front 2024-07-27T03:20:55.876705+0000 (oldest deadline 2024-07-27T03:21:21.776437+0000) 2024-07-27T03:21:25.000 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:24 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:24.800+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:25.000 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:24 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:24.800+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:25.000 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:24 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:24.800+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:25.000 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:24 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:24.800+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:25.000 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:24 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:24.800+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.097429+0000 front 2024-07-27T03:20:56.097464+0000 (oldest deadline 2024-07-27T03:21:21.997518+0000) 2024-07-27T03:21:25.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:24 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:24.835+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:25.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:24 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:24.835+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:36.377872+0000 front 2024-07-27T03:20:36.378048+0000 (oldest deadline 2024-07-27T03:21:07.578572+0000) 2024-07-27T03:21:25.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:24 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:24.835+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.579090+0000 front 2024-07-27T03:20:47.579104+0000 (oldest deadline 2024-07-27T03:21:10.479029+0000) 2024-07-27T03:21:25.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:24 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:24.835+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.380104+0000 front 2024-07-27T03:20:56.380077+0000 (oldest deadline 2024-07-27T03:21:21.680070+0000) 2024-07-27T03:21:25.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:24 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:24.898+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:25.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:24 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:24.898+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:25.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:24 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:24.898+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:25.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:24 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:24.898+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:25.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:24 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:24.926+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:25.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:24 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:24.926+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:25.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:24 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:24.926+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:25.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:24 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:24.926+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:25.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:24 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:24.926+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.876660+0000 front 2024-07-27T03:20:55.876705+0000 (oldest deadline 2024-07-27T03:21:21.776437+0000) 2024-07-27T03:21:26.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:25 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:25.880+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:26.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:25 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:25.880+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:36.377872+0000 front 2024-07-27T03:20:36.378048+0000 (oldest deadline 2024-07-27T03:21:07.578572+0000) 2024-07-27T03:21:26.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:25 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:25.880+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.579090+0000 front 2024-07-27T03:20:47.579104+0000 (oldest deadline 2024-07-27T03:21:10.479029+0000) 2024-07-27T03:21:26.125 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:25 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:25.880+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.380104+0000 front 2024-07-27T03:20:56.380077+0000 (oldest deadline 2024-07-27T03:21:21.680070+0000) 2024-07-27T03:21:26.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:25 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:25.869+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:26.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:25 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:25.869+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:26.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:25 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:25.869+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:26.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:25 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:25.869+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:26.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:25 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:25.869+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:26.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:25 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:25.954+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:26.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:25 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:25.954+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:26.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:25 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:25.954+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:26.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:25 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:25.954+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:26.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:25 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:25.954+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.876660+0000 front 2024-07-27T03:20:55.876705+0000 (oldest deadline 2024-07-27T03:21:21.776437+0000) 2024-07-27T03:21:26.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:25 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:25.799+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:26.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:25 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:25.799+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:26.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:25 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:25.799+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:26.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:25 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:25.799+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:26.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:25 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:25.799+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.097429+0000 front 2024-07-27T03:20:56.097464+0000 (oldest deadline 2024-07-27T03:21:21.997518+0000) 2024-07-27T03:21:27.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:26 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:26.820+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:27.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:26 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:26.820+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:27.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:26 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:26.820+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:27.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:26 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:26.820+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:27.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:26 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:26.820+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:27.125 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:26 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:26.918+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:27.125 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:26 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:26.918+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:36.377872+0000 front 2024-07-27T03:20:36.378048+0000 (oldest deadline 2024-07-27T03:21:07.578572+0000) 2024-07-27T03:21:27.125 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:26 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:26.918+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.579090+0000 front 2024-07-27T03:20:47.579104+0000 (oldest deadline 2024-07-27T03:21:10.479029+0000) 2024-07-27T03:21:27.125 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:26 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:26.918+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.380104+0000 front 2024-07-27T03:20:56.380077+0000 (oldest deadline 2024-07-27T03:21:21.680070+0000) 2024-07-27T03:21:27.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:26 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:26.916+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:27.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:26 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:26.916+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:27.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:26 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:26.916+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:27.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:26 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:26.916+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:27.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:26 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:26.916+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.876660+0000 front 2024-07-27T03:20:55.876705+0000 (oldest deadline 2024-07-27T03:21:21.776437+0000) 2024-07-27T03:21:27.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:26 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:26.832+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:27.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:26 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:26.832+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:27.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:26 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:26.832+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:27.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:26 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:26.832+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:27.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:26 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:26.832+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.097429+0000 front 2024-07-27T03:20:56.097464+0000 (oldest deadline 2024-07-27T03:21:21.997518+0000) 2024-07-27T03:21:27.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:26 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:26.832+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:01.998230+0000 front 2024-07-27T03:21:01.998260+0000 (oldest deadline 2024-07-27T03:21:26.598644+0000) 2024-07-27T03:21:28.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:27 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:27.877+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.177161+0000 front 2024-07-27T03:20:31.177086+0000 (oldest deadline 2024-07-27T03:20:54.677075+0000) 2024-07-27T03:21:28.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:27 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:27.877+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:36.377872+0000 front 2024-07-27T03:20:36.378048+0000 (oldest deadline 2024-07-27T03:21:07.578572+0000) 2024-07-27T03:21:28.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:27 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:27.877+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.579090+0000 front 2024-07-27T03:20:47.579104+0000 (oldest deadline 2024-07-27T03:21:10.479029+0000) 2024-07-27T03:21:28.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:27 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:27.877+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.380104+0000 front 2024-07-27T03:20:56.380077+0000 (oldest deadline 2024-07-27T03:21:21.680070+0000) 2024-07-27T03:21:28.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:27 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8[55674]: 2024-07-27T03:21:27.877+0000 7f8c20f22640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.380807+0000 front 2024-07-27T03:21:03.380845+0000 (oldest deadline 2024-07-27T03:21:27.480849+0000) 2024-07-27T03:21:28.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:27 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:27.784+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:28.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:27 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:27.784+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:28.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:27 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:27.784+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:28.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:27 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:27.784+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:28.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:27 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:27.784+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:28.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:27 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:27.886+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:28.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:27 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:27.886+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:28.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:27 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:27.886+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:28.220 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:27 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:27.886+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:28.220 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:27 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:27.886+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.876660+0000 front 2024-07-27T03:20:55.876705+0000 (oldest deadline 2024-07-27T03:21:21.776437+0000) 2024-07-27T03:21:28.220 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:27 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:27.886+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:02.877482+0000 front 2024-07-27T03:21:02.877436+0000 (oldest deadline 2024-07-27T03:21:26.977241+0000) 2024-07-27T03:21:28.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:27 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:27.872+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:28.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:27 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:27.872+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:28.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:27 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:27.872+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:28.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:27 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:27.872+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:28.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:27 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:27.872+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.097429+0000 front 2024-07-27T03:20:56.097464+0000 (oldest deadline 2024-07-27T03:21:21.997518+0000) 2024-07-27T03:21:28.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:27 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:27.872+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:01.998230+0000 front 2024-07-27T03:21:01.998260+0000 (oldest deadline 2024-07-27T03:21:26.598644+0000) 2024-07-27T03:21:29.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:28 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:28.786+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:29.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:28 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:28.786+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:29.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:28 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:28.786+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:29.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:28 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:28.786+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:29.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:28 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:28.786+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:29.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:28 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:28.850+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:29.220 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:28 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:28.850+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:29.220 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:28 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:28.850+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:29.220 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:28 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:28.850+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:29.220 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:28 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:28.850+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.876660+0000 front 2024-07-27T03:20:55.876705+0000 (oldest deadline 2024-07-27T03:21:21.776437+0000) 2024-07-27T03:21:29.220 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:28 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:28.850+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:02.877482+0000 front 2024-07-27T03:21:02.877436+0000 (oldest deadline 2024-07-27T03:21:26.977241+0000) 2024-07-27T03:21:29.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:28 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:28.874+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:29.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:28 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:28.874+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:29.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:28 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:28.874+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:29.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:28 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:28.874+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:29.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:28 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:28.874+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.097429+0000 front 2024-07-27T03:20:56.097464+0000 (oldest deadline 2024-07-27T03:21:21.997518+0000) 2024-07-27T03:21:29.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:28 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:28.874+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:01.998230+0000 front 2024-07-27T03:21:01.998260+0000 (oldest deadline 2024-07-27T03:21:26.598644+0000) 2024-07-27T03:21:29.512 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:29 smithi175 podman[71844]: 2024-07-27 03:21:29.189958639 +0000 UTC m=+5.561592081 container died 76aabc85fd36782ab67a4ff2fc4b02f5d3656653925dc374ba992df45cc6ed25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0) 2024-07-27T03:21:29.835 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:29 smithi175 podman[71844]: 2024-07-27 03:21:29.592599328 +0000 UTC m=+5.964232760 container cleanup 76aabc85fd36782ab67a4ff2fc4b02f5d3656653925dc374ba992df45cc6ed25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-07-27T03:21:29.835 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:29 smithi175 bash[71844]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8 2024-07-27T03:21:30.124 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:29 smithi175 podman[71860]: 2024-07-27 03:21:29.852227469 +0000 UTC m=+0.659330365 container remove 76aabc85fd36782ab67a4ff2fc4b02f5d3656653925dc374ba992df45cc6ed25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-27T03:21:30.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:29 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:29.833+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:30.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:29 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:29.833+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:30.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:29 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:29.833+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:30.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:29 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:29.833+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:30.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:29 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:29.833+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:30.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:29 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:29.872+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:30.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:29 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:29.872+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:30.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:29 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:29.872+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:30.220 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:29 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:29.872+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:30.220 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:29 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:29.872+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.876660+0000 front 2024-07-27T03:20:55.876705+0000 (oldest deadline 2024-07-27T03:21:21.776437+0000) 2024-07-27T03:21:30.220 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:29 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:29.872+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:02.877482+0000 front 2024-07-27T03:21:02.877436+0000 (oldest deadline 2024-07-27T03:21:26.977241+0000) 2024-07-27T03:21:30.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:29 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:29.913+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:30.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:29 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:29.913+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:30.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:29 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:29.913+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:30.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:29 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:29.913+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:30.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:29 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:29.913+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.097429+0000 front 2024-07-27T03:20:56.097464+0000 (oldest deadline 2024-07-27T03:21:21.997518+0000) 2024-07-27T03:21:30.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:29 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:29.913+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:01.998230+0000 front 2024-07-27T03:21:01.998260+0000 (oldest deadline 2024-07-27T03:21:26.598644+0000) 2024-07-27T03:21:30.548 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:30 smithi175 podman[71951]: 2024-07-27 03:21:30.202986799 +0000 UTC m=+0.103498361 container create 245a25419c406a7dd135956ccf64caa448485ac5746bd46ed90256cae05eef44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724) 2024-07-27T03:21:30.548 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:30 smithi175 podman[71951]: 2024-07-27 03:21:30.116348055 +0000 UTC m=+0.016859580 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c 2024-07-27T03:21:30.549 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:30 smithi175 podman[71951]: 2024-07-27 03:21:30.44702871 +0000 UTC m=+0.347540230 container init 245a25419c406a7dd135956ccf64caa448485ac5746bd46ed90256cae05eef44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-07-27T03:21:30.549 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:30 smithi175 podman[71951]: 2024-07-27 03:21:30.450118353 +0000 UTC m=+0.350629882 container start 245a25419c406a7dd135956ccf64caa448485ac5746bd46ed90256cae05eef44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf) 2024-07-27T03:21:30.549 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:30 smithi175 podman[71951]: 2024-07-27 03:21:30.546941126 +0000 UTC m=+0.447452647 container attach 245a25419c406a7dd135956ccf64caa448485ac5746bd46ed90256cae05eef44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.schema-version=1.0) 2024-07-27T03:21:30.822 INFO:journalctl@ceph.osd.8.smithi175.stdout:Jul 27 03:21:30 smithi175 podman[71951]: 2024-07-27 03:21:30.638924957 +0000 UTC m=+0.539436487 container died 245a25419c406a7dd135956ccf64caa448485ac5746bd46ed90256cae05eef44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-8-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-07-27T03:21:31.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:30 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:30.820+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:31.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:30 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:30.820+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:31.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:30 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:30.820+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:31.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:30 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:30.820+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:31.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:30 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:30.820+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:31.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:30 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:30.859+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:31.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:30 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:30.859+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:31.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:30 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:30.859+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:31.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:30 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:30.859+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:31.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:30 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:30.859+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.876660+0000 front 2024-07-27T03:20:55.876705+0000 (oldest deadline 2024-07-27T03:21:21.776437+0000) 2024-07-27T03:21:31.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:30 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:30.859+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:02.877482+0000 front 2024-07-27T03:21:02.877436+0000 (oldest deadline 2024-07-27T03:21:26.977241+0000) 2024-07-27T03:21:31.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:30 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:30.906+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:31.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:30 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:30.906+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:31.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:30 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:30.906+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:31.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:30 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:30.906+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:31.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:30 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:30.906+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.097429+0000 front 2024-07-27T03:20:56.097464+0000 (oldest deadline 2024-07-27T03:21:21.997518+0000) 2024-07-27T03:21:31.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:30 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:30.906+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:01.998230+0000 front 2024-07-27T03:21:01.998260+0000 (oldest deadline 2024-07-27T03:21:26.598644+0000) 2024-07-27T03:21:31.287 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.8.service' 2024-07-27T03:21:31.327 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:21:31.328 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-07-27T03:21:31.328 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-07-27T03:21:31.328 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.9 2024-07-27T03:21:31.645 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:31 smithi067 systemd[1]: Stopping Ceph osd.9 for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:21:31.969 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:31 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:31.642+0000 7fc091778640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:21:31.969 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:31 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:31.642+0000 7fc091778640 -1 osd.9 95 *** Got signal Terminated *** 2024-07-27T03:21:31.970 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:31 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:31.642+0000 7fc091778640 -1 osd.9 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-27T03:21:31.970 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:31 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:31.819+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:31.970 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:31 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:31.819+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:31.970 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:31 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:31.819+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:31.970 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:31 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:31.819+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:31.970 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:31 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:31.819+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.876660+0000 front 2024-07-27T03:20:55.876705+0000 (oldest deadline 2024-07-27T03:21:21.776437+0000) 2024-07-27T03:21:31.970 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:31 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:31.819+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:02.877482+0000 front 2024-07-27T03:21:02.877436+0000 (oldest deadline 2024-07-27T03:21:26.977241+0000) 2024-07-27T03:21:32.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:31 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:31.869+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:32.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:31 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:31.869+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:32.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:31 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:31.869+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:32.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:31 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:31.869+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:32.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:31 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:31.869+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:32.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:31 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:31.914+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:32.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:31 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:31.914+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:32.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:31 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:31.914+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:32.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:31 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:31.914+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:32.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:31 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:31.914+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.097429+0000 front 2024-07-27T03:20:56.097464+0000 (oldest deadline 2024-07-27T03:21:21.997518+0000) 2024-07-27T03:21:32.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:31 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:31.914+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:01.998230+0000 front 2024-07-27T03:21:01.998260+0000 (oldest deadline 2024-07-27T03:21:26.598644+0000) 2024-07-27T03:21:33.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:32 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:32.778+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:33.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:32 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:32.778+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:33.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:32 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:32.778+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:33.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:32 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:32.778+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:33.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:32 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:32.778+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.876660+0000 front 2024-07-27T03:20:55.876705+0000 (oldest deadline 2024-07-27T03:21:21.776437+0000) 2024-07-27T03:21:33.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:32 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:32.778+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:02.877482+0000 front 2024-07-27T03:21:02.877436+0000 (oldest deadline 2024-07-27T03:21:26.977241+0000) 2024-07-27T03:21:33.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:32 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:32.897+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:33.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:32 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:32.897+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:33.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:32 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:32.897+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:33.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:32 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:32.897+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:33.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:32 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:32.897+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.097429+0000 front 2024-07-27T03:20:56.097464+0000 (oldest deadline 2024-07-27T03:21:21.997518+0000) 2024-07-27T03:21:33.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:32 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:32.897+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:01.998230+0000 front 2024-07-27T03:21:01.998260+0000 (oldest deadline 2024-07-27T03:21:26.598644+0000) 2024-07-27T03:21:33.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:32 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:32.894+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:33.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:32 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:32.894+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:33.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:32 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:32.894+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:33.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:32 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:32.894+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:33.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:32 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:32.894+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:34.151 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:33 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:33.856+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:34.151 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:33 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:33.856+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:34.151 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:33 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:33.856+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:34.151 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:33 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:33.856+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:34.151 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:33 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:33.856+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.097429+0000 front 2024-07-27T03:20:56.097464+0000 (oldest deadline 2024-07-27T03:21:21.997518+0000) 2024-07-27T03:21:34.151 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:33 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:33.856+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:01.998230+0000 front 2024-07-27T03:21:01.998260+0000 (oldest deadline 2024-07-27T03:21:26.598644+0000) 2024-07-27T03:21:34.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:33.760+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:34.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:33.760+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:34.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:33.760+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:34.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:33.760+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:34.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:33.760+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.876660+0000 front 2024-07-27T03:20:55.876705+0000 (oldest deadline 2024-07-27T03:21:21.776437+0000) 2024-07-27T03:21:34.220 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:33 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:33.760+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:02.877482+0000 front 2024-07-27T03:21:02.877436+0000 (oldest deadline 2024-07-27T03:21:26.977241+0000) 2024-07-27T03:21:34.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:33 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:33.934+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:34.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:33 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:33.934+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:34.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:33 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:33.934+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:34.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:33 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:33.934+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:34.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:33 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:33.934+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:34.908 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:34 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:34.858+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:34.909 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:34 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:34.858+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:34.909 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:34 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:34.858+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:34.909 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:34 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:34.858+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:34.909 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:34 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:34.858+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.097429+0000 front 2024-07-27T03:20:56.097464+0000 (oldest deadline 2024-07-27T03:21:21.997518+0000) 2024-07-27T03:21:34.909 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:34 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:34.858+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:01.998230+0000 front 2024-07-27T03:21:01.998260+0000 (oldest deadline 2024-07-27T03:21:26.598644+0000) 2024-07-27T03:21:35.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:34 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:34.748+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:35.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:34 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:34.748+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:35.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:34 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:34.748+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:35.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:34 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:34.748+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:35.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:34 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:34.748+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.876660+0000 front 2024-07-27T03:20:55.876705+0000 (oldest deadline 2024-07-27T03:21:21.776437+0000) 2024-07-27T03:21:35.220 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:34 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:34.748+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:02.877482+0000 front 2024-07-27T03:21:02.877436+0000 (oldest deadline 2024-07-27T03:21:26.977241+0000) 2024-07-27T03:21:35.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:34 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:34.908+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:35.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:34 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:34.908+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:35.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:34 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:34.908+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:35.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:34 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:34.908+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:35.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:34 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:34.908+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:36.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:35.723+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:23.569764+0000 front 2024-07-27T03:20:23.569570+0000 (oldest deadline 2024-07-27T03:20:48.869788+0000) 2024-07-27T03:21:36.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:35.723+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:31.770906+0000 front 2024-07-27T03:20:31.770882+0000 (oldest deadline 2024-07-27T03:20:56.471018+0000) 2024-07-27T03:21:36.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:35.723+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.872563+0000 front 2024-07-27T03:20:39.872623+0000 (oldest deadline 2024-07-27T03:21:04.572441+0000) 2024-07-27T03:21:36.219 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:35.723+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:47.874885+0000 front 2024-07-27T03:20:47.874935+0000 (oldest deadline 2024-07-27T03:21:10.774853+0000) 2024-07-27T03:21:36.220 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:35.723+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.876660+0000 front 2024-07-27T03:20:55.876705+0000 (oldest deadline 2024-07-27T03:21:21.776437+0000) 2024-07-27T03:21:36.220 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:35 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9[71093]: 2024-07-27T03:21:35.723+0000 7fc089d5b640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:02.877482+0000 front 2024-07-27T03:21:02.877436+0000 (oldest deadline 2024-07-27T03:21:26.977241+0000) 2024-07-27T03:21:36.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:35 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:35.810+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:36.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:35 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:35.810+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:36.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:35 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:35.810+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:36.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:35 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:35.810+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:36.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:35 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:35.810+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.097429+0000 front 2024-07-27T03:20:56.097464+0000 (oldest deadline 2024-07-27T03:21:21.997518+0000) 2024-07-27T03:21:36.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:35 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:35.810+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:01.998230+0000 front 2024-07-27T03:21:01.998260+0000 (oldest deadline 2024-07-27T03:21:26.598644+0000) 2024-07-27T03:21:36.275 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:35 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:35.891+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:36.275 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:35 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:35.891+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:36.275 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:35 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:35.891+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:36.276 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:35 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:35.891+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:36.276 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:35 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:35.891+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:37.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:36 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:36.829+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:37.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:36 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:36.829+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:37.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:36 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:36.829+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:37.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:36 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:36.829+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:37.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:36 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:36.829+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.097429+0000 front 2024-07-27T03:20:56.097464+0000 (oldest deadline 2024-07-27T03:21:21.997518+0000) 2024-07-27T03:21:37.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:36 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:36.829+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:01.998230+0000 front 2024-07-27T03:21:01.998260+0000 (oldest deadline 2024-07-27T03:21:26.598644+0000) 2024-07-27T03:21:37.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:36 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:36.907+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:37.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:36 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:36.907+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:37.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:36 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:36.907+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:37.375 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:36 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:36.907+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:37.375 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:36 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:36.907+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:37.412 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:37 smithi067 podman[106630]: 2024-07-27 03:21:37.108124187 +0000 UTC m=+5.572570020 container died 075353a7d1856e708a83021d24ad6d1c3ec6b9aacd8b95cfc72def819f52ed3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-27T03:21:37.719 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:37 smithi067 podman[106630]: 2024-07-27 03:21:37.567533048 +0000 UTC m=+6.031978907 container cleanup 075353a7d1856e708a83021d24ad6d1c3ec6b9aacd8b95cfc72def819f52ed3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-07-27T03:21:37.719 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:37 smithi067 bash[106630]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9 2024-07-27T03:21:38.165 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:37 smithi067 podman[106645]: 2024-07-27 03:21:37.827227576 +0000 UTC m=+0.717551002 container remove 075353a7d1856e708a83021d24ad6d1c3ec6b9aacd8b95cfc72def819f52ed3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-07-27T03:21:38.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:37 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:37.830+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:38.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:37 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:37.830+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:38.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:37 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:37.830+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:38.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:37 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:37.830+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:38.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:37 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:37.830+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.097429+0000 front 2024-07-27T03:20:56.097464+0000 (oldest deadline 2024-07-27T03:21:21.997518+0000) 2024-07-27T03:21:38.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:37 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:37.830+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:01.998230+0000 front 2024-07-27T03:21:01.998260+0000 (oldest deadline 2024-07-27T03:21:26.598644+0000) 2024-07-27T03:21:38.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:37 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:37.830+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-27T03:21:12.201081+0000 front 2024-07-27T03:21:12.200947+0000 (oldest deadline 2024-07-27T03:21:37.500849+0000) 2024-07-27T03:21:38.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:37 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:37.873+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:38.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:37 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:37.873+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:38.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:37 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:37.873+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:38.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:37 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:37.873+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:38.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:37 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:37.873+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:38.418 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:38 smithi067 podman[106735]: 2024-07-27 03:21:38.066344915 +0000 UTC m=+0.018081806 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c 2024-07-27T03:21:38.418 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:38 smithi067 podman[106735]: 2024-07-27 03:21:38.169601885 +0000 UTC m=+0.121338767 container create 5d9f9b08335e7167a6cb7f31a089e688c9c6f3922e501fdfb36072398a1775e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-07-27T03:21:38.418 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:38 smithi067 podman[106735]: 2024-07-27 03:21:38.413494531 +0000 UTC m=+0.365231421 container init 5d9f9b08335e7167a6cb7f31a089e688c9c6f3922e501fdfb36072398a1775e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9-deactivate, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240724) 2024-07-27T03:21:38.418 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:38 smithi067 podman[106735]: 2024-07-27 03:21:38.416807237 +0000 UTC m=+0.368544124 container start 5d9f9b08335e7167a6cb7f31a089e688c9c6f3922e501fdfb36072398a1775e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9-deactivate, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf) 2024-07-27T03:21:38.719 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:38 smithi067 podman[106735]: 2024-07-27 03:21:38.505644007 +0000 UTC m=+0.457380897 container attach 5d9f9b08335e7167a6cb7f31a089e688c9c6f3922e501fdfb36072398a1775e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-07-27T03:21:38.719 INFO:journalctl@ceph.osd.9.smithi067.stdout:Jul 27 03:21:38 smithi067 podman[106735]: 2024-07-27 03:21:38.607757645 +0000 UTC m=+0.559494531 container died 5d9f9b08335e7167a6cb7f31a089e688c9c6f3922e501fdfb36072398a1775e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, ceph=True, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-27T03:21:39.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:38 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:38.848+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:39.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:38 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:38.848+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:39.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:38 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:38.848+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:39.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:38 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:38.848+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:39.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:38 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:38.848+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:39.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:38 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:38.848+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-27T03:21:12.212653+0000 front 2024-07-27T03:21:12.212670+0000 (oldest deadline 2024-07-27T03:21:38.112639+0000) 2024-07-27T03:21:39.175 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.9.service' 2024-07-27T03:21:39.216 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:21:39.216 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-07-27T03:21:39.216 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-07-27T03:21:39.216 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.10 2024-07-27T03:21:39.243 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:38 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:38.871+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:39.243 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:38 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:38.871+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:39.244 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:38 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:38.871+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:39.244 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:38 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:38.871+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:39.244 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:38 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:38.871+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.097429+0000 front 2024-07-27T03:20:56.097464+0000 (oldest deadline 2024-07-27T03:21:21.997518+0000) 2024-07-27T03:21:39.244 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:38 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:38.871+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:01.998230+0000 front 2024-07-27T03:21:01.998260+0000 (oldest deadline 2024-07-27T03:21:26.598644+0000) 2024-07-27T03:21:39.244 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:38 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:38.871+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-27T03:21:12.201081+0000 front 2024-07-27T03:21:12.200947+0000 (oldest deadline 2024-07-27T03:21:37.500849+0000) 2024-07-27T03:21:39.499 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:39 smithi149 systemd[1]: Stopping Ceph osd.10 for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:21:39.921 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:39 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:39.513+0000 7f841b4ce640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:21:39.922 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:39 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:39.513+0000 7f841b4ce640 -1 osd.10 95 *** Got signal Terminated *** 2024-07-27T03:21:39.922 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:39 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:39.513+0000 7f841b4ce640 -1 osd.10 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-27T03:21:40.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:39 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:39.919+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:40.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:39 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:39.919+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:40.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:39 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:39.919+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:40.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:39 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:39.919+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:40.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:39 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:39.919+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.097429+0000 front 2024-07-27T03:20:56.097464+0000 (oldest deadline 2024-07-27T03:21:21.997518+0000) 2024-07-27T03:21:40.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:39 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:39.919+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:01.998230+0000 front 2024-07-27T03:21:01.998260+0000 (oldest deadline 2024-07-27T03:21:26.598644+0000) 2024-07-27T03:21:40.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:39 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:39.919+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-27T03:21:12.201081+0000 front 2024-07-27T03:21:12.200947+0000 (oldest deadline 2024-07-27T03:21:37.500849+0000) 2024-07-27T03:21:40.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:39 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:39.897+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:40.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:39 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:39.897+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:40.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:39 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:39.897+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:40.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:39 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:39.897+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:40.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:39 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:39.897+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:40.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:39 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:39.897+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-27T03:21:12.212653+0000 front 2024-07-27T03:21:12.212670+0000 (oldest deadline 2024-07-27T03:21:38.112639+0000) 2024-07-27T03:21:41.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:40 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:40.877+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:41.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:40 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:40.877+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:41.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:40 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:40.877+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:41.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:40 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:40.877+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:41.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:40 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:40.877+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.097429+0000 front 2024-07-27T03:20:56.097464+0000 (oldest deadline 2024-07-27T03:21:21.997518+0000) 2024-07-27T03:21:41.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:40 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:40.877+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:01.998230+0000 front 2024-07-27T03:21:01.998260+0000 (oldest deadline 2024-07-27T03:21:26.598644+0000) 2024-07-27T03:21:41.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:40 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:40.877+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-27T03:21:12.201081+0000 front 2024-07-27T03:21:12.200947+0000 (oldest deadline 2024-07-27T03:21:37.500849+0000) 2024-07-27T03:21:41.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:40 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:40.936+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:41.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:40 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:40.936+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:41.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:40 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:40.936+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:41.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:40 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:40.936+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:41.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:40 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:40.936+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:41.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:40 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:40.936+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-27T03:21:12.212653+0000 front 2024-07-27T03:21:12.212670+0000 (oldest deadline 2024-07-27T03:21:38.112639+0000) 2024-07-27T03:21:42.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:41 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:41.891+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:42.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:41 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:41.891+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:42.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:41 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:41.891+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:42.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:41 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:41.891+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:42.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:41 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:41.891+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.097429+0000 front 2024-07-27T03:20:56.097464+0000 (oldest deadline 2024-07-27T03:21:21.997518+0000) 2024-07-27T03:21:42.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:41 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:41.891+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:01.998230+0000 front 2024-07-27T03:21:01.998260+0000 (oldest deadline 2024-07-27T03:21:26.598644+0000) 2024-07-27T03:21:42.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:41 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:41.891+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-27T03:21:12.201081+0000 front 2024-07-27T03:21:12.200947+0000 (oldest deadline 2024-07-27T03:21:37.500849+0000) 2024-07-27T03:21:42.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:41 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:41.967+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:42.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:41 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:41.967+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:42.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:41 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:41.967+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:42.375 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:41 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:41.967+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:42.375 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:41 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:41.967+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:42.375 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:41 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:41.967+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-27T03:21:12.212653+0000 front 2024-07-27T03:21:12.212670+0000 (oldest deadline 2024-07-27T03:21:38.112639+0000) 2024-07-27T03:21:43.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:42 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:42.872+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:43.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:42 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:42.872+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:43.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:42 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:42.872+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:43.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:42 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:42.872+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:43.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:42 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:42.872+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.097429+0000 front 2024-07-27T03:20:56.097464+0000 (oldest deadline 2024-07-27T03:21:21.997518+0000) 2024-07-27T03:21:43.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:42 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:42.872+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:01.998230+0000 front 2024-07-27T03:21:01.998260+0000 (oldest deadline 2024-07-27T03:21:26.598644+0000) 2024-07-27T03:21:43.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:42 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:42.872+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-27T03:21:12.201081+0000 front 2024-07-27T03:21:12.200947+0000 (oldest deadline 2024-07-27T03:21:37.500849+0000) 2024-07-27T03:21:43.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:42 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:42.925+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:43.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:42 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:42.925+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:43.375 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:42 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:42.925+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:43.375 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:42 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:42.925+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:43.375 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:42 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:42.925+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:43.375 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:42 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:42.925+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-27T03:21:12.212653+0000 front 2024-07-27T03:21:12.212670+0000 (oldest deadline 2024-07-27T03:21:38.112639+0000) 2024-07-27T03:21:43.375 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:42 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:42.925+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-07-27T03:21:19.813505+0000 front 2024-07-27T03:21:19.813548+0000 (oldest deadline 2024-07-27T03:21:42.713414+0000) 2024-07-27T03:21:44.249 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:43 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:43.911+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:22.793190+0000 front 2024-07-27T03:20:22.793116+0000 (oldest deadline 2024-07-27T03:20:48.093158+0000) 2024-07-27T03:21:44.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:43 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:43.911+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:32.194310+0000 front 2024-07-27T03:20:32.194354+0000 (oldest deadline 2024-07-27T03:20:57.494405+0000) 2024-07-27T03:21:44.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:43 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:43.911+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:37.494744+0000 front 2024-07-27T03:20:37.494765+0000 (oldest deadline 2024-07-27T03:21:02.695393+0000) 2024-07-27T03:21:44.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:43 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:43.911+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6814 osd.3 since back 2024-07-27T03:20:46.796258+0000 front 2024-07-27T03:20:46.796414+0000 (oldest deadline 2024-07-27T03:21:12.096209+0000) 2024-07-27T03:21:44.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:43 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:43.911+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:56.097429+0000 front 2024-07-27T03:20:56.097464+0000 (oldest deadline 2024-07-27T03:21:21.997518+0000) 2024-07-27T03:21:44.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:43 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:43.911+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:01.998230+0000 front 2024-07-27T03:21:01.998260+0000 (oldest deadline 2024-07-27T03:21:26.598644+0000) 2024-07-27T03:21:44.250 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:43 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10[62097]: 2024-07-27T03:21:43.911+0000 7f8413ab1640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-27T03:21:12.201081+0000 front 2024-07-27T03:21:12.200947+0000 (oldest deadline 2024-07-27T03:21:37.500849+0000) 2024-07-27T03:21:44.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:43 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:43.879+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:44.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:43 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:43.879+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:44.375 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:43 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:43.879+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:44.375 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:43 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:43.879+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:44.375 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:43 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:43.879+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:44.375 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:43 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:43.879+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-27T03:21:12.212653+0000 front 2024-07-27T03:21:12.212670+0000 (oldest deadline 2024-07-27T03:21:38.112639+0000) 2024-07-27T03:21:44.375 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:43 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:43.879+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-07-27T03:21:19.813505+0000 front 2024-07-27T03:21:19.813548+0000 (oldest deadline 2024-07-27T03:21:42.713414+0000) 2024-07-27T03:21:45.233 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:44 smithi149 podman[71481]: 2024-07-27 03:21:44.965340446 +0000 UTC m=+5.548123473 container died 0a250e1cdceb59f9aef863eabd18fb0324337955690792ccde9b6d10e49cabce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, GIT_CLEAN=True) 2024-07-27T03:21:45.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:44 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:44.881+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:45.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:44 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:44.881+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:45.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:44 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:44.881+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:45.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:44 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:44.881+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:45.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:44 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:44.881+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:45.375 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:44 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:44.881+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-27T03:21:12.212653+0000 front 2024-07-27T03:21:12.212670+0000 (oldest deadline 2024-07-27T03:21:38.112639+0000) 2024-07-27T03:21:45.375 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:44 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:44.881+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-07-27T03:21:19.813505+0000 front 2024-07-27T03:21:19.813548+0000 (oldest deadline 2024-07-27T03:21:42.713414+0000) 2024-07-27T03:21:45.499 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:45 smithi149 podman[71481]: 2024-07-27 03:21:45.314559372 +0000 UTC m=+5.897342390 container cleanup 0a250e1cdceb59f9aef863eabd18fb0324337955690792ccde9b6d10e49cabce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-27T03:21:45.500 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:45 smithi149 bash[71481]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10 2024-07-27T03:21:45.903 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:45 smithi149 podman[71498]: 2024-07-27 03:21:45.498959407 +0000 UTC m=+0.526792021 container remove 0a250e1cdceb59f9aef863eabd18fb0324337955690792ccde9b6d10e49cabce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724) 2024-07-27T03:21:46.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:45 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:45.866+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:46.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:45 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:45.866+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:46.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:45 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:45.866+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:46.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:45 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:45.866+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:46.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:45 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:45.866+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:46.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:45 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:45.866+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-27T03:21:12.212653+0000 front 2024-07-27T03:21:12.212670+0000 (oldest deadline 2024-07-27T03:21:38.112639+0000) 2024-07-27T03:21:46.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:45 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:45.866+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-07-27T03:21:19.813505+0000 front 2024-07-27T03:21:19.813548+0000 (oldest deadline 2024-07-27T03:21:42.713414+0000) 2024-07-27T03:21:46.171 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:45 smithi149 podman[71582]: 2024-07-27 03:21:45.837339136 +0000 UTC m=+0.017678894 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c 2024-07-27T03:21:46.171 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:45 smithi149 podman[71582]: 2024-07-27 03:21:45.975343507 +0000 UTC m=+0.155683338 container create 5fe043c10d54a243efe274b51334d0f91de8132f75951005729366c6172683df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-07-27T03:21:46.171 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:46 smithi149 podman[71582]: 2024-07-27 03:21:46.169620202 +0000 UTC m=+0.349959980 container init 5fe043c10d54a243efe274b51334d0f91de8132f75951005729366c6172683df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-07-27T03:21:46.500 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:46 smithi149 podman[71582]: 2024-07-27 03:21:46.173032326 +0000 UTC m=+0.353372082 container start 5fe043c10d54a243efe274b51334d0f91de8132f75951005729366c6172683df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0) 2024-07-27T03:21:46.500 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:46 smithi149 podman[71582]: 2024-07-27 03:21:46.244256873 +0000 UTC m=+0.424596631 container attach 5fe043c10d54a243efe274b51334d0f91de8132f75951005729366c6172683df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10-deactivate, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-07-27T03:21:46.500 INFO:journalctl@ceph.osd.10.smithi149.stdout:Jul 27 03:21:46 smithi149 podman[71582]: 2024-07-27 03:21:46.362512159 +0000 UTC m=+0.542851916 container died 5fe043c10d54a243efe274b51334d0f91de8132f75951005729366c6172683df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-07-27T03:21:46.967 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.10.service' 2024-07-27T03:21:47.045 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:21:47.045 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-07-27T03:21:47.046 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-07-27T03:21:47.046 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.11 2024-07-27T03:21:47.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:46 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:46.856+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:47.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:46 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:46.856+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:47.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:46 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:46.856+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:47.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:46 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:46.856+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:47.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:46 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:46.856+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:47.126 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:46 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:46.856+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-27T03:21:12.212653+0000 front 2024-07-27T03:21:12.212670+0000 (oldest deadline 2024-07-27T03:21:38.112639+0000) 2024-07-27T03:21:47.126 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:46 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:46.856+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-07-27T03:21:19.813505+0000 front 2024-07-27T03:21:19.813548+0000 (oldest deadline 2024-07-27T03:21:42.713414+0000) 2024-07-27T03:21:47.126 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:47 smithi175 systemd[1]: Stopping Ceph osd.11 for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:21:47.852 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:47 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:47.384+0000 7f06f0c13640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:21:47.853 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:47 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:47.384+0000 7f06f0c13640 -1 osd.11 95 *** Got signal Terminated *** 2024-07-27T03:21:47.853 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:47 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:47.384+0000 7f06f0c13640 -1 osd.11 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-07-27T03:21:47.853 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:47 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:47.850+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:47.853 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:47 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:47.850+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:47.853 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:47 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:47.850+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:47.853 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:47 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:47.850+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:47.853 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:47 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:47.850+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:48.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:47 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:47.850+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-27T03:21:12.212653+0000 front 2024-07-27T03:21:12.212670+0000 (oldest deadline 2024-07-27T03:21:38.112639+0000) 2024-07-27T03:21:48.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:47 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:47.850+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-07-27T03:21:19.813505+0000 front 2024-07-27T03:21:19.813548+0000 (oldest deadline 2024-07-27T03:21:42.713414+0000) 2024-07-27T03:21:49.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:48 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:48.803+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:49.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:48 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:48.803+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:49.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:48 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:48.803+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:49.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:48 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:48.803+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:49.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:48 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:48.803+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:49.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:48 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:48.803+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-27T03:21:12.212653+0000 front 2024-07-27T03:21:12.212670+0000 (oldest deadline 2024-07-27T03:21:38.112639+0000) 2024-07-27T03:21:49.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:48 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:48.803+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-07-27T03:21:19.813505+0000 front 2024-07-27T03:21:19.813548+0000 (oldest deadline 2024-07-27T03:21:42.713414+0000) 2024-07-27T03:21:50.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:49 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:49.757+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:50.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:49 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:49.757+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:50.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:49 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:49.757+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:50.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:49 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:49.757+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:50.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:49 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:49.757+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:50.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:49 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:49.757+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-27T03:21:12.212653+0000 front 2024-07-27T03:21:12.212670+0000 (oldest deadline 2024-07-27T03:21:38.112639+0000) 2024-07-27T03:21:50.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:49 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:49.757+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-07-27T03:21:19.813505+0000 front 2024-07-27T03:21:19.813548+0000 (oldest deadline 2024-07-27T03:21:42.713414+0000) 2024-07-27T03:21:51.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:50 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:50.772+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:51.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:50 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:50.772+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:51.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:50 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:50.772+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:51.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:50 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:50.772+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:51.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:50 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:50.772+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:51.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:50 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:50.772+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-27T03:21:12.212653+0000 front 2024-07-27T03:21:12.212670+0000 (oldest deadline 2024-07-27T03:21:38.112639+0000) 2024-07-27T03:21:51.125 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:50 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:50.772+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-07-27T03:21:19.813505+0000 front 2024-07-27T03:21:19.813548+0000 (oldest deadline 2024-07-27T03:21:42.713414+0000) 2024-07-27T03:21:52.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:51 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:51.767+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-07-27T03:20:24.505609+0000 front 2024-07-27T03:20:24.505498+0000 (oldest deadline 2024-07-27T03:20:49.205556+0000) 2024-07-27T03:21:52.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:51 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:51.767+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-07-27T03:20:29.206176+0000 front 2024-07-27T03:20:29.206054+0000 (oldest deadline 2024-07-27T03:20:55.106004+0000) 2024-07-27T03:21:52.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:51 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:51.767+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6804 osd.2 since back 2024-07-27T03:20:39.707348+0000 front 2024-07-27T03:20:39.707316+0000 (oldest deadline 2024-07-27T03:21:02.007407+0000) 2024-07-27T03:21:52.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:51 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:51.767+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-07-27T03:20:55.310375+0000 front 2024-07-27T03:20:55.310437+0000 (oldest deadline 2024-07-27T03:21:18.810335+0000) 2024-07-27T03:21:52.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:51 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:51.767+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.175:6812 osd.5 since back 2024-07-27T03:21:03.511233+0000 front 2024-07-27T03:21:03.511268+0000 (oldest deadline 2024-07-27T03:21:25.811277+0000) 2024-07-27T03:21:52.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:51 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:51.767+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.67:6822 osd.6 since back 2024-07-27T03:21:12.212653+0000 front 2024-07-27T03:21:12.212670+0000 (oldest deadline 2024-07-27T03:21:38.112639+0000) 2024-07-27T03:21:52.124 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:51 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11[62215]: 2024-07-27T03:21:51.767+0000 7f06ed1fe640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-07-27T03:21:19.813505+0000 front 2024-07-27T03:21:19.813548+0000 (oldest deadline 2024-07-27T03:21:42.713414+0000) 2024-07-27T03:21:53.029 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:52 smithi175 podman[72054]: 2024-07-27 03:21:52.736090716 +0000 UTC m=+5.498045476 container died c10699581d89143a34c46eaa385765af5a13b3cfc5c0f7d9cc13db9e905ba5b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-07-27T03:21:53.370 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:53 smithi175 podman[72054]: 2024-07-27 03:21:53.118032223 +0000 UTC m=+5.879986983 container cleanup c10699581d89143a34c46eaa385765af5a13b3cfc5c0f7d9cc13db9e905ba5b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-07-27T03:21:53.370 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:53 smithi175 bash[72054]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11 2024-07-27T03:21:53.624 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:53 smithi175 podman[72069]: 2024-07-27 03:21:53.369012391 +0000 UTC m=+0.629391372 container remove c10699581d89143a34c46eaa385765af5a13b3cfc5c0f7d9cc13db9e905ba5b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-07-27T03:21:53.991 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:53 smithi175 podman[72159]: 2024-07-27 03:21:53.586443745 +0000 UTC m=+0.017454924 image pull b2e97770704716ffa72fedd8f5cf9108f0148e655a9891b47bbdbb68f339dce3 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c 2024-07-27T03:21:53.991 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:53 smithi175 podman[72159]: 2024-07-27 03:21:53.694962528 +0000 UTC m=+0.125973746 container create ffa6abc3562a48bdbbdcfa623fba00912121ca1b568a37f182cc734967fb54a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-27T03:21:53.991 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:53 smithi175 podman[72159]: 2024-07-27 03:21:53.914420806 +0000 UTC m=+0.345431985 container init ffa6abc3562a48bdbbdcfa623fba00912121ca1b568a37f182cc734967fb54a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-07-27T03:21:53.991 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:53 smithi175 podman[72159]: 2024-07-27 03:21:53.917379532 +0000 UTC m=+0.348390711 container start ffa6abc3562a48bdbbdcfa623fba00912121ca1b568a37f182cc734967fb54a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=) 2024-07-27T03:21:53.991 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:53 smithi175 podman[72159]: 2024-07-27 03:21:53.989415853 +0000 UTC m=+0.420427043 container attach ffa6abc3562a48bdbbdcfa623fba00912121ca1b568a37f182cc734967fb54a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0) 2024-07-27T03:21:54.374 INFO:journalctl@ceph.osd.11.smithi175.stdout:Jul 27 03:21:54 smithi175 podman[72159]: 2024-07-27 03:21:54.102749588 +0000 UTC m=+0.533760778 container died ffa6abc3562a48bdbbdcfa623fba00912121ca1b568a37f182cc734967fb54a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-07-27T03:21:54.679 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@osd.11.service' 2024-07-27T03:21:54.715 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:21:54.715 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-07-27T03:21:54.715 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-07-27T03:21:54.715 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.a 2024-07-27T03:21:55.038 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:21:54 smithi067 systemd[1]: Stopping Ceph mds.a for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:21:55.469 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:21:55 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-a[78470]: 2024-07-27T03:21:55.035+0000 7f93e3319640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:21:55.469 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:21:55 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-a[78470]: 2024-07-27T03:21:55.035+0000 7f93e3319640 -1 mds.a *** got signal Terminated *** 2024-07-27T03:21:58.719 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:21:58 smithi067 podman[106837]: 2024-07-27 03:21:58.450301818 +0000 UTC m=+3.515462377 container died ae35ee434933d6980741f938a36213a5ff09d50daf78c79ab1cfef7a41674c46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0) 2024-07-27T03:21:59.071 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:21:58 smithi067 podman[106837]: 2024-07-27 03:21:58.868722033 +0000 UTC m=+3.933882650 container cleanup ae35ee434933d6980741f938a36213a5ff09d50daf78c79ab1cfef7a41674c46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-27T03:21:59.072 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:21:58 smithi067 bash[106837]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-a 2024-07-27T03:21:59.313 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.a.service' 2024-07-27T03:21:59.336 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:21:59 smithi067 podman[106861]: 2024-07-27 03:21:59.070304189 +0000 UTC m=+0.612794204 container remove ae35ee434933d6980741f938a36213a5ff09d50daf78c79ab1cfef7a41674c46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.license=GPLv2) 2024-07-27T03:21:59.336 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:21:59 smithi067 systemd[1]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.a.service: Deactivated successfully. 2024-07-27T03:21:59.336 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:21:59 smithi067 systemd[1]: Stopped Ceph mds.a for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:21:59.336 INFO:journalctl@ceph.mds.a.smithi067.stdout:Jul 27 03:21:59 smithi067 systemd[1]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.a.service: Consumed 1.594s CPU time. 2024-07-27T03:21:59.351 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:21:59.351 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-07-27T03:21:59.352 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-07-27T03:21:59.352 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.d 2024-07-27T03:21:59.675 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 27 03:21:59 smithi067 systemd[1]: Stopping Ceph mds.d for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:21:59.969 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 27 03:21:59 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-d[77248]: 2024-07-27T03:21:59.672+0000 7f612c552640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:21:59.969 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 27 03:21:59 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-d[77248]: 2024-07-27T03:21:59.672+0000 7f612c552640 -1 mds.d *** got signal Terminated *** 2024-07-27T03:22:02.219 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 27 03:22:01 smithi067 podman[106968]: 2024-07-27 03:22:01.949350151 +0000 UTC m=+2.370148064 container died a30b577e5ac20082a16ef5a179b674a6ca75e594b6d768b5f703c1da9501f7db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-d, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724) 2024-07-27T03:22:02.686 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 27 03:22:02 smithi067 podman[106968]: 2024-07-27 03:22:02.256150146 +0000 UTC m=+2.676948057 container cleanup a30b577e5ac20082a16ef5a179b674a6ca75e594b6d768b5f703c1da9501f7db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-d, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0) 2024-07-27T03:22:02.686 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 27 03:22:02 smithi067 bash[106968]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-d 2024-07-27T03:22:02.686 INFO:journalctl@ceph.mds.d.smithi067.stdout:Jul 27 03:22:02 smithi067 podman[106982]: 2024-07-27 03:22:02.448930256 +0000 UTC m=+0.497223657 container remove a30b577e5ac20082a16ef5a179b674a6ca75e594b6d768b5f703c1da9501f7db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240724) 2024-07-27T03:22:02.703 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.d.service' 2024-07-27T03:22:02.738 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:22:02.738 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-07-27T03:22:02.738 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-07-27T03:22:02.739 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.g 2024-07-27T03:22:02.969 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 27 03:22:02 smithi067 systemd[1]: Stopping Ceph mds.g for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:22:03.469 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 27 03:22:03 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-g[78048]: 2024-07-27T03:22:03.076+0000 7f17ea1c6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:22:03.469 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 27 03:22:03 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-g[78048]: 2024-07-27T03:22:03.076+0000 7f17ea1c6640 -1 mds.g *** got signal Terminated *** 2024-07-27T03:22:06.712 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 27 03:22:06 smithi067 podman[107090]: 2024-07-27 03:22:06.451272736 +0000 UTC m=+3.479637452 container died 2799be9e5935f99a2bc58273b326b5ba33eff03ddea6914ed5317920f76cb57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-g, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-07-27T03:22:06.969 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 27 03:22:06 smithi067 podman[107090]: 2024-07-27 03:22:06.809797431 +0000 UTC m=+3.838162146 container cleanup 2799be9e5935f99a2bc58273b326b5ba33eff03ddea6914ed5317920f76cb57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-g, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-07-27T03:22:06.970 INFO:journalctl@ceph.mds.g.smithi067.stdout:Jul 27 03:22:06 smithi067 bash[107090]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-g 2024-07-27T03:22:07.297 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.g.service' 2024-07-27T03:22:07.331 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:22:07.331 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-07-27T03:22:07.331 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-07-27T03:22:07.331 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.j 2024-07-27T03:22:07.469 INFO:journalctl@ceph.mds.j.smithi067.stdout:Jul 27 03:22:07 smithi067 systemd[1]: Stopping Ceph mds.j for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:22:07.970 INFO:journalctl@ceph.mds.j.smithi067.stdout:Jul 27 03:22:07 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-j[76886]: 2024-07-27T03:22:07.629+0000 7fbbe1284640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:22:07.970 INFO:journalctl@ceph.mds.j.smithi067.stdout:Jul 27 03:22:07 smithi067 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-j[76886]: 2024-07-27T03:22:07.629+0000 7fbbe1284640 -1 mds.j *** got signal Terminated *** 2024-07-27T03:22:11.316 INFO:journalctl@ceph.mds.j.smithi067.stdout:Jul 27 03:22:11 smithi067 podman[107212]: 2024-07-27 03:22:11.046058808 +0000 UTC m=+3.494008569 container died c8f67424c9aad325e97cb70ffc560d9ba02730b62f62523718c6996122837429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-27T03:22:11.674 INFO:journalctl@ceph.mds.j.smithi067.stdout:Jul 27 03:22:11 smithi067 podman[107212]: 2024-07-27 03:22:11.396580353 +0000 UTC m=+3.844530113 container cleanup c8f67424c9aad325e97cb70ffc560d9ba02730b62f62523718c6996122837429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-j, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240724, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-07-27T03:22:11.674 INFO:journalctl@ceph.mds.j.smithi067.stdout:Jul 27 03:22:11 smithi067 bash[107212]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-j 2024-07-27T03:22:11.837 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.j.service' 2024-07-27T03:22:11.870 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:22:11.870 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-07-27T03:22:11.870 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-07-27T03:22:11.870 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.b 2024-07-27T03:22:12.158 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:22:11 smithi149 systemd[1]: Stopping Ceph mds.b for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:22:12.426 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:22:12 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-b[67306]: 2024-07-27T03:22:12.156+0000 7fa5c3a65640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:22:12.426 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:22:12 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-b[67306]: 2024-07-27T03:22:12.156+0000 7fa5c3a65640 -1 mds.b *** got signal Terminated *** 2024-07-27T03:22:14.741 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:22:14 smithi149 podman[71686]: 2024-07-27 03:22:14.44923215 +0000 UTC m=+2.388754291 container died e6c2c871b8ee6267eb99eff183a84319999f3b8341281a7ffc0d4e615fdc9ef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-b, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.build-date=20240724, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-07-27T03:22:14.999 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:22:14 smithi149 podman[71686]: 2024-07-27 03:22:14.889356868 +0000 UTC m=+2.828879020 container cleanup e6c2c871b8ee6267eb99eff183a84319999f3b8341281a7ffc0d4e615fdc9ef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-b, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0) 2024-07-27T03:22:15.000 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:22:14 smithi149 bash[71686]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-b 2024-07-27T03:22:15.367 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:22:15 smithi149 podman[71713]: 2024-07-27 03:22:15.11545796 +0000 UTC m=+0.656569497 container remove e6c2c871b8ee6267eb99eff183a84319999f3b8341281a7ffc0d4e615fdc9ef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-b, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240724) 2024-07-27T03:22:15.367 INFO:journalctl@ceph.mds.b.smithi149.stdout:Jul 27 03:22:15 smithi149 systemd[1]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.b.service: Deactivated successfully. 2024-07-27T03:22:15.370 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.b.service' 2024-07-27T03:22:15.406 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:22:15.406 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-07-27T03:22:15.407 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-07-27T03:22:15.407 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.e 2024-07-27T03:22:15.736 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:22:15 smithi149 systemd[1]: Stopping Ceph mds.e for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:22:15.999 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:22:15 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-e[66689]: 2024-07-27T03:22:15.734+0000 7fdb00e12640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:22:15.999 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:22:15 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-e[66689]: 2024-07-27T03:22:15.734+0000 7fdb00e12640 -1 mds.e *** got signal Terminated *** 2024-07-27T03:22:18.719 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:22:18 smithi149 podman[71819]: 2024-07-27 03:22:18.456398339 +0000 UTC m=+2.825930527 container died 4ef3b65849c3912ae2210bc394e4dad42b6ea83c5325a3a4ea02bba9129e7857 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0) 2024-07-27T03:22:18.999 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:22:18 smithi149 podman[71819]: 2024-07-27 03:22:18.80101112 +0000 UTC m=+3.170543307 container cleanup 4ef3b65849c3912ae2210bc394e4dad42b6ea83c5325a3a4ea02bba9129e7857 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-e, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-07-27T03:22:19.000 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:22:18 smithi149 bash[71819]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-e 2024-07-27T03:22:19.232 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.e.service' 2024-07-27T03:22:19.254 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:22:19 smithi149 podman[71834]: 2024-07-27 03:22:19.00198234 +0000 UTC m=+0.540355430 container remove 4ef3b65849c3912ae2210bc394e4dad42b6ea83c5325a3a4ea02bba9129e7857 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-e, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-27T03:22:19.255 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:22:19 smithi149 systemd[1]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.e.service: Deactivated successfully. 2024-07-27T03:22:19.255 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:22:19 smithi149 systemd[1]: Stopped Ceph mds.e for c0782a00-4bc4-11ef-bcc7-c7b262605968. 2024-07-27T03:22:19.255 INFO:journalctl@ceph.mds.e.smithi149.stdout:Jul 27 03:22:19 smithi149 systemd[1]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.e.service: Consumed 1.521s CPU time. 2024-07-27T03:22:19.265 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:22:19.265 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-07-27T03:22:19.265 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-07-27T03:22:19.265 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.h 2024-07-27T03:22:19.589 INFO:journalctl@ceph.mds.h.smithi149.stdout:Jul 27 03:22:19 smithi149 systemd[1]: Stopping Ceph mds.h for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:22:19.999 INFO:journalctl@ceph.mds.h.smithi149.stdout:Jul 27 03:22:19 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-h[67002]: 2024-07-27T03:22:19.587+0000 7f3c125cd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:22:19.999 INFO:journalctl@ceph.mds.h.smithi149.stdout:Jul 27 03:22:19 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-h[67002]: 2024-07-27T03:22:19.587+0000 7f3c125cd640 -1 mds.h *** got signal Terminated *** 2024-07-27T03:22:22.749 INFO:journalctl@ceph.mds.h.smithi149.stdout:Jul 27 03:22:22 smithi149 podman[71943]: 2024-07-27 03:22:22.455710129 +0000 UTC m=+2.966283064 container died 7b2d25a41b9125fd0b4e619dfeb67515e479b736028fe144b562c84c9b4219fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240724, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-07-27T03:22:23.107 INFO:journalctl@ceph.mds.h.smithi149.stdout:Jul 27 03:22:22 smithi149 podman[71943]: 2024-07-27 03:22:22.846377512 +0000 UTC m=+3.356950432 container cleanup 7b2d25a41b9125fd0b4e619dfeb67515e479b736028fe144b562c84c9b4219fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240724, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-27T03:22:23.107 INFO:journalctl@ceph.mds.h.smithi149.stdout:Jul 27 03:22:22 smithi149 bash[71943]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-h 2024-07-27T03:22:23.280 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.h.service' 2024-07-27T03:22:23.311 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:22:23.312 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-07-27T03:22:23.312 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-07-27T03:22:23.312 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.k 2024-07-27T03:22:23.643 INFO:journalctl@ceph.mds.k.smithi149.stdout:Jul 27 03:22:23 smithi149 systemd[1]: Stopping Ceph mds.k for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:22:23.999 INFO:journalctl@ceph.mds.k.smithi149.stdout:Jul 27 03:22:23 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-k[66374]: 2024-07-27T03:22:23.640+0000 7f0ebe57f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:22:23.999 INFO:journalctl@ceph.mds.k.smithi149.stdout:Jul 27 03:22:23 smithi149 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-k[66374]: 2024-07-27T03:22:23.640+0000 7f0ebe57f640 -1 mds.k *** got signal Terminated *** 2024-07-27T03:22:26.749 INFO:journalctl@ceph.mds.k.smithi149.stdout:Jul 27 03:22:26 smithi149 podman[72064]: 2024-07-27 03:22:26.461444497 +0000 UTC m=+2.924469230 container died 15e7bc64f44a0d62fac47d8e090c6b46c0e8343719abe26cd2f9038791a6dcbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240724, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-07-27T03:22:27.167 INFO:journalctl@ceph.mds.k.smithi149.stdout:Jul 27 03:22:26 smithi149 podman[72064]: 2024-07-27 03:22:26.783231551 +0000 UTC m=+3.246256288 container cleanup 15e7bc64f44a0d62fac47d8e090c6b46c0e8343719abe26cd2f9038791a6dcbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-07-27T03:22:27.167 INFO:journalctl@ceph.mds.k.smithi149.stdout:Jul 27 03:22:26 smithi149 bash[72064]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-k 2024-07-27T03:22:27.167 INFO:journalctl@ceph.mds.k.smithi149.stdout:Jul 27 03:22:26 smithi149 podman[72078]: 2024-07-27 03:22:26.967449506 +0000 UTC m=+0.502295391 container remove 15e7bc64f44a0d62fac47d8e090c6b46c0e8343719abe26cd2f9038791a6dcbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0) 2024-07-27T03:22:27.182 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.k.service' 2024-07-27T03:22:27.213 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:22:27.213 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-07-27T03:22:27.214 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-07-27T03:22:27.214 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.c 2024-07-27T03:22:27.526 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jul 27 03:22:27 smithi175 systemd[1]: Stopping Ceph mds.c for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:22:27.874 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jul 27 03:22:27 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-c[67516]: 2024-07-27T03:22:27.523+0000 7f31d1fcf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:22:27.874 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jul 27 03:22:27 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-c[67516]: 2024-07-27T03:22:27.523+0000 7f31d1fcf640 -1 mds.c *** got signal Terminated *** 2024-07-27T03:22:30.859 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jul 27 03:22:30 smithi175 podman[72262]: 2024-07-27 03:22:30.451746475 +0000 UTC m=+3.039754231 container died 443e7d24d3cde7feb95c34f15d388f2a36e5d8378bd28296d038f8ceaa9b7976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=) 2024-07-27T03:22:31.124 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jul 27 03:22:30 smithi175 podman[72262]: 2024-07-27 03:22:30.931867071 +0000 UTC m=+3.519874816 container cleanup 443e7d24d3cde7feb95c34f15d388f2a36e5d8378bd28296d038f8ceaa9b7976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-07-27T03:22:31.124 INFO:journalctl@ceph.mds.c.smithi175.stdout:Jul 27 03:22:30 smithi175 bash[72262]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-c 2024-07-27T03:22:31.309 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.c.service' 2024-07-27T03:22:31.342 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:22:31.342 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-07-27T03:22:31.342 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-07-27T03:22:31.342 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.f 2024-07-27T03:22:31.874 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jul 27 03:22:31 smithi175 systemd[1]: Stopping Ceph mds.f for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:22:31.874 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jul 27 03:22:31 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-f[67211]: 2024-07-27T03:22:31.635+0000 7f07f7b9b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:22:31.874 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jul 27 03:22:31 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-f[67211]: 2024-07-27T03:22:31.635+0000 7f07f7b9b640 -1 mds.f *** got signal Terminated *** 2024-07-27T03:22:34.779 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jul 27 03:22:34 smithi175 podman[72385]: 2024-07-27 03:22:34.453373409 +0000 UTC m=+2.895641143 container died 73a994aac40f55053ce097e9c8d068cbd87107b501cdbc8ed0b1a7a31acc0afc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240724, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-07-27T03:22:35.070 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jul 27 03:22:34 smithi175 podman[72385]: 2024-07-27 03:22:34.867812893 +0000 UTC m=+3.310080659 container cleanup 73a994aac40f55053ce097e9c8d068cbd87107b501cdbc8ed0b1a7a31acc0afc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240724, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-07-27T03:22:35.070 INFO:journalctl@ceph.mds.f.smithi175.stdout:Jul 27 03:22:34 smithi175 bash[72385]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-f 2024-07-27T03:22:35.289 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.f.service' 2024-07-27T03:22:35.321 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:22:35.322 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-07-27T03:22:35.322 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-07-27T03:22:35.322 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.i 2024-07-27T03:22:35.624 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jul 27 03:22:35 smithi175 systemd[1]: Stopping Ceph mds.i for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:22:36.124 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jul 27 03:22:35 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-i[67827]: 2024-07-27T03:22:35.654+0000 7f185a46b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:22:36.124 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jul 27 03:22:35 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-i[67827]: 2024-07-27T03:22:35.654+0000 7f185a46b640 -1 mds.i *** got signal Terminated *** 2024-07-27T03:22:38.823 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jul 27 03:22:38 smithi175 podman[72507]: 2024-07-27 03:22:38.461343702 +0000 UTC m=+2.912709308 container died 2cf721d5493231d6ae04a66d2498d4caecf8a0a87b0b037ace8f21899528e900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-i, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240724) 2024-07-27T03:22:39.124 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jul 27 03:22:38 smithi175 podman[72507]: 2024-07-27 03:22:38.920768497 +0000 UTC m=+3.372134105 container cleanup 2cf721d5493231d6ae04a66d2498d4caecf8a0a87b0b037ace8f21899528e900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-i, org.label-schema.build-date=20240724, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-07-27T03:22:39.124 INFO:journalctl@ceph.mds.i.smithi175.stdout:Jul 27 03:22:38 smithi175 bash[72507]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-i 2024-07-27T03:22:39.493 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.i.service' 2024-07-27T03:22:39.525 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:22:39.525 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-07-27T03:22:39.525 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-07-27T03:22:39.525 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.l 2024-07-27T03:22:39.850 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jul 27 03:22:39 smithi175 systemd[1]: Stopping Ceph mds.l for c0782a00-4bc4-11ef-bcc7-c7b262605968... 2024-07-27T03:22:40.124 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jul 27 03:22:39 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-l[66909]: 2024-07-27T03:22:39.848+0000 7f6cf08da640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-07-27T03:22:40.124 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jul 27 03:22:39 smithi175 ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-l[66909]: 2024-07-27T03:22:39.848+0000 7f6cf08da640 -1 mds.l *** got signal Terminated *** 2024-07-27T03:22:42.759 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jul 27 03:22:42 smithi175 podman[72629]: 2024-07-27 03:22:42.469187106 +0000 UTC m=+2.719499054 container died c5738d3a87e5e5bd67334cbf61a45738faf18aaa0bd29c12417546a4f0019fac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-l, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, org.label-schema.build-date=20240724, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-07-27T03:22:42.759 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jul 27 03:22:42 smithi175 podman[72629]: 2024-07-27 03:22:42.757407992 +0000 UTC m=+3.007719941 container cleanup c5738d3a87e5e5bd67334cbf61a45738faf18aaa0bd29c12417546a4f0019fac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240724, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-07-27T03:22:43.117 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jul 27 03:22:42 smithi175 bash[72629]: ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-l 2024-07-27T03:22:43.118 INFO:journalctl@ceph.mds.l.smithi175.stdout:Jul 27 03:22:42 smithi175 podman[72644]: 2024-07-27 03:22:42.924913643 +0000 UTC m=+0.452191336 container remove c5738d3a87e5e5bd67334cbf61a45738faf18aaa0bd29c12417546a4f0019fac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9aaf4d60e8d705336836dbcbfec1a0782d3f0e61410a4eae49591e4cd808a15c, name=ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-pdonnell-testing-20240726.202642-debug-af8fdbf, ceph=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240724, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-07-27T03:22:43.132 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c0782a00-4bc4-11ef-bcc7-c7b262605968@mds.l.service' 2024-07-27T03:22:43.163 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-07-27T03:22:43.163 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-07-27T03:22:43.164 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 --force --keep-logs 2024-07-27T03:22:43.381 INFO:teuthology.orchestra.run.smithi067.stdout:Deleting cluster with fsid: c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:22:46.104 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 --force --keep-logs 2024-07-27T03:22:46.309 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:22:48.700 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 --force --keep-logs 2024-07-27T03:22:48.902 INFO:teuthology.orchestra.run.smithi175.stdout:Deleting cluster with fsid: c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:22:51.201 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-27T03:22:51.235 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-27T03:22:51.266 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-07-27T03:22:51.292 INFO:tasks.cephadm:Archiving crash dumps... 2024-07-27T03:22:51.294 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-07-26_23:45:44-fs-wip-pdonnell-testing-20240726.202642-debug-distro-default-smithi/7820977/remote/smithi067/crash 2024-07-27T03:22:51.294 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/crash -- . 2024-07-27T03:22:51.335 INFO:teuthology.orchestra.run.smithi067.stderr:tar: /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/crash: Cannot open: No such file or directory 2024-07-27T03:22:51.335 INFO:teuthology.orchestra.run.smithi067.stderr:tar: Error is not recoverable: exiting now 2024-07-27T03:22:51.337 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-07-26_23:45:44-fs-wip-pdonnell-testing-20240726.202642-debug-distro-default-smithi/7820977/remote/smithi149/crash 2024-07-27T03:22:51.338 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/crash -- . 2024-07-27T03:22:51.377 INFO:teuthology.orchestra.run.smithi149.stderr:tar: /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/crash: Cannot open: No such file or directory 2024-07-27T03:22:51.377 INFO:teuthology.orchestra.run.smithi149.stderr:tar: Error is not recoverable: exiting now 2024-07-27T03:22:51.379 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-07-26_23:45:44-fs-wip-pdonnell-testing-20240726.202642-debug-distro-default-smithi/7820977/remote/smithi175/crash 2024-07-27T03:22:51.379 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/crash -- . 2024-07-27T03:22:51.412 INFO:teuthology.orchestra.run.smithi175.stderr:tar: /var/lib/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/crash: Cannot open: No such file or directory 2024-07-27T03:22:51.412 INFO:teuthology.orchestra.run.smithi175.stderr:tar: Error is not recoverable: exiting now 2024-07-27T03:22:51.413 INFO:tasks.cephadm:Checking cluster log for badness... 2024-07-27T03:22:51.414 DEBUG:teuthology.orchestra.run.smithi067:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-07-27T03:22:51.450 INFO:teuthology.orchestra.run.smithi067.stdout:2024-07-27T03:20:00.000143+0000 mon.a (mon.0) 1250 : cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-07-27T03:22:51.451 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-07-27T03:22:51.451 DEBUG:teuthology.orchestra.run.smithi067:> sudo egrep '\[SEC\]' /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-07-27T03:22:51.531 DEBUG:teuthology.orchestra.run.smithi067:> sudo egrep '\[ERR\]' /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-07-27T03:22:51.607 INFO:teuthology.orchestra.run.smithi067.stdout:2024-07-27T03:20:00.000143+0000 mon.a (mon.0) 1250 : cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-07-27T03:22:51.608 INFO:tasks.cephadm:Compressing logs... 2024-07-27T03:22:51.608 DEBUG:teuthology.orchestra.run.smithi067:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-07-27T03:22:51.651 DEBUG:teuthology.orchestra.run.smithi149:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-07-27T03:22:51.654 DEBUG:teuthology.orchestra.run.smithi175:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-07-27T03:22:51.675 INFO:teuthology.orchestra.run.smithi067.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.log 2024-07-27T03:22:51.675 INFO:teuthology.orchestra.run.smithi067.stderr:: No such file or directory 2024-07-27T03:22:51.675 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.audit.log 2024-07-27T03:22:51.677 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.g.log 2024-07-27T03:22:51.677 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.log: /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.j.log 2024-07-27T03:22:51.677 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.3.log 2024-07-27T03:22:51.677 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.0.log 2024-07-27T03:22:51.678 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.j.log: /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mgr.x.log 2024-07-27T03:22:51.679 INFO:teuthology.orchestra.run.smithi067.stderr: 86.8% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.log.gz 2024-07-27T03:22:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mon.a.log 2024-07-27T03:22:51.679 INFO:teuthology.orchestra.run.smithi175.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.log 2024-07-27T03:22:51.679 INFO:teuthology.orchestra.run.smithi175.stderr:: No such file or directory 2024-07-27T03:22:51.679 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.2.log 2024-07-27T03:22:51.679 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.f.log 2024-07-27T03:22:51.679 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.log: /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.audit.log 2024-07-27T03:22:51.683 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.5.log 2024-07-27T03:22:51.683 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.c.log 2024-07-27T03:22:51.683 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.audit.log: /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mon.c.log 2024-07-27T03:22:51.683 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mgr.z.log 2024-07-27T03:22:51.683 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.i.log 2024-07-27T03:22:51.684 INFO:teuthology.orchestra.run.smithi175.stderr: 86.8% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.log.gz 2024-07-27T03:22:51.684 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mgr.z.log: 90.2% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.audit.log.gz 2024-07-27T03:22:51.690 INFO:teuthology.orchestra.run.smithi175.stderr: 92.5% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mgr.z.log.gz 2024-07-27T03:22:51.690 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.cephadm.log 2024-07-27T03:22:51.690 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.11.log 2024-07-27T03:22:51.690 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.8.log 2024-07-27T03:22:51.690 INFO:teuthology.orchestra.run.smithi175.stderr: 82.4% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.cephadm.log.gz 2024-07-27T03:22:51.690 INFO:teuthology.orchestra.run.smithi149.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.log 2024-07-27T03:22:51.690 INFO:teuthology.orchestra.run.smithi149.stderr:: No such file or directory 2024-07-27T03:22:51.690 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.audit.log 2024-07-27T03:22:51.691 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.7.log 2024-07-27T03:22:51.691 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.log: /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.b.log 2024-07-27T03:22:51.691 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.4.log 2024-07-27T03:22:51.691 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.1.log 2024-07-27T03:22:51.691 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.b.log: /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mgr.y.log 2024-07-27T03:22:51.691 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.h.log 2024-07-27T03:22:51.691 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose 86.8% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.log.gz 2024-07-27T03:22:51.691 INFO:teuthology.orchestra.run.smithi149.stderr: -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.10.log 2024-07-27T03:22:51.691 INFO:teuthology.orchestra.run.smithi149.stderr: 90.2% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.audit.log.gz 2024-07-27T03:22:51.691 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.cephadm.log 2024-07-27T03:22:51.691 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.h.log: /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.e.log 2024-07-27T03:22:51.691 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.cephadm.log: 82.5% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.cephadm.log.gz 2024-07-27T03:22:51.691 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-volume.log 2024-07-27T03:22:51.691 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.e.log: 92.4% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mgr.y.log.gz 2024-07-27T03:22:51.692 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.6.log 2024-07-27T03:22:51.692 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mgr.x.log: /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mon.a.log: /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.6.log: 90.0% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.audit.log.gz 2024-07-27T03:22:51.692 INFO:teuthology.orchestra.run.smithi067.stderr: 95.0% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.j.log.gz 2024-07-27T03:22:51.692 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.cephadm.log 2024-07-27T03:22:51.692 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.a.log 2024-07-27T03:22:51.692 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-volume.log 2024-07-27T03:22:51.692 INFO:teuthology.orchestra.run.smithi067.stderr: 82.6% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph.cephadm.log.gz 2024-07-27T03:22:51.692 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.d.log 2024-07-27T03:22:51.695 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.9.log 2024-07-27T03:22:51.697 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mon.b.log 2024-07-27T03:22:51.699 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-volume.log 2024-07-27T03:22:51.700 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.d.log: 91.0% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-volume.log.gz 2024-07-27T03:22:51.700 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-07-27T03:22:51.709 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.k.log 2024-07-27T03:22:51.711 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-volume.log.gz 2024-07-27T03:22:51.713 INFO:teuthology.orchestra.run.smithi067.stderr: 95.0%/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.9.log: -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.d.log.gz 2024-07-27T03:22:51.713 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-07-27T03:22:51.714 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.l.log 2024-07-27T03:22:51.720 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-07-27T03:22:51.723 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-07-27T03:22:51.724 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/cephadm.log: 94.6%/var/log/ceph/ceph-client.admin.log: -- replaced with /var/log/ceph/cephadm.log.gz 2024-07-27T03:22:51.726 INFO:teuthology.orchestra.run.smithi067.stderr: 94.2% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.g.log.gz 2024-07-27T03:22:51.729 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.l.log: 92.8% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.c.log.gz 2024-07-27T03:22:51.729 INFO:teuthology.orchestra.run.smithi067.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-07-27T03:22:51.735 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.k.log: 92.8% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.e.log.gz 2024-07-27T03:22:51.735 INFO:teuthology.orchestra.run.smithi175.stderr: 91.0% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-volume.log.gz 2024-07-27T03:22:51.737 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-07-27T03:22:51.737 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-07-27T03:22:51.738 INFO:teuthology.orchestra.run.smithi067.stderr: 94.2% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.a.log.gz 2024-07-27T03:22:51.739 INFO:teuthology.orchestra.run.smithi149.stderr: 92.5% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.h.log.gz 2024-07-27T03:22:51.743 INFO:teuthology.orchestra.run.smithi175.stderr: 92.6% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.i.log.gz 2024-07-27T03:22:51.756 INFO:teuthology.orchestra.run.smithi175.stderr: 93.6% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.f.log.gz 2024-07-27T03:22:51.766 INFO:teuthology.orchestra.run.smithi175.stderr: 94.1% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.l.log.gz 2024-07-27T03:22:51.780 INFO:teuthology.orchestra.run.smithi149.stderr: 94.2% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.k.log.gz 2024-07-27T03:22:51.852 INFO:teuthology.orchestra.run.smithi067.stderr: 90.0% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mgr.x.log.gz 2024-07-27T03:22:51.925 INFO:teuthology.orchestra.run.smithi175.stderr: 91.7% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mon.c.log.gz 2024-07-27T03:22:51.992 INFO:teuthology.orchestra.run.smithi149.stderr: 92.8% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mds.b.log.gz 2024-07-27T03:22:52.023 INFO:teuthology.orchestra.run.smithi149.stderr: 91.7% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mon.b.log.gz 2024-07-27T03:22:52.155 INFO:teuthology.orchestra.run.smithi175.stderr: 92.9% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.11.log.gz 2024-07-27T03:22:52.326 INFO:teuthology.orchestra.run.smithi175.stderr: 92.7% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.8.log.gz 2024-07-27T03:22:52.372 INFO:teuthology.orchestra.run.smithi067.stderr: 92.6% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.3.log.gz 2024-07-27T03:22:52.436 INFO:teuthology.orchestra.run.smithi175.stderr: 92.7% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.5.log.gz 2024-07-27T03:22:52.438 INFO:teuthology.orchestra.run.smithi149.stderr: 92.7% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.7.log.gz 2024-07-27T03:22:52.498 INFO:teuthology.orchestra.run.smithi067.stderr: 92.6% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.6.log.gz 2024-07-27T03:22:52.556 INFO:teuthology.orchestra.run.smithi067.stderr: 90.5% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-mon.a.log.gz 2024-07-27T03:22:52.857 INFO:teuthology.orchestra.run.smithi149.stderr: 93.5% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.4.log.gz 2024-07-27T03:22:52.910 INFO:teuthology.orchestra.run.smithi149.stderr: 93.4% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.1.log.gz 2024-07-27T03:22:52.911 INFO:teuthology.orchestra.run.smithi175.stderr: 93.4% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.2.log.gz 2024-07-27T03:22:52.913 INFO:teuthology.orchestra.run.smithi175.stderr: 2024-07-27T03:22:52.913 INFO:teuthology.orchestra.run.smithi175.stderr:real 0m1.246s 2024-07-27T03:22:52.914 INFO:teuthology.orchestra.run.smithi175.stderr:user 0m3.433s 2024-07-27T03:22:52.914 INFO:teuthology.orchestra.run.smithi175.stderr:sys 0m0.157s 2024-07-27T03:22:52.948 INFO:teuthology.orchestra.run.smithi067.stderr: 93.5% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.0.log.gz 2024-07-27T03:22:52.951 INFO:teuthology.orchestra.run.smithi067.stderr: 93.4% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.9.log.gz 2024-07-27T03:22:52.953 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-07-27T03:22:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:real 0m1.288s 2024-07-27T03:22:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:user 0m5.056s 2024-07-27T03:22:52.953 INFO:teuthology.orchestra.run.smithi067.stderr:sys 0m0.187s 2024-07-27T03:22:52.994 INFO:teuthology.orchestra.run.smithi149.stderr: 93.4% -- replaced with /var/log/ceph/c0782a00-4bc4-11ef-bcc7-c7b262605968/ceph-osd.10.log.gz 2024-07-27T03:22:52.996 INFO:teuthology.orchestra.run.smithi149.stderr: 2024-07-27T03:22:52.996 INFO:teuthology.orchestra.run.smithi149.stderr:real 0m1.328s 2024-07-27T03:22:52.996 INFO:teuthology.orchestra.run.smithi149.stderr:user 0m5.096s 2024-07-27T03:22:52.996 INFO:teuthology.orchestra.run.smithi149.stderr:sys 0m0.204s 2024-07-27T03:22:52.996 INFO:tasks.cephadm:Archiving logs... 2024-07-27T03:22:52.997 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-07-26_23:45:44-fs-wip-pdonnell-testing-20240726.202642-debug-distro-default-smithi/7820977/remote/smithi067/log 2024-07-27T03:22:52.998 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/log/ceph -- . 2024-07-27T03:22:53.698 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-07-26_23:45:44-fs-wip-pdonnell-testing-20240726.202642-debug-distro-default-smithi/7820977/remote/smithi149/log 2024-07-27T03:22:53.699 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/log/ceph -- . 2024-07-27T03:22:54.421 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-07-26_23:45:44-fs-wip-pdonnell-testing-20240726.202642-debug-distro-default-smithi/7820977/remote/smithi175/log 2024-07-27T03:22:54.422 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/log/ceph -- . 2024-07-27T03:22:54.976 INFO:tasks.cephadm:Removing cluster... 2024-07-27T03:22:54.976 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 --force 2024-07-27T03:22:55.190 INFO:teuthology.orchestra.run.smithi067.stdout:Deleting cluster with fsid: c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:22:55.472 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 --force 2024-07-27T03:22:55.681 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:22:55.947 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c0782a00-4bc4-11ef-bcc7-c7b262605968 --force 2024-07-27T03:22:56.148 INFO:teuthology.orchestra.run.smithi175.stdout:Deleting cluster with fsid: c0782a00-4bc4-11ef-bcc7-c7b262605968 2024-07-27T03:22:56.399 INFO:tasks.cephadm:Removing cephadm ... 2024-07-27T03:22:56.399 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf /home/ubuntu/cephtest/cephadm 2024-07-27T03:22:56.419 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf /home/ubuntu/cephtest/cephadm 2024-07-27T03:22:56.435 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf /home/ubuntu/cephtest/cephadm 2024-07-27T03:22:56.450 INFO:tasks.cephadm:Teardown complete 2024-07-27T03:22:56.451 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-07-27T03:22:56.463 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-07-27T03:22:56.463 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-07-27T03:22:56.466 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-07-27T03:22:56.478 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-07-27T03:22:56.583 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-07-27T03:22:56.593 INFO:teuthology.task.clock:Checking final clock skew... 2024-07-27T03:22:56.593 DEBUG:teuthology.orchestra.run.smithi067:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-27T03:22:56.596 DEBUG:teuthology.orchestra.run.smithi149:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-27T03:22:56.599 DEBUG:teuthology.orchestra.run.smithi175:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-07-27T03:22:56.610 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-07-27T03:22:56.611 INFO:teuthology.orchestra.run.smithi067.stderr:bash: line 1: ntpq: command not found 2024-07-27T03:22:56.612 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-07-27T03:22:56.615 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-27T03:22:56.615 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-07-27T03:22:56.615 INFO:teuthology.orchestra.run.smithi149.stdout:^* hv01.front.sepia.ceph.com 3 6 377 60 -216us[ -216us] +/- 62ms 2024-07-27T03:22:56.615 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 60 -304us[ -304us] +/- 60ms 2024-07-27T03:22:56.615 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 +568us[ +568us] +/- 68ms 2024-07-27T03:22:56.615 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-07-27T03:22:56.617 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-27T03:22:56.617 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-07-27T03:22:56.617 INFO:teuthology.orchestra.run.smithi175.stdout:^* hv01.front.sepia.ceph.com 3 6 377 59 -206us[ -207us] +/- 62ms 2024-07-27T03:22:56.617 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 60 -285us[ -287us] +/- 60ms 2024-07-27T03:22:56.617 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 +582us[ +581us] +/- 68ms 2024-07-27T03:22:56.617 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-07-27T03:22:56.617 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-07-27T03:22:56.618 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2024-07-27T03:22:56.618 INFO:teuthology.orchestra.run.smithi067.stdout:^* hv01.front.sepia.ceph.com 3 6 377 60 -219us[ -247us] +/- 62ms 2024-07-27T03:22:56.618 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 60 -288us[ -316us] +/- 60ms 2024-07-27T03:22:56.618 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 +570us[ +542us] +/- 68ms 2024-07-27T03:22:56.618 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-07-27T03:22:56.618 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-07-27T03:22:56.628 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-07-27T03:22:56.628 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-07-27T03:22:56.639 DEBUG:teuthology.orchestra.run.smithi067:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-07-27T03:22:56.745 DEBUG:teuthology.orchestra.run.smithi149:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-07-27T03:22:56.809 DEBUG:teuthology.orchestra.run.smithi175:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-07-27T03:22:56.873 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-27T03:22:56.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-27T03:22:56.905 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-27T03:22:56.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-27T03:22:56.932 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-07-27T03:22:56.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-27T03:22:56.959 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-07-27T03:22:56.970 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-07-27T03:22:56.980 INFO:teuthology.task.internal:Duration was 1580.812747 seconds 2024-07-27T03:22:56.980 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-07-27T03:22:56.991 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-07-27T03:22:56.991 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-07-27T03:22:56.994 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-07-27T03:22:56.996 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-07-27T03:22:57.030 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-27T03:22:57.033 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-27T03:22:57.034 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-07-27T03:22:57.575 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-07-27T03:22:57.575 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi067.front.sepia.ceph.com 2024-07-27T03:22:57.575 DEBUG:teuthology.orchestra.run.smithi067:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-07-27T03:22:57.598 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi149.front.sepia.ceph.com 2024-07-27T03:22:57.599 DEBUG:teuthology.orchestra.run.smithi149:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-07-27T03:22:57.642 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi175.front.sepia.ceph.com 2024-07-27T03:22:57.642 DEBUG:teuthology.orchestra.run.smithi175:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-07-27T03:22:57.662 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-07-27T03:22:57.662 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-07-27T03:22:57.665 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-07-27T03:22:57.685 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-07-27T03:22:57.728 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-07-27T03:22:57.728 DEBUG:teuthology.orchestra.run.smithi067:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-07-27T03:22:57.734 DEBUG:teuthology.orchestra.run.smithi149:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-07-27T03:22:57.750 DEBUG:teuthology.orchestra.run.smithi175:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-07-27T03:22:58.473 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-07-27T03:22:58.485 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-07-27T03:22:58.486 DEBUG:teuthology.orchestra.run.smithi067:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-07-27T03:22:58.512 DEBUG:teuthology.orchestra.run.smithi149:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-07-27T03:22:58.538 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-07-27T03:22:58.562 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-07-27T03:22:58.574 DEBUG:teuthology.orchestra.run.smithi067:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-07-27T03:22:58.577 DEBUG:teuthology.orchestra.run.smithi149:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-07-27T03:22:58.581 DEBUG:teuthology.orchestra.run.smithi175:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-07-27T03:22:58.599 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = core 2024-07-27T03:22:58.602 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = core 2024-07-27T03:22:58.625 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = core 2024-07-27T03:22:58.656 DEBUG:teuthology.orchestra.run.smithi067:> test -e /home/ubuntu/cephtest/archive/coredump 2024-07-27T03:22:58.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-27T03:22:58.672 DEBUG:teuthology.orchestra.run.smithi149:> test -e /home/ubuntu/cephtest/archive/coredump 2024-07-27T03:22:58.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-27T03:22:58.686 DEBUG:teuthology.orchestra.run.smithi175:> test -e /home/ubuntu/cephtest/archive/coredump 2024-07-27T03:22:58.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-07-27T03:22:58.711 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-07-27T03:22:58.722 INFO:teuthology.task.internal:Transferring archived files... 2024-07-27T03:22:58.723 DEBUG:teuthology.misc:Transferring archived files from smithi067:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-07-26_23:45:44-fs-wip-pdonnell-testing-20240726.202642-debug-distro-default-smithi/7820977/remote/smithi067 2024-07-27T03:22:58.723 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-07-27T03:22:58.776 DEBUG:teuthology.misc:Transferring archived files from smithi149:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-07-26_23:45:44-fs-wip-pdonnell-testing-20240726.202642-debug-distro-default-smithi/7820977/remote/smithi149 2024-07-27T03:22:58.776 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-07-27T03:22:58.811 DEBUG:teuthology.misc:Transferring archived files from smithi175:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-07-26_23:45:44-fs-wip-pdonnell-testing-20240726.202642-debug-distro-default-smithi/7820977/remote/smithi175 2024-07-27T03:22:58.811 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-07-27T03:22:58.845 INFO:teuthology.task.internal:Removing archive directory... 2024-07-27T03:22:58.845 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf -- /home/ubuntu/cephtest/archive 2024-07-27T03:22:58.848 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf -- /home/ubuntu/cephtest/archive 2024-07-27T03:22:58.850 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf -- /home/ubuntu/cephtest/archive 2024-07-27T03:22:58.893 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-07-27T03:22:58.906 INFO:teuthology.task.internal:Not uploading archives. 2024-07-27T03:22:58.906 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-07-27T03:22:58.917 INFO:teuthology.task.internal:Tidying up after the test... 2024-07-27T03:22:58.918 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-07-27T03:22:58.920 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-07-27T03:22:58.922 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-07-27T03:22:58.933 INFO:teuthology.orchestra.run.smithi067.stdout: 265686 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 27 03:22 /home/ubuntu/cephtest 2024-07-27T03:22:58.934 INFO:teuthology.orchestra.run.smithi149.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 27 03:22 /home/ubuntu/cephtest 2024-07-27T03:22:58.948 INFO:teuthology.orchestra.run.smithi175.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 27 03:22 /home/ubuntu/cephtest 2024-07-27T03:22:58.949 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-07-27T03:22:58.960 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-07-27T03:22:58.988 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/fsync-tester}} duration: 1580.8127467632294 failure_reason: '"2024-07-27T03:20:00.000143+0000 mon.a (mon.0) 1250 : cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline" in cluster log' flavor: default owner: scheduled_pdonnell@teuthology success: false 2024-07-27T03:22:58.988 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-07-27T03:22:59.068 INFO:teuthology.run:FAIL